Software Under the Regulatory Microscope: The Current and Future State of Enforcement for Regulated Computer

Wednesday, March 24, 2021

The Current Regulatory Framework

“Software,” as a term in the regulated life sciences industry, has a wide variety of definitions and uses. For example, software can be a stand- alone medical device devoid of dedicated hardware, embedded in an electro-mechanical medical device, embedded in manufacturing production equipment, used as an application for product design, or used to generate spreadsheets for making decisions regarding regulated Quality System processes. As compared to other software uses in the industry, software that is used to support manufacturing production or to automate work or decision-making within the Quality System is governed by a comparatively small library of regulatory literature and relatively inconsistent application of regulatory oversight globally.

The existing guidelines and requirements from regulatory competent authorities and industry groups generally recommend that software systems used to automate manufacturing production or process activities required by regulations (i.e., the Quality System) be designed, developed, validated, implemented, and maintained in a structured way that takes into consideration the principles of strong risk management, change management, and traceability.

Subscribe to our e-Newsletters
Stay up to date with the latest news, articles, and events. Plus, get special offers
from American Pharmaceutical Review – all delivered right to your inbox! Sign up now!

For example, the Food and Drug Administration’s (FDA) regulations contain three primary content elements that address the use of regulated computer systems:

  • 21 CFR § 820.70(i) regulates the use of computer systems used to support production or the Quality System in medical device manufacturers.
  • 21 CFR § 211.68 speaks to the use of computer systems for “manufacture, processing, packing, and holding of a drug product.”
  • 21 CFR Part 11 addresses the use of electronic records and electronic signatures across all industries regulated by the U.S. Food and Drug Administration (FDA).

Elaborating on these regulations, FDA has also authored, or participated in the development of a small number of guidance documents and technical reports. More specifically, FDA’s Centers for Drug Evaluation and Research (CDER), Biologics Evaluation and Research (CBER), and Veterinary Medicine (CVM) collaborated in 2016 to draft the “Data Integrity and Compliance with CGMP: Guidance for Industry),1 which was finalized in 2018. A footnote included in this guidance references the “General Principles of Software Validation”2 guidance that was finalized in 2002 by FDA’s Center for Devices and Radiological Health (CDRH). FDA has also collaborated with the International Society for Pharmaceutical Engineering (ISPE) in the development of “A Risk- Based Approach to Compliant GxP Computerized Systems” (GAMP)3 and served as co-chair of the Association for the Advancement of Medical Instrumentation’s (AAMI) AAMI TIR36:2007 (“Validation of software for regulated purposes”)4 and AAMI/ISO TIR80002-2:2017 (Medical device software – Part 2: Validation of software for medical device quality systems”).5

The above referenced documents are used by regulatory competent authorities internationally in concert with EudraLex (The Rules Governing Medicinal Products in the European Union), Volume 4 (Good Manufacturing Practice: Medicinal Products for Human and Veterinary Use), Annex 11: Computerized Systems.6


The Current State of Regulatory Enforcement

The United States and Australia have the clearest and most accessible data concerning inspectional citations relating to non-medical device regulated software. From a regulatory perspective, the data in these two countries reflect significantly different enforcement priorities.

In the United States, FDA drug facility inspections conducted between 2016 and 2020 cited firms for software issues 324 times. With respect to device facility inspections, within the same time period FDA cited software deficiencies 118 times. Across all FDA- regulated industries, FDA separately cited deficiencies relating to electronic records and signatures 74 times in the last five years. While the number of citations against software-related regulations may seem significant, these observations by FDA investigators constitute only 2.9% of the total number of inspection observations (11,136) against pharmaceutical companies and only 1.0% of the total (11,487) against medical device firms.7

These numbers also reflect different enforcement priorities for CDER and CDRH. For drug manufacturers, software deficiencies consistently fall within the top 10 (in terms of frequency) types of citations, while procedure issues, lab controls, and failure to investigate discrepancies consistently top the list. For device manufacturers, software deficiencies are more likely to fall within the top 25 to 35 types of citations, while procedure issues, complaints, and purchasing controls top the list.

By contrast, the statistics for Australia paint an entirely different picture. In a November 2019 presentation, given by Jefferey Wan of the Australian Therapeutic Goods Administration (TGA),8 inspection data for the combined 2018/2019 fiscal year reflects much more emphasis on “computerized systems.” Specifically, deficiencies relating to computerized systems (Annex 11), particularly audit trails and system security, constituted 24% of the total major deficiencies cited for domestic inspections during this period, and 37% of the total major deficiencies cited for international inspections. The only type of deficiencies cited more frequently relate to qualification and validation (Annex 15). It also appears that citations concerning validation of computerized systems and data integrity are trending upward.

It is worth noting here that increased use of the Medical Device Single Audit Program (MDSAP), where commercial notified body MDSAP audit results are recognized by Australia, Brazil, Canada, Japan, and the U.S., is starting to cloud this data because the findings from MDSAP audits are not recorded as 483 observations in the FDA datasheets or as deficiencies in TGA inspection data. Rather, MDSAP audit results are recognized in lieu of certain routine FDA9 and TGA10 inspections of medical device facilities.

Sources of Software-Related Observations, Findings, and Deficiencies

In my experience as a leader and advisor for companies in the life sciences industry, software-related issues spring from three typical sources: manufacturing floor walkthroughs, clinical study or Bioresearch Manufacturing (BIMO) inspections, and quality records generated from tools.

1. Manufacturing Floor Walkthrough

Items regulators and notified bodies tend to focus on during facility tours include manufacturing equipment, obvious usage of manufacturing execution systems, document control systems, and whether there are software-driven tools validating the system for intended use. Manufacturers often consider qualification and calibration of such equipment and systems during manufacturing process validation as sufficient evidence that the system is fit for use. The failure, however, to include clear software validation evidence in the process validation or equipment qualification scheme may result in citations.

2. Clinical Study or BIMO Inspection

Systems involved in the collection, analysis, and/or dissemination of data during a clinical study or trial are frequent targets during inspections, particularly FDA BIMO inspections. As part of the design and validation packages, manufacturers should expect investigators and auditors to look for evidence of effective cybersecurity (including security provisions in cloud-based systems or services), measures to ensure the integrity of data both in motion and at rest, rigorous specification and validation of interfaces (clear software architectures might be helpful here), and the vetting of algorithms incorporating machine learning to analyze data.

Data privacy issues overlap the above focus areas and can also get manufacturers into trouble during facility inspections. Although FDA is not directly responsible for regulating privacy compliance (e.g., Health Insurance Portability and Accountability Act (HIPAA)), investigators that notice violations may channel them to the appropriate government agency, such as the U.S. Department of Commerce. Similarly, notified bodies that support EU audits do not target the General Data Protection Regulation (GDPR) explicitly, but both cybersecurity and data integrity topics may reveal GDPR compliance gaps.

3. Quality Records Generated from Tools

Given the ubiquitous nature of software tools used to support Quality System procedures and execution, it is no surprise that these tools are also integral to inspections and audits. Many companies still provide FDA investigators with paper records during inspections, which are often generated by software systems. As a result, the natural next question for the investigator is whether such systems have been validated, and if so, whether there is documented evidence of that validation. Common processes driving this line of inquiry include document control, corrective action/preventive action (CAPA), design and development tools, spreadsheets used to support management reviews or regulatory decision-making, complaint management, product non-conformances and defects, and statistical trending and analysis.

In my experience, companies sometimes mistakenly believe that only software systems directly related to manufacturing must be validated. In order to ensure compliance with all applicable regulations, standards, and guidance, however, regulators expect both pharmaceutical and medical device companies to validate all systems that automate Quality System processes.

Lessons Learned from FDA Warning Letters

Although less common, deficiencies in regulated software systems occasionally appear in FDA warning letters, which the agency issues to warn companies of the potential for additional enforcement action when it believes a company’s responses to inspection observations are inadequate. To provide companies with some insight into what regulators are concerned about, listed below are examples of non- device regulated software deficiencies by topic that have been noted in FDA warning letters, with advice on how to address each.

Change Management

Manufacturers should be mindful that all changes must be validated. It is a common misconception that validation is only required for changes that are “significant” or impact the intended use of a system, but there is no support for this view in the regulatory literature.

Enhancements and Defects

Manufacturers should expect enhancement requests and defect re- cords to be scrutinized, as regulators will want to understand why any unincorporated changes were not implemented previously. Investigators and auditors may also expect to see a timeline for imple- menting solutions to any unresolved issues or “critical enhancements.”

Executing Validation Plans

Manufacturers should be mindful that investigators and auditors will expect a validation plan that defines the types of testing to be conducted during validation (as it should) to be executed, and for any deviations to be recorded in the validation report.

Data Integrity

Manufacturers should be aware that the data integrity of any system producing quality records (including clinical study data and analysis) may be an area of focus during an inspection or audit. Regulators are likely to evaluate data integrity using the five principals known as “ALCOA” (or similar), which stands for “Attributable, Legible, Contemporaneous, Original, Accurate.”

Spreadsheets

Manufacturers beware - the requirement for validation of spreadsheets used for compliance-related decision-making, or to otherwise support the Quality System, is real.

Retrospective Remediation and Systemic Thinking

To be considered acceptable by FDA, a response to an inspectional observation or finding must involve a systemic and retrospective remediation of issues (think CAPA). One of the most common criticisms FDA includes in its warning letters is, in fact, that manufacturers address only the examples of non-compliance stated in the inspection observations, without investigating and resolving the systemic root cause(s) of the issues.

A Look to the Future

As both the technology and regulatory landscapes evolve for life sciences companies, adjustments must be made to the way requirements for software validation are viewed and how lifecycle activities are executed in light of these changes.

During 2021, FDA is likely to publish a “Computer Software Assurance” guidance,11 which will redefine the expectations for regulated software lifecycles. The new guidance is expected to focus more on regulated software impacts to device safety, device quality, and the integrity of the Quality System. The new guidance will also likely provide a wider variety of acceptable objective evidence for substantiating that the system meets its intended use requirements, and it is anticipated that there will be an explicit acknowledgement of iterative lifecycle models. Altogether, manufacturers should expect the new guidance to place greater emphasis on both risk management and change management.

Additionally, as malicious attacks on healthcare-related software systems and networks become more frequent and sophisticated, manufacturers should also expect both implicit and explicit regulatory expectations regarding cybersecurity to increase and become more granular. Using the NIST “Framework for Improving Critical Infrastructure Cybersecurity”12 and existing global medical device cybersecurity guidance (even if not in the medical device space) may better prepare companies to tackle the ever-increasing technical and regulatory system security challenges.

Companies should also bear in mind that artificial/augmented intelligence and machine learning are playing an increasingly prominent role in the analysis of data for clinical studies, manufacturing capabilities, and product performance in the field. As FDA develops a regulatory framework for these technologies, the agency continues to engage with industry and publish thought papers that provide insight into their thinking. Although FDA’s current publications13,14 focus on Software as a Medical Device (SaMD), it is worth noting FDA’s evolution in this field and how it might relate to non-device computer systems in the future. For example, FDA has specifically discussed the following two potential requirements:

  • The incorporation of “pre-specifications” in product design to anticipate how machine learning algorithms will change as they learn from input data.
  • “Algorithm change protocols” to define methods and control risk associated with algorithm changes, and to ensure the continued safety and efficacy of the system/device.

Finally, life sciences companies are becoming ever more dependent on third-party cloud-based solutions and Software-as-a-Service (SaaS) as platforms for regulated software systems. As a result, companies should expect increased scrutiny from regulators when it comes to software lifecycle activities and objective evidence of validation related to these systems. Additionally, it is likely that more pressure will be placed on suppliers of software services and infrastructure platforms to provide documentation to their customers that supports effective execution of structured software development and validation, particularly with respect to cybersecurity, data center infrastructure, and change management.

References

  1. U.S. Food and Drug Administration. “Data Integrity and Compliance with Drug CGMP: Questions and Answers Guidance for Industry.” 2018
  2. U.S. Food and Drug Administration. “General Principles of Software Validation; Final Guidance for Industry and FDA Staff.” 2002
  3. The International Society for Pharmaceutical Engineering. “GAMP 5: A Risk-Based Approach to Compliant GxP Computerized Systems.” 2008
  4. Association for the Advancement of Medical Instrumentation. “AAMI TIR36:2007: Validation of Software for Regulated Processes.” 2007
  5. Association for the Advancement of Medical Instrumentation. “AAMI/ISO TIR80002- 2:2017: Medical Device Software – Part 2: Validation of Software for Medical Device Quality Systems.” 2017
  6. European Commission: Health and Consumers Directorate-General. “EudraLex (The Rules Governing Medicinal Products in the European Union), Volume 4 (Good Manufacturing Practice: Medicinal Products for Human and Veterinary Use), Annex 11: Computerized Systems.” 2010
  7. U.S. Food and Drug Administration. “Inspection Citation from 10/1/2008 through 12/9/2020 (Excel Format).” 23 December 2020. Inspection Citation. 26 January 2021. <https://www.fda.gov/media/107480/download>.
  8. Wan, Jefferey. “Common Inspection Deficiencies and Statistics.” November 2019. Presentation: Common inspection deficiencies and statistics. 26 January 2021. <https:// www.tga.gov.au/sites/default/files/presentation-common-inspection-deficiencies-and- statistics.pdf>.
  9. U.S. Food and Drug Administration. Medical Device Single Audit Program (MDSAP). 4 January 2021. 26 January 2021. <https://www.fda.gov/medical-devices/cdrh- international-programs/medical-device-single-audit-program-mdsap>.
  10. Therapeutic Goods Administration. Medical Device Single Audit Program (MDSAP). 6 September 2019. 26 January 2021. <https://www.tga.gov.au/medical-device-single- audit-program-mdsap>.
  11. U.S. Food and Drug Administration. CDRH Proposed Guidances for Fiscal Year 2021 (FY 2021). 16 October 2020. 26 January 2021. <https://www.fda.gov/medical-devices/ guidance-documents-medical-devices-and-radiation-emitting-products/cdrh-proposed- guidances-fiscal-year-2021-fy-2021>.
  12. National Institute of Standards and Technology. “Framework for Improving Critical Infrastructure Cybersecurity.” 2018.
  13. U.S. Food and Drug Administration. “Artificial Intelligence/Machine Learning (AI/ML)- Based Software as a Medical Device (SaMD) Action Plan.” 2021.
  14. U.S. Food and Drug Administration. “Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD).” 2019.

Author Biography

Eric Henry is a Senior Quality Systems and Compliance Advisor in the FDA and Life Sciences practice of the law firm King & Spalding. Eric has 30 years of global leadership and practitioner experience in quality and compliance organizations (20 years in medical device leadership), with a specialization in software quality (including cybersecurity), medical device design controls, risk management, audit management, and Quality System remediation.

  • <<
  • >>

Join the Discussion