Matteo Bernasconi- Institute of Technology Management, University of St. Gallen; Thomas Friedli- Institute of Technology Management, University of St. Gallen; Nuala Calnan - Pharmaceutical Regulatory Science Team, Technological University Dublin and The Quality Risk Management Institute
Abstract
Quality excellence is a key factor in pharmaceutical manufacturing that differentiates the high performers from the less successful companies. Quality Metrics research identified that ‘high performer’ manufacturing plants excel for most Pharmaceutical Quality System (PQS) effectiveness metrics, achieve this with lower inventory levels, all while handling more Stock Keeping Units (SKUs).6 On the other hand, the research also confirms that manufacturing plants with low PQS excellence, report a higher share of indirect QA/QC labor full time equivalents (FTEs), in response to underlying operational stability problems. Quality Excellence is described as an advanced approach to quality which goes beyond merely being compliant with regulations. Quality excellence is patient-driven, culturally embedded and built into the processes and behavior of an organization. This article provides an overview of quality metrics research undertaken by the authors on behalf of FDA and the development of a machine learning based, predictive risk surveillance model.
Introduction
For the pharmaceutical industry quality excellence is particularly crucial, since non-conforming drug products may lead to harmful effects to the patients.1 National regulatory agencies, such as the U.S. Food and Drug Administration (FDA), have a mandate to protect public health and bare responsibility for enforcing current regulations through product surveillance programs and by performing regular compliance inspections. In spite of international regulators’ efforts, the pharmaceutical supply chain continues to experience product recalls, drug shortages, and supply disruptions due to unsatisfactory quality in manufacturing.2 The FDA’s Center for Drug Evaluation and Research (CDER) recognizes the importance of fostering enhanced product quality and has launched several initiatives in recent years to improve the situation.3 Additionally, CDER is seeking to leverage advanced analytical methods to schedule inspections based on a safety risk model.4 The current FDA Site Selection Model primarily relies on compliance data, since the Agency already owns all required data. However, compliance data are not the only suitable data for assessing quality risk. Operational performance metrics and the maturity of continuous improvement practices provide the possibility to complement compliance history with operational plant data for a more comprehensive understanding of quality risk.5,6 Results are so promising, that the U.S. Government intends to commence metrics reporting from industry to the FDA.7–9
Quality Metrics Initiative Background
The Quality Metrics Initiative was launched with the publication of the FDA Safety and Innovation Act (FDASIA) in 2012, followed by the establishment of a public docket in 2013. In the public docket, the FDA announced its intention to examine the use of selected Quality Metrics to support their risk-based inspection program. In 2015 an initial draft guidance was released, and revised in 2016, focusing on the intention to collect data from covered establishment and proposed a data submission procedure to measure establishments’ performance.10 The document presented a Quality Metrics Reporting Program, in which companies would submit data to the Agency. The focus was on three metrics: Lot Acceptance Rate to measure manufacturing process performance, Invalidated Out-Of-Specification Rate to measure laboratory robustness, and Product Quality Complaint Rate to assess patient feedback. The comments received on the guidance highlighted the preference of starting the quality metrics reporting program as a pilot rather than as an industry wide implementation.
However, in the draft guidance the FDA did not provide scientific evidence for supporting the selection of the three nominated metrics nor their suitability. In 2016, the FDA awarded the University of St. Gallen an initial 1-year research grant, later extended to a second and a third year and leading to the publication of three research reports. The research team fully operationalized a Pharmaceutical Quality System (PQS) Excellence Model (Figure 1). The three-years research showed that quality excellence reaches beyond merely being compliant with current regulations and builds competence over numerous pillars which support an integrated approach to both PQS Effectiveness and PQS Efficiency. An effective PQS focuses on and ensures both manufacturing effectiveness (meeting patient and market needs) and compliance with the regulations (meeting regulatory requirements). PQS efficiency ensures the organization can also meet their business performance expectations.
In the beginning of 2022, the FDA published a new public docket on the Quality Metrics Reporting Program, sharing insights from the pilot project and asking for comments on the possible future direction of the program.8 The Agency recognized the low flexibility of the previous proposed approach with three fixed metrics, and proposes to move to four focus areas from which each company can select and deliver individual metrics: these focus areas are:
1. Manufacturing Process Performance
2. PQS Effectiveness
3. Laboratory Performance
4. Supply Chain Robustness
In some cases, the Agency suggests that multiple metrics from across these areas could be delivered to ensure a robust assessment. Additionally, the benefits of this data to the Agency are highlighted. Data might be used for example, to develop compliance and inspection policies, improve CDER’s ability to predict future drug shortages, and to encourage the industry to implement innovative PQS and continuous improvement approaches in manufacturing.
Quality Management Maturity
In 2019, the FDA analyzed drug shortages and identified three major root causes: (1) the lack of incentive to produce less profitable drugs, (2) missing market reward for mature quality management systems, and (3) logistical and regulatory constraints to make the market recover quickly after a disruption.2 The second root cause highlights the difficulty of assessing and rewarding sites based on the quality of drug products. This led the FDA to reveal their interest, in a 2022 white paper, on developing an assessment for rating organizational Quality Management Maturity (QMM) and creating incentives for industry to strive to achieve higher QMM.9 This QMM rating will not be a mere assessment of the level of implementation of quality and compliance related practices but rather provide the basis to incentivize continuous improvement programs.
However, the link between quality metrics and QMM has not been quantified yet.13 Some studies focus on the assessment of quality culture, proving a relationship between quality culture and operational performace.14,15 In 2020, a quality benchmarking study was performed to assess the current status of quality and quality related practices. The study shows a positive relationship between quality management practices and delivery performance.16 Nevertheless, a commonly shared model of operationalization of QMM has not yet been articulated.
Connecting Quality Metrics and Quality Management Maturity through Machine Learning
The Quality Metrics Reporting Program, as well as the QMM rating system and subsequent assessments would certainly provide more data to the FDA. One use of this data could be to strengthen CDER’s risk-based inspection scheduling model. The combination of these different data sources is fundamental to better analyze the complexity currently faced by a manufacturing facility and would provide greater accuracy to the risk assessment, and potentially contribute to reduce the risk of drug shortages and product recalls.6,17 Additionally, the submission of companies’ proprietary information would provide the Agency with the possibility to continuously assesses the operations of a manufacturing facility between inspections, offering the potential to reduce the frequency and/or length of routine surveillance inspections.8
In 2021, a Risk Surveillance (RiskSurve) research project was awarded to the University of St. Gallen to improve the current understanding of the relationship between operational indicators and quality risk, by creating a machine learning based predictive risk model.18 For the purpose of the research, the model under development relies on anonymized operational data currently available from the University of St. Gallen pharmaceutical industry benchmarking studies in lieu of reported site quality metrics. The predictive risk model is composed of four different dimensions:
1. Outcome Metrics,
2. Maturity (Quality),
3. Compliance History,
4. External Signals.
The analysis of these four dimensions is performed considering the Context Factors in which the analyzed facility is operating and provide additional specificity to the risk analysis.
The first two dimensions, Outcome Metrics and Maturity (Quality) refer to the quality performance and routine management practices assessed at a site level. These two dimensions are interrelated, where a higher maturity of implementation of a range of routine site management practices lead to a better operational performance.19 Additionally, while the suitability of both dimensions to predict quality risk has previously been discussed and tested, this has never been evaluated as a combination of them.5,6 The third Compliance History dimension contains information on the previous compliance situation for a given facility. This dimension provides the historical regulatory data to train the model, and relies on data already available to the Agency. The combination of historical compliance data coupled with other available data has already been proven as reliable to predict inspection outcomes.20 The fourth dimension, the External Signals, target publicly available information to improve the predictability of the quality risk by considering information originating from outside the facility. This external information is used to triangulate the data from the Outcome Metrics and Maturity (Quality) dimensions to provide proxies for these dimensions.
Conclusion
The FDA continues to invest in research and knowledge to improve the quality of drug products. Initially, the Agency started investigating the definition and suitability of quality metrics and management practices within a PQS. These efforts resulted in a Quality Metrics Reporting Program, which is currently under revision to provide more flexibility on the metrics to be reported. This flexibility will provide some challenges regarding metrics comparability and especially in the aggregation of metrics for the analysis. However, the development of a QMM assessment will support the Agency to define a unique QMM rating system for the industry. This combination of relevant quality metrics coupled with fostering enhanced quality management maturity has the potential to change the current FDA compliance risk-based approach towards a more quality performance-based assessment for scheduling of their inspection activities. For the Industry, this could help to shift the emphasis from reactive fire-fighting actions to more proactive preventive actions and contribute to greater resilience of global drug supply chains.
Literature
1. Yu LX, Kopcha M. The future of pharmaceutical quality and the path to get there. Int J Pharm. 2017;528(1-2):354-359. doi:10.1016/j.ijpharm.2017.06.039
2. FDA. Drug Shortages: Root Causes and Potential Solutions.; 2019. Accessed September 22, 2022. https://www.fda.gov/media/131130/download
3. Fisher AC, Lee SL, Harris DP, et al. Advancing pharmaceutical quality: An overview of science and research in the U.S. FDA’s Office of Pharmaceutical Quality. Int J Pharm. 2016;515(1- 2):390-402. doi:10.1016/j.ijpharm.2016.10.038
4. FDA. Understanding CDER’s Risk-Based Selection Model.; 2018. Accessed October 28, 2022. https://www.fda.gov/media/116004/download
5. Eich S, Friedli T. Analysis of the effects of Operational Excellence implementation on Inspection Outcomes in the Pharmaceutical Industry: An Empirical Study. Brazilian Journal of Operations & Production Management. 2021;18(3):1-15. doi:10.14488/ BJOPM.2021.021
6. Friedli T, Köhler S, Macuvele J, et al. FDA Quality Metrics Research: 3rd Year Report.; 2019. Accessed August 22, 2020. https://item.unisg.ch/en/divisions/production-management/ st-gallen-fda-quality-metrics-research
7. The White House. Building Resilient Supply Chains, Revitalizing American Manufacturing, and Fostering Broad-Based Growth.; 2021. Accessed October 28, 2022. https://www. whitehouse.gov/wp-content/uploads/2021/06/100-day-supply-chain-review-report.pdf
8. FDA. Food and Drug Administration Quality Metrics Reporting Program; Establishment of a Public Docket; Request for Comment.; 2022. Accessed October 28, 2022. https://www. govinfo.gov/content/pkg/FR-2022-03-09/pdf/2022-04972.pdf
9. FDA. Quality Management Maturity: Essential for Stable U.S. Supply Chain of Quality Pharmaceuticals.; 2022. Accessed October 28, 2022. https://www.fda.gov/media/157432/ download
10. FDA. Submission of Quality Metrics Data Guidance for Industry - Draft Guidance.; 2016. Accessed December 6, 2022. https://www.fda.gov/media/93012/download
11. Friedli T, Köhler S, Buess P, Basu P, Calnan N. FDA Quality Metrics Research.; 2017. Accessed August 22, 2020. https://item.unisg.ch/en/divisions/production-management/st-gallen-fda-quality-metrics-research
12. Friedli T, Köhler S, Buess P, Eich S, Basu P, Calnan N. FDA Quality Metrics Research: 2nd Year Report.; 2018. Accessed August 22, 2020. https://item.unisg.ch/en/divisions/production-management/st-gallen-fda-quality-metrics-research
13. VanDuyse SA, Fulford MJ, Bartlett MG. ICH Q10 Pharmaceutical Quality System Guidance: Understanding Its Impact on Pharmaceutical Quality. AAPS J. 2021;23(6):117. doi:10.1208/ s12248-021-00657-y
14. Patel P, Baker D, Burdick R, et al. Quality Culture Survey Report. PDA J Pharm Sci Technol. 2015;69(5):631-642. doi:10.5731/pdajpst.2015.01078
15. Friedli T, Buess P, Köhler S, Chen C, Mendivil S, Baker D. The Impact of Quality Culture on Operational Performance—An Empirical Study from the Pharmaceutical Industry. PDA J Pharm Sci Technol. 2018;72(5):531-542. doi:10.5731/pdajpst.2018.008771
16. Fellows M, Friedli T, Li Y, et al. Benchmarking the Quality Practices of Global Pharmaceutical Manufacturing to Advance Supply Chain Resilience. AAPS J. 2022;24(6):111. doi:10.1208/ s12248-022-00761-7
17. FDA. Resiliency Roadmap for FDA Inspectional Oversight.; 2021. Accessed October 28, 2022. https://www.fda.gov/media/148197/download
18. Bernasconi M, Friedli T, Calnan N. Holistic Risk-Based Site Surveillance - A Data-Based Approach to Site Quality Risk Identification and Assessment in the Pharmaceutical Industry. Level3. 2021;16(1). Accessed October 28, 2022. https://arrow.tudublin.ie/level3/ vol16/iss1/8
19. Voss C, Blackmon K, Hanson P, Oak B. The Competitiveness of European Manufacturing? A Four Country Study. Business Strategy Review. 1995;6(1):1-25. doi:10.1111/j.1467-8616.1995.tb00169.x
20. Seiss M. FDA CDER ORA Site Selection Model Improvement Pilot Study.; 2018.
Author Biographies
Matteo Bernasconi is a research associate at the University of St. Gallen. At the Institute of Technology Management, he works in the Operational Excellence team with a special focus on the pharmaceutical industry. Currently, his academic work concentrates on preventive risk quality management and data-driven decision-making.
Prof. Dr. Thomas Friedli is a director at the Institute of Technology Management. He leads a division of 14 PhD students and one post-doc. His research areas include strategic management of production companies, management of industrial services, and operational excellence. He is editor, author and co-author of numerous books and articles.
Dr. Nuala Calnan is a veteran of the biopharmaceutical industry. Her work as a consultant, academic, author and commentator is directed toward the development of practical, patient-focused excellence and has included research for FDA(USA), HPRA(Ireland) on patient safety and product quality in the manufacture and distribution of drugs. As founder of The Quality Risk Management Institute, Nuala works with organizations on delivering effective end-to-end risk management by transforming organizational culture, implementing behavior-based quality excellence strategies and enhancing performance management through people. Nuala is also a Research Fellow and Arnold F. Graves scholar at TU Dublin.
Subscribe to our e-Newsletters
Stay up to date with the latest news, articles, and events. Plus, get special
offers from American Pharmaceutical Review delivered to your inbox!
Sign up now!