Independence and Dependence in Calibration: A Discussion of FDA and EMA Guidelines

Introduction

Two recently published FDA and EMA documents describe expectations for the submission of spectroscopic methods to the regulatory agencies and are important advancements for the implementation of modern non-destructive analytical methods in the pharmaceutical industry.^1,2 Both documents are focused on the description of near infrared spectroscopic methods. However, the FDA Draft Guidance indicates that the same fundamental concepts of validation may be applied to Raman, X-ray, and other Process Analytical Technology (PAT) analytical techniques. These expectations must become part of the plans or procedures for the development of PAT spectroscopic methods.

The FDA and EMA documents should not be considered a checklist for the validation of methods to be submitted to regulatory agencies. Regulatory submission must include justifications of the strategies followed for the development of calibration models; strategies that define how scientists work. The documents encompass chemometrics; a scientific discipline that is based on careful planning and observation of data to extract the maximum information. The documents describe the information needed to demonstrate the validation of the methods. Validation requires evidence that prediction performance is valid, and a calibration model should be evaluated according to its purpose.³ Validation requires thinking, planning and establishing clear objectives. Thus, the FDA and EMA documents should not be viewed as checklists for the submission of PAT and spectroscopic methods.

The development of a calibration model for a pharmaceutical product requires extensive planning.⁴ Model development starts with a learning phase where scientists study the composition of a formulation and the pharmaceutical process and define method requirements and performance criteria. This initial planning has been described as “general steps in an analytical Quality by Design strategy”.⁵ Thus, the planning entails the selection of an analytical technique and may also include a risk assessment to identify factors that may affect the method’s performance.⁵ This planning stage is essential to obtain a calibration model that will be suitable to predict future production samples. Spectral acquisition and modeling occur in the second phase, turning instrument signals into information. The third phase is the use of the calibration model to predict samples that come from the manufacturing process. This work process is captured in the FDA and EMA documents which state that submissions should clearly describe:

• a calibration model for creating the mathematical relationship between the changes in signal and the property of interest. This calibration model is the deliverable from the planning and modeling steps.
• a calibration test set for internal validation and optimization of the model. EMA defines internal validation as “the application of resampling statistics to cross-validate and provide an “internal check” of the performance of the model for the purposes of optimization”
• an independent validation for external validation of the model.

The main focus of this article is to examine the steps in the development of calibration models and discuss the challenges of obtaining independent validation samples. The FDA and EMA documents rightfully request independent validation samples; however, this “independence” requires further discussion.

Calibration Models

Calibration models require planning and a projection to the future. The calibration should contain all formulation components, include the expected concentration and be as similar as possible to the future samples that will be predicted.^3,4,6 These tenets are well described in the FDA and EMA documents. However, samples from the industrial process are usually available with only one drug concentration. Thus, one of the first challenges in model development is to prepare calibration samples similar to the process but with an expanded drug concentration.^4,7

The range in drug concentration has been expanded in several ways.⁴ This expansion may be performed by following different experimental designs, permitting the calibration to encompass variations in excipient and active pharmaceutical ingredient (API) composition.^8,9 The expansion may also be performed by taking granules or blends from the pharmaceutical process and spiking with API or excipients.^7,10 The drug concentration may also be varied in batches prepared within smaller production scale equipment.^11,12 These approaches make it possible to include the physical variation of the process within the calibration model and also expand the concentration range. Thus, the approach followed in developing calibration models capable of handling process variations should be explained within the regulatory submission.

Calibration Test Set

The EMA indicates “A calibration test set may be used (instead of cross validation within the calibration set) to provide the first ‘test’ or check of the validity of the model”. The EMA document also indicates that the calibration test set does not represent independent validation of the NIRS procedure, since the samples are taken from different batches within the same historical population of batches. The EMA document further indicates that the calibration set often consists of two thirds of the available sample population and the calibration test set is the remaining third. The calibration test set is similar to the use of crossvalidation and this is acknowledged by the EMA and FDA documents. The EMA states in its glossary section that internal validation is not a substitute for the external validation of the model.

The term “calibration test set” is used to describe the first test of the validity of the model and acknowledges that multivariate calibration models, such as partial least squares regression, are rich in diagnostic tools that are very useful to optimize a calibration model.¹³ The calibration test set samples may be used to optimize the calibration model and progress towards the prediction of an “independent validation set for external validation of the model”.

Validation Set

The EMA guideline indicates: “Chemometric data analysis works by correlating the variance in the NIRS signal to a number of latent variables or factors, constrained by a set of calibration reference data. There is always a risk that the correlations identified by the software are due to chance only and not to changes in the analyte; therefore chemometric models should always be validated with an independent set of samples”. Thus, the mission of the “independent data set” is clearly established as assuring that predictions are due to changes in the analyte and not a result of chance. Previous studies have shown that this is a reasonable concern.^14,15 However, the FDA and EMA documents do not clearly describe the expected “independent data set”.

The two documents appear to indicate that a first requirement for external validation (“independence”) is demonstration that productionscale batches are adequately predicted. The EMA document states “the external validation set should cover the calibration range of the NIRS model, including all variation seen in the commercial process and should include pilot and production-scale batches, where possible”. The FDA draft guidance indicates that “samples used for external validation should span a suitable range of operating conditions (i.e. ranges expected during commercial production) and should be obtained independently from the calibration and internal validation samples used during the development of the NIR models.” The documents appear to acknowledge that many calibration models are developed in laboratories where it is easier and more cost efficient to vary the concentration of samples than in a pilot or production facility. However, the end goal is to use this calibration model for the prediction of samples in a manufacturing facility. This prediction in the manufacturing facility is thus considered independent of the laboratory efforts.

The EMA guidelines also indicate an expectation that spectroscopic results be compared to a reference analytical procedure based on destructive testing. The EMA guidelines state: “Interpretation of the complex spectra of unprepared samples generated by NIRS measurement usually requires the use of chemometric calibration models. These models are developed using carefully selected and representative samples, which normally require qualification by independent, reference analytical procedures (normally requiring destructive sample preparation to extract or isolate the analyte of interest and calibration and validation using analytical reference standards”. The words “normally requiring destructive sample preparation to extract or isolate the analyte of interest”, are indicative of the use of High Performance Liquid Chromatography (HPLC). HPLC is able to quantify both an active pharmaceutical ingredient (API), process impurities and degradation products. The use of HPLC usually requires analysis in a separate laboratory, and this can be considered an “independent” result. However, both NIR and reference methods depend on the absolute calibration and use of analytical balances.

The importance of the analytical balance is frequently forgotten. This author has attended conferences where a speaker has mentioned the need to use HPLC, that gravimetric calibration samples are not sufficient. However, HPLC methods also depend on analytical balances and the weight standards used. HPLC methods often calculate drug concentration based on the weighing of 10 milligrams or less of a costly reference standard. In turn, the gravimetric preparations for the calibration blends in many NIR methods may involve the weighing 1 – 100 grams of the excipients and API. Thus, the gravimetric measurements in the preparation of NIR calibrations models could be more reliable than those in HPLC methods. HPLC methods also have a number of errors that are not found in NIR spectroscopy such as incomplete extractions and dilution errors.

The approach recommended by this author is develop the calibration model with samples prepared gravimetrically in laboratory facilities. The calibration model could then be challenged with a one or more test sets prepared gravimetrically within the developer’s laboratory. The test sets could involve:

• samples prepared with batches of excipients and API different from those used in the calibration set, or prepared using different sieve cuts. These determinations would also be part of the robustness challenges to the calibration model.
• samples prepared with separate weighing steps (should not be prepared from a single stock blend).
• samples with different proportions of excipients and API. These samples will challenge the method's ability to predict adequately in spite of subtle process variations.
• spectra that are not obtained on the same day as the calibration samples, to determine whether the calibration model is able to handle subtle variations in instrument noise. Noise spectra may also be obtained by using the reference material as sample. This noise spectrum may be multiplied by a factor of 2 or 3 and added to the NIR spectra of validation samples.¹⁶ In this way a spectrum with more noise may be obtained and its effect of the predictions determined. This approach may also be part of the robustness study.

The model could then be used to predict samples from a manufacturing facility. The samples from the manufacturing facility would then be analyzed by an HPLC system within the QC lab of the manufacturing facility. Researchers at the Universitat Autònoma de Barcelona have provided examples of this approach.^17,18 Thus, the least expensive gravimetric calibration samples are prepared first. The more expensive and laborious HPLC testing was performed after a number of test sets indicated that the method should perform well. The QC laboratory at the manufacturing facility is usually separate from the laboratory that developed the calibration model. The QC laboratory provides an “independent” assessment, even though it still depends on the calibration and use of the analytical balance. Independence stops at the analytical balance, all results depend on the proper use of analytical balances based on absolute calibration with internationally recognized weight standards.¹³

The end goal is still clearly expressed in the current Good Manufacturing Practices (cGMPs). Section 211.194 a2, specifies that “The suitability of all testing methods used shall be verified under actual conditions of use.” A previous publication indicated “Sample independency means that samples are not prepared under the same conditions as the calibration set samples. Validation samples should come from the process that will be monitored and be prepared with excipient and API batches that differ from those used in the calibration set.”⁴ This definition is consistent with the expectations of the FDA and EMA documents.

Final Comments

Validation efforts should not be restricted by the definition of the calibration test set given in the EMA and FDA documents. The calibration test set is seen as an initial check of the validity of the model, and the application of resampling statistics to cross-validate the performance of the model for the purposes of optimization. Validation may involve several test sets (validation samples) to progressively challenge the model,^3,4 in a strategy that does not fit the calibration test set definition described in the EMA and FDA documents. The EMA and FDA documents are important summaries of the progress made in developing non-destructive analytical methods for pharmaceutical processes. These documents are worthy of further discussion. Hopefully this article contributes to this needed discussion.

Acknowledgement

The work supported by National Science Foundation through Grant NSF-EEC0540855 was the seed for many helpful discussions that resulted in this article.

References

1. U.S. Department of Health and Human Services FDA. Development and Submission of Near Infrared Analytical Procedures Guidance for Industry Draft Guidance. 2015.
2. European Medicines Agency. Guideline on the use of Near Infrared Spectroscopy (NIRS) by the pharmaceutical industry and the data requirements for new submissions and variations. 2014:28.
3. Esbensen KH, Geladi P. Principles of Proper Validation: use and abuse of re-sampling for validation. J. Chemometrics. 2010;24(3-4):168-187.
4. Romañach R, Román-Ospino A, Alcalà M. A Procedure for Developing Quantitative Near Infrared (NIR) Methods for Pharmaceutical Products. In: Ierapetritou MG, Ramachandran R, eds. Process Simulation and Data Modeling in Solid Oral Drug Development and Manufacture: Springer New York; 2016:133-158.
5. Corredor C, Lozano R, Bu X, et al. Analytical Method Quality by Design for an On-Line NearInfrared Method to Monitor Blend Potency and Uniformity. J Pharm Innov. 2015;10(1):47- 55.
6. Kramer R. Chemometric Techniques for Quantitative Analysis. Taylor & Francis; 1998.
7. Càrdenas V, Blanco M, Alcalà M. Strategies for Selecting the Calibration Set in Pharmaceutical Near Infrared Spectroscopy Analysis. A Comparative Study. J Pharm Innov. 2014;9(4):272-281.
8. Bondi RW, Jr., Igne B, Drennen JK, 3rd, Anderson CA. Effect of experimental design on the prediction performance of calibration models based on near-infrared spectroscopy for pharmaceutical applications. Applied spectroscopy. 2012;66(12):1442-1453.
9. Sulub Y, Wabuyele B, Gargiulo P, et al. Real-time on-line blend uniformity monitoring using near-infrared reflectance spectrometry: a noninvasive off-line calibration approach. J. Pharm. Biomed. Anal. 2009;49(1):48-54.
10. Blanco M, Romero MA, Alcala M. Strategies for constructing the calibration set for a near infrared spectroscopic quantitation method. Talanta. 2004;64(3):597-602.
11. Mark H, Ritchie GE, Roller RW, Ciurczak EW, Tso C, MacDonald SA. Validation of a nearinfrared transmission spectroscopic procedure, part A: validation protocols. J. Pharm. Biomed. Anal. 2002;28(2):251-260.
12. Ritchie GE, Roller RW, Ciurczak EW, Mark H, Tso C, MacDonald SA. Validation of a nearinfrared transmission spectroscopic procedure. Part B: Application to alternate content uniformity and release assay methods for pharmaceutical solid dosage forms. J. Pharm. Biomed. Anal. 2002;29(1-2):159-171.
13. Martens H, Naes T. Multivariate Calibration. Wiley; 1992.
14. Small GW. Chemometrics and near-infrared spectroscopy: Avoiding the pitfalls. TrAC Trends in Analytical Chemistry. 2006;25(11):1057-1066.
15. Xiang D, Berry J, Buntz S, et al. Robust calibration design in the pharmaceutical quantitative measurements with near-infrared (NIR) spectroscopy: Avoiding the chemometric pitfalls. Journal of pharmaceutical sciences. 2009;98(3):1155-1166.
16. Colon YM, Florian MA, Acevedo D, Mendez R, Romanach RJ. Near Infrared Method Development for a Continuous Manufacturing Blending Process. J Pharm Innov. 2014;9(4):291-301.
17. Blanco M, Bautista M, Alcala M. Preparing calibration sets for use in pharmaceutical analysis by NIR spectroscopy. Journal of pharmaceutical sciences. 2008;97(3):1236-1245.
18. Blanco M, Bautista M, Alcala M. API Determination by NIR Spectroscopy Across Pharmaceutical Production Process. AAPS PharmSciTech. 2008;9(4):1130-1135.

About the Author:

Rodolfo J. Romañach - is Professor of Chemistry and Site Leader for the Engineering Research Center for Structured Organic Particulate Systems at Mayagüez. His research involves near infrared and Raman spectroscopy and multivariate methods for continuous improvement in manufacturing.