Microbiological Data Integrity: Points to Consider Regarding Second Analyst Verification

Data integrity is a topic of great importance in our industry. Robust data integrity is critical to ensuring patient safety by assuring that correct and appropriate decisions are made, and that all test results etc. accurately reflect the quality of products being manufactured. To this end, all firms must ensure that all generated records are complete, accurate and trustworthy. Ensuring and maintaining “data integrity” involves protecting original data from modification (either accidental or intentional), falsification and/or deletion. All data and records are expected to be:

• Attributable: all data is assignable to the individual who generated the data,
• Legible: data can be read either electronically or by the naked eye, and retained in a permanent format,
• Contemporaneous: all data and records must be generated the moment an activity or measurement etc. is performed,
• Original: data is either in the form that it was originally generated, or as a “verified copy” that retains all content, intent and meaning, and
• Accurate: data is complete, free from errors, truthful and reflective of the activity or measurement performed.¹

However, assuring the integrity of microbiological data presents some unique challenges. Most microbiological test results are still generated manually, either by counting the number of microbial colony forming units (CFU) present on a plate, examining a broth culture for signs of microbial growth (e.g., turbidity) or identifying the microscopic morphology on a slide preparation (e.g., a Gram stain), all with the naked eye. Being so manual in nature, these methods are highly subject to human error that can easily lead to an inaccurate result. Common examples include, but are not limited to, data inaccuracies resulting from:

1. Counting error, particularly when there are large numbers of colonies or confluent growth on a plate.
2. The presence of secondary growth, which is common when mold colonies present on the plate have begun to sporulate. The resulting spores can generate new colonies in addition to those recovered from the original sample.
3. Failure to visually detect very small (i.e., “pinpoint”) or transparent colonies on sample plates, or trace amounts of microbial growth in a liquid (e.g., in sterility testing vessels or media-filled vials from an aseptic process simulation). This issue can be exacerbated if the light by which the plates, vessels or vials are examined is not sufficiently bright, or at an incorrect angle, or if the background against which the sample is read is incorrect. This can also be an issue when there are colonies embedded within the plate agar, as they are with a pour plate.
4. Mistaking a colony for an artifact or vice-versa. Whether an object present on a plate is a microbial colony or not can sometimes be a matter of opinion between analysts.
5. Sometimes the microscopic morphology of an organism can be altered by the presence of antibiotics or other toxic substances, making accurate identification of the cell type (e.g., cocci, coccobacilli or rods) much more difficult, even for an experienced analyst. Some organisms may yield a Gram-stain result that contains both Gram-positive and Gram-negative cells (known as Gram-variable) which can sometimes be mistaken for poor staining technique, and vice-versa.

To help reduce or minimize the amount of inaccuracies due to human error, some firms have opted to employ a second analyst verificationfor these types of test results. At some firms, ALL microbiological test results require a second operator verifi cation, including all environmental monitoring (EM) samples (including process monitoring, water systems and gasses) and all other microbiological test results (e.g., microbial limits/bioburden, absence tests, Gram stains, etc.). Such an undertaking can easily require a significant increase in staffing and other related costs, but is there an adequate return on investment?

The simple answer is “sometimes”.

It is acknowledged that there have been documented instances of intentional fraud committed by individual analysts in the industry. There have also been cases where analysts have been under significant pressure from their management to generate only passing results. Because these are very complicated and multi-faceted issues, it is debatable whether second analyst verification would be truly effective in assuring data integrity under these circumstances. Therefore, such situations will not be addressed further in this article. This article will instead focus on some of the practical points to consider in determining where second analyst verification might be helpful in improving the quality of microbiological data integrity under normal operating conditions in a cGMP facility.

Subscribe to our e-Newsletters
Stay up to date with the latest news, articles, and events. Plus, get special offers
from American Pharmaceutical Review – all delivered right to your inbox! Sign up now!

In making such a determination, there are several inherent weaknesses associated with second analyst verification that must be considered.

These include, but are not limited to, the following.

Susceptibility to Human Error

First, the second analyst is equally subject to the same human error as the first analyst. There are also additional circumstances that may suggest an error has occurred when in fact there was none.

Consider a standard eight-hour shift where the first operator performs colony counts at the start of the shift and the second verification reading occurs at the end of the shift. If the samples were not immediately refrigerated after the first reading, it is possible that colonies too small to be visually detected during the first reading may have sufficiently increased in size to be detected by the second analyst. In this scenario, neither analyst is in error. Similarly, one of the two analysts may identify an object on a plate as an air bubble or other artifact while the second identifies it as a colony, or vice-versa. In this scenario, either one of the analysts could be in error.

Discrepancies in the results obtained by the first and second analysts usually result in the need for a third analyst reading as a tiebreaker. It is possible that the third read may result in a count that verifies neither of the previous two, especially if there is a large number of colonies and either confluent or secondary growth on the sample.

The Imprecision of Microbial Counts in General

It is generally accepted as scientific fact that the microbial counts obtained on a given sample plate, especially on a plate collected during environmental monitoring, are probably not entirely accurate in terms of representing what was actually present at the time of sampling. USP<1116> “Microbiological Control and Monitoring of Aseptic Processing Environments” states that, in general, surface monitoring is estimated to recover <50% of the organisms actually present on a given surface even when relatively high inoculum levels are present on standardized coupons. Recovery rates may be even lower where organisms are stressed to varying degrees due to desiccation, residual disinfectants and other stressors present in the actual production environment.2

Alert and Action Limits

Alert and action limits for viable parameters such as surface counts recovered by contact plates, settle plates or swabs are usually determined using a statistical calculation; often the mean plate count plus one to two standard deviations, the calculation of the 95th , 97^th or 99th percentile, or other appropriate calculation depending on the parameter being monitored. The purpose of alert and action limits is to aid in maintaining environmental and process control by detecting drift in collected data from the established baseline or norm.

Consider a Grade D room where the alert level is >10 CFU/plate and the action level is >20 CFU/plate. From a practical microbiological perspective, there is really no difference between 19, 20 or 21 CFU on a plate. Other than the need to open an investigation if 21 CFU have been recovered, there is little practical significance, especially as the colonies actually present on the plate are only an estimate of what was actually present on the surface at the time of sampling, as previously discussed.

Criticality of the Result in Question

Clearly, some microbiological plate counts or test results are more critical than others. For example, any sign of microbial contamination resulting from a sterility test sample constitutes a failure, and therefore it is imperative that the slightest evidence of microbial growth be detected. The same applies to tests for the absence of certain organisms. When a plate count result is at or near an established alert or action limit, the accuracy of the count is critical in determining whether formal action, such as an investigation, is required. In most Grade A areas, the presence of 1 CFU constitutes an exceeded action limit, requiring comprehensive investigation and corrective and preventative action. In this scenario, it is critical that no colonies are overlooked. In these cases, a second analyst verification can be helpful in ensuring an accurate count and therefore assuring that the appropriate cGMP decision is made.

However, in other situations the accuracy of the result, while still important, may not be as impactful. Consider the scenario presented previously concerning the alert and action limits in a Grade D room. If an analyst counts 7 CFU on a plate, and a second analyst counts 8 CFU there is little to no impact resulting from the error. In most cases, unless an analyst is intentionally miscounting, a count will be off by only 1 or 2 colonies, although the risk of an inaccurate count increases with the number of colonies present on the plate.

Additional Methods to Help Assure Result Accuracy

There are additional methods and controls available that can help to increase the accuracy of an analyst’s reported results that ought to be considered along with second analyst verifications.

• The use of Quebec colony counters, hand-held tally counters, colony counter pens etc. can aid in manual counting. There are also some automated colony counters on the market that may be considered.
• Examining plates under a stereo-microscope can be helpful in identifying pinpoint colonies and separating true colonies from artifacts.
• As part of their training, analysts should be required to demonstrate that they can accurately identify and count different colony types (especially those that may be problematic, such as pinpoint or clear colonies) and detect very low levels of microbial contamination in liquid samples, etc. Demonstration of proficiency is also critical to ensuring consistent and reliable Gram-stain results.

So, what makes sense?

In the opinion of this author, second operator verification makes good practical sense when a cGMP decision needs to be made, or when the result may be critical to assuring patient safety and/or product quality.

As with all decision we make, there should be a good reason for it and that reason should make good scientific sense.

Areas where a second analyst verification is likely to be valuable include but are not limited to:

• Verification of Gram stains reactions and cellular morphologies to prevent misidentification
• Verification of EM plate counts near, at or exceeding established alert or action limits, or test plate counts similarly near the specification limits.
• Verification of all plate counts (including zero counts) collected from Grade A areas where the action limit is >1CFU/Plate
• Verification of all sterility test results
• Verification of any suspect media-filled vials from aseptic process simulations
• Verification of absence test results

However, second analyst verification of plate counts or other test results where no cGMP decision is going to be made may not provide enough benefit to offset the significant costs of additional staffing and processing times. Unannounced, second analyst “spot checks” may be more appropriate in these situations as they would provide a means of monitoring overall analyst accuracy without unnecessarily burdening the lab.

In the end, in all cases, the justification for where second analyst verification might be applied should be based on:

• the likelihood an inaccurate result may occur,
• the likely degree of inaccuracy, and
• the severity of the impact that inaccurate result could potentially have

In considering the criticality and risks associated with each microbiological result type, a firm can reliably determine the practical benefit or lack thereof of second analyst verification. In this way, data integrity and patient safety can be assured, the correct cGMP decisions can be made, and unnecessary effort and associated costs can be avoided.

References

1. “Data Integrity and Compliance with Drug CGMP Questions and Answers Guidance for Industry”, December 2018. U.S. Department of Health and Human Services Food and Drug Administration Center for Drug Evaluation and Research (CDER) Center for Biologics Evaluation and Research (CBER) Center for Veterinary Medicine (CVM), Silver Spring, MD, accessed on January 24, 2020 from https://www.fda.gov/media/119267/download
2. General Chapter <1116>, “Microbiological Control and Monitoring of Aseptic Processing Environments”. USP 42- NF 37 2S, 2020, U.S. Pharmacopoeia, Rockville, MD. Accessed on January 24, 2020 from https://online.uspnf.com/uspnf