Introduction
Though not an afterthought by any means, validation is seldom integrated into every phase of serialization project planning, particularly the early stages, when the project’s scope and objectives are being defined. Thinking about validation at the project’s inception means the test scripts and documents that follow are more aligned with the manufacturer’s requirements. It all starts with the important Validation Master Plan. As its name implies, it lays the groundwork for how the system is to be qualified.
Validate Early and Throughout
The validation phase for many pharmaceutical manufacturers is often initiated late in the project cycle – sometimes even after Factory Acceptance Testing (FAT) has occurred and sometimes without a Validation Master Plan. When this is the case, project teams are often left with an OEM’s standard set of validation documents. These are typically non-robust testing scripts that the manufacturer then needs to edit and massage to fit their requirements – resulting in a mad scramble to finish and approve the documents in time for the validation execution.
In the case of serialization projects, this approach often results in the startup and validation teams facing avoidable and possibly unanticipated issues during these important phases of the project. The result is often costly extended downtimes.
If the validation team is included up front, Factory Acceptance Test plans and Site Acceptance Test (SAT) plans can be developed with the foreknowledge of required Operational Qualification (OQ) and Performance Qualification (PQ) test scripts. In other words, the FAT and SAT can be molded to include these tests or at least ensure they will pass during validation execution. This can’t be accomplished if the validation team is on-boarded late in the process (e.g., after the FAT is conducted).
Why is careful validation planning so critical? Unlike previous production line projects, serialization projects have a significant IT component. The type and level of IT integration required to facilitate serialization introduces a variety of potential problems that engineers have not previously encountered. Resolving them while installing a serialization solution is a blueprint for extended downtime, increased stress and significant project cost overruns. For instance, most serialization solution providers include a server (each OEM gives it a different moniker) that bridges the gap from the plant floor to the manufacturing software used to run production and handle serial number management. Most of the solution provider’s servers can also function as number management systems should a manufacturer not have one of the aforementioned systems in place (this occurs usually in smaller contract manufacturers and/or contract packagers). Just the seemingly simple task of getting connected with Active Directory has caused delays during implementation. Corporate firewalls are also notorious for causing connection and other time-consuming issues.
A new way to approach testing and validation can save pharmaceutical companies time and money. Built around the idea of “onsite but offline” testing, this workflow gives engineers, IT and validation personnel the flexibility to address potential start-up issues with serialization solutions before ever taking the packaging line out of service. Integrated into projects right from the beginning, offline testing can eliminate weeks of unnecessary downtime from these implementations.
The Serialization Imperative
This validation planning challenge stems from the Drug Supply Chain Security Act (DSCSA), which mandates that manufacturers begin serializing all prescription drug products intended for distribution to the U.S. market. These unique identifiers must be printed on the saleable unit level (bottle, carton, tray) and homogenized case level starting November 27, 2017.
This regulation will enhance the U.S. Food and Drug Administration’s ability to help protect consumers from exposure to drugs that may be counterfeit, stolen, contaminated or otherwise harmful. To the uninitiated, serialization appears to be the straightforward process of printing an identifying number on products and shipping cases – similar to the lot and date codes that are currently applied to each product.
In serialization, however, it’s not the complexity of the technology being installed on the packaging line that causes potential problems (although there can be issues here as well). The true issue lies in the integration of the plant floor systems with existing corporate IT systems that run manufacturing and now need to be upgraded to handle serial number management and reporting requirements (often called Level 4 or L4 systems as shorthand for their place in the hierarchy of IT solutions). Integrating the production line with Level 4 data makes it subject to every quirk and blip that can occur on the corporate network, any of which can potentially shut down the serialization system and the production line. Historically, a typical glitch or malfunction in a corporate manufacturing program would not normally affect plant floor production. This is because the software modules usually do not interact on a real-time basis with manufacturing systems (process and packaging). Process control systems and packaging automation are designed to function regardless of what happens to the plant’s corporate system.
The introduction of serialization to the packaging line now requires these systems to work together in “near” real-time. If you have trouble logging in or cannot download a batch of serial numbers, you can’t run the packaging line. If you lose a connection while running and cannot replenish your cache of serial numbers to the line, you are stuck. There are also several standard operating procedures that require modifications to maintain the integrity of the serialization scheme. For instance, when performing quality checks on the target line, the Quality Assurance (QA) team can no longer simply remove an item from anywhere on the line for testing. Serialization protocols require additional processes to allow for any sampling or disposal of products once the number has been applied. All of these elements combine to require extensive testing and validation to yield a fully operational production line.
Low-Impact Validation Model
So what is the solution to this potentially costly mode of operation? The good news is that it is not all that difficult to avoid prolonged production shutdowns while validating serialization solutions. And manufacturers can turn to the same best practice for serialization projects of all shapes and sizes. Here are some typical examples we have seen in the pharmaceutical industry:
- New equipment purchased (e.g., a case packer, a labeler, a cartoner, etc.) with cameras and printers already integrated.
- Stand-alone modules from serialization solution providers that are made to “fit into” an existing packaging line.
- Retrofitting existing equipment on the plant floor with the serialization solution hardware and software.
A long-known but often-ignored best practice does the trick: onsite but offline testing and validation. All of the above examples can benefit from this methodology to differing degrees. In the first example, the new equipment can be set up in a spare bay or separate space and the proper utilities connected. The serialization server can then be connected to the corporate IT software (or third-party software) and functional testing can commence.
The second example is set up the same way as the first. You may need some surplus equipment and/or interconnecting conveyors – for instance, maybe there is a spare bottle labeler to use if your solution requires serialized bottles.
The third example can be a bit trickier (and parts of this can also apply to the second example as well). The solution typically consists of equipment similar to what’s existing on the floor – perhaps spares or older versions that are kept in reserve, or a packaging line not currently in use. If possible, purchasing used equipment is also feasible (there are many websites providing such services). The shorter production line downtime as a result of employing this method many times will justify the cost of using surplus equipment as part of the solution. Ideally, the new serialization solution is set up behind the same firewall as the target packaging line.
By setting up this “simulated” packaging line or test environment, startup and validation teams can do what they do best – Site Acceptance Testing (SAT), Installation Qualification (IQ) and parts of Operational Qualification (OQ) – all while the targeted packaging line continues on uninterrupted. Using a risk-based approach to validation, many manufacturers find that they can justify leveraging most of these tests, even though the execution had occurred while the equipment was not in its final installation location. This is a tremendous cost savings over taking the equipment down once it is installed.
This onsite but offline environment also addresses a major shortcoming of the Factory Acceptance Testing (FAT) that serialization vendors sometimes provide. Those tests (especially as in the second example above) are designed to validate the performance of the individual components, but not their performance as part of an integrated solution. As stated earlier, involving the validation team up-front in the project allows the FAT plans to be customer-driven rather than provider-driven.
Perhaps one of the most important benefits derived from this is it gives start-up and validation teams a risk-free environment for testing and validating IT (Level 4) connections.
Employing an onsite but offline testing method has been proven to cut weeks of downtime from a potentially lengthy start-up and validation phase, even if the validation team doesn’t truly execute any of the protocols during this offline testing. Setting up cameras and printers, testing failure, rejection and recovery modes, and loading software to ensure connectivity with Level 4 systems helps minimize problems that could keep a packaging line down for days, if not weeks.
Let’s say that Company A has ordered a new labeler and new case packer from OEM B as part of their serialization implementation. They have also chosen to work with serialization solution provider C. OEM B and solution provider C have worked together to integrate the serialization technology with the new equipment. The equipment arrived at Company A and they proceeded to set it up in an unused portion of the warehouse.
As they began their on-site but offline testing program, both software and hardware issues appeared, as well as corporate IT problems. There was no resolution for over a month. Had this equipment been installed directly onto the packaging line, the loss of production would have been devastating. Instead, the packaging line continued to run while these unfortunate issues were addressed.
Planning onsite but offline testing into the serialization project from its inception can be a crucial factor in a project’s success. Another efficiency gained with this method is that it gives operations a forum to plan for operator and technician training. They have the ability to train on equipment and system functionality as well as train on new standard operating procedures that are part of every serialization implementation.
Conclusion
Pharmaceutical manufacturers that adopt this testing and validation strategy, even those who are late to start their serialization programs, give themselves a better chance to be in compliance by November 27, 2017 than those who don’t. They will also enjoy shorter downtimes and less cost of ownership, while helping ensure a safer prescription drug supply chain for all – which is really what it’s all about.
Author Biography
Mr. Pirrera is responsible for Vantage Consulting’s operations, maintaining high levels of technical excellence, and directs the firm’s expansion into new geographies. He also contributes his 25-plus years of manufacturing and engineering operations to customer engagements in the food, pharmaceutical and biotechnology industries.