5 Misconceptions About Data Integrity In Pharmaceutical And Device Development & Operations

By Peter H. Calcott, Ph.D., president and CEO, Calcott Consulting LLC

Data integrity (DI) as a topic or focus is not new: It is the foundation of basic research as well as the development of medicines. Patients and customers expect us to approach the science of drug development honestly and trust us to do it correctly. It is the cornerstone of our business, which is based on trust. However, in 2015, the Medicines and Healthcare products Regulatory Agency (HMRA) issued a landmark guidance on the topic of DI and problems it had seen in the industry during inspections.¹ This was reinforced by further guidances issued subsequently by the Food and Drug Administration and the European Medicines Agency and again by MHRA, both in 2016 and 2018.^2,3,4,5 But DI issues are not new. I have personally experienced them over my career, starting almost 40 years ago. They have always been there, but it appears they may have increased in frequency more recently, thus moving the issues to the forefront.

This two-part article series is not going to be a rehash of these very valuable guidances, which have been issued and implemented in most companies over the last few years. Rather, this article series is going to focus on my experience as a consultant who goes into companies to audit or perform gap assessments of systems and will illustrate some of the pitfalls companies fall into in the development of a rugged DI strategy. That includes misconceptions (this article, part 1) as well as poor implementation practices (my next article, part 2).

Misconception 1: DI Issues Can Always Be Traced Back To Numbers.

One of the first misconceptions is that DI issues always involve problems with numbers, so they are limited to the QC labs and the production shop floor. While there are numerous cases of DI transgressions involving numbers, there are many where numbers do not feature at all in the real issue at hand. A case in point is in the writing of validation reports and investigations. Often in validations and investigations, we get information that does not fit the preconceived expectations. This must always be evaluated, discussed, and featured prominently in the discussions and conclusions. I have found a tendency to “hide” these issues, hoping people just do not notice. The DI issue can start as a number issue, but often it escalates to wrong conclusions and wrong decisions that are not supported by the numbers that are there. These anomalies or deviations must be analyzed and persuasively discussed so we can truly assure they do not impact the conclusions and validity of the reports.

Misconception 2: DI Issues Are Intentional.

When you examine warning letters, you are left with the feeling that in these cases it is most probably intentional events that have been uncovered. However, in my experience, the majority of cases I have encountered have been unintentional errors – honest mistakes. A case in point happened when I examined a validation report of a computer system the company claimed was Part 11 compliant ⁶. In this case, a stress test of the system was ill conceived and executed such that the conclusion was not supported by the testing. When pointed out, the IT professional was shocked and immediately set the ball in motion for remediation of the system as well as others where they suspected the same issue might apply. So, when you start an investigation on a suspected DI issue, do not assume the worst of the persons involved. Give them the benefit of the doubt until you are really certain it is intentional.

Misconception 3: DI Issues Are Confined To E-Systems.

We all know that DI issues can occur in paper-based systems as well as e-systems and we know that the issues will manifest in different ways. We also know that the remediation can be quite different in the two situations. However, I have seen in many cases when the FDA has cited a company for DI issues; the remediation has focused overwhelmingly on assessments of e-systems. The company will create inventories of e-systems and methodically check them against Part 11 compliance criteria.⁶ However, not all DI issues are labeled DI issues in FDA citations. In some cases, the FDA may just reference poorly executed investigations that do not support the conclusions or they may reference quality issues by not reviewing documents adequately enough. There is no mention of DI issues; they are just not doing their job adequately. Yet these are paper-based examples of DI issues. What is often missing in many responses is the real recognition that DI issues can occur in paper-based systems that have nothing to do with Part 11 compliance. In these cases, there is no risk assessment of these paper-based systems or processes to identify weak points worthy of remediation.

Misconception 4: It’s A Computer Problem.

The agencies have been very forthright in identifying DI issues as a people problem.^1-5,7 What they mean is that it is people who cause the DI issues. With paper-based systems, it is very much clearer because people are actively engaged in the activities. The causes of these “human errors” are broad and can be due to many factors, including workload, effectiveness of training, and culture in the company, just to name a few. In the case of e-systems, many people indicate that it is the e-system that has failed. But that misses the point. It is people who evaluate systems, choose systems, configure them, validate them, test them, and ascribe access to the users. A failure in any one of these elements can render the system to being vulnerable to DI issues. I have seen many cases where the system administrator has assigned the wrong access level to an employee, giving them more access than the job requires or needs. Stress testing of systems is often found lacking in validations. I have also seen acceptance of vendor IQ/OQ that is substandard. As the saying goes, “buyer beware.” In most cases of DI problems, it can be traced to a human error or incomplete assessment.

Misconception 5: Management And Company Culture Don’t Play A Role.

In the guidances quoted on DI and its issues, the regulatory agencies are adamant: Management sets the tone and culture of the company and determines whether DI will be problematic or not. It is paramount that the prevailing culture in the company is one that focuses on the patient in assuring the delivery of a safe, efficacious product on time. With this focus on the patient, a blame-free culture that is open, where operators can raise issues without fear, must prevail. Operators in production or the QC labs must recognize that the right result of an operation or test may not result in the product being released. The right result may end up rejecting the material. The right culture is one major step in assuring an effective DI strategy that works.

References

1. MHRA GMP Data Integrity Definitions and Guidance to Industry March 2015
2. FDA Data Integrity and Compliance with GMP April 2016
3. EMA Data Integrity August 2016
4. MHRA “GX” Data Integrity Guidance and Definitions March 2018
5. FDA Data Integrity and Compliance with Drug CGMP December 2018
6. 21 CFR Part 11 Electronic Records; Electronic Signatures — Scope and Application
7. PIC/S Good Practices for Data Management and Integrity in Regulated GMP/GDP Environments July 2021

About The Author:

Peter H. Calcott, D.Phil., is president and CEO of Calcott Consulting LLC, which delivers solutions to pharmaceutical and biotechnology companies in the areas of corporate strategy, supply chain, quality, clinical development, regulatory affairs, corporate compliance, and enterprise e-solutions. He has also served as an expert witness. He also teaches at the University of California, Berkeley in the biotechnology and pharmaceutics postgraduate programs. Previously, he was executive VP at PDL BioPharma, chief quality officer at Chiron and Immunex Corporations, and director of quality assurance for SmithKline Beecham and for Bayer. He has also held positions in R&D, regulatory affairs, process development, and manufacturing at other major pharmaceutical companies. He has successfully licensed products in the biologics, drugs, and device sectors on all six continents. Calcott holds a doctorate in microbial physiology and biochemistry from the University of Sussex in England. He has been a consultant for more than 20 years to government, industry, and academia.