By Allan Marinelli and Howard Mann
The pharmaceutical industry, under the FDA and other regulatory bodies, has been utilizing computer software/systems validation testing methodology to validate software under GxP environment in alignment with the principles of good automated manufacturing practices (GAMP) since the first published guidance in 1994.1,2
Part 1 of this article series dived into the use of Agile methodology with a focus on the planning phase of a software development plan. In this article, which is part 2 of three, we will dive further into the use of Agile methodology, including a discussion of practices toward alignment among the pharmaceutical, biopharmaceutical, vaccine, medical device, and cell/gene therapy industries. We will look at the following topics:
- Team structure and collaboration phase
- Defining requirements and product definition phase
- Software architecture phase
- Detailed design phase
- Implementation and unit verification phase
- Integration and integration testing phase
In part 3 of this series, we will dive into the following topics:
- Rationale for software system testing phase
- Software release
- Configuration management and change management
- Corrective and preventive action
Aligning On Practices3,4
Attempting to align Agile practices with regulatory goals, concepts, and practices entails accounting for the following elements prior to introducing any Agile approaches into your medical device quality management system (QMS). This can also be applied within the pharmaceutical, biopharmaceutical, cell & gene therapy, and vaccine industries.
Team Structure And Collaboration Phase
The ideal team structure requires hiring the right team members who understand the Agile approach while continuously collaborating throughout the software development cycle. This entails four key aspects:
Pairing two open-minded technical people who have different, independent perspectives means they can brainstorm their ideas while simultaneously adding value to the mission of developing the software throughout its life cycle.
However, in cases involving more complex software development and more demanding industry expectations, pairing may be inadequate. Instead, additional peers or cross-functional teams may be required to ensure that all the angles are satisfied with respect to the scope of work mandated by the customer in terms of the level of risk, while maintaining alignment with the applicable regulations.
2. Stop the line
The notion of “stop the line” involves creating tools that allow the stakeholders to monitor processes or quality gates during the development of the software. These can include burndown charts, information transparency for all applicable stakeholders, acting on the input information provided to resolve the anomaly, and creating a positive work culture where alterations are welcomed.
In addition, these monitoring tools can be used as mechanisms to identify significant process issues in a formal CAPA (corrective and preventive action), as required by medical device regulations per FDA QSR 21 820.100, Subpart J, and ISO 13485, Clause 8.5.
These can pinpoint excursions from the initial intended use of the device by identifying abnormalities in the form of a punch list or other traceable means known/used by the stakeholders.
Retrospectives and reflections are practices that enable the team to regularly inspect their performance and take proactive actions to increase their overall effectiveness.
Retrospectives are event-based (e.g., at the end of an iteration, a release of a product, or any significant milestone) and are part of what the learned approach encompasses (i.e., what happened during this event that will help us improve?), whereas reflections are performed periodically, depending on the nature of the team and the circumstances during a specific timeframe. The essential question is "How is the team currently working to complete the overall project?"
It is recommended that these practices be performed regularly and frequently during the completion of each iteration, thus creating a positive atmosphere for the team to practice self-awareness inasmuch as attempting to shorten the software development cycle while maintaining effectiveness and providing quality outputs to increase user friendliness.
In summary, retrospectives and reflections enable ownership of continuous process improvement.
4. Collective ownership
Collective ownership involves the team taking ownership, assuming accountability and responsibility for the quality of the delivered software product.
With respect to the medical device industry (FDA’s QSR and standards such as ISO 13485), the team members are required to be trained in the QMS tasks that they perform as a foundation to Agile.
However, Agile takes a step beyond requiring the team members to have expertise in software design, implementation, and testing: It requires the ability to know when additional design changes are warranted and justified and when it is time to “freeze the design.” In the case of the latter, team members will potentially focus on putting supplemental inputs in a placeholder state to represent a sub-version of the current version or naming another version altogether to be put in the backlog, with the intent of potentially using those ideas in the future.
Defining Requirements And Product Definition Phase
In this phase, there are six elements to factor in:
- When does a story have enough definition for the team to begin work? Stories must be detailed enough to allow the developer to effectively elaborate and implement them.
- Requirements are done when a story is done. Use the completion criteria for a story/increment/release to identify where a requirement is formally documented, controlled, and reviewed/approved.
- Requirements documentation: Demonstrate that requirements documentation covers all areas that are required by regulatory stakeholders.
- Can stories/acceptance tests be used for final requirements? Well-elaborated stories can be used as part of the final requirements documentation mechanism such as installation qualification (IQ), operational qualification (OQ), performance qualification (PQ), requirements traceability matrix (RTM), configuration verification, etc.
- Can executable requirements be a valid part of the requirements definition and documentation? Executable requirements can be used as part of a final requirements documentation mechanism (IQ, OQ, PQ, RTM, configuration verification, etc.).
- How does Agile help with requirements for verification and validation?
- Agile promotes the active involvement of the customer (or customer proxy) by making their role part of the project team.
- The practice of pair programming helps with the process of requirements verification by enabling pairs to refine the requirements as the story is implemented.
- Demonstrations are a mechanism for feedback from the users. If the customer can be involved in the demonstrations, feedback on the adequacy of the requirements can be gained. This technique also helps reduce the risk of incorrect requirements definition, which can occur when a customer proxy is used. Use Agile practices for continuous requirements verification and validation.
Software Architecture Phase
In this phase, there are three elements:
1. Evolving architecture (Assess how much architecture work is needed in the context of a story.)
There are two main known architectures, Waterfall and Agile.
Known by some as an old-school approach, Waterfall entails a linear methodology whereby the architecture is an activity that usually begins early and ends just before other design activities are completed, such as coding, before commencing any validation activities.
In contrast, Agile uses an “emergent architecture” and recommends avoiding a big up-front design activity that includes specifying and locking down an architecture too early. Architecture is an activity that permeates the entire life cycle of a product and changes as a function of the project layer and story layer for each increment and iteration zero (“sprint zero” in scrum).
It is important to note that the endorsement of an architecture in the software development life cycle is a team effort. Every team member is responsible for the architecture in relation to the product’s features, building team-wide common knowledge, and identifying architectural problems.
2. Architecture planning (In software development plans, specify when architecture work is performed.)
3. Architecture verification
In this sub-phase, there are four sub-elements:
- Test-driven development with continuous integration and automated testing
- Refactoring (reworking or companion to simplify the design)
- Pair programming
- Iteration demos
NOTE: Agile methodology focuses on identifying architecture problems very quickly at the start of the development while endorsing continuous verification paradigms by incorporating continuous integration and verification. This includes the verification of software of unknown provenance (SOUP) while factoring in the associated risks and regression testing.
It is important to note that the goal of this efficient architecture verification process is to arrive at a successful implementation of the requirements indirectly or to tangibly demonstrate that the architecture supports the requirements. While the success of refactoring is dependent on the maturity and correctness of the architecture, it is important to ensure that all team members work in a collaborative way.
Detailed Design Phase
This phase has three elements:
1. Activities of detailed design (Define the activities that contribute to the detailed design.)
Since the Agile model is not as prescriptive and exhaustive as the Waterfall model, the activities involved in the detailed design and whether the situation involves upstream or downstream activities of the architecture and its implementation entail alignment with IEC 62304.
2. Emergent design (Perform regular verification of emerging design.)
During the development of the software, additional simple software design concepts will occur over time in small amounts. Focus on encapsulating the functionalities involving the simple design at this stage. The “nice to haves” or “additional complex designs” that have not been fully scrutinized during this phase should be reserved as options for future functionality that is not yet in the product.
3. Documentation of detailed design
It is often understood that it is only documentation, and one should only focus on continuously redesigning or singularly designing the functionalities/capabilities of the software to a point where it meets the projected intended uses, with minimal iterations between the sprints, including the least amount of documentation as possible.
The detailed design documentation (including the verification of testing, scripts, test scenarios, test cases, well-written code using modern programming languages, along with a complete set of well-written tests, etc.) and marketability of the software must be congruent with the functionalities/capabilities of the software. This is achieved by allowing the stakeholders to challenge the value of the documentation, and its potential gaps, which can further increase the likelihood of improving the functionalities of the software design features/performances to their utmost.
Hence, define the detailed design documentation to be produced in alignment with the intended functionalities/performance of the software, while considering the needs of development teams and regulators.
Implementation And Unit Verification Phase
When using Agile methodologies, development of implementation activities commences with planning activities in the project layer, which encompasses the release layer and increment layer.
The project team selects a story to be delivered that is incremented to reach the intended uses.
Some Agile practices will endorse the principles of test-driven development (TDD), pair programming, continuous integration, and continuous automated testing aligned to deliver high-quality software.
- Test-driven development (Create a unit test, prior to writing the code, that can be used to generate the functionality of the story.)
- Pair programming (Pair two developers/programmers that initialize the writing of the code with a goal of minimizing rework after the fact.)
- Continuous integration (Using executable requirements, tests are automatically performed each time changes are made to the product at the code level. For more complex systems, this activity is performed in the release or increment layers with potential manual integration testing.)
- Continuous automated testing (Unit testing ensures that all code is tested at a design level and the typically automated tests are repeatable.)
Integration And Integration Testing Phase (Hardware/Software Testing)
This is for systems comprising various software units or software applications, including the hardware components (e.g., firmware components).
Integration entails the verification that all software, interconnected software interfaces, and hardware components function as intended.
The integration process is delineated below:
- At the inception of developing the software, the stories are generated by factoring in the test-driven development techniques by writing up detailed design tests and, at times, writing executable requirements.
- A continuous integration process will automatically execute tests that were initially set up and, every time there is a software change to the code, the process will continuously test and will be subsequently reviewed on a periodic basis based on an established risk assessment cycle commensurate with the potential risk at question, in conjunction with manual testing as warranted.
- For the more complex systems, many hardware components can be tested through hardware emulators or stubs, which would allow the inclusion of the testing of hardware components over and above the software components/applications.
- Typically, complex systems are tested and verified for correct operation during the release layer.
- As part of the three layers (story, increment, and release), the integration testing serves as an input to the completion criteria.
It is important to understand and execute all the elements described in this article, so all the accrued work accomplished by the stakeholders will increase the odds of achieving a successful shippable product.
- Guidance On The Use Of Agile Practices In The Development Of Medical Device Software, https://webstore.ansi.org/Standards/AAMI/AAMITIR452012R2018
About The Authors:
Allan Marinelli is the president of Quality Validation 360 and has more than 25 years of experience within the pharmaceutical, medical device (Class 3), vaccine, and food/beverage industries. His cGMP experience has cultivated expertise in quality assurance, compliance, quality systems, quality engineering, remediation and validation roles controlled under FDA, EMA, and international regulations. His experience includes quality systems, CAPA, change control, QA deviation, equipment, process, cleaning, and computer validation, as well as quality assurance management, project management, and strategies using the ASTM-E2500, GAMP 5, and ICH Q9 approaches. Marinelli has contributed to ISPE baseline GAMP and engineering manuals.
Howard Mann works as an independent consultant and/or contractor in the operational, regulatory, and quality assurance arenas. He has extensive experience in the healthcare industry and provides technical leadership guidance to the business development process, including the product development process in all areas of GxP compliance.