A Simplified, Five-Step Approach To Applying FDA's Human Factors Guidance

FDA’s most recent guidance on human factors, Applying Human Factors and Usability Engineering to Medical Devices, highlights the importance of enhancing patient safety by adopting human factors engineering processes in the design and manufacturing of medical devices. This topic has become more relevant in recent years, as the benefits of properly implementing human factors in medical device designs have become more evident.

FDA’s implies the need for Human Factors through 21 CFR 820.30 (Design Controls) by stating that design input should include needs of the user and patient, performance criteria should be met as part of design verification, and design validation shall ensure that devices conform to defined user needs and intended uses — including testing of production units under actual or simulated conditions.

Human factors engineering used in design validation activities focuses on the interaction between people, technology, and work environment. Processes may include: task analysis, to identify the job demands and performance challenges faced by users; use of design controls, to ensure that devices are compatible with human capabilities and limitations; continuous involvement of representative users throughout the design process; feedback from postmarket surveillance activities, to identify unanticipated threats to safety that can be removed or reduced by redesign; and analysis of field observations, to understand the actual conditions under which users work with the device.

Since FDA considers human factors a valuable component of product development, a guidance was developed to allow manufacturers to understand the agency’s perspective. In addition, FDA has identified circumstances under which human factors validation testing should be submitted in premarket submissions, due to harm potential from use error. To provide clarity, FDA also has published a draft guidance with a list of devices that meet this criteria: List of Highest Priority Devices for Human Factors Review. Through this simplified, five-step approach, I intend to summarize requirements needed to fulfill FDA’s human factors expectations in Premarket Regulatory Affairs submissions.

Step #1 — Conduct A Robust Identification Process

Identify Device Users:  All users involved should be identified. For example, it’s important to include who uses the device, purchases the device, unpacks the device, sets up or installs the device, disposes of the device, etc. The ability of a user to operate the device depends on personal characteristics that also should be evaluated and understood by the design and development teams. The medical device organization should take into account any physical or cognitive limitations, education level (to understand reading ability and/or training needed), and any other relevant characteristics that might affect how users interact with the device.

Identify Device Use Environment: An evaluation of the environment in which the medical device will be used provides insight into many different conditions, allowing device manufacturers to design more optimal user interfaces. Some questions that should be answered at this stage include: What is the lighting like? What is the noise like? What is the room where the equipment will be used like? Are there similar models of the same device in the same room? Will the device be used in a moving vehicle? What is the temperature like where the device will be used? How many people will be in the room where the equipment will be used? These conditions, which could affect safety and effectiveness, should be taken into account.

Identify User Interfaces:  Interfaces should be evaluated to include their interaction between the user and the device. Interfaces can be used while users set up, use, or perform maintenance on the device. Examples of interface considerations include displays, device shapes, accessories, switches, buttons, training material, etc. Medical device manufacturers can identify potential hazards from these interfaces, leading to design improvements that prevent patient or user harm.

Identify Critical Tasks:  Identifying all device critical tasks that users should perform correctly for the device to be safe and effective is an important step in the identification process. Critical tasks are those that, if performed incorrectly, could cause serious harm. These tasks can change as design evolves and more information is learned about user interactions with the device interface. Through the use of a fault tree analysis or a failure mode and effect analysis, critical tasks can be adequately identified. Design and development groups should be aware that, while it can be frustrating, in some cases, some critical tasks will not be identified until validation testing is conducted.

Identify Known Issues:  Known user-related problems also should be identified and captured to be considered during user interface design. This data may include faults with devices made by other manufacturers, feedback from post-market surveillance systems, training and sales personnel knowledge, etc. Journal articles and databases are very helpful in the identification of known issues. By taking a look at the product lifecycle and researching products with similar classifications, design developers can identify many problems associated with related devices, and work to mitigate hazards at an early stage.

Step #2 — Conduct A Streamlined Formative Usability Study

Through formative usability testing, medical device manufacturers can evaluate a final product — or interface elements of the product — in prototype form during the design and development process. This helps ensure that the design elements meet expectations, as well as identify areas that can be improved.

These studies are used to refine results of initial data, and to determine if design modifications are necessary. They should also focus on aspects where user interface design options are still under evaluation. This step is very important, because it can reveal previously unrecognized user-related hazards and errors that can become critical tasks, allowing the design team to determine if improvements implemented are effective in reducing or eliminating hazards or potential user errors.

Formative studies are conducted when design and development activities are still in progress, and therefore are less formal, allowing teams more flexibility with sample sizes and the data collection methods they decide to use, which could include: simulations, interviews, cognitive walkthroughs (asking participants to state aloud their thought processes, and to explain any difficulties they experience with use of the device), or contextual inquiry (observing representatives of intended users interacting with a currently marketed device, similar to that being developed, as they normally would in a real environment), among other techniques.

A test plan protocol used in formative evaluation typically includes: evaluation purpose, a description of interface to be evaluated, use scenarios and tasks involved, an evaluation of the end users, the data collection and data analysis methods to be used, and an explanation how the results will be used to trigger design modifications that will mitigate hazards.

Formative evaluation should be used to perfect the device design in preparation for validation testing; using iteration to ensure related risks are mitigated, changes are successfully implemented and no new risks are introduced. It is important to note that, if design problems are identified during a validation, then that validation becomes a formative evaluation. Finally, when conducting formative evaluations, organizations should take into account all factors affecting the evaluation, such as participant training or lack thereof, to avoid arriving at incorrect conclusions.

Step # 3 — Eliminate or Reduce Use-Related Hazards

Although they can be burdensome and costly, design modifications generally are the most effective process to mitigate use-related hazards. Organizations should consider putting emphasis on risk severity.

Use-related hazards can be managed with the use of risk management tools, such as a) Incorporating inherent safety by design (for example: removing features that can be mistakenly selected, eliminating interactions that could lead to user errors, or using specific components that cannot be connected incorrectly), b) Incorporating protective measures in the medical device itself or in the manufacturing process (for example: incorporating warning screens, display alerts, interlocks, or safety guards), or  c) Incorporating information for safety (for example: including caution statements in the instructions for use, explaining potential hazards or providing user training).

Incorporating Information for safety and providing user training is an acceptable and widely used practice. However, it is not always the most effective, since it requires users to rely on memory, or it might not be available during use of the medical device. Studies on labeling and training are highly recommended, because they assess the clarity of all labeling — packaging, instructions for use — and the level and type of training that will be necessary, if it is required.

Step # 4 — Conduct a Validation Testing

FDA encourages manufacturers to submit a draft of the human factors validation protocol for feedback before it is executed. The main goal of human factors validation testing is to demonstrate that the device can be used by its intended users without serious use errors, or problems with the intended uses under the expected use conditions.

The validation test should focus on the final critical task list. At this stage, it’s important to ensure participants reside in the geographical region where the device will be commercially available, and that labeling and training materials correspond with those to be used in the regions being evaluated. A robust test protocol should describe the number of times participants will use the device and the critical tasks to be evaluated, including data collection and evaluation techniques.

In addition, training delivery and dwell time between training and testing provided to the study participants should be described. The design of training promoted for safe device use should reflect the training that will be used commercially. If intended users will receive no training before using the device, then participants in the human factors validation should not be trained.

All interface elements should be available during the validation. The validation should be conducted using realistic simulated-use scenarios. Some critical factors during this step include ensuring that the testing involves at least 15 representative participants per user group, and that those participants represent the range of characteristics within their user group (for example: age, occupation, impairments, literacy level, etc.). Different user groups may have different knowledge or limitations that could affect how they interact with the device interface, and therefore should be separated into distinct users groups (for example: health care providers vs. patients with sensory disabilities, pediatric users vs. adult users, etc.).

During the validation, device manufacturers should consider carefully documenting data obtained by observing the participant’s performance, as well as data from interviews conducted after the use scenarios are completed. In the event that users report any errors, it is important to try to obtain as much information as possible about the issues observed by asking specific questions about how each problem occurred.

Step #5 — Conduct Data Analysis to Create a Usability Report

The results of the human factors validation test should show no user errors or problems that could result in serious harm; any errors or potential problems should be eliminated or reduced through redesign. Use-related risks should be assessed to understand whether design modifications are needed and how use errors occurred, as well as to determine if design change modifications will reduce risks to acceptable levels. An easy way to do this is to create a table where each task associated with use of the device is analyzed, taking into account user errors or problems, observations from the study, root causes, clinical consequences, potential harm, and mitigation measures to reduce risk.

For each one of these tasks, it also is important to determine if redesign is needed, and to retest as needed after any redesign activities, ensuring that any modifications made will not incorporate new unacceptable risks. While periodic reviews of postmarket surveillance already are a regulatory requirement, use of this information to gather relevant data for analysis of human factors in a systematic process can allow companies to build a dynamic and interactive mechanism, feeding into the human factors evaluation process.

Finally, a human factors evaluation report should be prepared for inclusion in a regulatory premarket submission, discussing a summary of safety related considerations, processes used, and conclusions. The FDA provides a recommended structure under Appendix A of the Guidance; it includes conclusions drawn from the human factors evaluation study, description of users, uses, use environments, training and user interface, a summary of known problems, analysis of hazards and risks, a summary of preliminary analysis and evaluations, description and categorization of critical tasks, and details of the human factors validation testing.

The report does not need to include test data, but the details should be sufficient to clearly explain to regulatory reviewers how all serious use-related hazards were identified, evaluated, and mitigated.

Conclusion

This process has become critical to demonstrating safety and effectiveness as part of medical device engineering development. Human factors evaluations also allow FDA to verify that the device has been in the hands of users early in the development process, where unanticipated hazards could be assessed and mitigated. FDA expects human factor validation to utilize representative users and realistic testing conditions, a clear description of the protocol and results, and a robust analysis and justification of the residual risks.

Medical device organizations with robust human factors processes are leading the market, proving to have better clinical outcomes, and better quality — which can provide medical device manufacturers with a market edge. By following this five-step approach and ensuring the process is standardized across all new products under development, or when design changes are made to improve functionality, companies can leverage usability engineering to significantly reduce risks and increase efficiencies.

About The Author

Marcelo Trevino is the President, Global Regulatory Affairs and Quality Systems, at TregMedical, a life sciences group focused on global medical device regulatory, quality, and compliance. Marcelo can be reached at: marcelotrevino@outlook.com

Marcelo has 23+ years’ experience in quality and regulatory affairs, serving in multiple senior leadership roles with different organizations while managing a variety of medical devices: surgical heart valves, patient monitoring devices, insulin pump therapies, surgical instruments, orthopedics, medical imaging/surgical navigation amongst others. He has an extensive knowledge of medical device management systems and medical device regulations worldwide (ISO 13485:2016, ISO 14971:2019, EU MDD/MDR, MDSAP). Mr. Trevino holds a B.S. degree in Industrial and Systems Engineering and an MBA in Supply Chain Management from the W.P. Carey School of Business at Arizona State University. He is also a certified Quality Management Systems Lead Auditor by Exemplar Global.

He has experience working on Lean Six Sigma Projects and many Quality/Regulatory Affairs initiatives in the US and around the world including Third Party Auditing through Notified Bodies, Supplier Audits, Risk Management, Process Validation and remediation activities.

Additionally, he is a Certified Six Sigma Black Belt and Biomedical Auditor through the American Society for Quality (ASQ) and holds Certificates in Environmental & Sustainability Management Regulatory Affairs Management from University of California, Irvine.

He regularly publishes articles to assist corporations in their quest for exceptional quality and regulatory compliance.