As artificial intelligence (AI) advances, it is transforming industries at a rapid pace. The medical device and healthcare software industries utilizing AI are no different. Because of this, the FDA has developed guidance to provide expectations to keep up with this new device development pace.
Machine learning’s impact
Machine learning is a sub-category under the larger AI umbrella. Machine learning’s distinguishing feature is an algorithm’s ability to mimic the way a human learns over time. As such, machine learning algorithms continuously improve upon themselves by incorporating new datasets into an existing model. This means that a new algorithm is continuously being created that, in the medical device software arena, needs FDA authorization.
Regulators have wrestled with the question: How do you regulate something that is constantly changing? The FDA refers to these as “machine learning enabled device software functions” (ML-DSF) in its guidance documents, and the FDA’s Predetermined Change Control Plan (PCCP) attempts to address this issue. In this blog we describe the main components of the PCCP submission and provide recommendations for how incorporating human factors principles in developing the PCCP approach can improve the chances of successfully implementing a PCCP.
What is the PCCP?
Released in April 2023, aims to reduce the number of potential submissions that would otherwise be necessary in an inherently iterative process like software development. In the PCCP, manufacturers document planned modifications to the ML-DSF that affect the software’s use, safety, or effectiveness. If planned modifications are not specified within the PCCP, they may require a new submission.
There are three main components to any PCCP submission:
Description of Modifications – This section identifies specific, planned modifications that are intended for the ML-DSF. The description should include any changes to the device’s characteristics or performance that will occur with a specific modification, limited to those that can be verified and validated. Manufacturers should utilize usability testing to identify potential future modifications and evaluate potential UI design patterns of these planned modifications prior to including them in the PCCP. Data from this testing is then used to support modifications on the front end, reducing the potential testing burden on the back end. With this approach, the development team is more informed about which potential modifications would be most effective, which can improve decision making when choosing between multiple modification options.
Modification Protocol – This section describes the methods to be followed when developing, validating, and implementing modifications delineated in the Description of Modifications This includes data management practices, re-training practices, performance evaluation protocols, and updated procedures, including communication and transparency to end users. Manufacturers can develop pre-defined usability testing protocols such that when a certain type of modification (e.g., changes to the user interface) triggers the need for usability testing, the development team is prepared for the effort required to conduct testing. Incorporating the time required and tasks associated with testing into the development schedule prepares teams to integrate the necessary testing effort and reduces potential disruptions of unexpected testing needs.
Impact Assessment—This section assesses the risks and benefits of implementing a PCCP for an ML-DSF and how those risks will be mitigated. It will compare the device with each modification implemented to the version of the device without the modification implemented. This section also requires a discussion about the risks and benefits of the modification, including any potential social harms. Manufacturers can use human factors principles to define guardrails that identify which modifications would or would not require additional usability testing based on use-related risk and usability parameters. Defining modifications with this pre-determined criterion makes providing justification in an impact assessment more efficient and effective. Additionally, knowing which modifications require testing informs the development team of the potential impact associated with desired modifications, allowing for effective planning.
The PCCP is important because it acknowledges the FDA’s current read on a continuously evolving aspect of medical device technology; it will likely dramatically impact the development of these products. This new documentation also forces AI-enabled software developers to think about the end user from the beginning of the development process, saving time and frustration down the road.
Need support navigating this guidance? We have over 20 years of experience planning, managing, and conducting research within the highly regulated medical device testing space. Let’s talk! [email protected]