In today’s fast-evolving healthcare landscape, understanding and implementing effective validation strategies for medical devices and biotech products is not just beneficial—it’s essential. One such strategy that merits a closer look, especially if you’re involved in healthcare UX and human factors research, is the implementation of a knowledge task (KT) during a research session and how best to tailor this to your research objectives.
Let’s dive into why this methodology is included in medical device testing and how to leverage it to further assess product safety and effectiveness.
Understanding knowledge tasks: A crucial component in medical device testing
First off, what exactly are knowledge tasks according to recent FDA guidance documents and HF standards? Although the definition varies across these references, to put it simply, knowledge tasks are to evaluate users’ understanding of critical safety information. This could range from interpreting warnings and caution statements in a user manual to identifying the storage requirements printed on the carton. The consistent element across all definitions of knowledge tasks is the focus on safety information comprehension found within a product’s labeling and packaging.
When to assess knowledge tasks in medical device testing
The KT approach has three building blocks: when to assess, how to set up, and what metrics to assess. These elements provide HF practitioners with options that build upon each other to create a successful approach for knowledge task execution.
When to assess – This phase refers to when to ask knowledge task questions to a participant during a test session. There are a couple of options:
- OPTION 1- During a separate, dedicated section after simulated-use tasks are complete, or
- OPTION 2 – During both simulated-use tasks and a separate knowledge task section.
Each of these options has its pros and cons. If you assess during a separate knowledge task section, you follow the CDRH 2016 guidance directly. Still, you might miss out on opportunities to leverage observation-based insights, in which case it could be beneficial to proceed with Option 2. Option 2 provides a versatile approach but adds coding complexity to a study session.
How to set up – Building upon ‘when to assess’, if you plan to assess knowledge tasks during a separate knowledge task portion of a study session, you’ll want to think about how to set up that knowledge task activity. There is a direct setup option and an indirect setup option.
- A direct setup focuses on participants using the labeling to answer each knowledge task question and assesses a direct relationship between labeling effectiveness and understanding.
- An indirect setup focuses on the participants behaving and answering questions as if they were in the real world, where the labeling may or may not be used. It tests an indirect relationship between labeling effectiveness and understanding.
Neither is better or worse than the other; it all just depends on what you want to test and how.
What to assess – The final building block of the KT approach is to determine what aspects to assess. According to FDA guidance and regulatory documents, “comprehension” (or understanding) and “perception” (or locating) are vital components, with comprehension being the most emphasized across guidance documents and standards. Assessing both location and comprehension ensures that participants can find and understand information effectively. However, this approach may lengthen test sessions and pose challenges in assessing location accurately. On the other hand, only assessing comprehension may overlook labeling attributes (e.g., labeling saliency) and the importance of finding information. Both approaches have their rationale and challenges, but practitioners must choose the method that best suits their testing objectives and regulatory requirements.
The importance of knowledge tasks in healthcare UX research
Bold Insight emphasizes that while simulated use is often considered the gold standard in human factors research, it’s not always possible or practical. This is especially true for tasks or information that rely on what the user knows rather than what they do. Here, knowledge tasks come into play, offering a viable, often underutilized, alternative.
Despite their value, knowledge tasks are complex and require nuanced understanding and application. They are not a one-size-fits-all solution but demand customization and careful consideration of the specific context and user needs.
Tailoring the knowledge task methodology to fit your study needs can significantly enrich your human factors validation studies, particularly in healthcare, where safety and effectiveness must be evaluated. By understanding when to assess, how to set up, and what to assess, you can unlock more profound insights into user comprehension and, ultimately, contribute to developing safer, more effective medical devices and biotech products. Remember, the goal is not just to complete a validation process but to genuinely enhance user understanding and application of critical safety information.
Interested in planning a study? We’d love to hear from you. Let’s talk!