As the market for over-the-counter (OTC) drugs and non-prescription medical devices continues to expand, there is a need for human factors (HF) research to ensure these new products are safe and effective for use.
Study design
When evaluating the use of these types of products, both self-selection and usability should be considered. Although this approach was not specifically recommended in the 2013 FDA guidance, it aligns with the best practice of comprehensively assessing all foreseeable use cases, has been recommended by the FDA in recent years, and was discussed during the 2023/2024 HFES conferences.
Why this approach?
Mirroring a combination of self-selection and simulated-use testing activities more accurately reflects real-world use of these products. The incorporation of this self-selection step in the process is important to uncover use-related issues that may present as potential users decide whether or not they should use a product (e.g., individuals with contraindications).
Here’s how it works:
Self-selection phase: Participants first undergo a self-selection activity to determine whether they are appropriate users for the device. Those who correctly identify themselves as intended users move on to the simulated use portion of the evaluation.
Dismissal and probing: If intended users fail to self-select correctly or if non-intended users do self-select, they are dismissed from further testing. However, before dismissal, root-cause probing is conducted to understand the reasons behind their incorrect self-selection, offering valuable insights for potential design improvements.
Session duration & timing considerations
Typically, self-selection activities take about 15 minutes, including time to complete a health assessment questionnaire, the self-selection activity, and any necessary probing.
Manufacturers have a few different options with how to implement self-selection activities with respect to timing of the subsequent validation testing for their OTC products. Each of these options has their own advantages and disadvantages:
Option 1 – Combined self-selection & validation: Participants participate in self-selection and immediately participate in OTC validation following the self-selection activities.
- Advantages: Participants will only have one session, which decreases logistical effort for scheduling.
- Disadvantages: There are unknowns with recruitment with this approach as many may screen out. This could result in additional down time and test days, and the total number of participants to be recruited is unknown.
Option 2 – Self-selection as a recruitment tool: Participant self-selection process is used as a recruitment tool. For example, participants are intercepted in a public place (e.g., grocery store, mall) to see if they want to participate. They complete the self-selection process (~15 minutes) and then schedule a time for their follow-up validation study session at a later date.
- Advantages: By decoupling the self-selection from the validation study activities, there are efficiencies gained in scheduling and recruitment for the research team.
- Disadvantages: There may be some participant dropout for folks who cannot or are uninterested in participating in a larger study.
Option 3 – Separate user groups for self-selection & validation: A hybrid approach where option 2 is used to complete the self-selection testing, but separate participants are used for any additional simulated-use validation testing.
- Advantages: Avoids logistical issues with rescheduling participants from the self-selection phase to the validation phase.
- Disadvantages: Doubles recruitment size, which adds additional expenses.
Recruitment and sample size: striking the right balance
Recruitment is a critical aspect of any HF study, and for OTC medical devices, the FDA recommends recruiting participants from the general population using flyers or advertisements. Once participants arrive, they complete a questionnaire to determine if they are intended or non-intended users.
The FDA guidance suggests continuing the study until at least 15 participants successfully self-select for the simulated-use portion. However, given the variability in self-selection outcomes as well as the variety of approaches a manufacturer can take with study timing, there are a few different strategies for participant recruitment:
Option 1 – Combined self-selection & validation: When planning to run self-selection activities and usability validation testing together, we recommend recruiting around 25 participants to ensure a sufficient number reach the testing phase due to participant attrition resulting from failures to self-select. This approach provides a buffer, as predicting how many participants will be dismissed during the self-selection process when working with a general population sample can be challenging. Depending on the contraindications / indications for the product, this could also impact the total volume of recruited participants.
Option 2 – Self-selection as a recruitment tool: When using self-selection as an early recruitment tool, we recommend running intercepts in a public place for 1-3 days to identify a minimum of N=15 participants who successfully self-select for the simulated-use portion and agree to participate in a follow-up session. An additional n=2-3 participants may be used as buffer for last minutelast-minute cancellations.
Option 3 – Separate user groups for self-selection & validation: When splitting out self-selection and validation into separate activities, we recommend using option 2’s recruitment strategy (i.e., intercepts in a public place) to complete the self-selection portion. For the validation portion, we recommend following the guidance by the FDA by recruiting a minimum of N=15 participants for sessions (with an additional 2-3 as a buffer in case of last-minute cancellations).
Other recruitment strategies
Recently, the HF community has suggested that manufacturers may want to include 15 intended users and 15 non-intended users. This proposed change would allow for a more definitive sample size of 30 participants, facilitating a more comprehensive evaluation of the product from both ends of the self-selection spectrum. This approach does not currently appear in FDA guidance but represents a potential future direction for study designs.
We strongly advise engaging with the FDA through a pre-submission for protocol review ahead of validation for OTC medications. Early engagement can ensure that your study design aligns with FDA expectations, ultimately saving time and resources.
Need support for your next study? We’re here to help! Email us at [email protected]