We developed a mixed methods approach that used IDIs, survey research, desk research, and real-time process mapping to understand the landscape of telehealth. Methodology specifically considered the wants and needs of patients and healthcare providers as well as the effects of COVID-19 on the global adoption, trust, access, and usage of telehealth technologies.

We conducted 90-minute individual interviews that incorporated differentiation, simulated-use tasks, and knowledge assessment tasks. Participants were patients with type 2 diabetes across a range of body mass indexes (BMIs) and nurses who administer insulin regularly. The instructions for use (IFU) was assessed and updated iteratively during week 1, and one final version of the IFU for each prototype was used during week 2.

We recruited representative members of surgical teams to conduct simulated implantations in porcine subjects at a simulation lab managed by a local hospital.
Our researchers observed the simulated implantations to capture the surgical team dynamics and interactions with the revised implantation procedures.

The validation study included HCP training, done by our on-staff RN to demonstrate techniques, a 24-hour decay, and a 60-min validation session, during which participants had to prepare and administer two doses of the kit.

For this exploratory research, we simulated at-home environments in a lab setting, including a mock living room, bedroom, and bathroom. Set up in the same test space, we used room dividers, furniture, sink, and mirrors to make it feel as realistic as possible. The goal was to encourage the pediatric patient and caregiver to do what they do at home with the medical system and accompanying mobile app. They simulated the entire process to interact with the app and medication system. Test preparations unique to this study included informal interviews with child specialists familiar with this population to ensure the lab setup would make the participants feel comfortable; modifications included stress relief balls, sensory bin, and wall signs. This specialized population also required we have an on-site clinician throughout the test sessions.

We conducted 90-minute usability sessions with adult patients with rheumatoid arthritis and/or polymyalgia rheumatica and their healthcare providers. Participants completed two injection tasks, the second requiring use of the IFU to ensure comprehensive evaluation of platform materials, and then completed a series of knowledge tasks. When time permitted, patient participants completed a force assessment using prefilled syringes with placebo of differing viscosities.

We conducted an expert review of the devices to determine how the company’s injection device compared to other injection devices. The goal of this review was to determine whether the device has novel features and its user interface components are consistent with competitor devices currently on the market.

We conducted an ethnography study across multiple countries in which we observed laboratory technologists in-person as they interacted with the diagnostic machines. Researchers observed various sizes of laboratory operations in order to compare workflows across hospital size and geographic regions. Where possible, we asked follow up questions after observations to confirm findings.

Three experts were chosen by HFES to design and deliver a two-day training, including Korey Johnson (Bold Insight), Dr. Anthony Andre, and Michael Wiklund. The team presented a series of modules that included the science behind HF, such as human perception, cognition, memory, and learning, as well as best practices in root cause analysis and dealing with the biases associated with subjective data.

Our team provided a broad overview of how feedback from end users should be leveraged early and often during product development, and how that feedback should be integrated into use specifications, task analyses, risk analyses, and the overall human factors plan.

We conducted 45-minute usability testing sessions with adult patients, caregivers, and healthcare providers. Participants were tasked with administering a dose of medication using the autoinjector and administered knowledge task questions to assess their understanding of the instructions for use (IFU) and labeling.

We utilized one-on-one moderated usability tests in a simulated-use environment. Our lab provided items commonly found in a clinical setting including a sink, personal protective equipment, trash bins, and cleaning and disinfectant solutions. Participants were instructed to do what they would do in their own practice to prepare the device for their next patient. Instructional materials were provided, but participants were not specifically directed to use them.

Throughout this research, the goals were to evaluate the new system for safety, effectiveness, and unforeseen use risks, think through touchpoints and pain points of updated releases to ensure this is accounted for in the new system, consider alternate designs to make both versions of the system more usable and understood, and bridge patients to the new app (training, integration).
To reach this specialized patient population, we tested in multiple locations across the country, both in English- and Spanish-speaking in-lab sessions. We also utilized our on-staff RN to advise on considerations during testing for this protected patient population to ensure we approached testing scenarios and environments in a way that participants will understand and respond to. One part of the research required clinician-assisted training to simulate real-world training of the system.

We asked current and past users of the drug delivery pump to evaluate images and videos of three new designs. We identified pain points and avenues for improvement by asking users about specific instances where they struggled with their current carrying accessory.

We tested the redesigned device with a surgical team trained on the current system.
We conducted in-person testing in an outpatient clinic, in the surgical team’s environment. We evaluated system setup, tool programming, and use during a simulated surgery.

We conducted 90-minute remote usability sessions with adult patients throughout the UK using mid- to high-fidelity prototypes. Participants interacted with the prototypes using remote screen control to evaluate the value of the content, the usability of the app, and their overall sentiment towards digital therapies. We worked closely with the client throughout the project to ensure information security and ethical best practices were followed at all times.

We conducted a four-day ethnographic (in situ) study in the UK and the US. In both countries, we visited a small clinic and a large hospital to capture lab technicians’ interactions with the analyzers within highly automated and manual workflow settings. Throughout the observation, we recorded friction points, troubleshooting instances and potential areas for improvement. We also worked closely with the lab technicians to create a journey map of their overall workflow, where we identified key processes involved in each stage of user interaction in both environments.

We conducted the study at a hospital where we asked nurses to perform tasks typically executed on an EHR under simulated conditions. The nurses then provided feedback on a post-test questionnaire for qualitative and quantitative data.

We took a multi-method approach to the study. We first visited orthopedic surgeons’ offices to observe current practices while they administered intraarticular injections into the knees of patients, view the spaces in which they work, and talk to staff that assist with the injection process. After the site visits were completed, we conducted one-on-one interviews with HCPs to further learn about their routines with intraarticular injections, their thoughts on injecting to other sites, and opinions on reducing the total number of injections needed for a full dose. We concluded the sessions with the HCPs administering a simulated injection with a prototype syringe into a model of a knee and providing high-level directional feedback for future development.

We conducted remote exploratory interviews with patients and healthcare providers, using their journeys from diagnosis to treatment to guide the development of potential interfaces. Patient recruiting included adults and minor participants (aged 5-17) with amblyopia, a type of visual impairment; some also had additional diagnoses, such as epilepsy or cerebral palsy, which impacted their mobility and cognitive abilities.

We conducted one-on-one, in-depth interviews with participants currently taking the medication in pill form. We asked participants to complete a simulated-use scenario using the prototype device. We collected feedback on ease of use, physical attributes, need for instructions, and overall acceptability of the device. We also captured participant data around the new liquid formulation, including appearance, taste, and color.

We conducted in-lab testing to determine how much feedback and involvement was acceptable and desired by the user. Using graphical cards, users identified the ideal flow to receive, store, administer, and dispose of the device. One month later, additional testing with a smaller group of new and previous testers refined the user flow based on feedback from the first session, delving deeper into features, drawing on their current journey to determine how best to utilize connectivity.

We recruited nurse educators who routinely train patients to use infusion pumps and invited them to share their insights and unique experience during 90-minute remote sessions. We evaluated their feedback about how they structure their educational sessions and about how training needs are determined.

We conducted simultaneous in-lab sessions with both pharmacists and pharmacy technicians. They each performed their normal roles in a simulated pharmacy setting we designed. Our research sessions utilized simulated pharmacy shelves with realistic distractor cartons and prescription mock-ups.

We interviewed more than 500 health care professionals from the US, UK, and Ireland, ensuring variety in our sample collection.
To learn how HCPs might interact with the system, we asked them to naturally request patient data in their own words, collecting both text and voice samples of various scenarios for each participant.

We conducted 60-minute remote usability sessions with adult patients and caregivers throughout the United States. Participants were asked to interact with the app to evaluate its ease of understanding and presentation of relevant content. We observed participants as they performed key tasks and collected their feedback as they interacted with the application.

We conducted 90-min individual interviews with specialty healthcare providers. Participants interacted with a clickable prototype of both the desktop and mobile websites. In addition to testing the usability of the site, the various types of content were also evaluated.

Our research approach included a certified diabetes educator who administered a 2-hour group training session with participants. After a minimum decay period of 24 hours, we observed participants using a non-functioning prototype.

Our research approach involved a 90-minute in-depth interview with participants from the target population, who were observed as they followed the IFU to set up and used the device. We then asked them questions about their experiences and probed on any aspects of confusion at each stage of their interaction with the device.

For the US portion of this international study, we recruited MS patients for 60 minute in-person sessions. Participants were asked to complete a series of tasks with guidance from the application. After their attempts, participants were asked to evaluate the ease of understanding the application’s instructions and user interface.

Our 4-year program of research involved usability testing, iterative design research, and simulated-use environments. We designed and operated both low-fidelity (Wizard of Oz) and high-fidelity (scenario-based trauma center simulation) test environments over the course of 10+ projects carried out in both the United States and United Kingdom.

Before one-on-one interviews with potential users, our team conducted a heuristic evaluation of the instructions for use. We then completed a simulated-environment usability study to collect data that guided revisions and updates to the instructional materials. We then conducted a post-iterative study where we collected further feedback on the product and redesigned instructional materials.

We created and executed a documentation plan based on FDA and international HF guidance documents to fulfill regulatory agency expectations and to ensure readiness to demonstrate safe and effective use of the device. We then conducted an in-person validation study, including a training session and a test session, with healthcare providers (HCPs). ​

We developed an HF plan and associated documentation and provided consulting on the HF development and regulatory approach. This included identification of target users and specified use cases per ANSI/AAMI/ISO 62366-1, identification of use-related risks and proposed strategies for risk mitigation, and development process documents.​

Both studies conducted one-on-one, in-lab usability sessions with three different groups of specialized HCPs using prototypes of the portal. Our ability to adapt and be flexible, debriefing in real time, was key to successfully accomplishing the study goals: When we discovered that session time was inadequate to collect the desired data, we debriefed after the sessions to refocus testing for the following day. We already had clear trends, so it was possible to adjust the protocol and moderators guide to cover all of the manufacturer’s objectives.

We conducted interviews with subject matter experts (SMEs) from the manufacturer’s organization, as well as contextual inquiries in the field with clinical teams to understand known opportunities for improvement and current challenges.
We used these insights to create a number of simulated-use scenarios and conducted lab-based research to refine and prioritize design opportunities.

We conducted a mixed-method study involving a series of remote, individual interviews and a longitudinal online diary study with participants to gather detailed qualitative insights into patients’ treatment behaviors, preferences, use environments, and how patients use connected devices and applications in their daily disease management.