UX research reveals common concerns from HCPs about telehealth technology

UX research reveals common concerns from HCPs about telehealth technology

U

BOLD INSIGHT

Our research has shown a trend in healthcare provider feedback regarding mHealth alerts and features. While generally positive about the concept, liability is top of mind with this user group and common questions were raised during testing.

HCPs and Telehealth

Part 1 ∙ UX research reveals common concerns from HCPs about telehealth technology

By Chris Goesel, RN and Kamila Lada

Pharmaceutical manufacturers and health IT developers are looking for ways to incorporate features into products that increase and improve communication channels between healthcare providers (HCPs), patients, and the patient care team. The goal is to facilitate quicker access to patient data and allow the care team to communicate remotely.

Advancements in technology have undeniably changed the way HCPs communicate with each other and the way they care for patients. While these changes may be viewed as positive by some, it is not the case for everyone. In the minds of HCPs, how does connectivity (and telehealth) improve the way they care for patients and collaborate with colleagues? What issues are solved vs. created? What problems are HCPs still seeking to solve in relation to the multitude of issues they face on a regular basis?

While conducting user experience (UX) research on mobile health (mHealth) applications, we noticed a trend in HCP feedback regarding the concept of alerts and telehealth that is not always positive and potentially indicative of a gap between user need and product design.

Why HCPs like the growing connectivity with their patients

During some recent UX research, HCPs said connectivity was a useful aide when it came to patient care. The ability to instantly obtain critical clinical information allowed them to diagnose and treat patients in a timelier manner, especially when on the go.

Further, patients may have various physicians (e.g., PCPs, specialists, etc.) assigned to their case and it is necessary for HCPs to communicate and discuss clinical findings with consulting doctors. The option to share specific findings across HCPs allows for immediate collaborative feedback resulting in better quality care.

However, while the idea of connectivity has its advantages, we also found HCPs raised some red flags about the technology.

Telehealth and alerts deemed problematic for some HCPs

Healthcare professionals are faced with many issues daily; patient safety, HIPAA compliance, and liability are top of mind. The idea of being accessible and having patient data and clinical information at their fingertips around the clock causes some HCPs to consider the potential downside of connectivity. Who is liable for responding to critical parameters when multiple HCPs have access to patient data at the same time? What if time-sensitive clinical information is missed when received as a mobile app notification? What if HCPs are overheard while dictating pertinent patient information to a mobile app? What if the wrong information is entered or shared such as medication dose, patient name, diagnoses, or test results? The idea of 24-hour accessibility made HCPs contemplate the potential shift in expectations for how they provide care to their patients.

The challenge for product designers is to keep these pros and cons in mind while creating a productive and safe environment for telehealth. Ultimately, doctors want to be doctors. They are in the profession to do no harm and engage in face to face with patients to improve their lives. The products designed to increase connectivity and make 24-hour anywhere care available to everyone should not be designed without the input from this user group who bring forward concerns and needs.

Our next blog in this series will dive a little deeper into liability and telehealth and its impact on the UX design of these applications. Stay tuned!

So, your recruit fell apart… now what?

So, your recruit fell apart… now what?

U

BOLD INSIGHT

With the thousands of projects managed over the years, we feel like we’ve “seen it all”. While rare, in spite of the amount of preparation and planning, sometimes things don’t go as expected. Allison details tips on how to recover from a research participant recruit that falls into that category.

You have written the screener, kicked off your recruit weeks before the study, you keep in regular communication with the recruiter and things seem to be going well. You are at the Friday before your study and you only have one slot to fill. Your recruiter is confident and you are feeling good!

Imagine your dismay when testing hits and participants begin dropping like flies. The flu has taken over and those participants who are showing up are less than desired. Participants lied about their background and aren’t eligible to participate. They can’t find a parking space so decided to just not come. All these unanticipated events can take a seemingly ‘easy’ recruit and turn it into a bit of a challenge.

What do you do when you find yourself short on participants?

Over recruit and backups

Okay, so this is a preventative option. Over recruit when possible to avoid coming up short. If you need to have 15 completed sessions, schedule at least 18 participants. Recruiters local to the area often have an idea of how many participants you will need to recruit in order to fill your quota.

On a recent study in Atlanta, the recruiter recommended we schedule 19 participants in order to get 15 completed sessions. Due to the location of the facility, traffic, and parking we experienced a high no-show rate and certainly would have missed our quota had we not over-recruited. Luckily, we were prepared and ended the study on target. Trust your recruiter’s expertise!

When filling the recruit, you could also consider scheduling backup participants, typically referred to as ‘floaters’, that come to the testing facility but do not partake in the session unless there is a no-show, or another participant is ineligible. This may be more difficult for populations that require a higher compensation, such as surgeons, but paying for an few extra participants could save you a lot of time and money in the long run if you end up not filling your quota.

Keep in constant communication with the recruiter

This may seem obvious, but your first step when you notice participants not showing up or filling requirements should be to keep in regular contact with the recruitment company. As recruiters try to fill the empty slots, ask for consistent updates. Try to get to the root of the issue to prevent it from happening in future studies.

Take a closer look at requirements

Now that you are in a bit of a bind, it may be beneficial to loosen some of the screening requirements. Consider lowering your age range or modifying any requirements on demographics, if possible.

On a recent study this approach proved invaluable at filling a recruit that had otherwise stalled. I’d been recruiting users for a new home comfort app, but the study had very narrow demographic criteria and was targeted exclusively at iPhone users. I knew that if we were able to open the recruit up to Android users, we would easily be able to meet our quota. I decided to have a follow-up conversation with my client to better understand why we couldn’t recruit Android users. During our discussion he disclosed that the app would be released on Android shortly following the iPhone release and agreed to open the recruit to Android users which solved our problem.

Engage new resources

There is no shame in reaching out to more recruiters if you are finding the original company just isn’t getting the job done. Engage new resources as early as possible and let them know that you might need some help filling the slots. You can likely pay the recruiter per each session they fill.

If appropriate, ask participants if they know anyone that would be interesting in participating in the study. Earlier this year, I was working on a study that targeted specialty pharmacists and pharmacy technicians. During the week of fieldwork, participants started dropping due to conflicts. My client originally asked that we only recruit a certain number of participants from the same workplace, but once we found ourselves in a bind, they released that requirement. After a session, when escorting participants from the testing room, we asked them to reach out to colleagues to see if they would be interested in participating in the study. By the end of the week, we reached our quota thanks in part to their referrals. Just remember to remind participants to keep the contents of the study confidential if they signed an NDA.

Coworkers may also be able to cast a net to friends or acquaintances to see if anyone qualifies to participate in the study. Reach out to them to see if they can lend a hand!

Extend testing hours

Testing often occurs during regular working hours of 9-5. If participants are being recruited on short notice, these hours may not work for them. Consider adding in evening sessions to create more opportunities for participation.

Increase compensation

A little more money can make a world of difference. Give participants incentive to take part in the study!

 

Remember, occasionally losing a few participants is an unavoidable part of doing user research: what matters most is how we respond to such situations. Recognition of what is happening along with rapid corrective action can pull a study back from the brink and help you fulfill your research goals!

7 insights from the 2019 HFES Health Care Symposium

7 insights from the 2019 HFES Health Care Symposium

By Korey Johnson and Jackie Ulaszek

The Human Factors and Ergonomics Society (HFES) hosted its annual healthcare symposium last week in Chicago. Our team was out in force and had a great time connecting and reconnecting with friends and colleagues around the industry. If you missed the conference, in addition some great content, you also missed your chance to test your claw machine skills with us.

Aside from having fun, we also had the opportunity to hear about some great work that our human factors colleagues are doing around the industry, and present some of our own work. We attended many sessions and heard good discussion around FDA and regulatory perspectives, use of technology in research, interesting research methods employed, and cross-industry collaboration, among others.

Here are a few of the most interesting and useful insights our team brought back from the conference, most from the medical and drug delivery device track, but some general insights as well. Thanks to those responsible for the contributions summarized below, as well as those who made contributions elsewhere!

Statistics on HFE submission success rate

(From Pre-Symposium WK1: Improving the Safety & Effectiveness of Medical Devices Through Human Factors Considerations; MDD8 – CDRH/CDER FDA PANEL)

U

BOLD INSIGHT

Note the common HFE deficiencies in submissions to help plan realistic timelines for human factors related efforts during development.

During the Center for Devices and Radiological Health (CDRH) workshop, the Human Factors Pre-Market Evaluation Team (HFPMET) provided the usual overview of the FDA organization and regulatory review pathways. They also provided some hypothetical case studies to point out common Human Factors Engineering (HFE)-related things that go wrong in submissions. According to the HFPMET team, some of the more common deficiencies in submissions from an HFE perspective that require at least one request for additional information (and therefore delay the development process) included:

  • Incomplete / missing data for user groups or use environments (i.e., insufficient exploratory research or documentation thereof)
  • Incomplete / missing analysis of known use issues (i.e., insufficient post-market surveillance or documentation thereof)
  • Critical tasks not clearly identified, or not all critical tasks assessed (i.e., insufficient use-related task/risk analysis or documentation thereof)
  • Insufficient linkage between use-related risk analysis (URRA), critical task definition, and task success criteria
  • Reporting only performance rates and/or using preference ratings as a component of acceptance criteria

This year, the CDRH team went a step further and provided some statistics around HFE submissions. Of 184 510(k) HF/UE reviews in 2018, only 21 made it through without requests for additional information from the HFPMET reviewer. Of 157 Q-Subs, only 6 resulted in agreement on a human factors validation protocol. Not surprisingly, these 6 are all included in the 21 submissions that did not require additional information. These types of statistics can be useful to highlight the importance of planning for realistic timelines around human factors-related efforts during development, and we are looking forward to seeing more of this type of reporting from the FDA in the future.

Interesting regulatory and legal perspectives

(From MDD3 – PRE-CERTIFYING EXCELLENCE IN DIGITAL HEALTH: HUMAN FACTORS AND USER EXPERIENCE PERSPECTIVES; MDD7 – HUMAN FACTORS AND MEDICAL DEVICE INSTRUCTIONS FOR USE: IT’S NOT JUST GOOD BUSINESS IT’S THE LAW; MDD9 – REGULATORY IMPERATIVES; Posters, and others.)

U

BOLD INSIGHT

Applicants in the FDA’s Pre-Cert Program for SaMD will be assessed on the extent to which they have robust human factors processes in place, and the FDA is seeking input from industry to help define how a manufacturer can demonstrate this!

U

BOLD INSIGHT

Trying to design to avoid litigation makes no sense – litigation will happen regardless. But acknowledging this eventuality can serve as another arrow in the quiver for human factors practitioners vying for the resources to document a robust human factors effort during product development.

U

BOLD INSIGHT

The general message for the largely US-based audience from the EU-based speaker regarding the new EU MDR was to keep calm and carry on.

There were a few panels and presentations that highlighted forward-looking regulatory / legal perspectives.

In one Bold Insight-led panel, human factors and regulatory experts from industry and the FDA described the Software As a Medical Device (SaMD) Pre-Certification Program, focusing largely on the Excellence Appraisals that will be required in order for a manufacturer of SaMD to become pre-certified. The panelists addressed the question – What does “good human factors” look like from an organizational perspective and what types of processes and metrics can manufacturers present during an Excellence Appraisal to demonstrate a culture of quality and excellence as it pertains to human factors? The panel was intended to be informational for those unfamiliar with the Pre-Cert Program, as well as a call to action for the entire HF community to contribute their examples of organizational processes and metrics directly to the FDA docket for the program. (This is a topic we have been following closely; read one of our past posts, FDA’s digital health precertification program emphasizes importance of post-market surveillance, and find more information available about the program on the FDA’s website.)

In another Bold Insight-led panel, instructional material was discussed from the perspective of human factors experts contributing to the design of the materials, and the perspective of litigators and expert testimony from the HF domain. The intent of the panel was to provide the predominantly development-focused attendees with the novel perspective of how documented human factors efforts during development can help protect a manufacturer from liability in future litigation. The attendees were presented with a number of fascinating (and in most cases, tragic) examples of cases where design for safe and effective use was called into question during litigation. The take home message was that trying to design to avoid litigation makes no sense – litigation will happen regardless. But acknowledging this eventuality can serve as another arrow in the quiver for human factors practitioners vying for the resources to document a robust human factors effort during product development.

Finally, an update was provided on the new EU Medical Device Regulation (MDR). The general message for the largely US-based audience from the EU-based speaker was – keep calm and carry on. The new MDR is largely bringing medical device regulation in the EU into more alignment with how the FDA regulates medical devices in the US, at least from a human factors perspective. Under the new regulation, “The new rules significantly tighten the controls to ensure that medical devices are safe and effective and at the same time foster innovation and improve the competitiveness of the medical device sector” (http://europa.eu/rapid/press-release_MEMO-17-848_en.htm). This initiative will be enforced starting in 2020, with all medical products needing to be recertified by 2024 at the latest.

Use of VR in research

(From CCIT5 – MIXED REALITY; MDD6 – CONTEXTUAL RESEARCH AND VIRTUAL REALITY; Posters, and others.)

U

BOLD INSIGHT

The industry is increasingly incorporating VR and/or AR into device prototyping and research deliverables with positive results.

While virtual reality (VR) has been leveraged in many industries over the past several years, including healthcare, it is gaining traction not just as a tool to be used in the field, but also one to be used during research and development. Many universities are utilizing “mixed reality”, a combination of augmented reality (AR) and VR, as an advanced training tool. For example, overlaying patient data, such as the progression of a gunshot wound over time, on top of a physical mannikin and viewing these injuries through a headset while performing the physical actions on the mannikin. In research and development, device prototyping software can be easily integrated into a VR program to conduct expert evaluations and comparative analyses, as well as understand spatial relationships and context of use without the costly expense of physical prototypes.

Lastly, for contextual research, a demonstration was provided in one session of how stakeholders can be provided with a more impactful deliverable than a standard research report. By leveraging VR, stakeholders can review insights generated by the research as they virtually walk through the research conducted, jumping to physical locations to review research findings tagged to physical objects in the environment.

Training decay

(From MDD5 – DEVICE TRAINING PROGRAMS; Many posters, and others.)

U

BOLD INSIGHT

As an industry, we are making progress towards a more consistent and practical approach to adequately simulating memory decay in operational context of use without unnecessarily burdening research timelines and budgets. But we still have a ways to go.

Of course, training decay was a popular topic in both the workshops and panels with the FDA, as well as in several other sessions and posters. Many perspectives were presented, including:

  • The cost associated with extended decay periods that may not be necessary.
  • The need to more effectively leverage seminal research such as the Ebbinghaus forgetting curve, as well as the extremely large body of memory-related research that has been conducted since the 1880’s in finding a reasonable solution for adequately simulating extended decay periods.
  • Primary research that is currently being conducted with support from the FDA to assess the extent to which performance decrements exist for variable decay periods in the context of training specifically for medical device use.

There was quite a bit more presented and discussed on this topic. We are making progress, and hopefully some of the ongoing initiatives will give us some recognized empirical basis for decay periods in medical device human factors research that adequately simulate operational context of use without unnecessarily burdening development timelines and budgets.

Cross-industry collaboration

U

BOLD INSIGHT

While there seems to be increasing acknowledgement of the value that a strong human factors presence within hospitals and clinical networks offers, there were few solutions to overcome the budgetary constraints that often stifle this effort.

There seemed to be more discussion at this symposium around efforts to increase human factors presence within hospitals and clinical networks. Though many hospitals and clinics have recognized the importance of including human factors in their processes and procedures, many barriers for complete integration still exists. Both HF professionals and clinicians agree that what the other does is important but neither fully understands the ins and outs of the other role, making it challenging to find common ground for measuring improvement.

Hospitals with an HF champion have had the most successful integration; however, the role and impact of change may be limited depending on the champion’s leadership role in the organization. Even for hospitals that have the stakeholder buy-in and access to HF professionals, many times the HF expert or team is stretched thin across engagements and cannot attend to on-site demands.

Lastly, after all the above is considered, HF personnel working in hospitals most often do not have the budget necessary to seek external support when demands on the HF function are high. Most of the discussions we participated in and overheard throughout the symposium were focused on the recognition that this is an area for improvement, as opposed to discussions around solutions. However, there were a few promising cases where it seems that the right relationships have been formed to really get buy-in for human factors from a hospital administration perspective – hopefully this becomes more of a trend.

On the cross-industry collaboration front, hfMEDIC continued to gain momentum and interest as it organizes under the NSF’s Industry-University Cooperative Research Centers (IUCRC) Program. There are several pre-competitive research opportunities that routinely emerge at this conference and others, for which a vehicle such as hfMEDIC could be extremely valuable. We hope to support the development of this program and work with the existing members to drive research forward that will be of value to the industry as a whole.

What did you take away from the symposium?

These are just a few of the topics that stood out to our team, and there were many others! These also sparked discussion amongst our team:

  • Attention was drawn to the inadequacy of currently available ergonomic and biomechanical normative data for use in the design of medical devices (e.g., for specific patient populations).
  • Deception was used in a study presented to enhance the fidelity of simulated use, which sparked some interesting conversation within our team.
  • Some very interesting case studies were presented where stress was simulated with a high degree of fidelity.

And more! Let us know what your key takeaways were!

Product design: Why it’s important to see behavior

Product design: Why it’s important to see behavior

As designers, we assume users will behave the way we expect them to behave. Unfortunately for us, humans are unpredictable … attempts to model behavior typically predict less than 10% of actions. Consider the Theory of Planned Behavior, which suggests behaviors depend on an individual’s attitude, subjective norm (how do others feel about the behavior), and perceived behavioral control (do I believe I can successfully perform the behavior). All of these factors may differ from person to person, be influenced by their context and the objects with which they are working, and typically can only account for a small portion of the variability.

Therefore, we need to see behavior.  Putting a product in front of a human and giving them a chance to use it – to behave – yields insights that cannot be predicted or hypothesized without testing. Lesson: User testing and iteration matters.

Is a product still usable if the user has to “get used to it”?

Is a product still usable if the user has to “get used to it”?

During usability, when participants have trouble using something we often hear ‘I could do better but I’d just have to get used to it.’ This is a common phrase; it typically means there are usability issues. But maybe something more subtle is at play.

Sometimes the designs we test are not expected to be ‘walk up and use’ – in which case, ‘getting used to it’ might be a real thing. The design might be good, but it requires training. We have to know if the user is expected to have prior knowledge, require training, or need a ‘breaking in period’ (getting used to it!); if we don’t give the ‘breaking in’, the designs we test are not getting a fair evaluation. Experts perform differently from novices. Sometimes (often) user interfaces that novices find easy to use are frustrating to experts.

Research shows big payoff when design includes voice of the user

Research shows big payoff when design includes voice of the user

U

BOLD INSIGHT

A recent McKinsey report outlines the ROI of UX research, offering a business case to invest in building designs centered around the user.

Arriving at a fantastic design rarely happens by magic or luck. More typically, it occurs through a great deal of work … including revisions, iterations, and seeking out the voice of the user. We know this. It makes sense. Yet, stakeholders still cut design cycles short because they can’t be sure that one more iteration, or that running a user study, will have a positive return on investment. Being in the field of user experience research, we know the positive impact that user experience (UX) research has on product design, but it can be challenging to convince a board room to invest. But now we have evidence! McKinsey recently released a report showing the business benefits of design.

The report found a robust correlation between companies that are strong in design and the financial performance of each company. These results held true across the industries they studied, covering physical products, digital products, and services. As you read into the report, UX research was at the heart of what put companies in the top quartile.

The report goes on…

“The importance of user-centricity, demands a broad-based view of where design can make a difference … In practice, this often means mapping a customer journey (pain points and potential sources of delight) rather than starting with “copy and paste” technical specs from the last product. This design approach requires solid customer insights gathered firsthand by observing and—more importantly—understanding the underlying needs of potential users in their own environments. … Yet only around 50 percent of the companies we surveyed conducted user research before generating their first design ideas or specifications…

And on…

… many companies have been slow to catch up. Over 40 percent of the companies surveyed still aren’t talking to their end users during development. … With no clear way to link design to business health, senior leaders are often reluctant to divert scarce resources to design functions. That is problematic because many of the key drivers of the strong and consistent design environment identified in our research call for company-level decisions and investments.

In our experience, designers are amazed by the many ways consumers interpret their designs. Knowledge of consumer interpretations empowers designers to improve their designs in subsequent iterations. Thus, design benefits from the voice and interpretation of the people who will adopt it.

Plain and simple, companies that invest in user experience research perform better financially than those that don’t. This data has been anecdotal in the past, but McKinsey’s report should serve as the foundation for any UX manager trying to build a business case for UX.

Pin It on Pinterest