Bold Insight’s Korey Johnson co-presents human factors training to members of FDA’s DMEPA

Bold Insight’s Korey Johnson co-presents human factors training to members of FDA’s DMEPA

Representatives from the Human Factors and Ergonomics Society (HFES) provided a two-day training course on human factors science, user-centered design principles, and root cause analysis on September 25-26, 2018 at the Food and Drug Administration’s (FDA) White Oak Campus in Silver Spring, Maryland. This course was attended by representatives of the Division of Medication Error Prevention and Analysis (DMEPA) from the Center for Drug Evaluation and Research (CDER). Korey Johnson, Managing Director of Bold Insight, along with Dr. Anthony Andre, and Michael Wiklund were the three experts selected by HFES to provide this training course.

The training was intended to provide attendees with deeper knowledge of and appreciation for the scientific roots of human factors research and user-centered design. Training drew on seminal research and literature from the fields of psychology, engineering, and product design, and leveraged a wide range of practical examples of both good and bad application of human factors science.

“There are many reviewers within CDER tasked with assessing the extent to which submissions adequately support a decision indicating that a product can be used safely and effectively. Many of those reviewers have abundant experience assessing human factors and user-centered design related shortcomings. With this training we hoped to provide a bit more of the scientific underpinnings of our practice – on one hand to amplify instances where requests for additional research are warranted, and on the other hand to provide the scientific context for when that additional research may not be warranted,” said Johnson.

For more information about the Human Factors and Ergonomics Society (HFES), visit

Connect with Korey Johnson on Linked In, Twitter, or email.

Bold Insight is a user experience (UX) research agency based in Chicago. Our team offers clients the expertise and professionalism of a large agency, with the imagination and agility of a startup; specializing in medical device research, human factors validation, and large-scale global testing.

Bold Insight to exhibit at Human Factors Excellence for Medical Device Design conference

Bold Insight to exhibit at Human Factors Excellence for Medical Device Design conference

Bold Insight will exhibit at the 4th Annual Human Factors Excellence for Medical Device Design on August 1-2, 2018 in Minneapolis, Minnesota. Bringing together human factors engineering and user experience (UX) professionals, the conference will highlight emerging trends in the field of human factors and usability as it relates to compliance, testing strategies, training, new technologies, and submission requirements. Bold Insight Managing Directors Korey Johnson and Gavin Lew will be on hand to showcase the latest approaches to medical device research.

“Every time I have attended this conference, I have enjoyed discussing extremely relevant topics with my clients and human factors colleagues,” said Johnson. “It’s important for the medical device industry and human factors and UX leaders to have a place to connect and collaborate to advance the safety, efficacy, and efficiency of medical devices. Lately, as digital therapeutics and connected devices become more ubiquitous, we take opportunities like this conference to discuss ecosystem-level considerations and best practices to ensure engaging and satisfying overall experiences as much as safe and effective use of a single component alone.”

Be sure to visit Bold Insight’s booth throughout the conference for special giveaways and lively discussion about the latest research technology, the FDA’s human factors engineering guidance, and the future of digital therapeutics.


About Bold Insight

Bold Insight is a UX and human factors research agency. From user research informing early product design to global human factors validation, we specialize in applying human factors engineering to all phases of product development. We have the expertise of a large agency, with the imagination and agility of a startup. Learn more

Enough about us, let's talk about you!

We’d love to have an opportunity to bid on your project.
Responsiveness is important to us so we promise to get back to you immediately!


FDA’s digital health precertification program emphasizes importance of post-market surveillance

FDA’s digital health precertification program emphasizes importance of post-market surveillance



Finding efficient ways to leverage post-market surveillance data to inform product development at an institutional level will be key for SaMD manufacturers seeking FDA precertification.

Post-market surveillance is already an important component of human factors analyses that inform the development of any safe and effective medical device. For manufacturers of Software as a Medical Device (SaMD) that elect to participate in the FDA’s Digital Health Software Precertification Program, post-market surveillance data will become even more important. SaMD in many cases presents opportunities for a manufacturer to monitor Real World Perfomance Data (RWPD) directly via data recorded by the SaMD itself, but analysis of the events catalogued in publicly available adverse event databases remains a key source of post-market surveillance. The FDA’s MAUDE database can be complicated and time-consuming to mine. To make the most of this data, manufacturers would benefit from having a more efficient means by which to extract meaningful insights.


By implementing the pre-certification program, the CDRH at FDA hopes to establish a regulatory model that will “assess the safety and effectiveness of software technologies without inhibiting patient access to these technologies.” Under the FDA’s program, manufacturers of SaMD who successfully obtain precertification can benefit from streamlined premarket review, and, in cases of some lower-risk software or certain types of modifications, can bypass FDA review all together.

In order to allow for streamlined (or sometimes bypassed) premarket review, but still achieve its mandate to ensure the safety and efficacy of medical devices, the FDA must verify through the pre-certification program that a manufacturer has established a “robust culture of quality and organizational excellence” and is “committed to monitoring real-world performance of their products once they reach the U.S. market.” In their working model of the pre-cert program, the FDA outlines the key components to the program (Figure below taken from Developing a Software Precertification Program: A Working Model).


From a human factors perspective (and likely from other perspectives), the details of the excellence appraisal and certification are critical to the value and the success of the program itself. The FDA is working with the pilot participants in the program to define those details, which will be centered around five key principles:

  1. Product Quality
  2. Patient Safety
  3. Clinical Responsibility
  4. Cybersecurity Responsibility
  5. Proactive Culture

One can hope that sound human factors practices (e.g., documented institutionalization of getting feedback from end users early and often, and grounding design decisions in this feedback) are threaded throughout the assessment of an organization against these principles.

Post-market surveillance data is key

Real World Performance Data (RWPD, i.e., post-market surveillance data), and the extent to which regular monitoring of this data to is relied upon to inform continuous improvement, is an important component of the pre-certification program. With a streamlined (or bypassed) regulatory review, implementing a robust process by which RWPD is constantly monitored, analyzed, and acted upon will be critical to manufacturers’ demonstration to the FDA that their organization is committed to producing SaMD that is both safe and effective.

RWPD includes three key components in and of itself:

  1. Real World Health Data (RWHD)
  2. User Experience Data (UXD)
  3. Product Performance Data (PPD)

In many cases, software by its very nature presents opportunities for monitoring RWPD that require minimal effort to at least obtain the data on the part of the manufacturer. For example, error reports that are automatically sent to a manufacturer from a software program can be monitored for reliability analysis. In other cases, where data are (at least for now) generated externally, such as is the case with adverse event and product complaint reporting, more effort is required to obtain RWPD.

The data are made publicly available in one way or another – whether formally via databases such as the FDA MAUDE or FAERS databases, or informally via other channels such as social media. In both cases, independent of the effort required to obtain the data, a manufacturer of SaMD seeking precertification through this program must have a robust process in place to collect, analyze, and act on this data; any medical device manufacturer, for that matter, should also have this process in place. As is increasingly the case in the information age, the problem becomes not how to obtain data, but how to manage the incredible amount of data available.

Making post-market surveillance data easier to consume

While there are multiple sources for post-market surveillance data, the MAUDE database remains the more relevant “official” database of publicly available AE/PC data relevant for SaMD manufacturers. Unfortunately, the MAUDE database is not very accessible and requires a database engineer to plumb the depths of the data before any meaningful analysis can be conducted.

That said, with the right skills brought to bear, there is a wealth of information available in MAUDE for SaMD manufacturers to leverage in their efforts to ground their designs in RWPD. This seems to be an opportunity for a subscription service that regularly summarizes trends in AE/PC data from the MAUDE database and other sources. The MAUDE database in particular is not without its limitations, several of which are stated very clearly on the FDA website itself, but as long as it is the official database of AE/PC data for medical devices it would seem beneficial to have easy access to emerging and historical trends in this data. This way, medical device manufacturers in general could be better able to institutionalize the incorporation of robust post-market surveillance data analysis into their design process, and SaMD manufacturers in particular could better prepare themselves for precertification and streamlined regulatory approval process in doing so.


Related post: The time for a better UX in digital therapeutics is now

Pin It on Pinterest