Be the student: 3 tips to be a better UX research moderator

Be the student: 3 tips to be a better UX research moderator



Moderators should put themselves in the position of learning from the participant. Keeping a few tips in mind can facilitate this process and produce a more fruitful UX research session.

I recently attended a conference on human factors and medical devices. In one session, on the topic of moderating usability test sessions, the speaker used the phrase “Be the student” to describe the attitude the moderator should have when conducting this type of research. That is, while the participant is interacting with the device / app / website for likely the very first time and attempting to learn it, we, as moderators, should also be of the mindset to learn (not teach, or assume) about the participant-stimuli interaction for the first time.

This led to me to think about some of the other tips that we give to our UX researchers when we conduct moderation training:

  1. Flip questions back to the participant – I think I’ve used this tactic in just about every session I’ve ever moderated. The participant will invariably say something to the effect of, “I’m not sure how to do this; am I doing this right?” or “This feature should be changed, don’t you think?” And in every one of those cases, the question should be flipped right back to the participant with the moderator replying, “What do YOU think?”
  2. Use the participant’s terminology – when conducting a study on, for example, a syringe, I’ve seen participants refer to the flanges as the “finger grippers” or even “grippies.” Rather than correct the participant by informing them that they are using the wrong term, the moderator should adopt that term for the rest of the session when discussing the syringe and the process of the injection. This can apply to parts of a website or device or app where we, as the researchers, know the proper engineering or designer jargon, however, when talking with participants, we should be using the words they use.

  3. Leave questions open-ended – this one is a two-for-one and has two pieces of advice wrapped into it:
    • First, avoid yes/no questions. This is a basic tip for new moderators but it’s still very important. Simple, yes/no questions elicit simple yes/no answers. As qualitative researchers, we’re interested in the WHY and the HOW, so questions need to be crafted to go beyond the yes and no and get to the why and how.
    • Second, avoid baking the correct / preferred answer into the question. Instead of saying, “I noticed you did x instead of y” (which clearly states what you wanted the participant to do), say something along the lines of “I noticed you did x, tell me what you expected to happen when you did that.” This leaves the participant free to explain what they did rather than justify why they didn’t do what you were hoping for them to do.

All of this is to establish that the participant is the expert; not the moderator. Hence “Be the student.” The moderators are there to learn from the participants. And the participants are there to show us how the device should be used or at the very least, how they expect it to be used.

Bold Insight’s Korey Johnson featured in AAMI’s latest BI&T journal focusing on early use-related research for medical devices

Bold Insight’s Korey Johnson featured in AAMI’s latest BI&T journal focusing on early use-related research for medical devices

A roundtable of experts was featured in the July/August 2019 issue of the Association for the Advancement of Medical Instrumentation (AAMI) bimonthly journal, Biomedical Instrumentation & Technology (BI&T). The discussion, Early Research into the Operational Context of Device Use Is Needed, highlighted trends, value, and various roles of early use–related research of medical devices. Bold Insight’s Managing Director Korey Johnson, a frequent presenter and active member in the human factors research community, was asked to lend his expertise to the dialog.  

 “It was a great experience to connect and discuss with my peers a topic we are all passionate about,” said Johnson. “Most of us are involved in developing standards and defining best practices for this type of research, so we appreciate any opportunity to answer questions and share our perspectives with members of the healthcare technology management community who are tasked with executing this research. Our goal, as human factors practitioners in healthcare, is to do more than ensure safe and effective use at the conclusion of the product development process. We aim to make humans and the human experience central to the entire development process. AAMI is a respected resource for furthering industry standards and best practices. Convening this roundtable discussion on early use-related research is a great example of the organization helping to push the industry in the right direction.” 

 AAMI members can access the full article by visiting  

 Connect with Korey Johnson on Linked In, Twitter, or email (korey.johnson [at] 

 About Bold Insight 

Bold Insight is user experience (UX) and human factors (HF) research agency based in Chicago. Our team offers clients the expertise and professionalism of a large agency, with the imagination and agility of a startup; specializing in medical device research, human factors validation, and large-scale global testing. 

 About AAMI and BI&T publication 

The Association for the Advancement of Medical Instrumentation (AAMI) publishes BI&T (Biomedical Instrumentation & Technology) a bimonthly peer-reviewed journal dedicated to the developers, managers, and users of medical instrumentation and technology. Twice each year, AAMI also publishes the award-winning Horizons series. Each issue focus on a single topic of  interest to the medical technology community.  

Founded in 1967, AAMI is a unique alliance of nearly 7,000 members united by the common goal of increasing the understanding and beneficial use of medical instrumentation. Members include clinical engineers, biomedical equipment technicians, manufacturers, sterile processing professionals, researchers, quality assurance and regulatory affairs experts, and other healthcare technology professionals.  Visit to learn more.

Research shows big payoff when design includes voice of the user

Research shows big payoff when design includes voice of the user



A recent McKinsey report outlines the ROI of UX research, offering a business case to invest in building designs centered around the user.

Arriving at a fantastic design rarely happens by magic or luck. More typically, it occurs through a great deal of work … including revisions, iterations, and seeking out the voice of the user. We know this. It makes sense. Yet, stakeholders still cut design cycles short because they can’t be sure that one more iteration, or that running a user study, will have a positive return on investment. Being in the field of user experience research, we know the positive impact that user experience (UX) research has on product design, but it can be challenging to convince a board room to invest. But now we have evidence! McKinsey recently released a report showing the business benefits of design.

The report found a robust correlation between companies that are strong in design and the financial performance of each company. These results held true across the industries they studied, covering physical products, digital products, and services. As you read into the report, UX research was at the heart of what put companies in the top quartile.

The report goes on…

“The importance of user-centricity, demands a broad-based view of where design can make a difference … In practice, this often means mapping a customer journey (pain points and potential sources of delight) rather than starting with “copy and paste” technical specs from the last product. This design approach requires solid customer insights gathered firsthand by observing and—more importantly—understanding the underlying needs of potential users in their own environments. … Yet only around 50 percent of the companies we surveyed conducted user research before generating their first design ideas or specifications…

And on…

… many companies have been slow to catch up. Over 40 percent of the companies surveyed still aren’t talking to their end users during development. … With no clear way to link design to business health, senior leaders are often reluctant to divert scarce resources to design functions. That is problematic because many of the key drivers of the strong and consistent design environment identified in our research call for company-level decisions and investments.

In our experience, designers are amazed by the many ways consumers interpret their designs. Knowledge of consumer interpretations empowers designers to improve their designs in subsequent iterations. Thus, design benefits from the voice and interpretation of the people who will adopt it.

Plain and simple, companies that invest in user experience research perform better financially than those that don’t. This data has been anecdotal in the past, but McKinsey’s report should serve as the foundation for any UX manager trying to build a business case for UX.

UX and sustainability: What is our role as researchers?

UX and sustainability: What is our role as researchers?



As UX researchers and designers, we should consider the entire product lifecycle, including disposal. If possible, also thinking ahead to future product iterations and how product replacement will be handled.

Earlier this year, I was a part of a formative study for a pharmaceutical manufacturer to investigate patients’ opinions about a new liquid medication delivery device. It was a daily dose that came in one-time-use bottles made of plastic, meaning patients would have to dispose of approximately 30 bottles per month. One of the unexpected insights we gathered from this study was that participants were highly concerned about the waste involved and almost unanimously voiced the desire for the bottle to be reusable, recyclable, or smaller at the very least.

This is an interesting first-hand example of sustainability becoming a factor in user experience (UX). Users often want the latest technology, not only because the capabilities of new devices are helpful in critical ways, but also because our society sees newness as fashionable. And design is often structured around that concept. For example, new software often can’t be run on old devices, requiring the purchase of new hardware – and resulting in a continuous cycle of disposal. This frequent disposal of goods has a significant impact on the environment, and technology may have further environmental effects in the form of energy-demanding websites and electronic waste. So how do we as UX professionals consider the needs and desires of users while also factoring in the broader human value of environmental sustainability? Should that be part of our responsibility?

Design with sustainability in mind from day one

Some experts, including the reputable Don Norman, suggest that designers should consider the full life-cycle of a product as they build it, with the end goal of creating well-cared-for systems. This kind of design not only has a positive impact on the environment, but it can often ultimately improve the user experience (as noted in the example above) and therefore contribute to commercial success.

There are some actionable ways that sustainability can be included in design. For one, product innovators and designers can think ahead to the disposal of the product. How will users get rid of the product when they no longer need or want it? Furthermore, does the invention of this product displace an older model, and what impact will the disposal of the older version have on the consumer? On the environment? Considering these concepts early in the innovation cycle means avoiding greater problems later down the line.

Adding options to reduce waste

On the other end, designers can also look at the ramifications of behavior induced by a certain product or system. For example, food delivery apps might promote waste because of the large amount of plastic and packaging that are often inherently involved in the process. Building in options that provide the user opportunities to reduce waste or save energy is a simple way that designers can promote sustainability and allow users to make more informed decisions about their consumption, which at least for some users, elevates their overall experience with a product.

As UX researchers, we are responsible for investigating what is important to users in order to improve their overall experience with products and services. But humans are complex, and their desires are not always straightforward. Sometimes, the insights we uncover remind us that technology does not exist in a bubble, it exists in societies with values and norms (like concern for the environment). What are some other ways UX professionals might be able to address the intersection between sustainability and design?

Four things to keep in mind when conducting on-the-road UX research

Four things to keep in mind when conducting on-the-road UX research



To successfully execute on-the-road driver experience research, it’s important to account for factors unique to the driving environment, such as the route or geographic location.

There are many facets of the technological renaissance currently underway in the automotive industry. Development in areas such as infotainment, connectivity, and autonomous driving are fundamentally altering the in-car experience; the extent to which these developments are implemented successfully hinges in part on informing their design with a thorough assessment of impacts to that experience. While it is possible to test vehicle systems in a parked car or on a closed track, neither will fully simulate the experience of driving on the road. Conducting UX research on the open road allows stakeholders to test product or system performance in dynamic conditions that match those faced by drivers every day.

There are several factors UX researchers must consider before conducting an on-the-road study, and by thinking ahead, researchers can ensure they will capture the richest data possible. As is the case with any simulated-use research, many of these factors bubble up to understanding the importance of context of use. That being said, there are some unique considerations for the driving environment so here are four things to keep in mind when conducting on-the-road UX research:

1. Know your route

Just as we (UX researchers) know the products and systems we are researching inside and out, when conducting on-the-road UX research we should also know the test routes and roads equally well. To do this, drive the route multiple times prior to your research. Become aware of how the route drives at different times of day to account for variables such as street parking. Make note of detours and alternative routes in case drivers are unexpectedly diverted from the original plan during your research. If the product or system of interest is meant to be used on the highway, plan a route that goes on the highway. If it’s meant to be used on rough roads, seek out a road with potholes, speedbumps, or rough patches. This will result in a more realistic simulation and ultimately in richer data. Aside from accounting for these factors when planning routes, becoming comfortable with the routes beforehand ensures the research team can focus more on observing behavior and interaction and less on getting research participants from point A to point B.

2. Time of day can impact driver experience

The time of day during which on-the-road UX research is conducted can considerably impact the context of use for in-vehicle systems. When conducting a highway road test in metropolitan areas, research conducted during evening rush hour versus earlier in the day may result in fundamentally different experiences. One driver may experience bumper to bumper traffic, while the other may be able to speed down a relatively open highway. The amount of cognitive bandwidth available to devote to interaction with onboard systems can vary greatly depending on external environmental factors such as traffic, and it is important to account for this variation and how it could impact the driver experience.

3. Location, location, location

Depending on how the system is meant to be used, some regions will be better than others in which to conduct the on-the-road research. If the system is meant to be used over rolling hills, Southern California could be an ideal test site. If the system is meant to be used in snow, testing in Chicago in December could be ideal. The vehicle category should also play a role in selecting an appropriate test site. A system only available in luxury vehicles should be tested in more affluent locations where sales of luxury vehicles are higher, thereby making recruitment of research participants easier. This is where you utilize the expertise of your recruiters to ensure it’s possible to locate the desired population.

4. Focus participants on the stimuli

While this is a standard expectation for any research study, during on-the-road studies, keeping participants focused on the vehicle systems being tested can be particularly challenging. Participants’ attention can be pulled in several directions, from systems inside the vehicle to road distractions. If participants are meant to provide feedback on the experience of the infotainment system, it doesn’t help if they discuss the seat warmers or the slick roads. Conducting in-vehicle UX research on the open road can yield great insights to operational context of use, but it does come at a cost. It can be difficult for research participants to separate out the individual aspects of the in-car experience when providing qualitative feedback, so it is essential that researchers specify and clarify the elements for which feedback is relevant. Something as simple as mentioning the system being tested in follow-up questions can help keep participants focused on appropriate parts of the vehicle during their session.

In summary, context of use is key and the above are just a few examples of elements that are important to account for in a driving environment. Whether the artifact of interest is the infotainment system, or a mechanical system intended to improve the quality of the ride, it is essential to understand the context in which the system will be used so that context can be replicated in your research. Will the system be used while parked? On the highway? When going off-road? When research objectives include obtaining any sort of understanding around how a system will be used in operational context, researchers must attempt to replicate the road, traffic, and route conditions in which the system will be used.

There will always be factors we cannot control, like weather or other drivers. However, planning ahead and thinking through the points outlined above will enable researchers to get the most out of on-the-road UX research and obtain the valuable insights needed to develop best-in-class in-vehicle experiences.

Pin It on Pinterest