Four things to keep in mind when conducting on-the-road UX research

Four things to keep in mind when conducting on-the-road UX research



To successfully execute on-the-road driver experience research, it’s important to account for factors unique to the driving environment, such as the route or geographic location.

There are many facets of the technological renaissance currently underway in the automotive industry. Development in areas such as infotainment, connectivity, and autonomous driving are fundamentally altering the in-car experience; the extent to which these developments are implemented successfully hinges in part on informing their design with a thorough assessment of impacts to that experience. While it is possible to test vehicle systems in a parked car or on a closed track, neither will fully simulate the experience of driving on the road. Conducting UX research on the open road allows stakeholders to test product or system performance in dynamic conditions that match those faced by drivers every day.

There are several factors UX researchers must consider before conducting an on-the-road study, and by thinking ahead, researchers can ensure they will capture the richest data possible. As is the case with any simulated-use research, many of these factors bubble up to understanding the importance of context of use. That being said, there are some unique considerations for the driving environment so here are four things to keep in mind when conducting on-the-road UX research:

1. Know your route

Just as we (UX researchers) know the products and systems we are researching inside and out, when conducting on-the-road UX research we should also know the test routes and roads equally well. To do this, drive the route multiple times prior to your research. Become aware of how the route drives at different times of day to account for variables such as street parking. Make note of detours and alternative routes in case drivers are unexpectedly diverted from the original plan during your research. If the product or system of interest is meant to be used on the highway, plan a route that goes on the highway. If it’s meant to be used on rough roads, seek out a road with potholes, speedbumps, or rough patches. This will result in a more realistic simulation and ultimately in richer data. Aside from accounting for these factors when planning routes, becoming comfortable with the routes beforehand ensures the research team can focus more on observing behavior and interaction and less on getting research participants from point A to point B.

2. Time of day can impact driver experience

The time of day during which on-the-road UX research is conducted can considerably impact the context of use for in-vehicle systems. When conducting a highway road test in metropolitan areas, research conducted during evening rush hour versus earlier in the day may result in fundamentally different experiences. One driver may experience bumper to bumper traffic, while the other may be able to speed down a relatively open highway. The amount of cognitive bandwidth available to devote to interaction with onboard systems can vary greatly depending on external environmental factors such as traffic, and it is important to account for this variation and how it could impact the driver experience.

3. Location, location, location

Depending on how the system is meant to be used, some regions will be better than others in which to conduct the on-the-road research. If the system is meant to be used over rolling hills, Southern California could be an ideal test site. If the system is meant to be used in snow, testing in Chicago in December could be ideal. The vehicle category should also play a role in selecting an appropriate test site. A system only available in luxury vehicles should be tested in more affluent locations where sales of luxury vehicles are higher, thereby making recruitment of research participants easier. This is where you utilize the expertise of your recruiters to ensure it’s possible to locate the desired population.

4. Focus participants on the stimuli

While this is a standard expectation for any research study, during on-the-road studies, keeping participants focused on the vehicle systems being tested can be particularly challenging. Participants’ attention can be pulled in several directions, from systems inside the vehicle to road distractions. If participants are meant to provide feedback on the experience of the infotainment system, it doesn’t help if they discuss the seat warmers or the slick roads. Conducting in-vehicle UX research on the open road can yield great insights to operational context of use, but it does come at a cost. It can be difficult for research participants to separate out the individual aspects of the in-car experience when providing qualitative feedback, so it is essential that researchers specify and clarify the elements for which feedback is relevant. Something as simple as mentioning the system being tested in follow-up questions can help keep participants focused on appropriate parts of the vehicle during their session.

In summary, context of use is key and the above are just a few examples of elements that are important to account for in a driving environment. Whether the artifact of interest is the infotainment system, or a mechanical system intended to improve the quality of the ride, it is essential to understand the context in which the system will be used so that context can be replicated in your research. Will the system be used while parked? On the highway? When going off-road? When research objectives include obtaining any sort of understanding around how a system will be used in operational context, researchers must attempt to replicate the road, traffic, and route conditions in which the system will be used.

There will always be factors we cannot control, like weather or other drivers. However, planning ahead and thinking through the points outlined above will enable researchers to get the most out of on-the-road UX research and obtain the valuable insights needed to develop best-in-class in-vehicle experiences.

In-vehicle UX research: Here’s one recommendation that hasn’t changed in 10 years

In-vehicle UX research: Here’s one recommendation that hasn’t changed in 10 years



I found myself discussing what can be done to increase the extent to which voice recognition systems are seen as a benefit rather than an annoyance with the research sponsors, and I said the same things as I said 10 years ago… improve the system to support and recognize more natural speech patterns.

After recently conducting research on drivers’ experience with technology in their vehicles, I reflected on what has changed about that experience in the last 10 years…. and what hasn’t changed. The availability of customized, on-demand, and context-appropriate information in in-vehicle displays seems to be one of the biggest differences that drivers now benefit from, compared to their experiences 10 years ago. However, in-vehicle voice recognition was a particularly interesting topic during the research. In some cases, these systems have progressed considerably in the past decade such that they are useful now. But in many cases, I honestly couldn’t tell the difference between current voice interfaces and those of the late 2000’s.

What’s impacting drivers now

There are a lot of cool things on the horizon for the auto industry with autonomous vehicles gaining momentum, but I’m talking specifically about features equipped in late model mass market vehicles that are having a pretty big impact on a large portion of the population’s driving experience right now. The integration of customizable and context-sensitive safety, diagnostic and convenience information into instrument clusters, and even the heads-up display (HUD) has changed the typical driver experience.

In an affordable $20,000-$30,000 vehicle, drivers can get a technology and communications package that allows them to have a wealth of information at their fingertips. Diagnostic and safety sensors providing alerts whenever vehicle status is sub-optimal, lane departure warning alerts and driver attention (or rather, lack thereof) alerts, numerous cameras and sensors on the exterior of the vehicle to assist maneuvering in tight quarters, or just park the car for you, haptic indicators to orient attention to directional alerts, and of course, the ability to control connected media devices without ever taking your hands off the wheel. These are just a few of the things that might not be exactly revolutionary anymore, but are currently changing how mainstream drivers experience their vehicles.

HUDs seem to be on the border – they are not new in automotive application but have not yet achieved the mainstream deployment that you see for other technologies. HUDs can be valuable even if they are only capable of showing a driver their speed and the current speed limit for the road. When turn-by-turn navigational cues, safety alerts, and customizability are added in, the HUD feature starts to be more than just a ‘nice to have’.

A “real” voice system

With respect to voice recognition – these systems still leave much to be desired. I was a bit surprised to recently find myself having similar conversations with research participants about their experience using voice recognition systems as I had with research participants 10 years ago. There are some that HAVE gotten significantly better and allow for fairly natural speech patterns to be used when interacting with the system. But mostly, the systems still require clunky one- or two-word commands to precede any sort of action other than making a phone call.

Interestingly, now drivers have Amazon Echo, Google Home, and even Siri to use as a benchmark. They are even less willing to accept a system that doesn’t allow for them to converse at least as naturally they do when they use those systems. Ask drivers to compare their in-vehicle voice recognition systems to their experience with voice assistants and prepare for them to go on a diatribe about how cool it would be to have a “real” voice system in their vehicle to make commanding that system while driving easier.

Eventually, it will all come together. Personally, I am waiting for the day when I can say “Alexa, take me home.”

Pin It on Pinterest