Four things to keep in mind when conducting on-the-road UX research

Four things to keep in mind when conducting on-the-road UX research



To successfully execute on-the-road driver experience research, it’s important to account for factors unique to the driving environment, such as the route or geographic location.

There are many facets of the technological renaissance currently underway in the automotive industry. Development in areas such as infotainment, connectivity, and autonomous driving are fundamentally altering the in-car experience; the extent to which these developments are implemented successfully hinges in part on informing their design with a thorough assessment of impacts to that experience. While it is possible to test vehicle systems in a parked car or on a closed track, neither will fully simulate the experience of driving on the road. Conducting UX research on the open road allows stakeholders to test product or system performance in dynamic conditions that match those faced by drivers every day.

There are several factors UX researchers must consider before conducting an on-the-road study, and by thinking ahead, researchers can ensure they will capture the richest data possible. As is the case with any simulated-use research, many of these factors bubble up to understanding the importance of context of use. That being said, there are some unique considerations for the driving environment so here are four things to keep in mind when conducting on-the-road UX research:

1. Know your route

Just as we (UX researchers) know the products and systems we are researching inside and out, when conducting on-the-road UX research we should also know the test routes and roads equally well. To do this, drive the route multiple times prior to your research. Become aware of how the route drives at different times of day to account for variables such as street parking. Make note of detours and alternative routes in case drivers are unexpectedly diverted from the original plan during your research. If the product or system of interest is meant to be used on the highway, plan a route that goes on the highway. If it’s meant to be used on rough roads, seek out a road with potholes, speedbumps, or rough patches. This will result in a more realistic simulation and ultimately in richer data. Aside from accounting for these factors when planning routes, becoming comfortable with the routes beforehand ensures the research team can focus more on observing behavior and interaction and less on getting research participants from point A to point B.

2. Time of day can impact driver experience

The time of day during which on-the-road UX research is conducted can considerably impact the context of use for in-vehicle systems. When conducting a highway road test in metropolitan areas, research conducted during evening rush hour versus earlier in the day may result in fundamentally different experiences. One driver may experience bumper to bumper traffic, while the other may be able to speed down a relatively open highway. The amount of cognitive bandwidth available to devote to interaction with onboard systems can vary greatly depending on external environmental factors such as traffic, and it is important to account for this variation and how it could impact the driver experience.

3. Location, location, location

Depending on how the system is meant to be used, some regions will be better than others in which to conduct the on-the-road research. If the system is meant to be used over rolling hills, Southern California could be an ideal test site. If the system is meant to be used in snow, testing in Chicago in December could be ideal. The vehicle category should also play a role in selecting an appropriate test site. A system only available in luxury vehicles should be tested in more affluent locations where sales of luxury vehicles are higher, thereby making recruitment of research participants easier. This is where you utilize the expertise of your recruiters to ensure it’s possible to locate the desired population.

4. Focus participants on the stimuli

While this is a standard expectation for any research study, during on-the-road studies, keeping participants focused on the vehicle systems being tested can be particularly challenging. Participants’ attention can be pulled in several directions, from systems inside the vehicle to road distractions. If participants are meant to provide feedback on the experience of the infotainment system, it doesn’t help if they discuss the seat warmers or the slick roads. Conducting in-vehicle UX research on the open road can yield great insights to operational context of use, but it does come at a cost. It can be difficult for research participants to separate out the individual aspects of the in-car experience when providing qualitative feedback, so it is essential that researchers specify and clarify the elements for which feedback is relevant. Something as simple as mentioning the system being tested in follow-up questions can help keep participants focused on appropriate parts of the vehicle during their session.

In summary, context of use is key and the above are just a few examples of elements that are important to account for in a driving environment. Whether the artifact of interest is the infotainment system, or a mechanical system intended to improve the quality of the ride, it is essential to understand the context in which the system will be used so that context can be replicated in your research. Will the system be used while parked? On the highway? When going off-road? When research objectives include obtaining any sort of understanding around how a system will be used in operational context, researchers must attempt to replicate the road, traffic, and route conditions in which the system will be used.

There will always be factors we cannot control, like weather or other drivers. However, planning ahead and thinking through the points outlined above will enable researchers to get the most out of on-the-road UX research and obtain the valuable insights needed to develop best-in-class in-vehicle experiences.

When designing an experience, are you really thinking about the customer first?

When designing an experience, are you really thinking about the customer first?



While every organization has constraints to consider when designing experiences for customers or users, designing the ideal customer experience is completely possible within those constraints. Begin by identifying the experience you want your customer to have and work “backward” from there.

I recently completed a training to become a certified UX-PM trainer. While I’ve worked in the UX field for the last 7 years, going through this training was a nice, compact reminder of how to integrate UX into the overall product design experience. It was a great opportunity to think big picture, to think about how to maximize the customer’s experience, and to think about the various touchpoints and individuals that contribute to this effort. As the trainer put the following image on the screen, something seemed to click.

While many companies have existing systems they have to work within, consider what would happen if that model were turned upside down. Instead of thinking about how to mold a service or product experience within the constraints of existing systems, what if the systems were designed to make for a seamless experience for the customer?

You may be thinking to yourself that you or your business do consider the customer/user, that you have friendly staff, and that you strive to make each interaction with your customers a positive one. You may even have done work to design and test interfaces so that your customers are able to work within and use your systems effectively. I challenge you to think big picture, to think about whether there are parts of your service that could be reimagined, starting with the experience you want your customer to have and working “backward” from there.


Consequences of not designing for customers

I had an experience that came to mind as soon as I saw those diagrams. It was a perfect illustration of the systems out approach. Allow me to set the scene:

I was scheduled for my annual car maintenance. A few weeks before my scheduled appointment, I started receiving alerts on my dash that 1) I was due for an oil change and 2) that my keyfob battery needed to be replaced. I had a road trip planned and the battery warning was especially disconcerting to me, as I didn’t want to be locked out of the car three states from home. As a result, I moved up the appointment a month to get it taken care of. I even brought the batteries with me to the appointment because I wasn’t sure if they would be provided.

Here’s where the fun started.

To spare you some of the detail, I have broken down my experience into the four main issues that I encountered below, alongside some takeaways.


1. Information provided to the customer was incomplete and unclear

In my first meaningful interaction with the mechanic/service coordinator (which took place after they had looked at the car), he told me that beyond the service check, I would need to replace an “impacted” tire and adjust the car’s automatic alignment, all for a hefty $500+ price tag. Despite my attempts to get additional information about the necessity of making these repairs immediately, I couldn’t get an answer beyond “all the repairs are recommended.”

It would have been much more helpful to tell me which repairs were required (as in, which could have a safety-impact), which might be superficial, and what consequences I would face if I didn’t address them. More information would have made me feel as though I could make a properly informed decision, especially in light of the high price tag.

I authorized the tire replacement and later received an invoice. Because of the way the charges were listed on the invoice, I initially thought there was an accidental overcharge, and ended up calling to get more information about the additional item on the invoice (it ended up being a tax).

Transparency up front and during the appointment and clear itemization of each invoice charge could have resolved my questions and left me not questioning the transaction.

My questions and concerns could have been preempted, or at the very least acknowledged, by:

  • Use of layman’s terms in place of technical language
  • Assistance in assessing criticality of issues
  • Acknowledgment of budgetary constraints
  • Clear presentation of charges and costs in the invoice

2. The main repair center customer service line had no access to information

When I drove my car the next time, I saw a warning in the dash telling me that my keyfob battery was low. This was particularly upsetting because this was the thing I had asked explicitly to be fixed.

I called the service center to find out what the problem was. I would have expected them to have access to a record of my recent appointment and a list of the repairs that were done but was told instead that someone would reach out to me later that day with an answer. I never got a return call.

The customer should not be expected to understand and juggle different parts of an organization that don’t communicate with each other. If it’s necessary for there to be multiple systems, they must appear as one to customers, so that from the customers’ perspective, there is one cohesive organization and experience.


3. There was no way for me to escalate my complaint

After waiting for the call that never came, I received a customer satisfaction survey. Me being a UXer, I was curious whether filling it out would escalate the issue, so I completed the survey. I was honest about my experience.

I subsequently received a follow-up text from the service coordinator I had been communicating with during my appointment (I received text updates during the appointment, which was nice and made a positive impression at the time). It struck me as odd to receive a text follow up to my satisfaction survey from the individual with whom I was dissatisfied. Also, I thought the text channel was available for appointment updates only. This could have been a nice feature of this service model if the purpose of the text channel was made clear up front (back to point #1), but equally important, this didn’t really feel like my complaint was being escalated at all. I had just complained about the interaction with this guy in a survey, now here he was texting me to resolve the issue. Where I expected a quick reply via text, it took me six days and four separate messages to get the mechanic to tell me that the batteries had been changed in the key fobs.

When responding to a complaint, consider who the customer would want to communicate with and give the sense that the complaint was addressed and was being listened to. It’s unlikely that a customer will feel appeased if they had a dissatisfactory interaction with an employee, complain, then get rerouted to that person.


 4. Scheduling and reminder calls were being handled by a separate call center, that was not being updated by the service center staff

Last but not least, while trying to get an answer to the battery question, I began receiving reminder calls for my original appointment that I had cancelled weeks before. I even went so far as to call their general number and remind them that the appointment had been cancelled. I then received a message after the appointment was supposed to have taken place that told me I had missed my appointment.

Beyond confirming for me that the service number was a black hole, there was some issue with their internal communications where one side clearly didn’t update another and I was the one somehow coming out feeling as though I had dropped the ball somewhere.

Back to point #2. The customer should not be able to tell that there are multiple entities, much less be able to see communication rifts between the different entities. Most importantly, the customer should not be made to feel responsible for errors that are occurring on the company’s side.


Improving the experience with the customer in mind

This blog is not intended to be a rant about my negative experience, but an acknowledgement that creating a positive customer experience often requires attending to the multiple touchpoints at which the customer engages with your product or service. Systems that seem adequate (the text communication channel could have been great) can make for a negative overall experience when expectations are not set appropriately.

There will always be constraints to consider when designing experiences for customers or users. There will be things that can’t be changed. Designing the ideal customer experience is completely possible within those constraints. It might just mean sitting down with, listening to, and empathizing with your customers, to understand what they need from you. Once you design that experience and build that trust with your customers, they will stick with you.


In-vehicle UX research: Here’s one recommendation that hasn’t changed in 10 years

In-vehicle UX research: Here’s one recommendation that hasn’t changed in 10 years



I found myself discussing what can be done to increase the extent to which voice recognition systems are seen as a benefit rather than an annoyance with the research sponsors, and I said the same things as I said 10 years ago… improve the system to support and recognize more natural speech patterns.

After recently conducting research on drivers’ experience with technology in their vehicles, I reflected on what has changed about that experience in the last 10 years…. and what hasn’t changed. The availability of customized, on-demand, and context-appropriate information in in-vehicle displays seems to be one of the biggest differences that drivers now benefit from, compared to their experiences 10 years ago. However, in-vehicle voice recognition was a particularly interesting topic during the research. In some cases, these systems have progressed considerably in the past decade such that they are useful now. But in many cases, I honestly couldn’t tell the difference between current voice interfaces and those of the late 2000’s.

What’s impacting drivers now

There are a lot of cool things on the horizon for the auto industry with autonomous vehicles gaining momentum, but I’m talking specifically about features equipped in late model mass market vehicles that are having a pretty big impact on a large portion of the population’s driving experience right now. The integration of customizable and context-sensitive safety, diagnostic and convenience information into instrument clusters, and even the heads-up display (HUD) has changed the typical driver experience.

In an affordable $20,000-$30,000 vehicle, drivers can get a technology and communications package that allows them to have a wealth of information at their fingertips. Diagnostic and safety sensors providing alerts whenever vehicle status is sub-optimal, lane departure warning alerts and driver attention (or rather, lack thereof) alerts, numerous cameras and sensors on the exterior of the vehicle to assist maneuvering in tight quarters, or just park the car for you, haptic indicators to orient attention to directional alerts, and of course, the ability to control connected media devices without ever taking your hands off the wheel. These are just a few of the things that might not be exactly revolutionary anymore, but are currently changing how mainstream drivers experience their vehicles.

HUDs seem to be on the border – they are not new in automotive application but have not yet achieved the mainstream deployment that you see for other technologies. HUDs can be valuable even if they are only capable of showing a driver their speed and the current speed limit for the road. When turn-by-turn navigational cues, safety alerts, and customizability are added in, the HUD feature starts to be more than just a ‘nice to have’.

A “real” voice system

With respect to voice recognition – these systems still leave much to be desired. I was a bit surprised to recently find myself having similar conversations with research participants about their experience using voice recognition systems as I had with research participants 10 years ago. There are some that HAVE gotten significantly better and allow for fairly natural speech patterns to be used when interacting with the system. But mostly, the systems still require clunky one- or two-word commands to precede any sort of action other than making a phone call.

Interestingly, now drivers have Amazon Echo, Google Home, and even Siri to use as a benchmark. They are even less willing to accept a system that doesn’t allow for them to converse at least as naturally they do when they use those systems. Ask drivers to compare their in-vehicle voice recognition systems to their experience with voice assistants and prepare for them to go on a diatribe about how cool it would be to have a “real” voice system in their vehicle to make commanding that system while driving easier.

Eventually, it will all come together. Personally, I am waiting for the day when I can say “Alexa, take me home.”

Pin It on Pinterest