One of the most fascinating developments amidst continued advancements in generative AI (GenAI) is the rise of conversational AI technologies, from voice-activated smart speakers in the home to web-based customer service chatbots. These digital entities are designed to engage with users in a natural manner, offering assistance, answering questions, and even providing companionship. However, evaluating these assistants is far from straightforward and requires a nuanced understanding of context, tone, flow, and personality.
This challenge is fundamentally one for user experience (UX) researchers to tackle. UX researchers must unravel the complexities of conversational AI UX, and go beyond success metrics and recognition accuracy. In other words, they must consider how those responses are delivered: a conversational AI needs to strike the right balance between being helpful and being personable, between being efficient and being empathetic, and so on. In our experience, we always consider three key factors: an AI’s personality, the context the AI might find itself in, and how its conversations flow.
The personality and tone of an AI’s responses is vital for user engagement. Is it friendly and approachable, or more formal and professional? The tone should be consistent with the brand identity and the expectations of the user. A mismatch in tone can lead to confusion and frustration, and undermine trust and credibility. A well-defined personality can make the AI more relatable and engaging, but it must also be consistent and appropriate for the context.
Thinking about context, you might ask yourself: is my AI chatbot providing customer support for a specific product? Is it acting as a virtual assistant for a busy professional? We’ve found the context an AI is situated in can greatly influence the expectations and preferences of users. It can also be used to inform the AI’s tone; however, context is not a fixed state and so perhaps tone needn’t be either. For example, your customers might approach your sales chatbot, but ask a sensitive support question that requires a more delicate tone. This means context is not just answering ‘what answers will my AI provide?’, but also ‘what questions might my customers ask?’.
Finally, conversational flow – the core of the interaction. The AI must be able to guide the conversation in a logical manner, anticipating the user’s needs and providing relevant information or assistance. A smooth flow can enhance the user experience, while a disjointed flow can lead to annoyance. We often think of conversation as back-and-forth, but we know from customer service that it often is a little more complex than that: customers might list multiple points up front and an agent will resolve them one-by-one (or hand off to someone who can). Can your chatbot handle the realities of how conversations flow?
Evaluating the UX of a conversational AI assistant is a multifaceted process that requires careful consideration of all of these factors. Our team of researchers is dedicated to helping businesses navigate these complexities and support the design of AI assistants that not only meet user expectations but exceed them.
Let’s connect about your chatbot to see how we can help!
In the London area? We’re hosting a conversational AI meetup to discuss trends, design, and more! The event is 8 March, 2024. Visit this page for more details!