allstate chat: ai conversation design
conversation flows for allstate chat, born from lots of problem-solving and data to back it up. half of all users were picking one option — and then hitting a wall.
the problem
while performing regular tuning exercises, the development team noticed that over half of users were selecting “policy changes” from the bot’s opening message.
once there, 35% were asking for a way to cancel a policy and another 20% were asking about payment changes. neither of these were available as valid utterances to the bot.
role
head conversation designer
timeline
jun 2022 — sep 2022
team
2 conversation designers, 10 developers
methods
wireframing in axure, prototyping in voiceflow, analysis of training data in amazon lex
my role
leading the effort, problem-solving solutions, and proposing the changes.
overlap in language
it wasn’t obvious why users weren’t seeing the option to “change policy” as part of the edit intent. we wanted them to be able to access the intent, so we looked into what the overlap was going to.
following the flow in the production environment, we found that when a user asked for “policy changes”, it was actually filling the “change a policy” slot in the edit intent. that overlap in utterance language meant users weren’t seeing a payment change option at all.
two phrases that mean different things to a person meant the same thing to the model. everything downstream followed from that.
user journey
before arriving at the solution, i mapped out what users were actually experiencing when they went to cancel a policy or change payment. that showed exactly what needed to change, and where the pain point was for so many of them.
the solution
to fill these gaps i made two major changes:
- added a payment change quick reply to the opening message, so users would see the relevant intents.
- for users wanting to cancel a policy, added a quick reply inside the policy changes flow — the business had instructed us not to put it in the opening message. it also matched the user’s mental model, since they were already selecting policy changes as their path to cancellation.
the outcome
55% of users redirected to the right flow — fewer stuck, frustrated people using the allstate chatbot.
reflections
before this project i hadn’t stepped into the ai/machine-learning world at all. leading it let me explore that growing space. what i learned:
- how to navigate amazon lex, axure, and voiceflow
- the importance of avoiding overlap and adapting flows
- to succeed as a conversation designer, attention to detail is key
the other half of this internship
the same summer i led the redesign of allstate’s final details page — fitting three separate flows onto one screen in the shop & buy purchase flow.