CUI for Automobile


Video Prototype


The most efficient application scenario is "the user clearly knows the target of the task and the voice interaction is faster than the touch interaction".

Driver distraction leads to so many fatalities in world everyday, what can we do from voice interaction point of view to address this issue in a more meaningful way that helps people have safer and happier travel experience.

User Personas


I interviewed three experience drivers to define my key stakeholders and created three personas for them.

Primary Research

After the analysis of the personas, I found that my stakeholders are drivers who need navigation, communication, information, and entertainment when they are driving. Then I decided to choose Peter to be the one character that best embodies my Action.

Secondary Research

I did rounds of secondary research to dive deeper into the challenges of the current voice system. And literature review helped me to touch more quantitative and qualitative results.

From quantitative research, I learned that the most demand of voice system in the automobile is navigation.

From qualitative research, there are 64 participants did the interview for their vehicle VCS (voice control system), and we got four categories of themes identified. They are

01    |   Users’ mental models of VCS and expectations about system performance
02    |  Usability problems and system design issues
03    |  Participants lack system knowledge
04    |  VCS usage patterns

I got these insights from my research and also these insights are the significant challenges of the current voice interaction system.


How might we improve users’ trust in the voice interaction and help users discover what the voice assistant can do for them?

How might we help driver have a safer and clear feedback of the voice interaction?


I focus on Peter's story to build a persona narrative journey map to understand all scenarios and the thinking, feeling and opportunity of each scenario. From the emotional experience states, I found that the navigate, drive and park states are the biggest pain-point and based on the quantitative research result, I decide to focus on navigation voice design.

User Journey Map

I re-organized the three states to two scenarios to help me build the dialogue scripts, which are departing and arriving scenarios.



Where is the restaurant?


How’s the traffic now?


It is hard to find a parking space near the restaurant.

How can I find a parking space?

User Journey Map

Main Opportunities

After the user journey map analysis, I summarized three main opportunities for the solution and also each opportunity targeted to my insight and challenge. 

Voice Assistant Persona

Not only user persona, a persona for the voice system is important as well. When people hear a voice, they instantly make assumptions about the speaker’s gender, age, social status, emotional state, and place of origin, as well as personality traits like warmth, confidence, intelligence, etc. So I also build a voice assistant persona to guide me write the simple dialogue.

Value Proposition

You focus on driving, I help you with the rest.

An automobile voice system delivers safer and happier travel experience with your trust and practical help by natural dialogue in context and clear feedback so that you only need to focus on the drive.

Let you know what I can do for you.

Proactively provides cognitive, operational, and informational assistant to you.

Dialogue naturally in context.

Know your language and say your language under your scenario. Understand context, such as ur cation or previous queries, to support natural interactions.

Clear feedback make you know where you are.

With sound and vision feedback combination, that to keep you in a smooth dialogue flow.

User Speaking

System Thinking

Simple Scenario Dialogue

Reimagine End of Life is a week of exploring big questions about life and death. The Art and Design section is hosted by California College of the Arts. It is a great honor for me to participate in this exhibition so that more people can see and experience my work.

Simple Dialogue Flow

Service Blueprint

When I have solid sample dialogs, I abstract the flow and logic of the conversation to an action flow chart, this provides the structure of my conversational interface. However, if I want to achieve my goals, just simple dialogs is not enough, the system of automobile voice control system is complex, to understand the whole system and how it works well combo with the voice action, I draw a service blueprint to show the whole system in the user journey.

Service Blueprint (System Diagram)


When the system logic ready, a storyboard is kind of necessary to help my run the usability testing and prototype iteration.

​Scenario 1 : Departing

​Scenario 2 : Arriving

Usability Testing

For usability testing, I prepared a context sheet to help tester jump into the role and scenario. The objectives of the testing are to look for 


1. Natural conversation

2. User confusion

3. Unexpected utterances and

4. Signs of frustration or impatience.


And I found 10 users helping me do the test and I got a bunch of valuable feedback.

User Testing

Context Sheet

Feedback Note

Prototype Iteration

I used a voice prototyping tool Getstoryline to build my prototype and based on my usability testing to iterate my prototype, which focus on common detours, intent coverage, and error handling.

Scenario 1 prototype

Visual Feedback

To help drivers know the context and where they are, visual feedback is an important component of the whole system. And based on my research, the most current voice visual feedback are a tiny line of sound wave or real-time text. However, I think these visual feedbacks are not fitting the driving scenario, because drivers need to focus on driving, but they have to distract a lot of attention to cognize those visual components. This situation is “Cart before the horse”.

Sound Wave Visual Feedback on Central Control Display

Real-time Text Visual Feedback on Central Control Display

So I designed a new series of visual feedback that display on the dashboard. I choose two states of the voice system. One is user speaking, which use audio spectrum to show the user’s voice input, and using animated circle to show the system thinking state.

The reason I chose these two forms of visual feedback is that simple and obvious visual components won't increase drivers cognitive pressure, but drivers still understand the context. And display on the dashboard could make drivers noticing the feedback by split vision, drivers don't need to move their sight but focusing on driving.


After the exploring of the voice system, I deeply felt that how important of the conversational interface design for the upcoming conversational interaction age. Meanwhile, I learned and practiced that always look back to check my initial goals and challenges, making sure that I won't lose the direction and always design for the real problem and challenges.


And although this is an intense class project, I want to keep developing this project to explore more chance which not only about navigation. So for the next step, I will keep usability testing and do more iteration, also design more skills to cover complex driving scenarios, then improve the visual feedback effect and iterate the real scenario video.







CUI Designer

Video Editor

Jun 2018 -

Aug 2018

CUI Design Process


CUI prototyping

System Thinking

Animation Design


Conversational Interaction

Design by Boyang Jiao © 2018

Last update: May 13th, 2018 @ San Bruno, CA