Use your voice. How to make everything natural

The dream: Talk to computers

If you know everyday desktop work, you know the frustrations. Typing text. Dragging and dropping media. Moving data to the wrong place. The many not-so-friendly user interfaces. More often than not, a computer is not the assistant we want. It is a cold number cruncher with a reference to a long lost career working as a Computer. Yes, that was the job title we named the machines after.

In popular sci-fi, we chat with our computers. Like in the movie Her (2013), or the infamous HAL 9000 in 2001: A Space Odyssey (1968). Back in 2000, you could install Dragon Speech and speak naturally to your computer. And by speaking naturally, someone finally delivered on their promise. As long as you spoke English, of course.

Great training supports acting naturally and using your voice.

Speaking naturally

Speaking naturally is different for everyone. We articulate differently. We need more time to think through choices. We need natural pauses, and most of all we need space to explain. We take command when we feel it is necessary. We interrupt our counterpart by choice, or just out of excitement.

In real life, your counterpart might object, or get hurt by your words, even downright mad at you. Like when you interrupt a flow. This has a social cost.

Your counterpart is more likely to be charmed, convinced or challenged in a good way, if you are good at communicating your thoughts and ideas.

So let’s be honest. Static training, which is how most training is carried out today, is not natural.

To support natural communication when speaking to computers, you have to be able to use every speech tool available to you, and the computer has to use them too. AI has given us the perfect opportunity for this.

This is why we built the key component of Apprendly RolePlay: Speak Naturally. An advanced audio component that translates your voice in real time into bits churned by our own character engine, ACE. What you experience is an advanced real-time flow that feels natural.

Speech is essential to a great experience. It is what drives your training session, virtualising the everyday conversation you have with other people.

Design for open ended conversations

Nobody wants to be the one hanging up first.

A key design for natural conversation is being able to end it when you want to. So instead of giving your role-plays a limited number of steps, it should feel natural to you and to the computer when the conversation is over.

Right now, you can support this experience by letting Apprendly actors decide when the conversation is over. We believe in this form of course design, so we have enabled it as a default option when you create role-play content.

The actor stops when the goal is reached, or when you have given cues to end the conversation.

Designing the next voice experience

Speak Naturally is key to designing a good experience. What we are working on next is more flow in the conversation, where the AI actor talks over you gently and responds to confirmations like “aha” and “mhm” as it goes.

It is tricky by design, because the flow is driven by what your mic records, in real time, giving you context as it happens. Topping that with compute that also knows when to use confirmations takes multi-layered tracking of context and probability calculations on top of the existing experience. Guessing, then coming up with injected words that are natural to the conversation. The “ahas” and the “mhms” everyone uses.

So the next time you run a role-play with RolePlay, look out for nuances like the real-time flow. Maybe soon you will notice that the AI actor interrupts you.

Going back to a lesser experience will be like walking in a digital museum.

If you want to know how we handle the finer details of language, you should read my colleague Anders Heivoll’s piece on dialects and language nuances in role-plays.