Try the voice guide
Case studyConversation designPart of Northstar

Northstar Voice Agent

Conversation design for a real-time AI voice guide, built on how people actually talk. It’s live in Northstar today.

Conversation designVoice agentTurn-takingBarge-inRepair strategiesDesigning with AI
Role
SoloConversation designAgent instructions and build
Methods
Conversation analysisTurn-taking and repairIterative prototyping
Status
Live
Tools
Deepgram Voice AgentClaude Haiku 4.5, GeminiLovable
Voice Conversation

Try it here

Talk to the real voice agent, right here. It uses your microphone.

Northstar

Hi, I’m Northstar. Curious whether AI can be biased? You can ask me anything, or run an experiment to watch it happen. What sounds good?

This is the live voice agent, not a recording. Each demo conversation ends after two minutes.

01 · Problem

Learners often don’t know their question yet.

After a surprising result, learners want to ask about it before they can phrase it. Saying “wait, why?” is easier than typing a question.

The hard part was making it feel like a real conversation. A guide that lags or talks over you would undercut a tool about questioning AI.

02 · Conversation principles

The guide follows how people really talk.

People follow unwritten rules when they talk. I used these rules as the brief and wrote each one into the guide’s instructions.

Grice’s maximsConversation analysisTurn-takingRepairPoliteness
Grice

Quantity

Say only what’s needed.

In the guideAnswers are 2 to 4 short sentences, easy to hold in your head.

Grice

Quality

Only say what you can back up.

In the guideReal-world claims are hedged. Numbers from the demo are never called real statistics.

Grice

Relation

Stay on topic.

In the guideOff-topic questions get a friendly nudge back to AI bias.

Grice

Manner

Be clear and brief.

In the guideEveryday words and real examples. No lists or code read aloud.

Sacks et al.

Turn-taking

Wait for your turn.

In the guideIt waits for a pause, hands the turn back with a question, and stops when the learner speaks.

Schegloff and Sacks

Adjacency pairs

A question expects an answer.

In the guideThe greeting ends with a question, so learners know it’s their turn.

Schegloff et al.

Repair

Fix mishearing quickly.

In the guideIt asks once: “Could you say that again?” Coughs and noise get no reply.

Clark and Brennan

Grounding

Check you understood each other.

In the guideA label shows when it’s listening or thinking, and every line appears as text.

Brown and Levinson

Face

Protect people’s self-image.

In the guideThe guide takes the blame when something fails and never tells anyone off.

The greeting

First visit · welcome and introduction

“Hi, I’m Northstar. Curious whether AI can be biased? You can ask me anything, or run an experiment to watch it happen. What sounds good?”

Returning · welcome back

“Welcome back. Ask me anything, or run an experiment. Where do you want to start?”

New learners hear who is talking and what they can do. Returning learners just get a welcome back. Both end with a question and use the learner’s name if they’ve set one.

03 · Turn-taking

Designing the pause.

Most of what makes a voice feel human happens between turns. I designed three moments: when the guide starts speaking, how it hands the turn back, and how quickly it stops when the learner cuts in.

Listening

Waits for a natural pause.

Backup voice: 0.7 s of silence ends a turn. Sounds under 0.25 s are ignored.

On screen

“Speak naturally. I’ll respond when you pause.”

Thinking

Prepares the answer.

On screen

“One moment…”

Speaking

Stops talking the moment the learner speaks.

On screen

“You can interrupt me anytime.”

Muted

Stops listening until the mic is back on.

On screen

“Your microphone is off.”

04 · Error recovery

Planning for when it breaks.

Voice breaks more often than text. I planned a response for each failure, from a cough to a full outage, so the learner always knows what happens next.

Microphone

A cough or background noise

Reacting to a cough would interrupt more than it helps.

Learner getsNothing. It keeps listening.
Microphone

Speech it couldn’t understand

The guide takes the blame for mishearing.

Learner getsOne short request to say it again.
Conversation

An off-topic question

Keeps the lesson going without telling anyone off.

Learner gets“That’s outside my focus,” and a way back to the lesson.
Service

The service is busy

Says what’s happening and what to do next.

Learner gets“Give me a second to catch up, then ask me again.”
Service

Real-time voice can’t connect

The conversation keeps going.

Learner getsThe older, slower voice takes over on its own.
Service

No voice service at all

Experiments still work, since they don’t need AI.

Learner gets“The guide is resting, try again in a bit.”

An example

Learner

So can a hiring AI be unfair even if nobody meant it to be?

Northstar

It can. If it learns from past hiring decisions, it can pick up the same patterns, including unfair ones. Want to try the hiring experiment and see who gets left out?

Learner

Wait, what do you mean by patterns?

Interrupts
Northstar

Things like which schools or neighbourhoods past hires came from. The AI can treat those as signs of a good candidate. What do you think that means for everyone else?

Learner

What’s the weather tomorrow?

Off-topic
Northstar

That’s outside my focus, I only know about AI bias. Want to see how a loan AI decides who gets approved?

05 · Iterations

Five versions to get it right.

Version 1Partial

Record, transcribe, answer, speak

Couldn’t be interrupted, and every turn used build credits.

Version 2Failed

The browser’s free speech tools

A robotic voice that cut people off after 0.35 s of silence.

Version 3Failed

Gemini’s free tier

Often busy, and its backup voice made the guide interrupt itself.

Version 4Partial

Claude writes the answers

Better answers, but still slow to arrive.

Version 5Live

A real-time voice agent

One live connection. Fast, natural, and it stops when you speak.

06 · Development

From four steps to one live connection.

I compared four real-time voice services. The one I chose had to be free, work without logins, and keep Claude writing the answers.

OptionCost for a demoDecision
Deepgram Voice Agent$200 free starter credit, no card, and Claude available inside itChosen
OpenAI RealtimePaid onlyNo
ElevenLabs Agents15 free minutes a month, then paid, with the language model billed separatelyNo
Gemini LiveFree, but on the same busy free tier as versions 3 and 4Shelved

Before and after

07 · Voice states

An orb that moves with the voice.

The orb shows whether the guide is listening, thinking or speaking. It grows and glows with the voice, rising on each word and settling in the pauses.

ListeningA calm drift that grows as you talk
ThinkingA faster swirl while the answer is written
SpeakingSwells with each phrase

These orbs follow a simulated voice. With reduced motion turned on, they stay still.

08 · Reflection

What I’d take forward.

Design the architecture with the conversation

I spent four versions adjusting timings, retries and wording when the real problem was a pipeline that could only do one thing at a time. Next time I’d decide how turns, interruptions and failures should feel before choosing any technology.

Test with real conversations

So far I’ve tested the guide in my own sessions. Next I’d run sessions with learners, with their consent, and review where turns broke down or answers missed the point.

Design every error in detail

With more time, I’d design each failure as carefully as the main flow: write the exact message for every error case, decide what the screen shows, and make sure the learner always knows what to do next.

Next challenges

ChallengeWhy it matters
A classroom full of voicesOther people talking nearby could interrupt the guide or end a turn early.
Other languagesThe guide only speaks English. Each new language needs its own rules, greeting and repair lines.
Microphone consentIt should be obvious when the mic is on, what is sent, and that nothing is kept.
Keyboard and screen readersVoice mode should work fully without a mouse or sight, beyond the transcript.