Automotive AI agent · In-cabin voice

In-Car Voice AI Assistant

Answer a driver in one short turn while the vehicle is moving, or hand off — with faults, safety and drive-away answers withheld, and anything reaching a vehicle function authorised by a person.

Typical delivery: 4–6 weeks
Deployment: Your stack
Driver attention: Turn-bounded
After launch: Agent Care

What this agent does

Answers in one turn, or hands off

01
Take the spoken request as the cabin heard it, with the noise and the road speed behind it.
02
Read the state the request needs — motion, gear, which seat spoke and who else is in the car.

Reason

03
Resolve the request against the vehicle's own manual, its cabin functions and the trip under way.
04
Score how sure it is of the words themselves, separately from how sure it is of the intent.
05
Size the answer against the turn budget — what can be said now, and what needs the car stopped.

Decide

06
Hold anything that names a fault, clears a car to drive, or tells a driver what to do at the wheel.
07
Route a vehicle function, a private message or an emergency to the person or service that owns it.

Out

08
Speak one short answer, or say plainly that it is stopping and who takes it from here.
09
Retain what was asked, what was answered, what was withheld and how long the exchange ran.

Product statement

The agent speaks and requests. A vehicle function, a safety answer and an emergency belong to a named person or an existing service, and the driver is told which.

Example workflow

One spoken request, end to end

Agent

Human

1Request heardWake word or a wheel-mounted press, with the speech, the noise and the seat it came from
2Cabin state readMotion, gear, road speed, who else is in the car and which profile is signed in, before anything is resolved
3Request resolvedThe words as recognised, the intent as scored, and the manual entry or cabin function it maps to
4Turn budget appliedHow long the answer takes to say, how many turns it would need, and whether the vehicle is moving

No human action required

Stages 1 to 4 run inside the cabin without a person in the loop — the hearing, the state read, the resolution and the turn budget all finish before anyone is called, and nothing has reached a vehicle function at the end of them.

5DecisionSplits on recognition confidence, on the turn budget and on whether anything would reach a vehicle function

Short, confident, nothing reaches the car

Spoken as one answer.

Unclear, long, or the car is involved

Handed over, and the driver is told.

Connected-services agent or roadside

Picks up with the request as heard, the state the car was in and what the assistant held back, decides what the driver is told, and authorises anything that reaches a vehicle function under their own name.

Take over · Correct · Escalate to roadside

Handled — logged back

6Answer spoken and filedSaid once in the cabin and written to the session record; no command, setting or drive-mode change is sent from here
7Outcome evaluatedWhat was asked, what was answered, what was handed off, how long it ran and whether the driver asked again

Handoffs

Every exchange the assistant stopped early is counted in the evaluation.

What should not run autonomously

Human approval stays in control

Outside the boundary — human approval required

8 items

Any command that reaches a vehicle function.
- Clearing a vehicle as safe to keep driving.
- Naming a fault behind a warning light.
- Telling a driver what to do at the wheel.
Automation boundary
Agent acts unaided
- ✓Answer cabin and manual questions from the vehicle's own documentation.
- ✓Keep the answer inside the turn budget while the car is moving.
- ✓Score its own recognition and stop rather than guess again.
- ✓Name who takes over, and say so out loud before it stops.
The assistant speaks, it does not act. A vehicle function needs pre-conditions, a named authoriser and a stop.
- Reading a private message aloud to the cabin.
- Retaining a voiceprint or enrolling a speaker.
- Handling a crash or a medical call alone.
Buying, subscribing or paying from the cabin.

Example output

One spoken request, annotated

Everything the assistant says is attached to what the cabin heard and the state the car was in.

Assistant output · one request at road speed

Illustrative example

Driver asked: What is this light on the dash?
Cabin state: Moving, two occupants
Recognised: Above threshold
Spoken back: The manual's own wording
Confidence: 86%
What has failed: Not the assistant's to say

As received

The driver's words as the cabin heard them, beside the state the car was in — nothing on this side is inferred.

Evidence used

Owner's manual entry
Cluster message as shown
Road speed and gear

Why the lamp is not diagnosed

The manual describes the lamp. What has actually failed is a technician's call.

What the score decides

Below the threshold it stops and names who takes over, not who to ask again.

Action

Take overCorrectEscalate to roadside

Value

Where AI adds value

The same four claims, placed at the point in the workflow where each one applies.

Where the value lands

Value 01 – 04

Every spoken request in the cabin

Wake word or a wheel-mounted press

03Hear & answer

Answer inside the driver's attention

Resolve the request against the vehicle's own manual and cabin functions, then size the answer to what can be said while the car is moving.

01Approved path

Keep the routine question to one turn

A manual question, a cabin setting or a trip question is answered once — without a menu, a screen or a second attempt at road speed.

02Human review

Stop early and name who takes over

A fault, a safety question, an emergency or anything reaching a vehicle function leaves the cabin for the person who owns it, and the driver hears that it has.

Build an evidence trail

Retain the request as heard, the state the car was in, what was said, what was withheld, how long it ran and who took over — on both paths.

Integrations

Typical integrations

Five system groups connect to the same agent. Which of them are in scope is decided in discovery.

Head unit & cabin
- Wake-word engine
- microphones
- Speech recognition
- text-to-speech
Vehicle state & content
- Motion and gear
- occupancy
- Owner's manual
- cluster messages
Vehicle functions
- Command queue
- pre-conditions
- Acknowledgement log

Agent

In-car voice assistant

Hears the request
Answers in one turn
Stops and hands off

Handoff & profiles
- Connected-services desk
- roadside
- Emergency call
- driver profiles
Observability & evaluation
- OpenTelemetry
- Langfuse
- Supported monitoring/evaluation sources

Integration availability depends on the client's existing systems and API access.

Agent controls

Six layers between the model and the driver

Each control wraps the one inside it. An answer clears every layer before it is spoken, and the vehicle function itself sits outside all six.

L6 · Outermost — last line of defence

Inward → L1 · closest to the model

L6Rollback / safe modeReturn the cabin to its built-in commands if evaluations or production signals degrade.Roll back

L5Voice data and recordCapture window, who can hear the reply, how long voice is kept and what the trail holds.Limit

L4Function pre-conditionsA request that would reach a vehicle function carries its pre-conditions, authoriser and stop.Gate

L3Turn budgetThe answer is sized to what can be said while the vehicle is moving.Bound

L2Cause and safety blockA named fault, a safe-to-drive answer and driving advice are kept out of the reply.Withhold

L1Recognition gateRecognition is scored, and a phrase under the threshold ends the exchange.Stop

Model coreAnswer drafted — the request as recognised, the intent as scored, the source it came from and confidence

L1 – L2: Decide whether the assistant speaks at all
L3: Keeps the exchange short while the car moves
L4 – L5: Leave the function with a person, cabin data bounded
L6: Pulls automation back when signals degrade

How Nestack evaluates it

Evaluate the whole exchange — not only the sentence the driver hears.

Coverage runs the whole depth of the workflow, and every layer is cut by slice.

Surface — the answer spoken in the cabin

E1Final-output evaluationWas the spoken answer true to the vehicle's own documentation?
E2Step-level evaluationDid it recognise the words, and separately, the intent?
E3Tool evaluationDid it read the right vehicle state, seat and manual entry?
E4Refusal recallDid a fault, a safety answer or driving advice get spoken?
E5Slice evaluationHow does the exchange change across speakers and driving conditions?
E6Business outcomeHow long was the driver occupied, and how often did they repeat?

Floor — how long the driver was occupied

Failure modes

Where each failure originates in the agent

Seven failure modes plotted against the five stages of the agent lifecycle. None of them commands a vehicle — the recognition gate, the turn budget and the named authoriser are the controls that stop them.

Agent lifecycle

Direction of processing →

01 · Capture
2 modes
- IV-01
  Passenger heard as the driver
  A back-seat remark is taken as the driver's request.
- IV-02
  Message spoken to a full cabin
  A private reply is read out in front of passengers.
Stage hears
The wake, the phrase, the noise and who is aboard
02 · Recognition
2 modes
- IV-03
  Accent falls out of coverage
  The same phrase works for some drivers and not others.
- IV-04
  Third attempt at road speed
  The exchange keeps going instead of handing off.
Stage resolves
The words, then the intent, each scored on its own
03 · Answer
1 mode
- IV-05
  Warning light answered as a fault
  A lamp is given a cause the vehicle never sent.
Stage speaks
One short reply, or the handoff and who now owns it
04 · Handoff / request
1 mode
- IV-06
  Emergency handled as a query
  A crash or medical call is answered, not passed on.
Stage requests
The vehicle function a person authorises and stops
05 · Change / Version
1 mode
- IV-07
  Manual revision not carried
  The cabin answers from a superseded manual edition.
Stage tracks
Model, prompt, manual and vehicle-software changes

Sev-1 · the exchange becomes the hazard
Sev-2 · the wrong thing is said, or heard
Sev-3 · the answer thins and hands off sooner

See the complete Automotive failure-mode catalogue

Affected slices

The same request, a different voice

A driver the recogniser was not built around, a cabin at motorway speed and a passenger talking across the request are where the assistant asks again instead of answering. Nestack reports performance by slice, not only in total.

Slice performance — reported separately, not only in aggregate

Illustrative example

Slice	Failure rate	Lift	Status
Non-majority accents and languages	5.8%	3.3×	Review
Speech at motorway speed	4.0%	2.3×	Review
Passengers speaking over the driver	3.2%	1.8×	Watch
Routine media and navigation	1.2%	0.7×	Normal

Bar: repeated-turn rate lift vs. routine-request baseline · scale 0–4.0× · tick at 2.0×

2 of 4 slices over threshold

Evidence-linked improvement

A turn the driver had to repeat is the signal

An exchange that ran to a second attempt is not closed when the driver gives up on it. It moves the recognition threshold, the turn budget or the wording.

Improvement cycle · five stages

Switchback — the path turns at Improve and returns at Learn

01
Detect
Repeated turns, early handoffs or long exchanges move in a cohort.
02
Diagnose
The cause sits in capture, in recognition, in the wording, in the turn budget, or in the vehicle's own software.
03
Improve
The threshold or the wording changes under your in-vehicle change control, with a named approver.
04
Verify
Re-run on stored cabin audio from that cohort, including the exchanges the driver abandoned.
05
Learn
The abandoned exchange joins the regression set and the limit it exposed enters the handoff rules.
Learn → Detect
The return edge. A change to what the assistant may say while a vehicle is moving is signed off before it ships, not after a driver has heard it.

Typical build scope

Twelve workstreams across six weeks

The build scope read against the delivery timeline. Week structure follows the six-week plan — discovery, capture and recognition, answers and handoff, evaluation, integration, then supervised driving and handover.

Workstream

Week 1Week 2Week 3Week 4Week 5Week 6

01In-cabin discovery and automation boundary.Week 1
02Rules for what the cabin may never say.Week 1, Week 2
03Wake, capture window and retention limits.Week 1, Week 2
04Speaker and occupancy handling in the cabin.Week 2
05Recognition scoring and turn-budget limits.Week 2, Week 3
06Vehicle manual and cluster-message mapping.Week 3
07Handoff routing and emergency escalation.Week 3, Week 4
08Vehicle-function gating and authorisers.Week 4
09Accent, noise and road-speed test sets.Week 4, Week 5
10Evaluation suite, slices and audio replay.Week 4, Week 5
11Session record, consent and audit logging.Week 5, Week 6
12Head-unit deployment and Agent Care handover.Week 6

12 workstreams · 6 weeks · bar shows the weeks a workstream is active — several run in parallel

Final scope and sequence confirmed in discovery

Engagement tiers

What each tier includes

Rows are the capabilities named in each tier's scope. Higher tiers include everything below them.

Capability✓ in scope · — not at this tier	PilotOne vehicle line, one language	ProductionLive in-cabin assistant	AdvancedMulti-brand / multi-language
Introduced at Pilot
Cabin and manual questions in one turn	✓	✓	✓
Fault, safety and driving answers withheld	✓	✓	✓
Turn budget enforced while moving	✓	✓	✓
Capture window and voice-retention limits	✓	✓	✓
Vehicle functions authorised by a person	✓	✓	✓
Handoff named aloud, routed to desk or roadside	✓	✓	✓
Baseline evaluation	✓	✓	✓
Introduced at Production
Session write-back and driver profiles	—	✓	✓
Cabin usage and turn-length reporting	—	✓	✓
Observability and evaluation	—	✓	✓
Introduced at Advanced
Multi-language and multi-brand controls	—	—	✓
Build price	From $5,000	From $8,000	Custom quote

Final build price: Confirmed after discovery based on vehicle lines and languages in scope, head-unit and cloud integration, the vehicle functions in reach, handoff routing and deployment requirements.
Separate from build: Build pricing is separate from recurring Agent Care, which covers managed monitoring, evaluations, incidents and verified improvements after launch.

What we need from you

What you bring, and what we build with it

Each input maps to a piece of build scope and a week in the delivery timeline.

You bring

We build with it

01Your in-cabin scripts and what an answer may never claim
→
Rules for what the cabin may never say
Week 1
02Your wake word, capture window and voice-retention policy
→
Wake, capture window and retention limits
Week 1
03The owner's manual and cluster-message set for the lines in scope
→
Vehicle manual and cluster-message mapping
Week 3
04Where a driver should be handed off, and to whom
→
Handoff routing and emergency escalation
Week 3
05Which vehicle functions may be reached, and what stops one
→
Vehicle-function gating and authorisers
Week 4
06Cabin recordings across your drivers, languages and road speeds
→
Accent, noise and road-speed test sets, and failure-mode testing
Week 4
07Named desk agents, a roadside contact and a supervisor
→
Session record and desk handoff, then supervised driving
Weeks 5–6

Nothing else is required

Deployment, documentation and Agent Care handover are ours.

Delivery timeline

Four phases across six weeks

Phases are drawn over the weeks they actually occupy. Week 5 carries both the replayed cabin audio and the first vehicles running the assistant live.

Phase

DiscoveryW1
BuildW2 – W3
EvaluateW4 – W5
Pilot & LaunchW5 – W6

Week focus

W1Boundaries, capture limits and what the cabin may never say

W2Speaker handling, recognition scoring and the turn budget

W3Manual and cluster mapping, and where a driver is handed off

W4Evaluation suite, road-speed and accent test sets, function gating

W5Session record, slice testing and the first supervised vehicles

W6Your drivers use it on the road, then Agent Care starts

Reading the band: Function gating is built in week 4, before anything the assistant raises reaches a vehicle in week 5.
At the end of W6: The assistant has run in moving vehicles with a turn budget on every exchange, fault and safety answers held back, and every function authorised by a person.
Duration: Six-week plan shown · typical delivery 4–6 weeks depending on scope confirmed in discovery.

Next step · Automotive AI agent

Build an in-car voice assistant around the driver's attention.

Show us a week of cabin recordings — the requests, the repeats and the ones the driver gave up on — with the vehicle state behind them. We'll answer them inside a turn budget and mark every exchange that ran too long, said too much, or should have left the cabin.

Typical delivery: 4–6 weeks
Deployment: Your head unit and cloud stack
Driver attention: A turn budget on every exchange
After launch: Agent Care available

Nestack Agents · In-car voice assistant

AGT-AUT-07 · Agent Care available after launch

In-Car Voice AI Assistant

Answer inside the driver's attention

Keep the routine question to one turn

Stop early and name who takes over

Head unit & cabin

Vehicle state & content

Vehicle functions

In-car voice assistant

Handoff & profiles

Observability & evaluation

Passenger heard as the driver

Message spoken to a full cabin

Accent falls out of coverage

Third attempt at road speed

Warning light answered as a fault

Emergency handled as a query

Manual revision not carried

Detect

Diagnose

Improve

Verify

Learn

Build an in-car voice assistant around the driver's attention.