Partnership

Model Selection for Voicebots: How to Choose the Right Stack

How to choose models for a voicebot: speech recognition, LLM, and speech synthesis, with latency targets, routing, test plans, and U.S. compliance checks.

Engineer evaluating a voicebot test call

The right model for a voicebot is the one that answers fast enough to feel present, calls your tools correctly under your own prompt, and hands off to a person when it should. It is rarely the biggest or the highest-scoring model on a public leaderboard. A voicebot is a chain of speech recognition, a language model, and speech synthesis. You select for the whole chain, measured on your own calls. Platforms such as Monobot take the plumbing off your plate, so your team spends its time on the part that decides success: which calls to automate and how to test them.

Monobot
Put Your Voicebot Stack to Work
Explore Monobot’s AI voice agents, integrations, and analytics for managing customer interactions across calls and chats.

Visit

TL;DR

  • Choose models for the full voice pipeline (STT, LLM, TTS) and judge them on end-to-end latency, not on one component’s benchmark.
  • Use a fast model for most turns and route only harder or higher-risk turns to a stronger one.
  • Test with your own system prompt, tool schemas, and unhappy-path calls: interruptions, API failures, policy questions, escalations.
  • Pick automation candidates that are high volume, low risk, and have a verifiable outcome.
  • Check recording consent, retention, and logging before launch. U.S. rules differ by state.

Table of Contents

What model selection means for a voicebot

In a text chatbot, you pick one language model and judge the answers. In a voicebot, you pick several components, and the caller judges the sum of them. Voice agents have long used a three-step pipeline: speech-to-text transcribes the caller, a large language model decides what to say or do, and text-to-speech turns the reply into audio. Each step adds delay, and each can fail in its own way.

So “model selection” covers at least four decisions:

  1. Speech recognition (STT): how accurately and how quickly it transcribes your callers, including accents, phone-line noise, names, and account numbers.
  2. Reasoning model (LLM): how well it follows your policy, extracts details, and calls the right tool with the right parameters.
  3. Voice (TTS): how natural it sounds and how soon the first audio arrives.
  4. Turn detection: how the system decides the caller has started or stopped speaking.

The fourth is the one buyers overlook. Precision and recall on turn start and end determine whether the agent cuts the caller off or sits in silence. A strong LLM cannot fix a pipeline that interrupts people.

Start with the calls, not the models

Model choice depends on the job. A bot that reads back an appointment slot needs a different profile from one that resolves a billing dispute. Before comparing any models, sort your call types.

A useful screen asks three questions of every automation candidate: is the call high volume, is it low risk, and does it have a verifiable outcome? Apply it to your last month of call reasons:

Call type Volume Risk Verifiable outcome Model need
Appointment scheduling High Low Yes, a booked slot Fast model, reliable tool calls
Order status High Low Yes, a matched record Fast model, strong integration
Lead qualification Medium Low Yes, fields captured Fast model, good extraction
Billing disputes Medium High Partly Stronger model, escalation path
Account closure or complaints Low High Varies Human first, bot assists

The first three rows are where most teams start. They are the routine tasks Monobot’s platform is built around: appointment scheduling, order status updates, inquiries, and lead qualification. Monobot’s claim is that it can automate up to 80% of inbound calls and chats. Your own mix of call types decides how close you get.

The five criteria that decide the choice

1. Latency that the caller can feel

Callers notice a pause long before they notice a wrong word. Voice evaluation should use voice-relevant timing, such as how long the final transcript segment takes to arrive, because transcription timing affects whether the agent feels present or lagging. Measure the whole turn: end of caller speech, to transcript, to first token from the LLM, to first audio out.

Set a budget for each stage rather than a single target for the total. Our guides on per-stage voice latency budgets and on streaming STT to keep p95 under 800 ms go deep on how to do that. Track p95, not the average: the slow calls are the ones people remember.

2. Task success under your own prompt and tools

Public benchmarks do not use your system prompt or your tool schemas. Both matter. Longer tool schemas can slow first-token latency, and a model that is slightly slower but more reliable at invoking the right tool can be the better trade for a support agent. A bot that answers in 400 ms but books the wrong slot costs you a second call.

So weigh two things together: time to first response when streaming, and the share of test calls that finish the task correctly.

3. Turn detection and interruptions

People interrupt, change their minds, and talk over the bot. Check how the speech recognition layer handles the start and end of turns. Turn-detection precision and recall shape whether the agent talks over the caller or fails to answer. In your pilot, script calls where the caller interrupts mid-sentence and where they pause to find an account number.

Caller pauses during a voicebot interaction

4. Safe behavior when things go wrong

A good voicebot understands intent, keeps context, completes the task, follows policy, and recovers or escalates safely. Score models on how they fail. Does the agent clarify, retry, hand off, or exit politely when an API times out? This is where model choice meets your escalation design.

5. Cost per resolved call

A bigger model costs more per turn, and voice turns add up. Compare cost per completed task, not per token. If a cheaper model needs two attempts and a human handoff on one call in five, it may cost more than it saves. We return to cost below.

Route turns instead of choosing one model

You do not have to pick a single winner. A common approach is to start with a faster model for most turns and route only the harder turns to a stronger one. Examples of harder turns are complex billing disputes, multi-constraint rescheduling, repeated failures, and anything with higher consequences. The voice flow continues while the stronger model works, so the caller does not sit in dead air.

A practical routing rule set looks like this:

  • Default path: a fast model handles greetings, identity capture, status lookups, and simple scheduling.
  • Escalate to a stronger model: the caller repeats themselves twice, a tool call fails validation, or the intent is on your high-consequence list.
  • Escalate to a person: policy-sensitive questions, expressed frustration, or any step you decided must have human sign-off.

This keeps perceived latency low where it matters and spends compute where reasoning pays off. It also gives you a clean way to test: each route has its own pass rate.

How to test candidates before you commit

Evaluate continuously, not only before launch. The most useful test set mixes happy paths with unhappy ones: interruptions, changed minds, partial information, API failures, and policy-sensitive questions. Score each call on conversation, task, policy, experience, and recovery.

A workable pilot plan:

  1. Build 30 to 50 scripted scenarios from real call reasons, with at least a third being unhappy paths.
  2. Record real audio, with varied accents, background noise, and speakerphone quality, not clean studio speech.
  3. Run each candidate configuration against the same scenarios with your real prompt and tool schemas.
  4. Log per-stage timing and report p50 and p95 for the full turn.
  5. Score outcomes: task completed, correct data written, policy followed, escalation handled.
  6. Load test before scaling. Concurrency changes latency, so ramp calls gradually and hold them for realistic call durations.

The tedious part is wiring recording, scoring, and dashboards together. On Monobot, interaction dashboards and analytics and voice analytics for call analysis are part of the platform, so the numbers you need for this comparison are in one place.

Deployment approaches: build, assemble, or use a platform

Teams usually land in one of three setups:

  • Assemble the pipeline yourself. You pick an STT vendor, an LLM, and a TTS vendor, then write orchestration, telephony, logging, and monitoring. This gives control but makes you responsible for every integration and every outage.
  • Use a fragmented set of voice AI services. You pay separate bills for each layer and reconcile them.
  • Use one platform. The models, telephony, tools, and analytics sit behind one interface.

Monobot is the third option. Its pricing is bundled with no hidden fees, according to the company, and its no-code builder lets teams configure agents without engineering time, often deploying within minutes. For most customer service teams, that moves the question from “which three vendors do we stitch together” to “which calls do we automate first.”

If you run a BPO or want to resell, Monobot also offers white-label and OEM options.

Security, privacy, and governance checks for U.S. teams

Model choice has legal consequences, because the model sees the call. Check these before launch:

  • Traceability. Some public-sector standards, such as Virginia’s Commonwealth AI standard, call for AI decision paths to be logged for traceability and replay, data sources to be registered and approved, and personal information to be protected from third-party dissemination. Even if you are not a Virginia agency, it is a sound model for your own logging.
  • Recording consent. California and several other states, including Florida, Illinois, Maryland, Massachusetts, Montana, New Hampshire, Pennsylvania and Washington, have all-party consent requirements for some recorded calls. Recording compliance and CCPA obligations are separate issues. Have your counsel confirm the disclosure script your bot reads.
  • Documentation. The same analysis advises businesses recording customer calls to document what is captured, where it is stored, how long it is kept, who can access it, and whether AI or analytics are applied.
  • Prompt injection and data handling. Voice input is untrusted input. See our guides on securing realtime voicebots against prompt injection, data security in AI chatbots, and meeting CCPA deadlines for chatbot data requests.

For healthcare workloads, Monobot offers a HIPAA option at $1,000 per month. Your compliance officer or attorney should confirm what your specific use case requires.

What drives cost, and how to measure return

Model choice is one line in the budget. A realistic cost model includes:

  • Platform subscription. Monobot’s published plans are Starter at $200 per month, Growth at $500 per month, and Business at $1,000 per month, with Enterprise available on request. Plans are tiered by usage and features. See current plans on the pricing page.
  • Usage. Chat and voice interactions, SMS, and workspace seats are listed as separate items on the pricing page. Phone numbers are $2.50 per month per number.
  • Add-ons. HIPAA is $1,000 per month.
  • Internal time. Integration, prompt tuning, test-set maintenance, and weekly review.

To measure return, pick three numbers before launch and track them weekly:

  1. Containment rate: the share of calls in your chosen categories that finish without a human.
  2. Cost per resolved contact: platform and usage cost divided by completed tasks, compared with your current loaded cost per call.
  3. First-call resolution and satisfaction on automated calls versus the baseline.

Start with the three or four call types that pass the screen above, so your first month shows a clean before-and-after.

Monitor and improve after launch

The model you select on day one will not be the right one in six months. Callers change, your policies change, and models are updated. Build a review loop:

  • Sample failed and escalated calls every week and tag the cause: recognition, reasoning, tool failure, turn detection, or policy.
  • Re-run your scenario set whenever you change a prompt, a tool, or a model.
  • Watch p95 latency and escalation rate together. A faster model that raises escalations is not an improvement.
  • Use sentiment analysis and conversation intelligence to find calls worth listening to, rather than sampling at random.

The same data supports your human agents. Monobot’s real-time agent assistance gives suggestions during live conversations, so the calls the bot escalates still benefit from the platform.

Where Monobot fits

If your goal is a working voicebot on your top call types without building the pipeline yourself, start with Monobot. You can configure an agent in the no-code AI voice agent builder, or begin from ready-to-use industry templates for healthcare, banking, retail, logistics, HR, and IT. Integrations with third-party tools connect the agent to your scheduling, order, and CRM systems, and real-time analytics show how each call went.

Monobot

Plans start at $200 per month on the Monobot pricing page, and you can see the platform in a demo before you decide.

Our perspective

Most teams treat model selection as a shopping trip: find the best speech recognizer, the best LLM, the best voice, and bolt them together. We think that is the slow road.

What callers experience is the pipeline, not any single model. A superb model behind an awkward turn-taking setup still sounds broken. So we build Monobot as one platform where voice, chat, integrations, analytics, and agent assistance work together, and where you configure agents without code.

Our advice is to spend your effort on the call list and the test set. Pick the routine calls with a clear outcome, run realistic scenarios, watch p95 latency, and let the dashboards tell you what to improve. We designed Monobot to let you do that from day one, with bundled pricing and no hidden fees.

Alex

FAQ

Should I use one model for every turn?

Not necessarily. Many teams run a fast model for most turns and route only harder or higher-risk turns, such as billing disputes or repeated failures, to a stronger one. This keeps response time short while giving difficult moments more reasoning power.

Which latency number should I track?

Track the full turn, from the end of the caller’s speech to the first audio of the reply, and report p95 as well as the average. The slowest calls shape how people feel about the bot. Break the total into stages so you can see which one to fix.

Which calls should a voicebot handle first?

Start with calls that are high volume, low risk, and have a verifiable outcome, such as appointment scheduling, order status, and lead qualification. Keep policy-sensitive or emotionally charged calls with a person, with the bot assisting where appropriate.

Do I need to worry about call recording laws?

Yes. Several states, including California, Florida, Illinois and Washington, have all-party consent requirements for some recorded calls, and CCPA obligations are a separate matter. Your attorney should approve the disclosure your voicebot gives at the start of each call.

How often should I re-test the model choice?

Re-run your scenario set after every change to a prompt, tool, or model, and review sampled calls weekly. Voice agents should be evaluated continuously, not just before launch.

Sources