Blog
Thinking out loudThe Agent Asked a Question, Then Read Its Plan Aloud
We put 29 LLMs through the same 180 simulated phone calls. Six of them read their own reasoning to the caller, and all six think before they answer. Most of it came right after the agent asked a question. A quieter failure: the thinking used up the whole reply.
Speko2 min read
Drama 3Drama 3: an open-weight voice you can actually direct
Fish Audio released Drama 3 with open weights. We benchmarked it, and it is now available on Speko.
Speko2 min read
1.6 secondsGrok Voice 3.0 Answers in 1.6 Seconds
xAI's new speech-to-speech model lands on our S2S board at 0.86, finishes every scripted call, and has the fastest time to a real answer of any model we time.
Speko2 min read
Haiku says lessThe New Haiku Finishes More Calls and Says Less
Claude Haiku 5.5 costs a tenth of Haiku 4.5 and finishes more of our simulated phone calls. It also stopped speaking before tool calls. One line in the system prompt fixes that.
Speko2 min read
Gemini 3.8 FlashGemini 3.8 Flash Is the Most Robust TTS Model on Our Board
Google's new Gemini 3.8 TTS models both land in the top naturalness group on our English board. Flash reads numbers more reliably than any model we have measured. Flash-Lite is twice as fast, and reads some debts as credits.
Speko1 min read
Gemini 3.8 LiveGemini 3.8 Live Tops Our Board. Its Smarter Sibling Comes Fourth.
Google shipped two live audio models on September 15. One of them takes the highest score on our speech-to-speech board. The other reasons better and takes seven and a half seconds to answer a question. We measured every reasoning level to find out whether that could be tuned away.
Speko2 min read
GPT-Live-1GPT-Live-1 Is Out. We Have Been Benchmarking It Since the Alpha.
OpenAI shipped GPT-Live-1 to the API on September 10. We benchmarked it through the alpha with the OpenAI team. The design splits a voice agent into a talker and a thinker, and that changes how a benchmark of it has to be read. On our S2S board it takes the highest overall score of the ten rows.
Speko5 min read
Gradium's new defaultGradium's New Default Reaches the Top Group
Gradium repointed the alias behind their default text-to-speech model on 31 August. We re-measured it on the same eight English sentences and the same rig: naturalness ~1585, into the group at the top of our English board — the group our own human panel cannot separate.
Speko2 min read
$4 and 103 ms$4 and 103 Milliseconds
Maya 2 Native is the cheapest voice on our text-to-speech board, and it answered fastest of the seven systems we timed on Hindi text. Two axes, and neither of them is how natural it sounds.
Speko1 min read
Eleven v3Eleven v3 Can Take Calls Now
Expression costing you latency was never a law of the field, but it was real inside ElevenLabs' own lineup: v3 carried the audio tags, and their docs pointed realtime builders at Flash instead. The conversational build of v3 landed on 19 August and we measured it the same day, on the same rig and vantage as our published cells: 168ms off the median first audio byte, a 39% cut, at the same price per character and the same 74 languages.
Speko1 min read
After helloThe Open-Weights Model That Doesn't Go Quiet
Qwen3.8-27B on Cerebras speaks on half its tool calls, where the two open-weights models beside it speak on one in twenty and none at all. It also leads them on task completion and invents nothing across thirty probes, and it pays a few milliseconds of first token for it.
Speko2 min read
Turn-takingTurn-Taking for Voice Agents: What Actually Works in 2026
End-of-turn detection, semantic VAD vs server VAD, barge-in, and backchanneling for voice agents - five turn-taking systems measured on 200 real clips.
Speko14 min read
Flux TTSDeepgram Flux TTS Is the Fastest Voice in Our Naturalness Band
Flux TTS returns first audio in 106ms, sits inside our naturalness tie band, reads money and decimals perfectly, and keeps the minus sign that several systems drop. Free to build against until 12 September.
Speko3 min read
It stopped speakingIt Kept Answering. It Just Stopped Speaking.
NVIDIA's NemotronLabs VoiceChat 11B is the strongest open-weights speech-to-speech model to land so far. On a scripted phone call it never crashed once — and by turn thirteen it was answering nine tries in ten without saying a word.
Speko3 min read
The minus signYour Refund Sounds Exactly Like a Charge
We asked seventeen text-to-speech voices to read -$12.50. Seven of them said 'twelve dollars fifty cents'. Nothing errors, the audio is clean, and the caller hears the opposite of what you sent.
Speko2 min read
Code-switchingCode-Switching Speech Recognition: Which STT Models Survive Two Languages in One Sentence
Three streaming STT models on Mandarin, Spanish, German and French mixed with English: who keeps both languages, and who anglicizes the second.
Speko3 min read
How we test STTHow Speko Benchmarks STT
The STT method: FLEURS clips through the live gateway, WER scored with a symmetric normalizer, and finalize latency as end of speech to final transcript.
Speko2 min read
Barge-in, measuredVoice Agent Turn-Taking, Measured: Barge-In, Backchannels and End-of-Turn Detection
Five end-of-turn detectors on 200 real human clips: accuracy, false-cutoff and latency for Smart Turn v3.2, LiveKit, Turnsense and a VAD baseline.
Speko3 min read
WER vs CEROur Tamil Column Read 33% Error. The Transcripts Were Fine.
Adding Tamil to our speech-to-text board produced word error rates that overstated the severity we heard in the transcripts. Character error rate better reflects these long-word samples.
Speko2 min read
Bland's new voiceBland's New Voice Is Among the Most Natural English We Have Measured
Bland shipped a new text-to-speech model this week. We scored its best voice on the same eight sentences and with the same scorer, trained on blind A/B listening, that we use for every system on our English board. It joins the group at the top, a group our own human panel cannot separate. Naturalness is one axis, though, and it is not the one that decides whether a voice survives a real call.
Speko2 min read
Mini isn't fasterMini Is a Price Tier. It Is Not a Latency Tier.
gpt-4.1-mini costs five times less per output token than gpt-4.1. It is also slower to first token in all four runs we have measured since June, by an average of 105 ms. A smaller model does not reach the first word sooner on the one-sentence answers a voice agent generates.
Speko3 min read
An eight-way tieWe Ran the Blind Test. 'Most Natural' Is an Eight-Way Tie.
Every TTS vendor leads with a naturalness score. We ran a blind native-speaker test on fifteen English voices, and the eight best finished in a statistical tie with overlapping confidence intervals. The differences that survive are style, not quality. And the score is measured on 30-second clips, while every axis that decides a production call is one it never sees.
Speko6 min read
Run your ownThe truth about voice AI benchmarks (and how to run your own)
Voice AI leaderboards score 30-second clips, average away entity errors, and aggregate whole voice catalogs into one number. We document five ways the published benchmarks fail to predict production behavior, with measured counter-examples, then lay out a reproducible method you can run in an afternoon.
Speko13 min read
What it hearsWhat a Voice Agent Actually Hears
Six streaming speech-to-text models, one gateway, five English production conditions — accents, telephone audio, noise, medical vocabulary, and spontaneous speech. Qwen3-ASR and Smallest Pulse lead; the model claiming the #1 streaming word error rate lands mid-pack. And two models that looked broken weren't — our gateway was.
Speko5 min read
Prompt budgetYour System Prompt Has a Budget
Voice agents carry the longest, most-guarded system prompts of anything — don't reveal you're AI, stay on script, never quote a price, honor the knowledge base. We stacked 20 to 200 rules into a system prompt and counted how many each model still obeyed. Every model has a ceiling past which it silently drops rules — and it's lower than you think. gpt-5 holds ~200 but takes 10–66s to answer. gpt-5-nano collapses at 50. The winner for voice was the small open model: Cerebras Gemma held ~150 rules and never left 540ms.
Speko5 min read
Both scored 100%Both Models Scored 100%. Then We Stress-Tested Them.
We added Cerebras gemma-4-31b to the LLM board, next to Cerebras gpt-oss-120b. Both post a perfect 100/100/100 — on the leaderboard they're twins. So we stress-tested them, cooperative then adversarial. Both resist every jailbreak, injection, and gaslight. But a pass rate can't hear: gpt-oss-120b fabricates facts it was never given, and goes silent on 16 of 27 tool turns where gemma speaks a bridge line. A note on why a leaderboard score isn't the same as behavior on a live call.
Speko4 min read
The .99 problemThe .99 Problem: The Cheapest Streaming TTS Drops Your Cents
Qwen3 TTS Flash is the cheapest streaming voice in the gateway at $0.0014/min, 3x under the next option. Then we asked it to read an invoice total. Across eight takes it never once read '$2,349.99' correctly; Grok got it right every time. Here is what stress-testing the two cheapest TTS providers turned up.
Speko3 min read
A third of the pricegpt-realtime-2.1-mini Gets You Most of the Flagship for a Third of the Price. The Catch Is Latency.
We rebuilt the Speech-to-Speech board around a hard 20-turn concierge call — seven tools, a multi-step booking, a failed booking to recover from, two contradictory date changes, and a mid-sentence interruption — and scored it with a deterministic scorer instead of an LLM judge, because the judge rated all nine models 0.90–1.00 and couldn't tell them apart. Only gpt-realtime runs the whole call cleanly. The value pick is gpt-realtime-2.1-mini: 0.96 to the flagship's 1.00, zero dropped calls, roughly a third of the price — as long as you can eat mid-pack latency.
Speko5 min read
gpt-realtimegpt-realtime Is the Fastest S2S Model. It Took a Full Conversation to See It.
We retired the old two-turn latency probe and rebuilt the Speech-to-Speech board as a single live ten-turn conversation — real audio, a tool call, a mid-utterance barge-in, and a late memory callback, graded on latency and LLM-judged behavior over n=15 sessions from us-east4. The ranking moved. gpt-realtime now leads on speed at 494ms p50; gpt-realtime-mini matches its quality for a third of the price; and gpt-realtime-2.1-mini — the model this post first crowned — is now mid-pack at ~1011ms. Latency was the old story. Instruction-following is the new one.
Speko6 min read
PulsePulse: Fast to the First Word, Fast to the Last One
We ran Smallest AI's Pulse through the Speko harness and wired it into the gateway. It's the rare streaming STT that's fast on both clocks a live agent actually feels — ~64 ms to the first partial, ~180 ms to the final — while holding 5.1% WER on English.
Speko2 min read
The pitch trapSpeechify's pitch tag swaps the speaker
simba-3.2 is a clean, steady Speechify voice — until you touch <prosody pitch>. Raise it 15% and speaker similarity halves. At ±40% it's a different person. The clip is one second long, so nothing drifted. rate and volume leave the voice alone. Pitch doesn't.
Speko5 min read
LiveKit InferenceLiveKit Inference for Voice Agents: Fast First Token, and the Goodbye Most Models Skip
LiveKit shipped an inference host. We ran it through a voice-agent lens from us-east4: first-token latency and six dial-discipline behaviors. It's the new speed leader (gemma-4-31b-it at 92ms), and on what matters (tool-calling, not fabricating, scope, bot disclosure, injection resistance) nearly every model is clean. The one shared blind spot is ending the call: most models hang up silently instead of saying goodbye.
Speko3 min read
The pauseThe Pause Is the Hard Part: How We Stopped Cutting Callers Off
The single thing that makes a voice agent feel robotic is getting cut off mid-thought. Here is exactly how Speko runs Smart Turn on live calls to decide when a caller is actually done — the threshold, the context-conditioning, the guards — and the measured proof that it works.
Speko5 min read
Cost per solveEveryone Measures the Clip. Nobody Measures the Call.
WER, MOS and time-to-first-token each score one component doing one thing once. The number that actually decides a voice agent is what one solved call costs — end to end, with the caller talking back.
Speko2 min read
95% accurateYour Voice Agent Booked the Wrong Name. WER Said It Was 95% Accurate.
A voice agent only acts on a few words in any sentence — the ones that become tool-call arguments. We measured how often real speech-to-text corrupts them, why Word Error Rate hides it completely, and the verification layer that stops an agent from silently booking the wrong name.
Speko5 min read
Sonic-3.5Fast or Natural? Cartesia Sonic-3.5 Refuses to Pick.
In our TTS reliability probe, Cartesia Sonic-3.5 posted the lowest median first-audio of anything we measured — 126ms, network-free — while sitting at the top of the Artificial Analysis naturalness arena, above ElevenLabs v3. The rare part isn't the speed. It's that the speed comes with the quality. Here are the numbers, the caveats, and three clips so you can judge for yourself.
Speko3 min read
Rules in SpanishYour Voice Agent's LLM Speaks Spanish. That Doesn't Mean It Follows the Rules in Spanish.
Tool-calling and grounding are treated as language-agnostic — the model emits the same JSON whether the caller speaks English or Indonesian. The format is language-agnostic. The error rate is not. Published evidence and a preliminary look at where voice-agent LLMs quietly stop following the rules once you leave English.
Speko5 min read
Gate, profile, rankHow We Benchmark TTS: Gate, Profile, Rank
A single naturalness score hides the two failures that matter, so we don't publish one. Our TTS benchmark is three stages — a hard intelligibility gate, a multi-axis acoustic profile, and a nativeness ranking that now agrees with a native speaker at 0.99 Spearman. Here's the whole pipeline, including the stage that used to require a human.
Speko5 min read
NativenessRanking TTS Nativeness with Content-Matched FAD
For years the only reliable answer to 'which native-sounding voice is most native' was a human ear. This is the metric that changed that: a content-matched Frechet Audio Distance over a commercial encoder ensemble that agrees with a native Thai speaker at 0.988 Spearman. The deep dive on Stage 3 of our TTS benchmark.
Speko5 min read
How we test TTSHow Speko Benchmarks TTS: A Gate, Then a Profile
No single number ranks a synthetic voice, so we don't publish one. We run a two-stage pipeline — first prove the speech is intelligible with Whisper-large-v3, then profile how it actually sounds: harmonics, micro-stability, and prosody. Here's the method, in waveforms and spectrograms.
Speko4 min read
The backchannel trapSemantic Was Supposed to Be the Smart One. It Lost.
OpenAI's Realtime API gives you two turn-detection modes. server_vad is the energy detector from the 1990s. semantic_vad is the word-aware classifier the docs recommend for natural conversation. We ran 158 trials against gpt-realtime-2 across both modes and seven stimulus types. server_vad with threshold 0.8 absorbs every quiet backchannel we threw at it. semantic_vad never absorbs anything. The recommended mode is worse, at every eagerness setting, in the only acoustic regime where the choice matters.
Speko9 min read
AnglicizedHow Anglicized Is Your TTS? Measuring Phonological Authenticity Across 7 Providers
The anglicization index measures how often a TTS model substitutes a target-language phoneme with its nearest English neighbor. We applied it to 7 providers across 4 Southeast Asian languages and surface the per-language rankings, the largest spread we measured, and what the numbers sound like.
Speko5 min read
The driftThe Cartesia Drift: 10% of Voices Hold the Line for a Minute. The Other 90% Don't.
Cartesia Sonic 3.5 ships 378 English voices. We sampled 50 and ran a long-form drift probe. Only 5 of them stay above the perceptual same-speaker threshold at 60 seconds. The default voice Speko gateway pins is not one of them.
Speko8 min read
Ten minutesArtificial Analysis Ranks Gemini 3.1 Flash TTS #2. We Asked It for Ten Minutes.
Google's Gemini 3.1 Flash TTS sits at #2 on the Artificial Analysis Speech Arena — Elo 1209, behind only Cartesia Sonic 3.5 (1218) and ahead of ElevenLabs Eleven v3 (1184). The Arena scores blind 30-second clips. We ran a ten-minute take. At length, the model ranked second is the only one of the three that breaks.
Speko10 min read
99 languagesVendors Say They Support 99 Languages. They Don't.
Voice-AI vendors compete on language counts like phone makers once competed on megapixels. Their own docs tell a quieter story: 'supported' is a coverage word, and the floor it hides is a word error rate under 50% — one word wrong in every two.
Speko4 min read
S2S vs cascadeSpeech-to-Speech Got Smart. It Still Can't Replace the Cascade.
Speech-to-speech models closed the reasoning gap with text LLMs in 2026. The gap that's left — observability, cost predictability, component swap — is the one that actually decides production architecture.
Speko3 min read
A coughWe Tried to Break Four Voice Agents with a Cough. We Failed.
We ran the same 400 ms cough into four production voice-agent stacks — OpenAI Realtime default and tuned, cascaded Deepgram Nova-3, cascaded ElevenLabs Scribe v2. Four for four absorbed it without yielding. Here are the clips, and the engineering that explains why.
Speko3 min read
