A voice agent in a browser tab

Hypothesis 1 · browser libraries + cache

Speech recognition, a language model and speech synthesis, all running on your own device. No server, no API keys; your audio never leaves the browser. Works best in a recent Chrome or Edge on a computer with a GPU; on phones and browsers without WebGPU, the page picks a lighter version (you can change it below).

Speech to text · GPU + CPU
Parakeet TDT 0.6B v3 (Redux)
ternary encoder, 2-bit ONNX · 208 MB
not loaded
Language model · GPU
LFM2 350M
4-bit · 293 MB
not loaded
Text to speech · GPU
Kokoro 82M
fp32 · 325 MB
not loaded
Models not loaded

Conversation

Load the models, press Start talking, and say something.

Latency, last turn

–from the end of your speech to the agent's first sound
Waiting for silence Speech to text (after the silence wait) LLM to first sentence First sentence to audio
TurnTotalYour audioSilence waitSTT (speed)LLM first tokenLLM first sentenceTTS first audioLLM tok/s

How it works

A voice activity detector (Silero VAD) listens to the microphone in 32 ms frames. After 160 ms of silence it starts transcribing speculatively; if the silence reaches 500 ms, that's the end of your turn and the transcript is usually already done. If you start talking again before the agent makes a sound, it treats the pause as mid-sentence and merges both parts.

Speech recognition is Parakeet, a transducer model whose encoder weights are all −1, 0 or +1. The encoder runs on the GPU through WebGPU; the token-and-duration decoding loop is hand-written and runs on the CPU in ONNX Runtime Web. The text goes to a small language model on the GPU that streams tokens, and as soon as the first sentence (or a long first clause) is complete, Kokoro synthesizes it while the model keeps writing the rest.

The Lite and Super-lite versions are for phones: Moonshine tiny (English) replaces Parakeet, the device's own voice replaces Kokoro, and the language model is LFM2.5 350M on the GPU (Lite) or SmolLM2 135M on the CPU (Super-lite). They load one model at a time and keep about a third of the memory, because phone browsers reload a tab that uses too much.

The two Hypo 2 engine versions are experimental (tested on one laptop). They swap the language model's library for the inference engine written in this repository (Hypothesis 2): the same Qwen2.5 0.5B, read from a 4-bit GGUF file and run by hand-written WebGPU kernels, or, without a GPU, by C code compiled to WebAssembly with SIMD and threads. On the test laptop that is 6–7× faster token generation than the library on the same model.