Articles
Voice first. Transport, timing, and the parts that break when a real person talks to your agent.
Deep dive
Sound is a 20ms clock. TCP was built for documents. This is a ground-up tour of WebRTC — sampling, RTP, ICE, jitter buffers — and why every serious voice agent ends up as a media participant, not a server behind a POST.
Start here
The demo is easy — speech-to-text, LLM, text-to-speech, done. Then a real person talks to it and everything falls apart: the silences are too long, it talks over them, it can't tell when they're done. The hard problems in voice aren't the models. They're latency, barge-in, and turn-taking.
Also
Your agent just called searchDocs() and got back 48KB of JSON. On the next turn it re-reads all of it — or worse, silently drops the detail it needs. Store signatures and pointers instead of blobs.
Two frameworks, working independently, reached for the same word — harness. But they bet on opposite shapes of work: one you hold and steer, the other you hand a goal and let run. Here's the line between them, in code.
Every agent project had the same skeleton at the top: call the model, run tools, loop, pray you got the stop condition right. v7 moves that — plus approvals, structured output, and telemetry — into the SDK. Here's what I reached for the week it landed.
An agent that writes code you can't run is a fancy autocomplete. But running model-generated code on your own box is how you get a crypto miner. I build a mini-app where the agent writes, executes, and self-corrects code inside a Vercel Sandbox microVM — and walk the parts that surprised me.
Topics