Voice agents can start tool calls while the caller is still talking, paper shows
Researchers at Ant International describe Heartbeat Thinking, a way for voice agents to prepare while the caller is still speaking. Every half second a "Thinker" model reads the partial transcript and can fire off tool calls, such as looking up an order. When the caller stops, a separate "Responder" model checks that work against the full request, reuses what fits and does the rest. On the FDBench v3 test, average time to the final spoken answer fell from 6.90 to 4.39 seconds compared with an earlier method, and was 45.7% lower than Pipecat. Task quality was mixed: better on FDBench v3, slightly worse on two other benchmarks, and in 4.0% of FDBench recordings the answer used wrong or outdated results.
This is a research preprint tested on benchmarks, not a shipped product. The Hugging Face page has one comment, from the paper's submitter, restating the abstract, so there is no outside discussion yet.
Sorted take: the speed gain looks useful, but the paper's own shopping cart example shows the catch. An early action, like adding an item, is not undone by a later correction, so keep early tool calls to read-only lookups.
Why it matters: If you run a voice agent for support or orders, callers waiting after they finish talking is a common complaint. This is a pattern your developers can test: start read-only lookups during the call and measure the wait.
arXiv ↗On Hugging Face ↗AI-assisted, reviewed by Sorted AI
▲ 0
0 comments