AI voice receptionist for live inbound calls

The problem

A missed call is a missed customer. Small operations can't staff a phone line all day, and generic answering services can't actually do anything — they take a message. The goal here was an agent that picks up a live call, holds a natural conversation, and completes real work: checking availability and booking the appointment before the caller hangs up.

What it does

• Answers inbound phone calls with a natural, low-latency voice.
• Understands the caller's intent, asks the follow-up questions a human receptionist would, and handles interruptions and corrections mid-sentence.
• Books appointments against real availability and confirms them back to the caller.
• Pushes every meaningful event into backend systems through webhooks, so bookings, transcripts and outcomes land where the business already works.

How it's built

The voice loop runs on VAPI for telephony and turn-taking orchestration, Deepgram for streaming speech-to-text, and ElevenLabs for text-to-speech. Business logic lives behind webhook handlers: the agent's decisions trigger calls into scheduling and record-keeping systems, keeping the conversational layer and the systems of record cleanly separated.

Why voice is the hard mode of LLM engineering

Text chat forgives latency; a phone call doesn't. Every design decision is shaped by the real-time budget:

• Latency stacking. STT, reasoning, and TTS each add delay — the architecture has to stream at every stage, because a two-second pause on a phone line feels broken.
• Turn-taking. Real callers interrupt, change their minds, and talk over the agent. Handling barge-in gracefully matters more than any single answer.
• Failure containment. When a tool call fails or the caller goes off-map, the agent has to degrade into something a human would still find polite and useful — not freeze into silence.

It's the same discipline as the rest of my work: the model is the easy part; the engineering around it is what makes it usable by strangers.