Skip to content
All work

AI voice agent · API

A real-time AI calling API for e-commerce. A store sends an order event and a natural-sounding voice agent calls the customer, records a structured outcome and posts it back to the store.

Role
Solo: design, backend, voice pipeline, dashboard and deployment
Year
2026
Status
Beta
Links
GitHub
VAD → STT → LLM → TTS
order events
6order events
telephony providers
3telephony providers
languages (EN, Hinglish)
2languages (EN, Hinglish)
automated tests
51automated tests

01

The problem

Indian e-commerce loses money on unverified cash-on-delivery orders, failed payments and missed deliveries. Calling every customer by hand doesn't scale, and generic voice bots don't return data a store's backend can act on.

02

Key features

  • 01Six order events (order confirmation, COD verification, payment success and failure, shipped, out for delivery), each with its own prompt, greeting and allowed outcomes
  • 02Structured outcomes through a record_outcome tool. The agent states only the order facts it was sent and never promises changes
  • 03Scheduler with per-merchant calling hours, time zones, retries on busy or no-answer, and concurrency limits
  • 04Signed call.completed webhooks (HMAC-SHA256) with transcript, timeline and a stereo recording
  • 05English and Hinglish (Sarvam Indian voices); per-merchant voice overrides, including consented ElevenLabs clones and recorded human greetings
  • 06Shopify and WooCommerce webhooks trigger calls automatically, one call per order per event
  • 07Twilio, Plivo and Exotel telephony, plus a browser WebRTC test call that needs no phone
  • 08Multi-tenant merchant dashboard: API keys, team roles, analytics, call log and call detail

03

Architecture highlights

A FastAPI control plane (merchant API, scheduler, webhooks) dispatches calls into per-call Pipecat pipelines: VAD → STT → LLM with tools → TTS, streamed over telephony WebSockets or WebRTC.

  • Cascade pipeline by default, so every stage is swappable per merchant through config, including speech-to-text, LLM and text-to-speech providers.
  • Call state lives in the database, not the worker, which keeps voice workers stateless.
  • Idempotency keys on call creation and one-call-per-order-per-event on store webhooks prevent duplicate calls.
  • 51 pytest tests cover the API, merchant flows, platform integrations and WebRTC networking.

04

Tech stack

Voice & AI

  • Pipecat 1.11
  • Deepgram STT/TTS
  • Groq
  • Sarvam
  • ElevenLabs
  • Cartesia
  • Silero VAD

Backend

  • Python 3.12
  • FastAPI
  • SQLAlchemy (async)
  • PostgreSQL
  • SQLite

Telephony

  • Twilio
  • Plivo
  • Exotel
  • WebRTC

Ops

  • Docker Compose
  • PM2
  • Nginx
  • GitHub Actions
  • pytest
  • Ruff

Next project

LR21 · Elonix