Avoca — Internal

Appearance

VAPI + ElevenLabs chat based testing will 100% support our goals with Blueprint CI/CD like workflow ​

TLDR: Both platforms' chat/text APIs are full featured enough to drive our real production tool-dispatch code. Confirmed on both, ground-truthed against our own logs, not vendor docs. The one blocker found (VAPI has no team-resolution path for ad hoc test squads) is now fixed and shipped as a draft PR. This unlocks fast, no-audio iteration against real business logic instead of a mocked/stripped-down simulation.

What's proven ​

PlatformMechanismEvidence
VAPIPlain Chat API (POST /chat) against a scratch squadfindCustomerDNS rows in ClickHouse (default.public_tool_calls) — real EXECUTION_ERROR/SUCCESS, real latency (100ms–2s), not an instant mocked placeholder
ElevenLabsSimulation-type test (tool_mock_config.mocking_strategy: "none")Same ClickHouse ground truth — real dispatch confirmed, distinct from the deprecated simulate-conversation and Tool-Call-type tests, which always mock (confirmed via docs + live testing)

Multiple runs, cross-checked against default.public_tool_calls, not just the platform's own reported status.

Foundation built ​

  • PR #16217 — VAPI has no phone number or registered assistant id for an ad hoc scratch squad, so tool calls failed team resolution before reaching any real code. Fix: last-resort x-team-id header, checked only after the three real-call paths fail — unreachable for production traffic. Verified end-to-end via a scratch squad + isolated tunnel + ClickHouse.
  • Reusable script pattern for both platforms (chat-call-replication-spike/scripts/) — build a scratch squad/agent, drive one message, assert on the response, log every run.
  • EL needed no code change — lib/elevenlabs/webhook-tool-config.ts already binds a team_id dynamic variable straight to the x-team-id header, and dispatch already reads it.

Still need ​

The smoke-test runbook has a load-bearing checklist mostly unfilled:

  • Multi-agent handoff confirmed in chat mode specifically, not just voice
  • Row 10 (side-effect hygiene) — do platform-native test runs pollute calls/analytics/customer-facing webhooks if not flagged as test traffic? Blocks running suites at volume until answered.
  • EL selective tool-mocking (mocked_tool_ids) scoping, verified as tight as advertised
  • Seeding an integration test from a recorded failing turn (EL's partialConversationHistory looks promising, not yet exercised live)