Avoca — Internal

Appearance

Tool Call Mocks ​

How Hamming tests run against a real Vapi assistant without hitting real downstream tools (AutoOps bookings, real transfers, real CRM mutations). The mocking layer lives in avoca-next, not Hamming.

What this is ​

When a Hamming test places a call to a Vapi test phone bridging the EAS assistant, avoca-next's tool dispatcher detects the call is a Hamming test (by Hamming caller-ID match or test-phone server-URL marker). For each tool invocation during that call, the dispatcher checks the tool_call_mocks Supabase table for an entry keyed on (team_id, test_case_id, tool_name) and returns the configured mock response instead of executing the real tool.

The agent's reasoning path is real (real LLM, real prompt, real Vapi assistant, real PSTN). Only the leaf tool calls are substituted.

Terminology ​

The thing has different names at different layers. Same concept:

LayerName
Avoca dashboard UI"Override"
API path/api/hamming/tool-call-mocks
Supabase tabletool_call_mocks
CodeMockResult, resolveToolCallMock
This documentation"tool call mock" or just "mock"

State ownership: what lives where ​

Test cases and their mocks are stored in different systems. The Avoca dashboard combines them in one dialog but the data goes to two places.

ThingStored inVisible at hamming.ai?Visible in Avoca UI?
Test case (persona, opening utterance, scenario)Hammingyesyes
Override / tool-call mockAvoca Supabase tool_call_mocksno, neveryes
Guardrails / assertionsHammingyesyes ("Assertions")
Test run resultsHamming, snapshotted to Avoca via Inngestyesyes

Consequences:

  • Mocks travel with the dispatcher, not the call origin. A test case run from Hamming directly still hits Avoca's dispatcher (because the test phone's server URL points at avoca-next), which still applies the configured mocks. The mock layer doesn't care where the call was initiated.
  • A test case authored only in Hamming has no mocks. You'd have to come to Avoca to configure them, OR you'd lose the mock interception (write tools would safe-block-default; reads would hit real APIs).
  • Avoca's Simulation Testing UI is the canonical authoring surface. Don't author in Hamming-direct for our work; you lose the auto-wiring and lose the mock-config affordance.

Architecture ​

Five-tier resolution ​

The resolveToolCallMock function in apps/web/lib/tools/test-call-mock.ts returns the first match in this order:

  1. Replay-matched entry (tool_call_mock_replays), for test cases pinned from a real call (see below). Wins over everything else when the current invocation's args match a recorded invocation from that real call.
  2. Explicit per-test-case mock. Looked up by (team_id, test_case_id, tool_name). Always wins if present.
  3. Team default. Looked up by (team_id, tool_name). Applies to any test case for this team that didn't specify an explicit mock.
  4. Safe-block default for write tools. If the tool is in the write set and nothing matches above, returns a generic "not available during test calls" string. Prevents accidental writes.
  5. Real execution for read tools. Reads with no mock fall through to the real tool. Acceptable because reads don't mutate state.

Tier 0: replay-matched entries ​

The static tiers above (1–4) are a hand-authored, generic mock config — the same string every time, regardless of what a call actually asks. Tier 0 is different: it replays the actual tool-call inputs/outputs from one specific real call, so a historical bug that depended on a specific customer record or availability response can be reproduced deterministically inside a real Hamming voice test call, instead of settling for a generic approximation.

  • Storage: tool_call_mock_replays (one row per recorded invocation, unlike tool_call_mocks' one row per (test_case_id, tool_name)) — needed because a real call can invoke the same tool more than once with different args and results.
  • Matching: exact match on normalized input_args (volatile fields — request IDs, timestamps, correlation values — stripped before comparing), not position. A candidate fix that changes when a tool is called, without changing what it's asked, still gets the historically-correct response. Recording order only breaks ties between otherwise-identical matches.
  • Consumption: each recorded entry is only returned once per replay — tracked via consumed_at on the row itself (not in-memory), since resolveToolCallMock runs fresh per tool call, often across separate serverless invocations.
  • No match found: falls through to the static tiers above rather than erroring — a candidate fix legitimately asking something the original call never asked is expected, not a bug in the replay mechanism.
  • Non-interference: a test case with no replay entries behaves exactly as it did before this tier existed — tier 0 only activates when the test case actually has replay data for the tool being called.

See .indusk/planning/hamming-real-call-tool-mocking/adr.md in the FDE workbench for the full design reasoning.

Mockable tool whitelist ​

The complete list lives in packages/api-contracts/src/hamming-scheduling.ts as MOCKABLE_TOOL_NAMES. 35+ tools across all integration verticals: AutoOps, ARS, ServiceTitan, MTG, plus call-control tools like transferCall and checkTransferWindow.

If a tool isn't in this list, you can't mock it. The API rejects unknown tool names.

Configuration surface ​

UI: "Simulation Testing" in the v2 Avoca dashboard. Two entry points to the same page:

  • Settings → Responder → Simulation Testing
  • Calls → Agent Testing

URL pattern: https://dashboard.avoca.ai/team/<TEAM_ID>/settings/responder/simulation-testing. Flag-gated on hasSimulationTesting; if you can see it, your team has the flag.

The two dialogs that set mocks: CreateTestCaseDialog.tsx (new test case + initial mocks in one save) and EditTestCaseDialog.tsx (add/edit mocks on an existing test case).

For direct-API access (bypassing the UI), POST to /api/hamming/tool-call-mocks with a logged-in browser cookie.

Detection logic ​

The dispatcher detects test traffic via either:

  1. isHammingTestCall(customerPhone): caller-id is in HAMMING_TEST_PHONE_NUMBERS (see apps/web/config/hamming-test-numbers.ts).
  2. opts.isTest === true: the ?isTest=true query param baked into the test phone's server URL by buildTestPhoneServerUrl. Used for chat-bridge sessions and any path without a customer phone number.

Either signal triggers mock resolution.

Source ​