Skip to content

The team model, and where tests hook in ​

Why this doc exists. Every testing effort in flight today tests agents. But the unit that delivers value (and the unit that fails in production) is the team: a caller experiences the whole pipeline, from routing to the CRM write, and incidents like Protractor (calendars synced into AutoOps, API didn't, double-bookings) are invisible to any agent-scoped test. This page defines the team as the unit under test and maps every seam where a test suite can hook in. Sister pages: call anatomy (the six-stage lifecycle) and the post-call dispatch chain (L0–L7).

What a team is ​

A team is a binding, not a thing with much code of its own. It connects three layers:

LayerWhat lives thereShared acrossWhere
CRM layerclient library, in-call CRM tools, post-call workflow stagesevery team on that CRMapps/web/lib/<crm>/, apps/web/lib/workflow/stages/booking-<crm>/ (paths per CRM; see internal CRM abstraction for the full inventory)
Blueprint layerthe base blueprint (sub-agents, edges, prompts, tool sets)every team inheriting that basemulti_agent_blueprints
Team layertypedConfig (vertical + crm), the override, agents, phones, credentials, KB, destinations, hoursthat team onlyassistant_configs, multi_agent_config_overrides, team config tables

Functionally: a team is a function from an inbound call to outcomes (CRM writes, transfers, messages, emails, call records), parameterized by the binding above. Testing a team means fixing the inputs (a scenario plus world-state fixtures) and asserting on outcomes at chosen observation points. Every test category is a choice of which segment of the function to isolate and which observation points to assert on.

Blast radius: what a change touches ​

The layering makes change-triggers precise. This is what makes per-team required testing affordable: a team's own test burden shrinks to its delta plus its bindings, because the expensive assurance runs upstream where the shared code lives.

ChangeBlast radiusThe test that catches it
CRM client / post-call stage codeevery team on that CRMCRM-layer suite, run once centrally (mock CRM + real-instance canary)
Base blueprint editevery team inheriting it, including their overridesbase × override matrix regression (the "override drift" case: base edits don't propagate into per-team overrides, so single-team regression cannot see the break)
Team override / config editthat team onlythat team's pre-save regression + config audit
CRM-side change (their API, their sync)every team on that CRM, invisible to our diffsscheduled real-CRM canary (the only seam that would have caught Protractor)

The seams ​

Mapped onto the six-stage call anatomy. A seam is a point where a test can inject inputs, substitute a dependency, or observe outputs without touching production systems.

Seam 1 — the compile seam (pre-save, no telephony) ​

The base blueprint merges with the team override in multi-agent-config-merge.ts (mergeMultiAgentConfigWithOverrides), and the agent factory (agent-factory.ts) turns the merged config into the actual Vapi/ElevenLabs assistant payload. Everything a save is about to change is computable right here, synchronously, with no call: the compiled prompt, the tool set, the edges.

What hooks in: the pre-save gates. Configuration audit (grade the compiled config against the team's stated expectations) and text-replay regression (replay stored scenarios against the compiled prompt with mocked tools, diff the behavior). This seam is also the fix location for the known post-live eval gap: eval currently scores the template-with-variables rather than the compiled prompt, so anything scoring "the prompt" must consume this seam's output, not the template.

Seam 2 — the tool dispatcher (per-test, in-call) ​

Already built and documented: Tool Call Mocks. Every tool invocation passes through resolveToolCallMock() (test-call-mock.ts) with four-tier resolution (explicit per-test-case mock → team default → write-tool safe default → read passthrough), keyed off test detection (test phone numbers, active_test_calls, explicit isTest). Tools carry a read/write classification in TOOL_CALL_CATEGORY (available-tools.ts).

What hooks in: scenario-driven per-test mocks, the "25 tests, 25 fixture sets" requirement. The interception point exists; what's missing is the fixture-authoring layer on top (generate mock sets from recorded real tool results, attach them to scenarios).

Seam 3 — the CRM HTTP boundary (client logic included) ​

Already prototyped: Mock CRM API. Point the CRM client's base URL at WireMock and the full client logic (retry, idempotency, pagination, fan-out) runs against a deterministic fake. Complements Seam 2: dispatcher mocks bypass the client; HTTP mocks exercise it.

What hooks in: end-to-end flows where client behavior matters, and post-call workflow tests that need the CRM to "exist" statefully. Current limitation: the redirect is hardcoded on a throwaway branch; productionizing it (per-call-context base URL + config short-circuit) is a prerequisite for making this a standing test surface.

Seam 4 — the post-call entry (the blackbox seam) ​

The post-call workflow is already a blackbox by construction: it consumes an end-of-call report payload plus team config, and everything downstream (L0–L7 in the dispatch chain) is a deterministic-ish function of those inputs. The entry points are scheduleEndOfCallReport() and the Inngest function it feeds (EndOfCallReportInngestFunction.ts).

What hooks in: post-call tests that need no call at all. Feed a stored or synthesized end-of-call payload (transcript, toolCalls, metadata) at this seam, run the real workflow against Seam 3's mock CRM, and assert on the writes. A library of real historical payloads becomes a post-call regression corpus for free. This is the highest-leverage under-used seam in the system: it converts "post-call testing requires a phone call" into "post-call testing is a replay."

Seam 5 — the CRM dispatch boundary (proposed, not yet a seam) ​

Where the workflow chooses CRM-specific code. Today this is enum-keyed branching scattered across several sites, not an interface: typedConfig.crm if/else chains in booking-triage.ts (getBookingResult), the per-vertical triagers (run-triage-auto-service.ts), and 14+ per-CRM stage directories under lib/workflow/stages/.

What could hook in: if this boundary becomes a canonical internal CRM interface (per-CRM adapters behind it), it becomes the single assertion point for "the workflow decided to book X at time Y for customer Z," independent of which CRM executes it. That is the subject of Internal CRM abstraction.

Seam 6 — the real-CRM canary (outside our code entirely) ​

A paid test instance of the real CRM, exercised on a schedule: book through the full stack, verify the appointment appears both in the CRM's own UI-facing state and through our API view, then cancel. No mock can catch a CRM whose two views of itself disagree (the Protractor failure class); only this seam can.

What hooks in: post-live scheduled canaries per CRM, and one-time integration certification for a new team's CRM binding.

Seam summary ​

#SeamIsolatesExists todayAnchor
1Blueprint compileconfig → compiled agent, no callcode path exists, no test consumes itmergeMultiAgentConfigWithOverrides, agent-factory.ts
2Tool dispatcherin-call reasoning from real toolsyes (tool-call mocks)resolveToolCallMock()
3CRM HTTPeverything from real CRMprototype (WireMock, throwaway branch)client base-URL substitution
4Post-call entrypost-call from telephony entirelycode path exists, unused for testingscheduleEndOfCallReport / Inngest event
5CRM dispatchworkflow intent from CRM executionno (enum branching, no interface)booking-triage.ts + per-CRM stages
6Real-CRM canaryour assumptions from CRM realitynopaid test instances

Verified observations

Seam anchors and the dispatch-shape characterization come from a code survey of avoca-next main run 2026-07-03 (file paths verified by reading; line-level details not pinned here since they drift). The dispatch scatter is corroborated independently by the post-call dispatch chain doc's L4/L5 analysis.

What this implies for a testing regime ​

  • Tests attach to layers, not just to teams. CRM-layer suites run centrally per CRM; blueprint-layer regression fans out base × overrides; team-layer tests cover the delta and the bindings. A team-level gate that re-tested shared code hundreds of times would be waste; a regime without CRM-layer and blueprint-layer suites has holes no team-level gate can close.
  • The cheap gates live at Seams 1 and 4. Both are synchronous code paths requiring no telephony: compile-and-audit before save, replay-a-payload for post-call. Voice (Hamming through Seams 2/3) is the expensive, high-fidelity tier reserved for go-live and targeted escalation.
  • Seam 5 is the missing keystone. Seams 2, 3, and 6 each mock or observe a specific CRM. A canonical dispatch interface would let post-call tests assert intent once, uniformly, for every CRM, and is what makes the post-call genuinely blackboxable. See Internal CRM abstraction.