Appearance
The team model, and where tests hook in
Why this doc exists. Every testing effort in flight today tests agents. But the unit that delivers value (and the unit that fails in production) is the team: a caller experiences the whole pipeline, from routing to the CRM write, and incidents like Protractor (calendars synced into AutoOps, API didn't, double-bookings) are invisible to any agent-scoped test. This page defines the team as the unit under test and maps every seam where a test suite can hook in. Sister pages: call anatomy (the six-stage lifecycle) and the post-call dispatch chain (L0–L7).
What a team is
A team is a binding, not a thing with much code of its own. It connects three layers:
| Layer | What lives there | Shared across | Where |
|---|---|---|---|
| CRM layer | client library, in-call CRM tools, post-call workflow stages | every team on that CRM | apps/web/lib/<crm>/, apps/web/lib/workflow/stages/booking-<crm>/ (paths per CRM; see internal CRM abstraction for the full inventory) |
| Blueprint layer | the base blueprint (sub-agents, edges, prompts, tool sets) | every team inheriting that base | multi_agent_blueprints |
| Team layer | typedConfig (vertical + crm), the override, agents, phones, credentials, KB, destinations, hours | that team only | assistant_configs, multi_agent_config_overrides, team config tables |
Functionally: a team is a function from an inbound call to outcomes (CRM writes, transfers, messages, emails, call records), parameterized by the binding above. Testing a team means fixing the inputs (a scenario plus world-state fixtures) and asserting on outcomes at chosen observation points. Every test category is a choice of which segment of the function to isolate and which observation points to assert on.
Blast radius: what a change touches
The layering makes change-triggers precise. This is what makes per-team required testing affordable: a team's own test burden shrinks to its delta plus its bindings, because the expensive assurance runs upstream where the shared code lives.
| Change | Blast radius | The test that catches it |
|---|---|---|
| CRM client / post-call stage code | every team on that CRM | CRM-layer suite, run once centrally (mock CRM + real-instance canary) |
| Base blueprint edit | every team inheriting it, including their overrides | base × override matrix regression (the "override drift" case: base edits don't propagate into per-team overrides, so single-team regression cannot see the break) |
| Team override / config edit | that team only | that team's pre-save regression + config audit |
| CRM-side change (their API, their sync) | every team on that CRM, invisible to our diffs | scheduled real-CRM canary (the only seam that would have caught Protractor) |
The seams
Mapped onto the six-stage call anatomy. A seam is a point where a test can inject inputs, substitute a dependency, or observe outputs without touching production systems.
Seam 1 — the compile seam (pre-save, no telephony)
The base blueprint merges with the team override in multi-agent-config-merge.ts (mergeMultiAgentConfigWithOverrides), and the agent factory (agent-factory.ts) turns the merged config into the actual Vapi/ElevenLabs assistant payload. Everything a save is about to change is computable right here, synchronously, with no call: the compiled prompt, the tool set, the edges.
What hooks in: the pre-save gates. Configuration audit (grade the compiled config against the team's stated expectations) and text-replay regression (replay stored scenarios against the compiled prompt with mocked tools, diff the behavior). This seam is also the fix location for the known post-live eval gap: eval currently scores the template-with-variables rather than the compiled prompt, so anything scoring "the prompt" must consume this seam's output, not the template.
Seam 2 — the tool dispatcher (per-test, in-call)
Already built and documented: Tool Call Mocks. Every tool invocation passes through resolveToolCallMock() (test-call-mock.ts) with four-tier resolution (explicit per-test-case mock → team default → write-tool safe default → read passthrough), keyed off test detection (test phone numbers, active_test_calls, explicit isTest). Tools carry a read/write classification in TOOL_CALL_CATEGORY (available-tools.ts).
What hooks in: scenario-driven per-test mocks, the "25 tests, 25 fixture sets" requirement. The interception point exists; what's missing is the fixture-authoring layer on top (generate mock sets from recorded real tool results, attach them to scenarios).
Seam 3 — the CRM HTTP boundary (client logic included)
Already prototyped: Mock CRM API. Point the CRM client's base URL at WireMock and the full client logic (retry, idempotency, pagination, fan-out) runs against a deterministic fake. Complements Seam 2: dispatcher mocks bypass the client; HTTP mocks exercise it.
What hooks in: end-to-end flows where client behavior matters, and post-call workflow tests that need the CRM to "exist" statefully. Current limitation: the redirect is hardcoded on a throwaway branch; productionizing it (per-call-context base URL + config short-circuit) is a prerequisite for making this a standing test surface.
Seam 4 — the post-call entry (the blackbox seam)
The post-call workflow is already a blackbox by construction: it consumes an end-of-call report payload plus team config, and everything downstream (L0–L7 in the dispatch chain) is a deterministic-ish function of those inputs. The entry points are scheduleEndOfCallReport() and the Inngest function it feeds (EndOfCallReportInngestFunction.ts).
What hooks in: post-call tests that need no call at all. Feed a stored or synthesized end-of-call payload (transcript, toolCalls, metadata) at this seam, run the real workflow against Seam 3's mock CRM, and assert on the writes. A library of real historical payloads becomes a post-call regression corpus for free. This is the highest-leverage under-used seam in the system: it converts "post-call testing requires a phone call" into "post-call testing is a replay."
Seam 5 — the CRM dispatch boundary (proposed, not yet a seam)
Where the workflow chooses CRM-specific code. Today this is enum-keyed branching scattered across several sites, not an interface: typedConfig.crm if/else chains in booking-triage.ts (getBookingResult), the per-vertical triagers (run-triage-auto-service.ts), and 14+ per-CRM stage directories under lib/workflow/stages/.
What could hook in: if this boundary becomes a canonical internal CRM interface (per-CRM adapters behind it), it becomes the single assertion point for "the workflow decided to book X at time Y for customer Z," independent of which CRM executes it. That is the subject of Internal CRM abstraction.
Seam 6 — the real-CRM canary (outside our code entirely)
A paid test instance of the real CRM, exercised on a schedule: book through the full stack, verify the appointment appears both in the CRM's own UI-facing state and through our API view, then cancel. No mock can catch a CRM whose two views of itself disagree (the Protractor failure class); only this seam can.
What hooks in: post-live scheduled canaries per CRM, and one-time integration certification for a new team's CRM binding.
Seam summary
| # | Seam | Isolates | Exists today | Anchor |
|---|---|---|---|---|
| 1 | Blueprint compile | config → compiled agent, no call | code path exists, no test consumes it | mergeMultiAgentConfigWithOverrides, agent-factory.ts |
| 2 | Tool dispatcher | in-call reasoning from real tools | yes (tool-call mocks) | resolveToolCallMock() |
| 3 | CRM HTTP | everything from real CRM | prototype (WireMock, throwaway branch) | client base-URL substitution |
| 4 | Post-call entry | post-call from telephony entirely | code path exists, unused for testing | scheduleEndOfCallReport / Inngest event |
| 5 | CRM dispatch | workflow intent from CRM execution | no (enum branching, no interface) | booking-triage.ts + per-CRM stages |
| 6 | Real-CRM canary | our assumptions from CRM reality | no | paid test instances |
Verified observations
Seam anchors and the dispatch-shape characterization come from a code survey of avoca-next main run 2026-07-03 (file paths verified by reading; line-level details not pinned here since they drift). The dispatch scatter is corroborated independently by the post-call dispatch chain doc's L4/L5 analysis.
What this implies for a testing regime
- Tests attach to layers, not just to teams. CRM-layer suites run centrally per CRM; blueprint-layer regression fans out base × overrides; team-layer tests cover the delta and the bindings. A team-level gate that re-tested shared code hundreds of times would be waste; a regime without CRM-layer and blueprint-layer suites has holes no team-level gate can close.
- The cheap gates live at Seams 1 and 4. Both are synchronous code paths requiring no telephony: compile-and-audit before save, replay-a-payload for post-call. Voice (Hamming through Seams 2/3) is the expensive, high-fidelity tier reserved for go-live and targeted escalation.
- Seam 5 is the missing keystone. Seams 2, 3, and 6 each mock or observe a specific CRM. A canonical dispatch interface would let post-call tests assert intent once, uniformly, for every CRM, and is what makes the post-call genuinely blackboxable. See Internal CRM abstraction.