JIT Service Context — how it was built

Implementation notes for an engineer who needs to reproduce, extend, or productionize this. Companion to the findings write-up; Knowledge-Base-side concerns live in .indusk/planning/jit-service-context-poc/kb-technical-notes.md.

All paths are relative to the jit-service-context-poc worktree (branch poc/jit-service-context, never merges). Local Supabase only.

1. Schema

Four migrations under packages/db/migrations/:

File	What
`1785954299583_poc_add_job_service_types.sql`	`job_service_types` — one row per service per team
`1785954299584_poc_add_service_groups.sql`	`service_groups` — named groups per team
`1785954299585_poc_add_service_group_members.sql`	join table
`1785954299586_poc_add_job_service_types_rich_qa_fields.sql`	adds `estimated_duration`, `cost_range`, `prep_instructions`

job_service_types carries the thin fields copied from system_prompt_configs.job_types plus the rich layer: follow_up_questions (jsonb, {question, condition}), conditional_guidance, tech_notes, and the three Q&A columns.

Identity. job_types entries have no stable id, so seed_key is synthesized:

seed_key = lower( (custom_service_type || service_type) + ' ' + (custom_appointment_type || appointment_type) )

with UNIQUE (team_id, seed_key) making re-seeds idempotent. This mirrors jobTypeBookingServiceKeyLower. It is content-derived, so renaming a service orphans its rich row — the main thing to fix before anything real (see KB notes).

Generated types. The POC tables aren't in packages/db/src/generated/supabase.generated.ts. Do not run avoca-dev types to fix that — it regenerates from local schema, which has drifted, and broke 23 unrelated files. Use a narrow createServiceClient() as any at the query boundary, which is the existing repo convention for this situation.

2. The tool

Core: apps/web/lib/tools/service-context/getServiceContextTool.ts

llmArgs { need1, need2?, need3? }  +  injectedArgs { teamId }
  → loadServiceRowsForTeam(teamId), loadGroupMembership(teamId)   [parallel]
  → per-need forced-enum classification over that team's rows
  → computeVerdict(resolvedRows, groups)
  → formatServiceSection(row, i) per matched service
  → { message, data: { needs, resolved, verdict, serviceContext } }

Classification uses generateObjectWithDefaults with a Zod z.enum over the team's service ids — the same forced-enum trick as determineJobTypeFromTranscript, but over KB rows instead of CRM job types. Needs classify concurrently, which is why three needs cost about the same wall-clock as one.

computeVerdict:

same resolved service twice → one appointment
distinct services sharing a service_groups row → may be combined
otherwise → separate appointments (safe default)

Fail-open throughout: an unclassifiable need degrades to "no additional context", never an error.

Params are flat scalars on purpose. ElevenLabs tool parameters cannot be arrays or objects, hence need1/need2/need3 rather than needs: string[]. need1 is required.

Registration (three places, per the repo's "adding a voice tool" guide):

Layer	File
Catalog	`lib/voice-assistants/agents/tool-catalog/available-tools.ts` → `GET_SERVICE_CONTEXT: 'getServiceContext'`
FunctionTool	`.../tool-catalog/service-titan/service-titan-tools.ts` → schema + `.withElevenLabsToolId('tool_7601kz9zajg7fy9vgwrwd93cb65n')`
EL handler	`lib/elevenlabs/tool-handlers/getServiceContext.ts`
VAPI handler	`lib/vapi/tools/handlers/getServiceContext.ts`

Both handlers are thin adapters over the one core.

The EL tool id tool_7601kz9zajg7fy9vgwrwd93cb65n was minted against the live ElevenLabs account. Delete it at teardown.

Attachment is by prompt mention. The agent factory attaches prompt-referenced tools — an @tool:getServiceContext mention in a prompt variable is what puts the tool on the agent. Remove the mention and the tool is not registered at all. This is how the FAT (non-JIT) A/B configs are built: they simply don't override execution_model, so they inherit a blueprint default with no mention.

3. Prompt thinning methodology

The part worth reusing. Start from the compiled prompt, not the source variables — service logic is scattered across blueprint modules and you cannot find it reliably by reading source.

Compile the baseline. apps/web/scripts/dump-compiled-prompt.ts <configId> → createAgentFromAssistantConfigId → validateLiquidTemplatesOrThrow() → getElevenLabsAgentConfig() → conversationConfig.agent.prompt.prompt. Requires _stub-server-only + _load-env imports first, NODE_OPTIONS="--require ./scripts/_mock-otel.cjs", and main().then(() => process.exit(0)) — imported modules hold the event loop open otherwise.
Mark spans in the rendered output — what it is, where its content goes, what replaces it. Recorded in evidence/span-map.md (human review) and evidence/span-map.json (machine).
Trace each span back to its source variable and author a config-scope assistant_variables override. Never edit blueprint-scope rows — they're shared.
Recompile and diff-gate. evidence/prompt-diff.mjs enforces the invariant:
```
thinned == baseline − marked spans + declared replacements,   and nothing else
```
1
Self-testable against a projection (thinned-prompt-projected.txt) before touching the live config. This is what stops accidental collateral edits.
One normalization: the render-time "Current Day of the Week / Date / Time" line differs on every dump — environment, not content.

Gotcha that cost real time: a config-scope override whose type doesn't match the blueprint-scope variable's type is silently inert — it compiles to the blueprint default with no error. The blueprint rows use type = 'STRING'; inserting 'text' produced a smaller compiled prompt than the unthinned one and no warning. Always copy the type from the blueprint-scope row.

Scripts: poc-seed-job-service-types.ts (seed from jsonb), poc-fill-rich-columns.ts (author rich content), poc-apply-thinned-overrides.ts (upsert overrides from a directory of .txt files named after variables).

4. The A/B harness

Four configs on one voice assistant, so the standing test-config override (phone_numbers.assistant_config_id, the FlaskConical admin toggle) can switch between them. Same voice assistant is required — the server-side guardrail in app/api/admin/voice-assistants/_actions.ts rejects cross-assistant pinning. Do not bypass it with direct SQL.

	JIT (tool attached)	FAT (all in prompt)
ElevenLabs	`c0000000…0100`	`c0000000…0200`
VAPI	`3d8b0f9d…`	`c0000000…0300`

Fairness control. scripts/poc-build-mega-fat-config.ts generates the FAT side's prompt block using the tool's own formatServiceSection, over the same job_service_types rows. The two sides therefore carry byte-comparable content and differ only in delivery. It also copies every override from the JIT config except execution_model (which holds the tool mention), so platform-required settings like VAPI's end_call_enabled are preserved.

Args: <limit> <sourceConfigId> <targetConfigId> <name> — the limit sweeps catalog size for the curve.

Keep comparisons within a platform. EL and VAPI compile materially different prompts (the factory injects platform-specific speaking-style and personality blocks), so EL-vs-VAPI sizes are not comparable. JIT-vs-FAT within a platform is.

5. Measurement

Prompt-size curve (evidence/prompt-size-curve.md): seed a large catalog (poc-seed-mega-catalog.ts, 25 families × 3 appointment types), then for each N regenerate the FAT block at limit = N and compile both sides. JIT needs compiling once — it does not vary with catalog size, which is the finding.

Latency (evidence/latency.md): poc-measure-service-context-latency.ts calls the core directly across two teams and 1–3 needs, reporting avg/min/max and p50/p90. Excludes the webhook hop. Compare against real tool_calls durations for the tools already in the flow.

6. Demo / live-call harness

Three things had to be built before a live call was demonstrable:

Tool-call persistence. The EL dispatch path (pages/api/elevenlabs/tools/dispatch.ts) only writes tool_calls rows for mocked responses. A real getServiceContext invocation left no trace. persistServiceContextCall in the tool wrapper writes a SUCCESS row via upsertToolCall (best-effort, .catch(() => {}), never blocks the response). This is what makes the tool visible in the existing call-review UI and to the watcher.

A mock tier for manual dials. Both existing mock tiers in lib/tools/test-call-mock.ts require a testCaseId resolved from active_test_calls, which only exists during a Hamming run. A human dialling the test number gets null and read tools execute live — useless for a team with no CRM. POC_TEST_PHONE_MOCKS fires only when isTestPhoneTraffic && !testCaseId, for an explicit tool list (findCustomer, findCustomerDNS, checkZipCodeAreas, checkZipCode, getBookingInfo). It cannot affect production (needs the test-phone marker) or Hamming runs (they always have a test case).

Note getBookingInfo is not in TOOL_CALL_CATEGORY, so it defaults to 'read' and runs live. Write tools are already auto-blocked on test traffic, but with FALLBACK_WRITE_RESPONSE ("This tool is not available during test calls") — a refusal, not a confirmation.

Live watcher. poc-watch-service-context.ts polls tool_calls and prints each invocation — caller's words, resolved services, verdict, latency. Starts from launch time so the screen is clean for a demo.

7. Findings that are really implementation lessons

A JIT tool competes with any static instruction that answers the same question. The biggest failure in testing: on ElevenLabs the agent narrated "let me see what questions I need to ask" and then didn't call the tool. Root cause was not EL reliability — the Mega config only overrode execution_model, so the blueprint's default intake module still said "ask how old their system is" and "ask how many units they have." The agent already knew what to ask, so the tool was redundant. Copying the thinned intake module over removed the alternative source.

Thinning must remove the competing instruction, not merely add a trigger. Adding the tool while leaving static equivalents in place produces silent degradation: the agent improvises plausible questions and completes the call, and nothing looks wrong unless you diff the questions against the record.

Make the tool a precondition, not a suggestion. The trigger was hardened to require the call before any service-specific question and before offering availability, and to place it immediately after customer lookup rather than before intent classification — agents want to identify the caller first, and fighting that instinct loses.

Catalog size degrades classification. With 79 synthetic services, "boiler issue" resolved to a generic Boiler Repair and an unrelated Air Conditioner Repair. Large catalogs need disambiguation (keyword prefilter or family grouping) before classification.

avoca-dev duplicate-team is a RESET, not a merge. Re-running it against a live POC team wipes the scratch config, its overrides, and the phone row. Recovery is manual.

8. Teardown

Delete the ElevenLabs tool id tool_7601kz9zajg7fy9vgwrwd93cb65n
DELETE FROM teams WHERE id = 999901; (cascades)
Drop job_service_types, service_groups, service_group_members, job_service_types_parked
Remove the scratch config 89751381-8151-4319-bd43-5362911961f1 and its overrides from team 3082
Deprovision the Twilio ISV subaccount for 999901 and its two numbers
The branch is never merged, so code teardown is just deleting the worktree

JIT Service Context — how it was built ​

1. Schema ​

2. The tool ​

3. Prompt thinning methodology ​

4. The A/B harness ​

5. Measurement ​

6. Demo / live-call harness ​

7. Findings that are really implementation lessons ​

8. Teardown ​