feat(adr-0054): harness special messages — file-change diffs + interactive AskUserQuestion

[gsdatta]

Implements ADR 0054 end-to-end — a family of special harness messages: first-class, harness-agnostic events the web renders or interacts with richly, with no per-agent decoder (consumers read structured fields, not agent bytes). Two flavors.

Flavor A — rich file-change diffs (one-way)

Write/Edit/MultiEdit used to render as opaque key/value blobs (the 1 KB args_summary cap even destroyed the data a diff needs). The harness now recognizes those tools in claude's stream-json and, on the successful tool_result, emits a typed FileChanged { path, change } event. The web swaps the generic tool card for a collapsed +N −M row (counts via jsdiff — cheap, no Shiki) that expands to a full Pierre (@pierre/diffs) diff — all-green for Write, red/green hunks for Edit/MultiEdit — lazy-loaded into its own chunk so Shiki's WASM stays out of the main bundle. A failed edit emits no FileChanged, so it keeps its generic error card (never a phantom diff).

Flavor B — interactive AskUserQuestion (round-trip)

Today AskUserQuestion is dead: the CLI auto-denies it ~3 s after the tool_use and the user never sees the question. This wires it up:

• A PreToolUse hook (the harness binary re-invoked as hook-bridge) replaces --dangerously-skip-permissions — ordinary tools auto-allow (explicit + auditable; the VM is still the safety boundary), AskUserQuestion bridges to the harness over a per-session unix socket (ENGRAM_HOOK_SOCK, bind-before-spawn).
• The harness always defers the question — ending the turn so the VM can idle-evict — and emits a durable UserQuestion event. The web renders a UserQuestionCard (radios for single-select, toggles for multi-select) and marks the session awaiting-input.
• The user's answer rides the existing prompt/command channel: AnswerQuestion → ensure_active (auto-resumes the FC snapshot if the session idled) → the harness stashes the answer and re-spawns claude --resume. The deferred tool re-fires id-stable, the hook returns the answer, and the run continues. QuestionAnswered confirms back onto the card.

Why defer/resume, not a held permission

A question is human-paced and engrams is a density service, so pinning a hot VM to wait on a click is the wrong trade. The round trip is grounded in a protocol spike — 13 empirical findings pinned against claude 2.1.181 / 2.1.183 + the first-party Agent SDK 0.3.183 (recorded in the ADR). Decisively: the first-party SDK also continues a deferred tool only via a fresh --resume spawn (canUseTool can't defer; the live control protocol has no verb to continue a deferred tool), so this is the canonical path, not an engram limitation.

Hardening for upstream claude bug #64389

A PreToolUse defer nondeterministically (~1-in-6 locally; any model/tool) runs a continuation inference that poisons claude's transcript — a hallucinated "internal error retrieving your answer", or a duplicate question. Four layered mitigations:

• Part A — suppress the hallucinated narrate-past assistant text/chunks from the UI while an AUQ is pending.
• Part B — if a --resume ends with the answer still unconsumed (claude abandoned the deferred tool), fall back to delivering the answer as a fresh user message and mark the card answered — the question never hangs.
• Part C — scrub_transcript deletes the exact suppressed narrate-past lines from claude's .jsonl (match by message.id, relink parentUuid, atomic temp+rename) so the resume starts byte-equivalent to a clean defer. Fail-safe: any miss falls back to Part B.
• Part D — a session-level "at most one outstanding question" invariant dedups the #64389 double-fire (within- and cross-run): while a card is outstanding every further AUQ defers with no card and is scrubbed from the transcript, leaving exactly one pending question on resume.

Wire & delivery

• New HarnessEvents (UserQuestion=11, QuestionAnswered=12, FileChanged=13) and HarnessCommand::AnswerQuestion=7 are trailing, append-only over the positional bincode wire. wire_golden.rs pins both the bytes and the u32 variant index for each — including a mixed-arity Answers map so a regression to an untagged/deserialize_any encoding fails loudly. Answers = BTreeMap<String, Vec<String>> (no untagged enums — they panic under bincode 1.x).
• Coordinator passes the new events through as JSONB SSE (#[serde(default)], no migration); an old coordinator degrades gracefully (drops the live NOTIFY, replays from PG).
• At-least-once answer delivery: the host's command-replay buffer gained undelivered_answers alongside undelivered_prompts, recorded on answer_question and retired on QuestionAnswered{tool_call_id}.
• Orchestrator authz: AnswerQuestion is owner-scoped, byte-identical to SendPrompt (fail-closed).

Layers touched

engram-harness-proto (wire types + golden) · engram-harness-claude (the bulk — hook server + hook-bridge subcommand + defer/resume engine + narrate-past suppression + transcript scrub) · engram-host-agent (answer ingress + replay) · coordinator (event passthrough + AnswerQuestion ingress reusing ensure_active) · orchestrator (passthrough + authz) · web (FileChangePart/PierreDiff + UserQuestionCard + buildMessages dedup by tool_call_id).

Full design, the 13 findings, and alternatives considered: docs/adr/0054-harness-special-messages.md.

---

🤖 PR description generated with Claude Code

[github-actions]

The latest Buf updates on your PR. Results from workflow CI / buf (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
✅ passed	⏩ skipped	⏩ skipped	✅ passed	Jun 23, 2026, 6:34 PM

[nikhilunni]

Bug: question_outstanding is never cleared on this Part B fallback path → the session-level AUQ dedup leaks and silently swallows every later question.

question_outstanding (the session-level "a card is awaiting an answer" flag) is set back to false in exactly one place — the hook's answer-consumption path (main.rs:567). But this Part B fallback is taken precisely when claude narrate-past's and abandons the deferred tool, so on --resume the hook never fires — line 567 is unreachable on this path. The answer is delivered as a user message and QuestionAnswered is emitted here, but the flag stays true for the rest of the session.

Consequence: after a single narrate-past fallback, every subsequent genuine AskUserQuestion in the session hits the duplicate branch (main.rs:583) → deferred with no card → silently swallowed (and later scrubbed from the transcript). That re-introduces the exact "user never sees the question, Claude gives up" failure this ADR set out to fix. Narrow trigger (needs the ~1/8 narrate-past plus a later question in the same session), but silent when it hits.

One-line fix — clear the flag here (question_outstanding is already a param of run_claude_session), before start_turn so a fresh question asked in response is still carded:

if !stale.is_empty() {
    // ADR 0054: the card is now answered via the fallback; the hook never
    // fired to clear it on this path, so clear the session-level flag here
    // or every later AUQ is wrongly deduped as a duplicate.
    *question_outstanding.lock().await = false;
    if let Some(s) = stdin.as_mut() {
        ...

The invariant becomes "the flag is cleared on every answer-delivery path," not just the hook path.

Test gap: hook_dedups_duplicate_auq_session_wide covers the hook clearing the flag, but answer_resume_unconsumed_falls_back_to_user_message never sets it, so this path is untested. Worth extending that test to assert the flag is false after the fallback (and that a follow-up question re-cards).