Skip to content

feat(placement): ADR 0072 phase 2 — authoritative affinity + retire local_snapshots (#548) - #587

Merged
nikhilunni merged 1 commit into
mainfrom
epic/548-phase2-affinity
Jul 7, 2026
Merged

nikhilunni merged 1 commit into
mainfrom
epic/548-phase2-affinity

Conversation

@nikhilunni

Copy link
Copy Markdown
Contributor

Phase 2 of epic #548 (GCS-free resume), stacked on #586 (phase 1). Move 1 of ADR 0072.

Placement's warm-host choice becomes derived from the durable authority (PG snapshots.host_id) and vetoed only by observed host state — never by a mirror nobody populates.

Tier-0 authoritative affinity (placement.rs)

  • ScheduleContext drops the dead prefer_snapshot_id (× the never-populated local_snapshots), gains snapshot_host: Option<HostId> from SnapshotRecord::host_id.
  • pick_from runs snapshot_host_veto before the soft tiers: an alive, schedulable snapshot-host with disk headroom (free work_dir ≥ the chunk-cache floor) and RAM/CPU fit is picked always — its NVMe already holds the divergent set. Every veto increments engram_resume_affinity_fallback_total{reason} (dead|cordoned|wire_skew|cap|digest_not_ready|disk_full|ram_full|cpu_full) — the fallback is never again silent.
  • The resume path feeds real mem/cpu budgets (best-effort via resolve_cold_boot_spec) so "hard-full" is a real veto, not a vibe.

Retire the dead mirror end-to-end

local_snapshots was Vec::new()-hardcoded in the heartbeat loop for its entire life — a no-op affinity tier. Deleted: the Heartbeat wire field, HostRecord/HostHeartbeat fields, HostLocalSnapshot/LocalSnapshotReport, the coord HTTP DTO + PG upsert/select + row decode, the fleet-view proto count (reserved 7, satisfies buf FIELD_NO_DELETE), the api DTO + CLI display, and the hosts.local_snapshots column (migration 0082).

Notes

  • One disk-floor owner: HOST_DISK_CACHE_FLOOR_BYTES/_MIB move to engram-core; the host-agent floor aliases it (no second threshold, per the issue).
  • WIRE_VERSION 9→10. Coord rolls first; skewed hosts drain off scheduling via host_wire_version_ok until the host MIG rolls — the accepted clean-break deploy window (0 users).
  • Queries are runtime sqlx::query — no .sqlx regen needed.

Testing

Placement table tests: tier-0 authority (authoritative_snapshot_host_wins_when_healthy) + every fallback reason (snapshot_host_vetoes_fall_through_to_soft_tiers). Full workspace compiles incl. --tests; clippy -D warnings + fmt clean; 91 placement/heartbeat/host unit tests green.

Phases 3 (peer-first fill, rides #547) and 4 (evict-time prestage + rung-3 parked-local) follow.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Jul 6, 2026 •

Copy link
Copy Markdown

The latest Buf updates on your PR. Results from workflow CI / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed⏩ skipped⏩ skipped❌ failed (1)Jul 7, 2026, 7:15 AM

@nikhilunni
nikhilunni force-pushed the epic/548-gcs-free-resume branch from be8f416 to 5b310e8 Compare July 6, 2026 13:30
@nikhilunni
nikhilunni force-pushed the epic/548-phase2-affinity branch from d42520d to 264bbcf Compare July 6, 2026 13:30
@nikhilunni
nikhilunni force-pushed the epic/548-gcs-free-resume branch 3 times, most recently from 8bd5563 to b26914c Compare July 7, 2026 06:45
Base automatically changed from epic/548-gcs-free-resume to main July 7, 2026 07:02
…ocal_snapshots (#548)

Rebased onto main past #586/#598. Renumbered at land: migration
0090_drop_hosts_local_snapshots -> 0091 (main took 0090); ADR refs
0075 -> 0078. WIRE_VERSION 10 -> 11 (lockstep coord+host roll; v10
hosts drain from scheduling until the fleet rolls).

Review fixes folded in:
- the TS tier now follows the retirement: buf-generated fleet_pb.ts
  regenerated (web + orchestrator) and the four web consumers of the
  always-zero localSnapshots field removed (the un-regenerated codegen
  would have failed CI's drift check; the Fleet page rendered a dead
  '0 snapshots' datum).
- snapshot_host_veto's cap arm now emits the promised `cap:<name>`
  label (bare "cap" made fc_snapshot_version-vs-bundle-stamp spikes
  undiagnosable from the metric).
- pick_from's tier doc no longer claims capacity-BLIND affinity (tier-0
  is capacity-checked by construction).
- HostUtilization's rustdoc un-interleaved from the disk-floor consts;
  migration comment corrected (said WIRE 9->10; actual 10->11) before
  checksum-freeze; README/DESIGN heartbeat field lists updated; ADR
  gains the possession-freshness divergence note.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jJi2vqAaxt3Q5UKxbe4Gx
@nikhilunni
nikhilunni force-pushed the epic/548-phase2-affinity branch from 264bbcf to 243d2f2 Compare July 7, 2026 07:14
@nikhilunni
nikhilunni marked this pull request as ready for review July 7, 2026 07:14
@nikhilunni
nikhilunni merged commit 76449a1 into main Jul 7, 2026
18 checks passed
@nikhilunni
nikhilunni deleted the epic/548-phase2-affinity branch July 7, 2026 07:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant