Skip to content

Epic #544: transactional snapshots + RuntimeSpec + one-RPC Revive + reachability GC (ADR 0071, stacked on #583) - #584

Merged
nikhilunni merged 1 commit into
mainfrom
epic/544-transactional-snapshots
Jul 7, 2026
Merged

Epic #544: transactional snapshots + RuntimeSpec + one-RPC Revive + reachability GC (ADR 0071, stacked on #583)#584
nikhilunni merged 1 commit into
mainfrom
epic/544-transactional-snapshots

Conversation

@nikhilunni

@nikhilunni nikhilunni commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Epic #544 — transactional snapshots + durable_head + RuntimeSpec + one-RPC Revive + reachability GC (ADR 0071). Stacked on #583 (→ #581 → #580); merge that chain first. Building the full epic on this branch, phase by phase.

Phases (this PR accumulates all implementable ones)

  • P1 durable_head (landed): sessions.durable_head_snapshot_id advanced in record_snapshot's existing transaction, monotonic by created_at. Row existence == durability substrate.
  • P2 fork-at-attach: disk manifest identity forks at attach, not first flush.
  • P3 RuntimeSpec: a session's boot inputs are one persisted document (create-tx, refreshed at finalize), consumed by create/resume/queue/evac — kills the re-derivation bug class (queued-skills TODO, forge-token FK-ordering, post-resume egress drift).
  • P4 Revive: one epoch-fenced host RPC replaces the resume pipeline; coordinator side is one PG transaction (bind + Idle→Active + events). This is the resume-latency core.
  • P6 reachability GC: one sweeper (roots = durable heads ∪ retained chain ∪ base snapshots) replaces chunk_gc + snapshot_blob_gc + base_snapshot_retention + checkpoint_retention. bundle_gc stays (different root).

Gated, not skipped

  • P5 trust-the-row (drop the 4 GCS HEADs/resume + the recoverable column): flips ONLY after (a) a one-time fleet audit that every recoverable=true snapshot's blobs exist, (b) restore-failure fallback tested, (c) the SLO canary asserts evict→resume per image. Critic gate C7 — the prod-vantage could not verify the recoverable-vs-blobs invariant. Documented in ADR 0071; the fallback + selector land here so the flip is a one-line follow-up once the audit runs.

just check green per phase; live-PG coverage for the durable-head advance; Revive gets an FC-lane integration test wired into ci.yml.

🤖 Generated with Claude Code

https://claude.ai/code/session_014jJi2vqAaxt3Q5UKxbe4Gx

@nikhilunni
nikhilunni force-pushed the epic/544-transactional-snapshots branch from 81b33ab to 8834ac8 Compare July 6, 2026 10:18
@nikhilunni
nikhilunni force-pushed the epic/547-single-writer branch from 3a20ee9 to b5c653d Compare July 6, 2026 13:30
@nikhilunni
nikhilunni force-pushed the epic/544-transactional-snapshots branch from f1705af to 00bd8e9 Compare July 6, 2026 13:30
@nikhilunni
nikhilunni force-pushed the epic/547-single-writer branch 3 times, most recently from 68375d6 to 5bee563 Compare July 7, 2026 03:39
Base automatically changed from epic/547-single-writer to main July 7, 2026 03:54
@nikhilunni
nikhilunni marked this pull request as ready for review July 7, 2026 04:14
@nikhilunni
nikhilunni force-pushed the epic/544-transactional-snapshots branch from 00bd8e9 to 630cf00 Compare July 7, 2026 04:14
@nikhilunni @claude
feat(snapshots): ADR 0077 — transactional snapshots, durable_head, Ru…
3ee7c6e 
…ntimeSpec, Revive (#544)

Rebased onto main past #583/#585/#590-#596. Renumbered at land:
ADR 0074 -> 0077 (main's 0074 = parking ladder); migrations
0087-0089 -> 0088-0090 (main's applied high-water = 0087). The
queue_prompt column references died in the rebase (#592 removed it).

Five pre-merge review findings fixed (see the ADR's divergence log):
- get_session projection: the selected_skills removal left a missing
  comma aliasing live_disk_manifest_version AS park_rung — every
  session read un-parked (frozen-VM-advertised-Active on the rung-2
  ascent) and live_disk_manifest read None. Live-PG round-trip test
  pins the projection now.
- fork-at-attach minted an UNPUBLISHED (uuid, v0) into manifest_ref;
  sessions evicted before their first flush were unevictable and
  zero-dirty captures unresumable. Split: fork_identity is adopted at
  v1 by the FIRST publish; manifest_ref stays the resolvable base
  until then (+ regression test).
- durable_head advanced on recoverable=false rows; now gated, and a
  demote of the current head re-points to the newest recoverable.
- prepare_from_row swallowed RuntimeSpec read errors into "no skills";
  now propagates (scanner/resume retry is the recovery).
- the Idle->Created harness-failed park emitted no StatusChanged and
  swallowed transition failures; now emits + propagates.

Accepted (zero users): no selected_skills backfill into
session_runtime_specs; pre-deploy sessions lose their skill selection
on the next re-prepare.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jJi2vqAaxt3Q5UKxbe4Gx
@nikhilunni
nikhilunni force-pushed the epic/544-transactional-snapshots branch from 630cf00 to 3ee7c6e Compare July 7, 2026 04:33
@nikhilunni
nikhilunni merged commit 6a37340 into main Jul 7, 2026
18 checks passed
@nikhilunni
nikhilunni deleted the epic/544-transactional-snapshots branch July 7, 2026 05:11
nikhilunni added a commit that referenced this pull request Jul 7, 2026
@nikhilunni @claude
feat(chunk-store): ADR 0078 phase 1 — write-through floor (#548)
c0d684e 
GCS-free resume moves 4+5, rebased onto main past #583/#584/#597. Close
the put_chunk exists()-arm write-through hole + add put_chunk_unchecked
to drop the per-dirty-chunk GCS HEAD on flush paths. ADR renumbered
0075 -> 0078 at land (main's 0075 = substrate-single-writer, #583).

Review fix folded in: the flush path's explicit cache.put after
put_chunk_unchecked was a duplicate 16 MiB local write per flushed
chunk (the store's internal write-through already warms the ONE cache —
prod wires it in host-agent main). Cut to one path; the
flush_write_throughs_chunks_into_local_cache test now mirrors the prod
wiring (store carries the cache).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jJi2vqAaxt3Q5UKxbe4Gx
nikhilunni added a commit that referenced this pull request Jul 7, 2026
@nikhilunni @claude
feat(chunk-store): ADR 0078 phase 1 — write-through floor (#548)
8bd5563 
GCS-free resume moves 4+5, rebased onto main past #583/#584/#597. Close
the put_chunk exists()-arm write-through hole + add put_chunk_unchecked
to drop the per-dirty-chunk GCS HEAD on flush paths. ADR renumbered
0075 -> 0078 at land (main's 0075 = substrate-single-writer, #583).

The flush path keeps its explicit cache.put alongside the store-internal
write-through: cache and store travel separately into the disk backend,
so the store carrying a cache is prod wiring, not a structural
guarantee — collapsing to one cache identity is ADR 0076 (substrated)
territory. (A review pass tried cutting it; reverted — the CI-proven
shape stands. Note: eviction_finalize_redrive fails on the DEV-VM for
main and this branch alike — environmental, tracked in the dev-vm
pitfalls; CI is the arbiter for that suite.)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jJi2vqAaxt3Q5UKxbe4Gx
nikhilunni added a commit that referenced this pull request Jul 7, 2026
@nikhilunni @claude
feat(chunk-store): ADR 0078 phase 1 — write-through floor (#548)
b26914c 
GCS-free resume moves 4+5, rebased onto main past #583/#584/#597. Close
the put_chunk exists()-arm write-through hole + add put_chunk_unchecked
to drop the per-dirty-chunk GCS HEAD on flush paths. ADR renumbered
0075 -> 0078 at land (main's 0075 = substrate-single-writer, #583).

The flush path keeps its explicit cache.put alongside the store-internal
write-through: cache and store travel separately into the disk backend,
so the store carrying a cache is prod wiring, not a structural
guarantee — collapsing to one cache identity is ADR 0076 (substrated)
territory. (A review pass tried cutting it; reverted — the CI-proven
shape stands. Note: eviction_finalize_redrive fails on the DEV-VM for
main and this branch alike — environmental, tracked in the dev-vm
pitfalls; CI is the arbiter for that suite.)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jJi2vqAaxt3Q5UKxbe4Gx
nikhilunni added a commit that referenced this pull request Jul 7, 2026
@nikhilunni @claude
feat(chunk-store): ADR 0078 phase 1 — write-through floor (#548) (#586)
b3e458c 
GCS-free resume moves 4+5, rebased onto main past #583/#584/#597. Close
the put_chunk exists()-arm write-through hole + add put_chunk_unchecked
to drop the per-dirty-chunk GCS HEAD on flush paths. ADR renumbered
0075 -> 0078 at land (main's 0075 = substrate-single-writer, #583).

The flush path keeps its explicit cache.put alongside the store-internal
write-through: cache and store travel separately into the disk backend,
so the store carrying a cache is prod wiring, not a structural
guarantee — collapsing to one cache identity is ADR 0076 (substrated)
territory. (A review pass tried cutting it; reverted — the CI-proven
shape stands. Note: eviction_finalize_redrive fails on the DEV-VM for
main and this branch alike — environmental, tracked in the dev-vm
pitfalls; CI is the arbiter for that suite.)


Claude-Session: https://claude.ai/code/session_014jJi2vqAaxt3Q5UKxbe4Gx

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@nikhilunni nikhilunni mentioned this pull request Jul 7, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@nikhilunni