Skip to content

fix(consensus): preserve recovery floors and reject aged-out requests - #4365

Merged
hubcio merged 1 commit into
masterfrom
fix/partition-recovery-and-old-requests
Oct 2, 2026
Merged

hubcio merged 1 commit into
masterfrom
fix/partition-recovery-and-old-requests

Conversation

@hubcio

@hubcio hubcio commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Recovery below the creation view could install an empty primary
behind acknowledged writes. A directory left by an unfinished delete
was loaded as the next partition with the same ids, and a directory
created during recovery could lose its parent entries on power loss.
A Normal backup re-adopted a late duplicate StartView of its own
view and truncated prepares it had acked. Aged-out IDs could receive
false success, and so could writes of an HTTP session whose minted
client id another node or boot had used.

Use the creation view as a floor without certifying the log,
preserve certified WAL history, and ignore a StartView below the
committed prefix or one that a Normal replica's log already holds.
A created.revision file ties a partition directory to its
incarnation. An older incarnation's directory is deleted, and the
partition probes as if it had none. A newer one waits for metadata.
The file is written after the hierarchy is persisted and stays out
of install backups. A per-boot nonce in minted client ids replaces
the reseeded counter. Return RequestTooOld for aged-out IDs, map it
over HTTP and in C#, and stop retries on it, because the outcome is
unknown. In Java, a poll deadline during a routing retry keeps the
non-admission result.

@github-actions github-actions Bot added the S-waiting-on-review PR is waiting on a reviewer label Sep 30, 2026
@codecov

codecov Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.11078% with 66 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.21%. Comparing base (e4ff95b) to head (97e4b0c).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
core/server/src/partition_helpers.rs 96.53% 14 Missing and 7 partials ⚠️
core/partitions/src/partition_storage.rs 91.12% 8 Missing and 7 partials ⚠️
core/consensus/src/observability.rs 70.00% 6 Missing ⚠️
core/simulator/src/lib.rs 97.23% 6 Missing ⚠️
foreign/go/errors/errors_gen.go 50.00% 5 Missing ⚠️
core/sdk/src/clients/producer.rs 96.89% 3 Missing and 1 partial ⚠️
core/consensus/src/impls.rs 98.31% 3 Missing ⚠️
core/partitions/src/install_backup.rs 85.71% 1 Missing and 2 partials ⚠️
core/partitions/src/state_transfer.rs 50.00% 0 Missing and 1 partial ⚠️
...DK/IggyClient/Implementations/HttpMessageStream.cs 80.00% 0 Missing and 1 partial ⚠️
... and 1 more
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #4365      +/-   ##
============================================
- Coverage     87.85%   87.21%   -0.65%     
- Complexity     1577     1578       +1     
============================================
  Files          1290     1289       -1     
  Lines        228844   224404    -4440     
  Branches     192194   187740    -4454     
============================================
- Hits         201049   195710    -5339     
- Misses        23071    23740     +669     
- Partials       4724     4954     +230     
Components Coverage Δ
Rust Core 88.19% <96.45%> (-0.77%) ⬇️
Java SDK 68.72% <91.66%> (-0.01%) ⬇️
C# SDK 77.61% <85.71%> (+0.16%) ⬆️
Python SDK 91.24% <ø> (ø)
PHP SDK 85.67% <ø> (ø)
Node SDK 96.59% <100.00%> (+0.08%) ⬆️
Go SDK 70.29% <50.00%> (-0.01%) ⬇️
Files with missing lines Coverage Δ
core/common/src/error/iggy_error.rs 100.00% <100.00%> (ø)
core/consensus/src/client_table.rs 91.94% <100.00%> (+0.04%) ⬆️
core/consensus/src/lib.rs 0.00% <ø> (ø)
core/message_bus/src/installer/tcp.rs 94.37% <ø> (ø)
core/message_bus/src/lib.rs 97.66% <ø> (ø)
core/partitions/src/iggy_partition.rs 93.01% <100.00%> (+0.03%) ⬆️
core/partitions/src/lib.rs 0.00% <ø> (ø)
core/sdk/src/http/http_client.rs 92.98% <100.00%> (+0.88%) ⬆️
core/sdk/src/vsr.rs 94.47% <100.00%> (+0.28%) ⬆️
core/server/src/boot/mod.rs 87.97% <ø> (+0.81%) ⬆️
... and 32 more

... and 141 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@hubcio
hubcio force-pushed the fix/partition-recovery-and-old-requests branch from c26d002 to 0d5b2b7 Compare October 1, 2026 16:09
Comment thread core/common/src/error/iggy_error.rs
Comment thread core/sdk/src/clients/producer.rs Outdated
Comment thread core/server/src/partition_helpers.rs
Comment thread core/consensus/src/client_table.rs
Comment thread core/consensus/src/client_table.rs Outdated
Comment thread core/shard/src/lib.rs Outdated
Comment thread core/sdk/src/clients/producer.rs Outdated
Comment thread core/consensus/src/impls.rs Outdated
Comment thread core/consensus/src/impls.rs Outdated
Comment thread core/sdk/src/clients/producer.rs Outdated
@github-actions github-actions Bot added S-waiting-on-author PR is waiting on author response and removed S-waiting-on-review PR is waiting on a reviewer labels Oct 1, 2026
@hubcio
hubcio force-pushed the fix/partition-recovery-and-old-requests branch 3 times, most recently from 6cd7d49 to ade6a01 Compare October 2, 2026 09:41
Recovery below the creation view could install an empty primary
behind acknowledged writes. A directory left by an unfinished delete
was loaded as the next partition with the same ids, and a directory
created during recovery could lose its parent entries on power loss.
A Normal backup re-adopted a late duplicate StartView of its own
view and truncated prepares it had acked. Aged-out IDs could receive
false success, and so could writes of an HTTP session whose minted
client id another node or boot had used.

Use the creation view as a floor without certifying the log,
preserve certified WAL history, and ignore a StartView below the
committed prefix or one that a Normal replica's log already holds.
A created.revision file ties a partition directory to its
incarnation. An older incarnation's directory is deleted, and the
partition probes as if it had none. A newer one waits for metadata.
The file is written after the hierarchy is persisted and stays out
of install backups. A per-boot nonce in minted client ids replaces
the reseeded counter. Return RequestTooOld for aged-out IDs, map it
over HTTP and in C#, and stop retries on it, because the outcome is
unknown. In Java, a poll deadline during a routing retry keeps the
non-admission result.
@hubcio
hubcio force-pushed the fix/partition-recovery-and-old-requests branch from ade6a01 to 97e4b0c Compare October 2, 2026 11:24
@hubcio
hubcio merged commit 071121c into master Oct 2, 2026
84 checks passed
@hubcio
hubcio deleted the fix/partition-recovery-and-old-requests branch October 2, 2026 11:45
@github-actions github-actions Bot removed the S-waiting-on-author PR is waiting on author response label Oct 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants