[mickvandijke]

Native chunk reads now run their own iterative lookup, and ask each peer they query for the chunk as well as for closer peers. Browsers keep today's read. This implements ant-node ADR-0020.

How it works.

• The lookup runs on saorsa-core's public lookup engine, with the same alpha, grace period and deadline as saorsa-core's own lookup.
• Every queried peer answers the lookup at once. Only peers that probably hold the chunk are asked for it, because each one asked sends the whole chunk.
  • A likely holder is a peer with fewer than a close group of known peers nearer the chunk. Until the read knows 14 peers, as in its first round, every peer counts as one.
  • The likely holder chosen to go next is asked for the chunk and for closer peers in one step. A peer announcing get-or-closer/1 gets one get-or-closer request; any other peer gets FIND_NODE and a GET at once, in the same round trip. Every node in service today takes that second route.
  • Other likely holders answer the lookup with FIND_NODE straight away, and wait for a GET. One the lookup has since learnt is not among the nearest is skipped.
  • Peers far from the chunk get FIND_NODE only, as in saorsa-core's own lookup.
• Likely holders are asked one at a time, the first chosen uniformly among them, as ADR-0023's ordinary read chooses its first GET. One that answers without the chunk hands over at once; one still sending after 500 ms is hedged. At most two are asked at once, as the ordinary read never runs more than two GETs.
• saorsa-core checks the peers in a Closer answer as a FIND_NODE answer, and rejects a payload over the 64 KiB FIND_NODE limit before parsing it. A payload that fails any check fails only that peer's step.
• The read stops at the first verified chunk.
• A peer whose content fails verification only drops out of the read. That covers a wrong address or a hash mismatch. In its first round any peer is a likely holder, so ending the read on one bad answer would let a dishonest node on a lookup path fail downloads. The client logs a warning naming the peer.
• Requests belong to the read, not to the lookup round. A round's grace period therefore never cancels a chunk that is still arriving.
• If the lookup finds no holder, or fails, the existing retrieve_progressive path runs, with its absence rule, retry round and rule that bad content ends the read. Its first round starts from the peers the lookup ended on, so reading a chunk that is not stored, as manifest builds check, costs one lookup.
• Browsers keep today's read. A browser variant was built and A/B'd on the dev testnet, and was not faster there (see Test evidence), so it is not part of this PR. Browser read tuning is in a separate PR, perf(browser): fail dead dials in 3 s and use the whole read budget #222.

Review fixes (2026-10-27). A review found the read could multiply copies and slow down:

• A slow holder was hedged every 500 ms without a limit, so a link slower than one chunk per 500 ms could carry three or four copies at once. Now at most two.
• The holder gate never applied in the first lookup round, so up to three seeds sent the whole chunk at once wherever the client already knew the close group. Counting seeds as holders would have made that round wait for the gate, as later rounds already did. So routing no longer waits for the gate at all: waiting holders answer with FIND_NODE, and far peers are no longer sent chunk requests.
• A non-retryable kad error ended the read instead of falling back (latent: every adapter error was retryable).
• A read of a missing chunk ran two lookups.
• Rebased onto main again: tokio 1.53.3, then perf(browser): fail dead dials in 3 s and use the whole read budget #222's browser read tuning and ci(release): fall back to generated notes, and add the changelog rotation script #237. Main's GET helper now takes a dispatch notice for the ordinary read's hedge timer; the kad read's own GETs leave it unobserved, as main's other non-hedged GETs do.

Code layout.

• client_engine::kad_read: the policy, transport-neutral and tested on a scripted network.
• data::client::kad_get: the adapter, over saorsa-core and the chunk protocol. It shares verified_chunk with the plain GET.
• Test harness: in-process nodes announce the token, so the e2e suite reads through get-or-closer. MiniTestnet::start_without_get_or_closer starts nodes that do not, for the legacy route. The read counters no longer decode PUTs.

Measurements. See the investigation: sequential reads on production, native.

	p50	p90	Whole 612 MB file
Before	5.95 s	13.0 s	199–205 s
After	~1 s	~2 s	49–55 s

Those numbers predate the likely-holder gate. Without it, this read fetched 2.5–3.3 copies of a chunk on production, a trade-off the ADR accepted; the gate trades a little latency for fewer copies (see Test evidence).

Stack, merge last:

1. feat(dht): expose lookup steps for caller-driven lookups saorsa-core#173
2. feat(chunk): add a get-or-closer request ant-protocol#44
3. feat(storage): answer get-or-closer requests (ADR-0020) ant-node#249
4. this PR

This PR pins all three by git rev: saorsa-core c3c4c81, ant-protocol 0c6ae27 and ant-node 6a14b53. It cannot be published as is. Before merging, release saorsa-core#173 and ant-protocol#44, merge and release ant-node#249, and replace the [patch.crates-io] pins with version bumps.

Linear issue

Closes V2-1358

Risk tier

• T0 — docs / tooling / CI / pure UX-output. Repo CI only.
• T1 — client-only, no network-facing behavior change. CI + prod compat smoke.
• T2 — node/client logic with behavioral surface, no protocol/format/economics change. Dev testnet + ADR.
• T3 — protocol / storage format / payments / routing. T2 evidence + adversarial testing.

Compatibility

• Wire: sends the new GetOrCloserRequest only to peers whose user agent carries get-or-closer/1. Every other peer gets FIND_NODE and GET, both existing messages.
• Storage: none.
• Wire, nodes: peers that only route get FIND_NODE, as in saorsa-core's own lookup, so they answer on the DHT path rather than in the chunk handler, where get-or-closer would queue behind PUTs.
• API: additive. #[doc(hidden)] Client::chunk_get_kad_only, behind test-utils.

Semver impact

• breaking
• feature
• fix

Test evidence

• cargo test --lib --all: 892 passed. kad_read has 19 tests on a scripted network. These are new since the review, and each fails with its mechanism removed:

  • at most two likely holders are asked at once
  • likely holders waiting their turn answer the lookup at once
  • a waiting peer the lookup has passed is not asked for the chunk
  • peers far from the chunk are only asked for closer peers
  • a lookup without a holder hands back the peers it ended on

• CI's e2e list (cargo test -p ant-core --features test-utils --test e2e_chunk --test e2e_data --test e2e_file --test e2e_payment --test e2e_native_payment --test e2e_security --test e2e_cost_estimate --test e2e_adr0004 --test e2e_manifest -- --test-threads=1): 69 passed. That includes the new test_chunk_kad_read_uses_get_on_older_nodes: nodes without the token are read with FIND_NODE and GET, and sent no get-or-closer.

• cargo fmt --check, cargo clippy --all-targets --all-features -- -D warnings, and the browser WASM cargo clippy job: clean.

• Dev testnet comparison against main, 2026-10-27, after the review fixes. Two 60-node DigitalOcean fleets with the same layout:

  • 30% symmetric NAT across 9 regions
  • per fleet, 4 uploaders (50 MiB; 2 merkle, 2 no-merkle), 2 native downloaders and 1 headless-Chromium WASM downloader
  • 60-minute windows
  • both fleets on main as of 5e01699 (before perf(browser): fail dead dials in 3 s and use the whole read budget #222 and ci(release): fall back to generated notes, and add the changelog rotation script #237 landed), differing only by the four PRs

  Three runs:

  1. closed loop, as staging runs its clients
  2. the same, with the code swapped between the fleets' droplets
  3. the swapped fleets, with native downloads paced to one every 40 s, so both fleets serve the same reads

  stack vs main	run 1	run 2, swapped	run 3, swapped, matched load
  native download p50	−52% (19.7 → 9.4 s)	−47% (18.5 → 9.9 s)	−43% (18.4 → 10.5 s)
  native downloads completed	733 vs 350	707 vs 372	177 vs 176
  merkle upload p50	+7.0% (p < 0.001)	+1.7% (p = 0.36)	−6.2% (p = 0.001)
  no-merkle upload p50	−0.2% (p = 0.90)	+8.6% (p = 0.001)	+0.9% (p = 0.72)
  WASM download p50	−2.0% (p = 0.43)	−4.2% (p = 0.10)	−2.8% (p = 0.13)
  node dial failures, minutes 10–60 (run 3: whole window)	21 vs 0	0 vs 12	0 vs 2
  failed transfers	0	0	0

  • Native downloads are 43–52% faster on whichever droplets run the stack.
  • At equal read load nothing regresses. In a closed loop, the stack's fleet serves about twice the reads, and uploads beside them run about 4% slower. Staging runs its clients closed loop, so it will see that.
  • Node dial and connection failures follow one fleet's droplets, not the code. Every run's steady-state tail sits on the same droplets (B), whichever code they run. saorsa-core#173 changes no node-side DHT path.
  • Duplicate copies: nodes sent 1.45–1.52 bytes per native byte downloaded, against 1.31–1.35 for main. Before the review fixes it was 1.54–1.68. The rest is the 500 ms hedge.
  • Reports: ant-testnet/state/comparisons/v2-1358-review-fixes-{vs-main,swapped,swapped-matched}-60m-1027/README.md.

Earlier evidence, from before the review fixes:

• Live, against today's production nodes, which do not announce the token. ant file download of a 612 MB file took 56 s, against 242 s for the current client. The output was byte-identical, and all 150 chunks came through the new lookup's FIND_NODE-plus-GET route.

• Likely-holder gate, A/B on the same kad fleet. The client binary was swapped on two downloader droplets (syd1, fra1), and arms alternated over 50 MiB files not read before, with 5–6 downloads per arm in each cell. Each cell gives the change against asking every likely holder at once, as download time p50 and then bytes received:

  hedge	syd1, fetch cap 4	fra1, cap 4	syd1, cap 16	fra1, cap 16
  1 s	+13%, −36%	−2%, −33%	+15%, −28%	+18%, −16%
  500 ms (this PR)	+9%, −14%	−2%, −22%	+3%, −17%	+8%, −24%

  On these datacenter links bandwidth is not the bottleneck, so the gate shows its latency cost and not its benefit. Client CPU per download fell 10–17%.

• Browser variant, tried and dropped (2026-10-12). It ran this read driver over WebRTC Direct, with nodes answering the request inside chunk_protocol (a missing chunk got the node's browser find_node reply). The kad fleet's nodes were swapped in place to that build, keeping their data. Headless Chromium on two downloaders, 50 MiB private files, a fresh client per download. That WASM build was interleaved with this PR's, which reads as today, over 8 runs. Pairs are by round; the intervals resample whole runs:

  region	pairs	variant / today, mean	delta	faster in
  fra1	84	16.6 / 14.1 s	+17% (−1..+43%)	30 of 84
  syd1	85	36.3 / 38.2 s	−5% (−14..+6%)	45 of 85

  • Why, from traced reads. On 60 nodes, a warm browser already holds sessions with the close group, so today's read sends its first GET straight to a holder. The variant cannot shorten that. Its first round asks the 2–4 recent routes, which often do not hold the chunk. The holder is then asked a round later, after a dial and a wait for one of the browser's 4 read slots: about 0.5 s later per chunk on fra1. Its delivering request is faster, which offsets that only where transfers are long (syd1).
  • Seven variants, none committed, changed what the first round asks, the hedge, and which steps hold read slots. None beat the plain variant on fra1.
  • Full report: ant-testnet/state/comparisons/v2-1358-kad-vs-main-matched-60m-1009/README.md, sections "Browser get-or-closer reads" and "Why the browser read is not faster here".

• Browser (WASM) downloads are unchanged by this PR. Headless Chromium on the testnet's downloader droplets, 50 MiB private files, a fresh client per download, with main's and this PR's WASM builds interleaved:

  • Interleaved, 8 downloads each: fra1 17.4 s against 17.1 s p50, syd1 31.4 s against 30.8 s.
  • Pooled over every run, with 32–38 downloads per build per region, the means were within 2%.
  • Both builds took the same time against main's nodes and against this stack's nodes.
  • For WASM, those builds differed only in ant-protocol's two added enum variants.

New dependency

none

ADR

ADR-0020: Kademlia-style chunk GET: return the chunk or closer peers (ant-node). Measurements are in the investigation.

Mitigation / rollback

Client-only with no stored-format change: revert this PR. When the new lookup finds nothing, the existing read path still runs.

🤖 Generated with Claude Code

[dirvine]

Changes requested — consume a size-bounded closer-peer decoder

ant-core/src/data/client/kad_get.rs:106–115 passes untrusted peers directly to the new core decoder. The exact core revision pinned at root Cargo.toml:11 lacks the 65,536-byte pre-deserialisation limit; the outer chunk message permits up to 5 MiB. A local exact-head probe accepted 1,050,035 bytes containing 10,000 nodes. No crash or memory exhaustion was demonstrated.

This is the downstream exposure of the same P2 finding in saorsa-core#173, not a separate read-engine bug. Fix the public decoder in core, update this PR's core pin and lockfile (and node pin as the stack advances), then rerun focused and real-transport E2E tests. A retryable client-local size guard can mitigate this caller, but the public decoder still needs its own invariant.

No other introduced blocker was established in the read engine. Legacy fallback, content verification and native/browser separation were reviewed.

Reviewed commit: 7bc58ccb70081b8c5c911cc87a821108d8aadcf4.

Review method: six substantive seats across the linked set, including a completed z-ai/glm-5.2 API review recovered from a worker that timed out after saving its response; coordinator source checks and local execution. Initial functional/security/test seats recommended approval. The coordinator probe, independent adjudicator, API seat and replacement concurrency/release seats corroborate the missing decoder bound. This is not unanimous approval. The API seat recommended client approval with concerns; the coordinator withholds it because the exact client pin still exposes the defect.

Local verification (macOS, rustc 1.96.0): core closer-peers 3 passed and DHT-manager regression 143 passed; protocol all-features 108 passed; node get-or-closer unit tests 3 passed and real-transport E2E 1 passed; client kad_read unit tests 10 passed and real-transport E2E 1 passed. Formatting passed in all four repositories. No production/testnet deployment or merge performed. Draft git pins were verified with cargo metadata --locked; this is not published dependency closure. ADR-0020 remains Proposed.

[mickvandijke]

Thanks. This is addressed in two commits.

25d7c5a: bounded decoder.

• Pins saorsa-core fd37775 (the bounded decoder) and ant-node ad88647.
• A payload the decoder rejects surfaces as Error::Protocol. The kad read treats that as retryable, so only that peer's lookup step fails.
• I didn't add a separate client-side size check, since every closer-peers payload goes through the core decoder.

7d9a248: bad content drops only that peer. Tracing the rest of the answer path turned up a larger exposure in the same request.

• A Found that failed verification (wrong address or hash mismatch) raised InvalidData.
• That ended the read, and the ? in chunk_get_from_closest_peers skipped the fallback. The file download doesn't retry InvalidData either.
• Since get-or-closer goes to every queried peer, any dishonest node on a lookup path could fail downloads.
• Now such a peer only drops out of the lookup, and the client logs a warning naming it.
• If no other peer sends the chunk, the ordinary read runs, with its existing rule that bad content ends the read.
• New tests cover a corrupt peer alongside an honest holder, and a corrupt only-holder. Both fail under the old policy.
• ADR-0020 is updated to match (ant-node#249, 900ff66).

Checks:

• cargo test -p ant-core --lib: 822 passed, with the kad tests run 25 times clean.
• The real-transport e2e test_chunk_get_or_closer_reads_from_upgraded_nodes passed.
• Native, test-utils and WASM clippy are clean.
• CI on 25d7c5a was green (18/18). CI on 7d9a248 is running.

[dirvine]

Re-review — blockers resolved

Fixes verified

1. Core pin: saorsa-core now fd377754 (the bounded closer-peer decoder) in Cargo.toml and Cargo.lock. The first-round blocker (client passing untrusted peers to an unbounded decoder) is resolved — the decoder now rejects payloads over 64 KiB before postcard parsing.
2. ant-node pin: now ad886472 (ADR-0020 branch, carries the fixed core pin — see the coordination note on ant-node#249 about mid-branch vs head).
3. New: bad-content peers are dropped, read continues. is_retryable (kad_get.rs:155–160) now includes Error::InvalidData; note_bad_content logs and passes it through. In kad_read, a retryable failure drops the peer (Ok(None)); non-retryable local errors (e.g. Config) still abort. Tests: corrupt holder + honest slower holder → chunk found; corrupt-only holder → Ok(None) → fallback to ordinary read; adapter-rejected error ends the read.

Local verification at head 7d9a248b

• client-engine 12/12, adapter 1/1, e2e 1/1 ok (macOS). CI: E2E ubuntu/macos, Merkle E2E ubuntu/macos, unit tests, builds all green.
• CI red check: Security Audit fails — cargo audit reports 19 vulnerable dependencies, all inherited from the pinned saorsa-core git rev (e.g. tokio-rustls 0.24.1 RUSTSEC-2024-0336/0337, serde 1.0.228 RUSTSEC-2025-0049). Not introduced by this PR's code; resolves when the core rev is updated. Flagging for the core maintainer.

Verdict

APPROVE at head 7d9a248b, subject to: (a) the ant-node pin reconciliation (see #249), (b) the core cargo audit failures being tracked in the core rev. Panel: GLM 5.3 seat billing-failed (402) — 2/3 reviewed, provisional.