Skip to content

perf(browser): fail dead dials in 3 s and use the whole read budget - #222

Merged
mickvandijke merged 15 commits into
mainfrom
mickvandijke/v2-1358-browser-read-tuning
Oct 27, 2026
Merged

mickvandijke merged 15 commits into
mainfrom
mickvandijke/v2-1358-browser-read-tuning

Conversation

@mickvandijke

@mickvandijke mickvandijke commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

Browser downloads from mainnet spent most of their time dialling, and left half the browser's read budget idle. Against main, a 64 MiB public file downloads in 21.6 s at the median instead of 37.6 s in Chromium, and 25.5 s instead of 31.5 s in Safari 26.5.2, faster in every round. A 612 MB video took 195 and 270 s in Chromium against main's 398 and 323 s. I found this while checking browser downloads for V2-1358, whose get-or-closer read (#221) is native only.

What was slow. These come from traced headless-Chromium downloads from mainnet, and earlier from a 60-node testnet.

  • About half the endpoints a browser dials never answer ICE. In traced downloads of the 64 MiB file, 42–50% of fresh dials went to WebRTC Direct endpoints that never answered. Each held a pool slot, and kept its lookup round waiting, until the 10 s channel-opening timeout: 18–30 of the 64 slots at a time. Live nodes answered ICE in 110 ms at the median; the slowest of 788 took 2.7 s. A file whose DataMap has child records reads no data until the slowest of them arrives, and one 418-byte record took 10 s.
  • Chromium dials queued for the page's ICE gathering turn. The turn exists for WebKit's socket cap (fix(browser): gather ICE candidates one connection at a time #216). A dial waited 0.6 s at the median at a fetch concurrency of 4, and 1.7 s at 8, before sending its offer. ICE, DTLS, SCTP and the PQ handshake together took about 0.4 s.
  • The browser admits at most read_limit() physical GETs. That is the fetch controller's cap, which starts at the native cold start of 4. Nothing is persisted between browser sessions, and for small files the controller never grows. The read budget itself allows 8: 128 MiB over a 15.2 MiB reservation per GET.
  • The shared read engine hedges an early GET after 1 s. A 4 MiB record takes 1–3 s over a WebRTC DataChannel, so the hedge raced nearly every chunk. The duplicate keeps its read slot until its response has drained.

On its own, starting at 8 GETs made mainnet downloads 15–26% slower: more lookups ran at once and filled the pool with dead dials. The first two changes below remove that in Chromium, where a fetch start of 8 then beats 4 in every round. WebKit keeps its gathering turn, so there more concurrent reads still queue more dials, and Safari keeps the native start of 4.

What changes.

  • A dial fails when ICE has not connected 3 s after its offer. A fresh connection whose iceConnectionState is still new or checking then fails, and enters the dial-failure cache like any other failed dial. Once ICE has connected, DTLS, SCTP and the channel keep the opening timeout. It races alongside main's failure of a dead association, so a node refusing for capacity (ant-node ADR-0023) is still detected at once and marked Refused. A browser without iceConnectionState keeps the old behaviour.
  • Chromium dials gather ICE candidates at once. Only Chromium exposes navigator.userAgentData, so a page that has it skips the gathering turn, unless it reports WebKit's navigator.vendor: a Client Hints polyfill in Safari must not lift the turn, since overlapping gatherings there cut established connections (fix(browser): gather ICE candidates one connection at a time #216). WebKit and Firefox keep it; Firefox was not measured.
  • Chromium clients start fetching at the read budget's capacity, 8 GETs, and no browser probes past it. The shared client's controller is built with that fetch start and ceiling (Client::from_network_with_start). Engines whose dials take turns to gather ICE, WebKit and Firefox, keep the native start of 4, and the controller can still grow to 8. Quote and store keep their defaults.
  • The transport sets the hedge delay: 3 s on WASM and 1 s native, through BrowserNetwork::read_hedge_delay. The shared read engine has no target_arch branch.
  • An early GET's hedge delay runs from when it is dispatched to its peer. A browser GET reserves the read budget before anything of its peer's, and is dispatched there, reported through a RequestDispatched notice. Waiting for the budget never makes a peer look stalled; waiting on the peer does: a redial, its session slots, the send and the reply. No GET holds a peer's session slot while it waits for the budget. Native GETs are dispatched at once.
  • The same hedge rule holds after discovery, on native too. When discovery finishes, an ordinary GET no longer starts beside an early GET still within its delay, and a raced GET must stall too before another candidate joins it. At most two GETs run, as before. In those two cases a native read now waits up to the 1 s hedge delay, where before it started at once.
  • Each node session bounds read-ahead. A session admits speculative GETs within half its slots (2 of 4), never ahead of a request queued for one, using the read budget's share. A GET that a read comes to wait for is promoted while it queues, so a seek's GET never waits behind read-ahead.
  • send_and_await_chunk_response takes a ChunkRequest options struct, which carries the dispatch notice, and retrieve_progressive takes an EarlyReads policy.
  • The branch merges current main: retrieve_progressive keeps main's holder spreading and its wait for refusing holders (feat(read): spread hot-record reads across the close group, and back off briefly from nodes that decline #225) alongside the policy above.

ADR-0004 gains a dated section with these measurements.

Linear issue

Closes V2-1483

Risk tier

  • T0 — docs / tooling / CI / pure UX-output. Repo CI only.
  • T1 — client-only, no network-facing behavior change. CI + prod compat smoke.
  • T2 — node/client logic with behavioral surface, no protocol/format/economics change. Dev testnet + ADR.
  • T3 — protocol / storage format / payments / routing. T2 evidence + adversarial testing.

T1 proposed: there is no wire, format or node change. Reviewers should note:

  • A Chromium client now starts with up to 8 GETs in flight instead of 4, and read-ahead may hold up to 4 of them (at most 2 per node). Other engines start at 4 as before. That is within what native clients already send, and within nodes' per-session multiplex limit of 4.
  • A browser gives up on an endpoint that has not answered ICE after 3 s instead of 10 s, and bars it for the usual 30 minutes. It opens more connections per download (243 instead of 187 for the 64 MiB file), and Chromium no longer serializes their ICE gathering.
  • Native reads keep their 1 s hedge, but follow the hedge rule after discovery and after a race described above. Native download time did not change measurably (see test evidence).

Compatibility

  • Wire: none. The same requests, at up to 8 in flight per browser client instead of 4.
  • Storage: none.
  • API: additive. data::network::RequestDispatched is a new public type in every build. BrowserNetwork (browser builds) gains request_notifying_dispatched and read_hedge_delay, both with default implementations, so existing adapters compile unchanged.

Semver impact

  • breaking
  • feature
  • fix

Test evidence

On 1cd6d99, which merges main ccd4106.

  • Mainnet A/B, same head (2026-10-26). downloadPublicFile from mainnet in local headless Chromium on macOS, with a fresh context and client per download and the arms in rotating order. Bytes received are WebRTC transport bytes from getStats. All downloads succeeded and were byte-identical.

    64 MiB public file (f6eeba94…), 5 rounds:

    main ccd4106 this PR, fetch start 4 this PR
    download p50 / mean 37.6 / 37.5 s 28.9 / 29.1 s 21.6 / 22.6 s
    vs main, paired mean −23% (4/5 faster) −40% (5/5 faster)
    bytes received p50 107.8 MiB 96.9 MiB 100.8 MiB
    peer connections p50 187 189 243

    The fetch start of 8 beat 4 in every round. 612 MB video (134e4537…, 147 records), 2 rounds: 195 and 270 s, against main's 398 and 323 s. These ran on 75a1176; bfc8e1f keeps the same start in Chromium.

  • Safari and WebKit (2026-10-26, same file and method; Safari driven through safaridriver):

    engine rounds main fetch start 8 (75a1176) this PR vs main, paired faster
    Safari 26.5.2 4 31.5 s 38.8 s 25.5 s −19% 4/4
    WebKit (Playwright) 5 32.3 s 25.9 s −18% 5/5
    Chromium (Playwright) 3 35.3 s 18.6 s −46% 3/3

    At a fetch start of 8, Safari was slower than main in all 4 rounds. In traced WebKit downloads a dial waited 1.9 s at the median before its ICE checks at a start of 8, against 0.7 s at 4.

    612 MB video in Safari, on 1cd6d99, 2 rounds: 270.7 and 227.4 s, against main's 358.5 and 394.0 s (−34%), with fewer bytes received (823 against 860 MiB) and fewer connections (1,776 against 1,898).

  • Native CLI (2026-10-26/27). ant file download from mainnet, main ccd4106 against 1cd6d99, each download with a fresh temporary HOME so no peer cache or adaptive-controller snapshot carries over, builds in rotating order. All downloads byte-identical.

    file rounds main p50 this PR p50 vs main, paired 95% CI faster
    64 MiB 14 53.8 s 53.5 s +4.9% −3% to +13% 5/14
    612 MB 5 214.1 s 239.6 s +8.6% −3% to +21% 2/5

    Neither difference is significant; the 64 MiB medians are equal, and its paired mean comes from two slow PR rounds (72.8 and 63.8 s). A build of this PR with main's rules for when a GET may join an early one, the only change native reads see, measured +5.6% (64 MiB, 8 rounds) and +11.7% (612 MB, 3 rounds), so the hedge rule is not what moves native time.

  • Separating the changes (same file, 4 rounds, on the branch before it merged main; main at ccd4106):

    build p50 vs main, paired faster
    main 39.1 s
    fetch start 8 alone (432df72) 45.0 s +15% 1/4
    fetch start 4 + ICE deadline 32.3 s −15% 3/4
    fetch start 4 + no gathering turn 35.7 s −4% 3/4
    fetch start 4 + both 25.3 s −34% 4/4
    fetch start 8 + both 19.7 s −49% 4/4
  • Mainnet traces (setBrowserTrace, 3 downloads per build, before the merge): with both changes, a successful dial took 0.4 s at the median instead of 1.9–2.7 s, a failed one 3.0 s instead of 10.4–11.0 s, a lookup round 1.2 s instead of 1.6–3.8 s, and the first data read started at 7–8 s instead of 13.5–14.3 s. False positives: an endpoint that failed in one download connected in another for 4% of failures with the 3 s deadline, against 5% with the 10 s timeout.

  • WASM bindings (node --test ./wasm-tests/*.test.mjs on a browser-wasm,test-utils build): 251 passed on 84f5a8f.

    • a dial fails once its endpoint has not answered ICE checks in time, a dial whose ICE checks succeeded waits for a slow channel to open, in Chromium, concurrent dials gather ICE candidates at once and a WebKit page that adds userAgentData itself still gathers one connection at a time are new; each fails with the behaviour it pins reverted.
    • A download run as a Chromium page keeps more than 4 GETs in flight, and as any other page at most 4.
    • a speculative GET takes the slots an abandoned exclusive request hands back (from review): an exclusive wait that times out or is cancelled now wakes speculative GETs; it fails without the fix.
    • The earlier ones still pin the 3 s browser hedge, dispatch timing and the per-session read-ahead share.
  • cargo test -p ant-core --lib: 873 passed.

  • cargo clippy --workspace --all-targets -- -D warnings: clean. WASM clippy (--target wasm32-unknown-unknown --features browser-wasm, with and without test-utils): clean. cargo fmt --all -- --check: clean.

  • Dev testnet A/B (2026-10-09), on the first revision of this PR (c526289): two 60-node fleets, headless Chromium from fra1 and syd1, 50 MiB files, 12 downloads per build per region. Downloads were 27–29% faster at the median and received 12–13% fewer bytes. The dial changes were not run on the testnet, where every node fits in the 64-connection pool.

  • Full testnet write-up: ant-testnet/state/comparisons/v2-1358-kad-vs-main-matched-60m-1009/README.md, section "Browser (WASM) downloads".

New dependency

none. ant-core enables the RtcIceConnectionState feature of web-sys, which it already depends on.

ADR

n/a (T1). ADR-0004 gains the section "Dial failure and ICE gathering (2026-10-26)".

Mitigation / rollback

Client-only: revert this PR, which restores the browser's previous read admission, its 1 s hedge, the 10 s dial timeout and serialized ICE gathering in every engine.

🤖 Generated with Claude Code

@dirvine dirvine left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed WithAutonomi/ant-client at baf4153. No blocking defect found.

Two groups of three review seats, including an independent z-ai/glm-5.2 API review, agreed on no introduced blocker. I reconciled the findings against the source and local execution.

Verified:

  • The browser fetch start derives from the same constants as the physical read budget. The memory cap and adaptive backoff remain in place.
  • The speculative admission limit keeps read-ahead at at most two physical GETs. Existing seek and read-ahead regressions pass.
  • Native early-read hedging remains at 1 s; WASM changes to 3 s. No wire, storage or public API change.
  • Local release WASM build and generated bindings: all 209 tests passed, zero failures. The focused download suite also passed 22/22. Formatting and diff checks passed.
  • Current-head CI is green, including Browser WASM and both real-Chromium integration tests. Native and E2E results are CI evidence, not local reruns.

Non-blocking caveats:

  • The comment that a multiplexed session 'always has a slot left' should be read as protection from speculative reads alone. Other awaited reads can still fill that session and make another read queue. Ordinary reads can hold a budget permit while waiting for a multiplex slot; the wider global budget does not remove that per-session bound.
  • A targeted WASM regression that pins hedge timing/duplicate-fetch behaviour would strengthen coverage of the 3 s change.
  • The speed measurements remain author-reported. I did not reproduce the testnet or mainnet benchmarks, and could not retrieve the referenced raw report through the attempted GitHub path. The PR reports roughly 23–26% lower mean download duration in its testnet comparison; this review does not certify a universal throughput gain.

Panel reconciliation: I discarded speculative constant-drift/configuration concerns and corrected an inaccurate permit-order claim in the initial reports. No blocking dissent remained. The final local 209-test release run supersedes the evidence seat's earlier snapshot, which only had the focused 22-test run. Initial local setup/debug-run failures are not counted as release validation.

Approval is for this exact code head; no merge or release action performed.

mickvandijke added a commit that referenced this pull request Oct 12, 2026
From review of #222:
- A new test pins the 3 s WASM hedge. Two nodes hold a record, and
  discovery never completes, so only the hedge can send a second GET.
  A first reply after 2 s is the only GET; after 1 s a hedge would have
  raced it. A first reply after 6 s is raced after about 3 s. The test
  fails with the former 1 s hedge.
- SPECULATIVE_READ_LIMIT's comment now says what it guarantees:
  speculative GETs alone never fill a node's multiplexed session. Other
  awaited GETs can still fill it, and a seek's GET then queues.

Part of V2-1483.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mickvandijke

Copy link
Copy Markdown
Member Author

Thanks for the review. The follow-ups are in c526289.

  • Comment: SPECULATIVE_READ_LIMIT's comment now says only what it guarantees. Speculative GETs alone never fill a node's multiplexed session. Other awaited GETs can still fill it, and a seek's GET then queues.

  • New WASM test: an early read is hedged only after a DataChannel transfer's time. With discovery stalled:

    • a first reply after 2 s is the only GET;
    • a first reply after 6 s is raced after about 3 s.

    It fails with the former 1 s hedge and passed 5 runs in a row. All 210 WASM tests pass, and CI is green.

  • Benchmarks: the report lives in an unpublished testnet workspace, which is why the path didn't resolve. The tables in the description are the results. Raw per-download results are available on request.

mickvandijke and others added 5 commits October 19, 2026 12:39
Browser downloads started at the native fetch cold start of 4, while the
browser's read budget admits 8 GETs. The shared read engine also hedged
an early GET after 1 s. A WebRTC DataChannel takes 1-3 s to carry a
4 MiB record, so the hedge raced nearly every read. Its duplicate held
one of the 4 read slots until its response drained.

- Browser clients start fetching at the read budget's capacity. The
  adaptive controller still backs off from there.
- Read-ahead keeps its bound of 2 GETs. At 4 it could fill a node's
  multiplexed session, and a seek's GET then queued behind it.
- On WASM, an early GET is hedged after 3 s.

On two 60-node testnets, headless Chromium downloads of 50 MiB files
took 12.2 s at p50 instead of 16.8 s from fra1, and 25.2 s instead of
35.3 s from syd1. They received 12-13% fewer bytes. That is 12 downloads
per arm per region.

Part of V2-1483.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review of #222:
- A new test pins the 3 s WASM hedge. Two nodes hold a record, and
  discovery never completes, so only the hedge can send a second GET.
  A first reply after 2 s is the only GET; after 1 s a hedge would have
  raced it. A first reply after 6 s is raced after about 3 s. The test
  fails with the former 1 s hedge.
- SPECULATIVE_READ_LIMIT's comment now says what it guarantees:
  speculative GETs alone never fill a node's multiplexed session. Other
  awaited GETs can still fill it, and a seek's GET then queues.

Part of V2-1483.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the read budget

Address the review of the browser read tuning.

- An early GET is raced only once it has waited the hedge delay since it
  was sent. Time queued behind the read budget, a session slot or the data
  lane no longer counts. The transport reports the send through a
  RequestSent notice; native GETs leave at once.
- The same rule now holds after discovery: the ordinary path joins an
  early GET still within its hedge delay only once it stalls, and a raced
  GET must stall too before a third candidate joins it.
- The hedge delay comes from the transport (1 s native, 3 s browser), so
  the shared read engine has no target_arch branch.
- Read-ahead no longer shares a global clamp. Each node's session admits
  speculative GETs within half its slots, never ahead of a request queued
  for one, and promotes a GET a read comes to wait for. The read budget
  keeps its speculative share of the cap.
- The browser client builds its controller with a fetch start and ceiling
  equal to the read budget's capacity, instead of warm-starting every
  channel. Quote and store keep their slow start, the hill climber no
  longer probes past GETs the budget cannot admit, and a disabled
  controller holds the same start.
- Without an installed limiter, the pool admits the budget's capacity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A peer session admitted speculative GETs within a share it defined
itself, a copy of the read budget's divisor. It now uses the read
budget's, so the two bounds cannot drift apart.

The session docs now say what the share bounds. Like the read budget's
permits, a GET is classed once, when it queues as a read's or is
admitted. After a seek, read-ahead may hold more than the share until
the cancelled read's GETs complete. A read's GET is still never queued
behind read-ahead that comes after it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Address the second review of the browser read tuning.

- A GET now reserves the read budget before anything of its peer's, and
  is dispatched there. The budget is the client's own, and a GET raced
  against this one would wait for it too, so it never makes a peer look
  stalled. Waiting on the peer does count: a redial, its session slots,
  the send and the reply. Before, a GET queued behind a busy or silent
  peer's session slots never started its hedge delay, so it was never
  raced. After discovery it also held back the ordinary path until it was
  sent or failed, which for a silent peer took its other GETs' timeouts.
- With one order for every GET, no GET holds a peer session slot while
  it waits for the budget. A hedge, or a GET after discovery, waits for a
  permit another fetch frees and holds nothing of its peer's meanwhile,
  so the fetch start and ceiling stay at the budget's capacity.
  Read-ahead that a node's speculative share holds back now stalls too,
  and is raced at another holder. A GET to a peer still connecting holds
  its permit while it connects; early GETs only go to connected peers.
- The notice is renamed from RequestSent to RequestDispatched, and
  BrowserNetwork::request_notifying_sent to request_notifying_dispatched,
  for what they now report.
- Stall timers are polled before discovery hint updates, so a burst of
  updates cannot delay starting one.
- retrieve_progressive takes its early-read limit and hedge delay as one
  EarlyReads policy, and settles each GET's result in one place.
- send_and_await_chunk_response takes a ChunkRequest options struct that
  carries the dispatch notice, replacing its 10-argument notifying
  variant.
- The docs say that the hedge rules hold on native too: a raced GET, and
  one in flight when discovery finishes, must stall before another
  candidate joins it.
- The browser hedge delay's docs say it covers the peer's own queue,
  including replies its session carries ahead of the GET.

Tests: a GET waiting for its peer's only session slot has already been
dispatched, which fails with the notice where it was. Physical reads
stay within an installed adaptive fetch cap, and fill the read budget
without one.

Breaking changes: none against main. RequestDispatched and
BrowserNetwork::request_notifying_dispatched rename RequestSent and
request_notifying_sent, which this PR added and no release carries. The
test-utils export test_budgeted_reads takes an optional fetch cap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mickvandijke
mickvandijke force-pushed the mickvandijke/v2-1358-browser-read-tuning branch from 46e0a57 to 432df72 Compare October 19, 2026 10:45

@dirvine dirvine left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review of WithAutonomi/ant-client #222

Reviewed head: 432df72bd6adbcac23b8eaf8233fb139a0e92e7b.
Merge-base: e47ea7a7803e8c1947b0b254f1d38e5cdd950198.
Previous approval: baf4153297860066c77f9859c8385aac5d63bace.

REQUEST_CHANGES — browser fetch cold start regresses the reported mainnet workload

Location: ant-core/src/browser/wasm_transport.rs:2394-2399, BrowserNetworkClient construction.

The browser still starts its fetch controller at read_capacity (8), rather than the previous start of 4. The PR description now contains mainnet A/B measurements at this exact head against e47ea7a that contradict the original testnet-only speed claim:

  • Run 1: download p50 32.3 → 40.7 s and mean 32.5 → 42.4 s; this PR was slower in all 10 rounds. Reported connection p50 rises from 176 to 281, despite fewer received bytes.
  • Run 2: this head gives 40.9 s p50. Changing only the start back to 4 gives 33.8 s, compared with main at 32.6 s, while retaining the byte saving. The review-fix commits are not what caused the slowdown.

These are author-reported results, not benchmarks I reproduced. They are nevertheless current-head evidence against enabling the higher startup concurrency as a default. The suggested mechanism—WebRTC pool eviction and redial—is explicitly unverified; the finding does not depend on asserting that mechanism as fact.

Required change: retain the safer fetch start of 4 while keeping the 3 s browser hedge and the physical read-budget bound, and update the startup test expectations; or provide a corrected adaptive/pool-aware change with same-head mainnet evidence that removes this regression. The read capacity is a memory bound, not proof that starting all its slots improves throughput.

Prior follow-ups

  • The session-slot documentation now distinguishes speculative read-ahead from awaited GETs. Current multiplexed sessions admit speculative GETs into at most 2 of 4 slots; ordinary requests may fill the session. This is not a guarantee that a session always has a free slot.
  • The new generated-WASM real-clock regression directly checks the browser hedge delay, including the fast-reply/no-hedge case. It passed locally.
  • Native clients still use a 1 s hedge duration, but this revision also applies the stall rule after discovery and after a race. Native timing behaviour is therefore not wholly unchanged from the previous approval.

Verified checks and limits

  • Fresh release wasm32-unknown-unknown build at the reviewed head, with browser-wasm,test-utils and matching wasm-bindgen 0.2.122: passed.
  • Generated WASM executed under Node with the required setup-wasm.mjs preload: 230/230 passed, no failures, skips or cancellations.
  • cargo test --locked -p ant-core --lib client_engine: 156 passed.
  • Focused controller-construction test: 1 passed.
  • cargo fmt --all -- --check and git diff --check: passed.
  • Current native/WASM/Chromium functional CI checks pass. Security Audit is red for zip 2.4.2 (RUSTSEC-2026-0367, -0368, -0369). Cargo.lock is byte-identical to the merge-base, so this is inherited, not introduced by this PR. It is not a passing security check: https://github.com/WithAutonomi/ant-client/actions/runs/39239681477/job/118017190501.
  • I did not run live-network benchmarks or repeat the full native suite locally. Streaming read-ahead performance remains unmeasured in the author's report.

Review coverage

5/6 substantive reviews — partial panel; treat its coverage as provisional. GLM 5.3 and Qwen 3.8 completed both browser and scheduler/adapter scopes. DS4 completed a narrowed browser-startup/performance review, but its scheduler seat produced no valid final verdict after one narrowed same-model retry (initial attempt exhausted its output; retry failed validation). No substitution was counted.

All three named models independently recommend changes for the reported mainnet startup regression; no completed review dissents on that finding. The missing DS4 scheduler opinion is a disclosed coverage limit, not an additional code finding. The primary checked the source and parsed execution logs independently; controller-test coverage is one passing test.

All changed production paths were included across the browser and scheduler/adapter scopes. Test-only helpers were reviewed separately by the primary and exercised by the test run; they were not sent as complete source to every panel seat. No merge or deployment performed.

Comment thread ant-core/src/browser/wasm_transport.rs Outdated
let mut config = crate::data::ClientConfig::default();
config.adaptive.max.fetch = read_capacity;
let start = crate::data::client::adaptive::ChannelStart {
fetch: read_capacity,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current-head mainnet A/B in the PR description reports download p50 32.3→40.7 s versus the merge-base, slower in all 10 rounds. Its follow-up ablation reports start=4 at33.8 s versus main32.6 s, while retaining the byte saving. These are author-reported measurements, not an independent reproduction. Starting at the memory capacity of8 is therefore not supported by the current mainnet evidence. Retain fetch start4 (and update the startup test expectations), or provide a corrected adaptive/pool-aware approach with current-head mainnet evidence that removes the slowdown. The 3s browser hedge and physical read-budget bound can remain.

mickvandijke and others added 5 commits October 26, 2026 13:45
On mainnet, 42-50% of the WebRTC Direct endpoints a browser dials never
answer ICE connectivity checks. Each such dial waited the 10 s
channel-opening timeout, holding a pool slot and keeping its lookup round
waiting: 18-30 of the 64 slots at a time in traced downloads. Live nodes
answered ICE in 110 ms at the median and 1.2-1.4 s at p99; the slowest
of 788 took 2.7 s. Every failed dial in those traces was one whose ICE
never connected.

A fresh connection whose iceConnectionState is still new or checking
3 s after its offer is answered now fails, and enters the dial-failure
cache like any other failed dial. Once ICE has connected, DTLS, SCTP and
the channel keep the opening timeout. A browser without
iceConnectionState keeps the old behaviour.

The mock RTCPeerConnection reports iceConnectionState, and a node with
iceUnreachable stays checking forever. Two tests check that such a dial
fails at the deadline, and that a slow channel opens once ICE connected.

Part of V2-1483.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The page-wide ICE gathering turn exists for WebKit, which opens many
more sockets for overlapping gatherings and closes the oldest past its
per-page cap. Chromium has no such cap, but its dials queued for the
turn too: in traced mainnet downloads a dial waited 0.6 s at the median
at a fetch concurrency of 4, and 1.7 s at 8, before sending its offer.
ICE, DTLS, SCTP and the PQ handshake together took about 0.4 s.

Only Chromium exposes navigator.userAgentData, so a page that has it
skips the turn. WebKit and Firefox keep it; Firefox was not measured.
With this change the wait before the offer fell to 5 ms at the median.

Together with the 3 s ICE deadline, the fetch start of 8 pays off on
mainnet. A 64 MiB public file downloaded in headless Chromium took
19.2 s at the median against main's 35.0 s, faster in 5 of 5 rounds,
and 19.7 s against 39.1 s in an earlier run of 4. The fetch start of 8
alone was 15% slower than main in that run.

Part of V2-1483.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ADR-0004 gains a dated section with the mainnet measurements behind the
3 s ICE deadline and Chromium's concurrent ICE gathering, and how they
let the browser's fetch start of 8 pay off.

Part of V2-1483.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in hot-chunk read spreading and the wait for refusing holders
(#225), browser manifest export and pointer recovery (#230), and the
dependency bumps for the RustSec advisories.

- read.rs: retrieve_progressive takes this branch's EarlyReads policy
  and dispatch notices, and main's holder choice, refused peers and busy
  error. Early GETs pick their hint with next_early_peer, and the
  dispatch-timed stall rules hold within each refusal-aware round.
- wasm_transport.rs: a dial races its channel opening against main's
  failure of a dead association and the 3 s ICE deadline. note_refusal
  still marks a STUN refusal Refused; a dial that never reached ICE
  connected keeps the full failure cooldown.
- network.rs keeps both read_hedge_delay and refused_peers.
- The mock reports ICE connected before connectDelay, which stands for
  DTLS and SCTP, and keeps main's refusal and DTLS-failure paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ADR-0004 section now gives the merged branch against main at
ccd4106: a 64 MiB file took 21.6 s at the median against 37.6 s, faster
in 5 of 5 rounds, and the fetch start of 8 beat 4 in every round. A
612 MB video took 195 and 270 s against main's 398 and 323 s.

Part of V2-1483.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mickvandijke mickvandijke changed the title perf(browser): use the whole read budget and hedge after a DataChannel transfer perf(browser): fail dead dials in 3 s and use the whole read budget Oct 26, 2026
mickvandijke and others added 3 commits October 26, 2026 15:24
…t once

The browser started its fetch channel at the read budget's 8 GETs in
every engine. Where dials take turns to gather ICE candidates, as in
WebKit, more concurrent reads queue more dials for the turn. In WebKit
on mainnet, a dial at that start waited 1.9 s at the median before its
ICE checks, against 0.7 s at the native cold start of 4, a lookup round
took 3.6 s against 1.3 s, and a 64 MiB download 31.5 s against 24.3 s.

Only a page whose dials may gather at once, Chromium's, now starts at
the read budget. Others keep the native cold start, and the adaptive
controller can still grow to the budget's ceiling. Against main, a
64 MiB download from mainnet took 25.9 s at the median in WebKit
against 32.3 s (5 of 5 rounds faster), and 18.6 s in Chromium against
35.3 s (3 of 3).

The wasm tests run a download as a Chromium page, through a shared
asChromium helper, and as any other, and check its peak GETs in flight.

Part of V2-1483.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ADR-0004 now says only Chromium starts fetching at the read budget, why
WebKit keeps the native cold start, and how the merged branch compares
with main in Safari 26.5.2, Playwright's WebKit and Chromium.

Part of V2-1483.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Address the review of the dial changes:

- A page could add navigator.userAgentData itself, as a User-Agent
  Client Hints polyfill does, and Safari would then gather ICE for
  concurrent dials at once. Overlapping gatherings open about thirty
  sockets each in WebKit, which past 256 per page closes the oldest,
  cutting established connections (#216). A page reporting WebKit's
  vendor now keeps the turn, and the native fetch start, whatever else
  it exposes.
- The test that a dial whose ICE connected may open its channel after
  the ICE deadline did not reach the deadline: the mock held up
  setRemoteDescription for connectDelay, so the timer started only after
  the channel was ready. The mock's openDelay delays only the opening,
  and the test now fails if the deadline fails a live dial.
- A speculative GET over its session's share never tried the slots, so
  a closed session failed it only once other GETs released theirs. It
  now checks for a closed session on every pass.

Both new and changed tests fail with the behaviour they pin reverted.

Part of V2-1483.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mickvandijke
mickvandijke marked this pull request as ready for review October 27, 2026 11:06

@MaidSafe-Hermes MaidSafe-Hermes left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST_CHANGES — speculative read can sleep after capacity returns

[P2] Wake admit_read when a cancelled exclusive admission returns permits

Location: ant-core/src/browser/wasm_transport/multiplex.rs:187-224, together with the exclusive acquisition at :241-251.

admit_read waits only for released or a read-priority change after try_acquire_owned() reports NoPermits. An exclusive PUT first drops its slot and awaits acquire_many_owned(capacity). Tokio's pinned semaphore implementation returns any partially acquired permits when that future is cancelled or times out, but this return does not send this session's released notification.

A reproducible path is a four-slot session with one active GET: an exclusive PUT can reserve the other three permits while it waits for the fourth. A speculative GET then waits in admit_read. If the PUT is cancelled, the three permits become available, but the speculative GET is not woken; it can remain blocked until its admission timeout or an unrelated release/priority change, despite free capacity.

Please notify released whenever cancellation or timeout of the all-slot acquisition returns permits (for example, with a drop guard around that wait), and add a regression test that cancels a partially satisfied exclusive acquisition and asserts the speculative GET proceeds without another exchange completing or a priority change.

The review seats differed on this finding. I confirmed the path against multiplex.rs and Tokio 1.52.3's Acquire::drop, which returns partially acquired permits without notifying this separate watch channel. The remaining reviewed scopes found no other confirmed blocker.

Coverage: 2/2 seats reviewed all six changed-file scopes — sol-1 (gpt-6.1-sol, xhigh) and glm-1 (z-ai/glm-5.3). sol-1's initial browser-transport uncertainty was retried once with the missing cross-file and test evidence and resolved to APPROVE. Both seats' opinions and the remaining dissent are preserved in the review record.

Validation: exact-head cargo test --locked -p ant-core --lib client_engine passed (166 passed, 0 failed); PR CI passed Browser WASM and all other listed checks except Security Audit. The audit failure is the new RUSTSEC-2026-0387 finding for crypto-bigint 0.5.5; Cargo.lock is identical to base ccd4106, so this is not introduced by this PR.

mickvandijke and others added 2 commits October 27, 2026 12:45
The browser's fetch ceiling was explained as fetches beyond the read
budget's GETs only queueing. They do not: a fetch spends most of its
time in discovery, holding no GET. The ceiling holds because more
lookups cost dials more than they overlap transfers. With the default
ceiling of 256, the controller probed 10 and came back, a successful
dial's p90 rose from 2.7 s to 4.8 s, and mainnet downloads were slower
in 6 of 8 pairs across Chromium and Safari. The code comment and
ADR-0004 now say so.

Part of V2-1483.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s back

From review of #222: an exclusive request drops its slot and waits for
all of its session's slots. If that wait timed out or its caller was
cancelled, Tokio handed back the permits it had gathered, but nothing
sent the session's `released` notification. A speculative GET waiting in
admit_read listens only to that and to its read priority, so it could
sleep with free slots until its admission timeout.

The exclusive wait now holds a guard that notifies `released` however it
ends. A regression test holds one of a multiplexed session's four slots,
abandons an exclusive request holding the other three, and checks that a
speculative GET queued meanwhile is admitted with no slot released and
no priority change. It fails without the guard.

Part of V2-1483.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@MaidSafe-Hermes MaidSafe-Hermes left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE — follow-up resolves the reported P2

Reviewed WithAutonomi/ant-client PR #222 at 84f5a8fa3266f79473b80c71b70764a32b8055a6 against base ccd4106dd8fd60545bbd31c40d46cc041f85fb8d.

WakeOnDrop is installed before the timed exclusive acquire_many_owned wait. On timeout or caller cancellation, Tokio returns partially acquired permits before the guard notifies released, so speculative readers wake and retry. The added WASM regression holds one of four slots, cancels the exclusive wait after it gathers the other three, and asserts that the speculative GET is admitted without another slot release or a priority change. I found no new blocker in the follow-up.

Review coverage: Both required seats reviewed the follow-up at this exact head: sol-1 (gpt-6.1-sol, xhigh) and glm-1 (z-ai/glm-5.3), both APPROVE. The original full-PR review covered all six changed-file scopes with both seats.

Validation: Local cargo test --locked -p ant-core --lib client_engine passed, 166/166. A first run had one timing-threshold failure; the isolated test and full rerun passed. At review time, all functional CI checks passed. Security Audit remains failed for inherited RUSTSEC-2026-0387 / crypto-bigint 0.5.5; the lockfile is unchanged from base. This review does not state that the audit passed. No merge or release action performed.

@mickvandijke
mickvandijke merged commit 7a432e7 into main Oct 27, 2026
21 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants