Repository navigation
benchmarking/nighthawk-egress: add the adaptive capacity runner with gateway counter checks - #3629
Closed
Yuan Gao (ygao-g) wants to merge 10 commits into
Closed
Yuan Gao (ygao-g) wants to merge 10 commits into
Yuan Gao (ygao-g) wants to merge 10 commits into
Conversation
2 tasks done
Yuan Gao (ygao-g)
force-pushed
the
nighthawk-egress-2-runner
branch
from
November 4, 2026 20:50
21af21a to
a56bf82
Compare
This was referenced Nov 4, 2026
Open
Collaborator
Author
Reviewer notes (Stage 1 stack guide)Stack position: Stacks on #3628 ( Commit-by-commit checks (
Local checks: hack/verify/benchmarking-python.sh
go test ./cmd/kubectl-ate/internal/cmd/ -run TestEgressPolicyFromManifest
|
Yuan Gao (ygao-g)
force-pushed
the
nighthawk-egress-2-runner
branch
2 times, most recently
from
November 5, 2026 22:14
1807b45 to
6629485
Compare
…f nighthawk-ingress Move nighthawk-ingress/output.py to nighthawk_common unchanged so nighthawk-egress can reuse it. The ingress Dockerfile copies the package into the image and the ingress tests put benchmarking/ on sys.path. Signed-off-by: Yuan Gao <ypgao@google.com>
… out of nighthawk-ingress Move nighthawk-ingress/spec.py to nighthawk_common unchanged and point its importers at the new path. actors.py imported it without using it. Signed-off-by: Yuan Gao <ypgao@google.com>
…ut of nighthawk_common build_session_spec in nighthawk_common now takes a finished traffic template. The router SNI, the certificate paths and build_spec_dict, which assembles that template, move back to nighthawk-ingress/spec.py. Signed-off-by: Yuan Gao <ypgao@google.com>
benchmarking/workloads/deploy.sh built kubectl-ate inline, and the nighthawk-egress deploy script was about to carry a copy. Move the function into benchmarking/lib/kubectl-ate.sh for both to source. Signed-off-by: Yuan Gao <ypgao@google.com>
…ration With a 2 vCPU actor sandbox and no requests on the worker pod or nighthawk-server, nothing kept the load generator and the origin server from saturating before atenet-egress, so a run could measure them instead of the gateway. Give the sandbox 16 vCPUs, request 12 vCPUs for the worker pod and 6 for nighthawk-server, run the server with --concurrency 8, and move the example nighthawk_client run from one worker to sixteen. Signed-off-by: Yuan Gao <ypgao@google.com>
deploy-traffic-driver.sh required kubectl-ate on PATH, so a fresh environment needed a manual go install step first. Source build_kubectl_ate from benchmarking/lib/kubectl-ate.sh instead, and say so in the README prerequisites in place of the install step. Signed-off-by: Yuan Gao <ypgao@google.com>
… per stage Add the queue_to_connect and sequencer.blocking statistics and the latency pstdev to each stage row, make actors optional in capacity_summary, accept Path for descriptor paths, and export the helpers nighthawk-egress uses. Compute achieved-rps and send-rate from the exact stage duration, because Nighthawk truncates execution_duration to whole seconds. Signed-off-by: Yuan Gao <ypgao@google.com>
…nter parser Define the egress-specific failure predicates and Nighthawk template for the adaptive load search, targeting the atenet-egress gateways. Add the prometheus output parser for the egress metrics. Signed-off-by: Yuan Gao <ypgao@google.com>
Yuan Gao (ygao-g)
force-pushed
the
nighthawk-egress-2-runner
branch
from
November 5, 2026 22:54
6629485 to
688514b
Compare
Yuan Gao (ygao-g)
marked this pull request as draft
November 5, 2026 22:54
…and run-dev.sh Add runner.py, spec.py and output.py glue, cluster_setup.py, the runner image and run-dev.sh, which drives one adaptive capacity run end to end and scrapes atenet-egress counters before and after it through the atenet-egress-metrics Service. Name nighthawk-server exactly in egress-policy.yaml, with and without the cluster.local suffix, because the actor resolv.conf sets ndots:5 and atunnel sees the suffixed name. Signed-off-by: Yuan Gao <ypgao@google.com>
…ters, and results Signed-off-by: Yuan Gao <ypgao@google.com>
Yuan Gao (ygao-g)
force-pushed
the
nighthawk-egress-2-runner
branch
from
November 6, 2026 06:58
688514b to
991cdf5
Compare
2 tasks done
Collaborator
Author
shankgan
pushed a commit
to shankgan/substrate
that referenced
this pull request
Nov 13, 2026
…to nighthawk_common (agent-substrate#3628) Part of agent-substrate#3624 (Stage 1, item 2): share the adaptive load session spec and output helpers between `nighthawk-ingress` and the coming `nighthawk-egress` runner. ### What changes Pure relocation, no behavior change. `nighthawk-ingress/output.py` and `spec.py` move to `benchmarking/nighthawk_common/`. The router-specific parts of the spec builder (router SNI, certificate paths, `build_spec_dict`) stay in `nighthawk-ingress/spec.py`, which now calls the generic `build_session_spec`. Callers and tests import `nighthawk_common` directly; the ingress Dockerfile copies the package into the image. ### Review guide 1. `output.py` rename (GitHub shows 100%), two import lines, the Dockerfile `COPY`, two `sys.path` lines in the ingress tests. 2. `spec.py` rename at 100%, importers repointed, an unused import dropped from `actors.py`. 3. The split: read `nighthawk_common/spec.py` as deletions and `nighthawk-ingress/spec.py` as the same lines moving back. ### Testing `hack/verify/benchmarking-python.sh`: the ingress suite passes at each commit, and fails without the new `sys.path` lines, so the import path is exercised. Independent of agent-substrate#3627. agent-substrate#3629 and agent-substrate#3630 stack on both. AI use: AI-assisted for the diff, the commit split and the test runs; I reviewed the diff. - [x] Tests pass - [ ] Docs updated (none needed) --------- Signed-off-by: Yuan Gao <ypgao@google.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #3624 (Stage 1, item 3): the adaptive capacity runner for
atenet-egress.Stacks on #3628 and #3627, in that order; review the last four commits.
What changes
nighthawk_common: per-stagequeue_to_connectandsequencer.blockingtelemetry, optionalactors,Patharguments, public helper names, and achieved-rps from the exact stage duration (Nighthawk truncates it to whole seconds).nighthawk-egress/spec.py,output.py: the egress session spec forhttp,mitmandpassthrough, the gateway counter parser and thecapacity.jsonsummary.runner.py,cluster_setup.py,Dockerfile,run-dev.sh: one adaptive run end to end, scrapingatenet-egressPrometheus counters before and after through a new headlessatenet-egress-metricsService.traffic-driver-invokergets its own ServiceAccount.egress-policy.yaml: exact server hostnames instead of*; the actor'sndots:5makes atunnel see the.cluster.localname.Review guide
nighthawk_commonbehavior commit, with the ingress test for the new fields.Testing
hack/verify/benchmarking-python.shgreen (egress 24, ingress 13). GKE runs onc3-standard-22behind the README numbers.AI use: AI-assisted (Claude Code) for the diff, the commit split and the test runs; I reviewed the diff.