Skip to content

benchmarking/nighthawk-egress: add egress spec and counter parser - #86

Open
ygao-g wants to merge 3 commits into
nighthawk-ingress-ratesfrom
nighthawk-egress-b1-lib
Open

ygao-g wants to merge 3 commits into
nighthawk-ingress-ratesfrom
nighthawk-egress-b1-lib

Conversation

@ygao-g

@ygao-g ygao-g commented Nov 6, 2026 •

Copy link
Copy Markdown
Owner

The egress gateway has a manual Nighthawk setup, but no benchmark that searches for its capacity. Such a benchmark needs a load definition for each egress mode: http, mitm and passthrough. It also needs proof that the load went through the gateway path of that mode. A rate that is measured on another path is not a capacity of the mode.

A new library, with tests, provides both parts. The first part builds the adaptive Nighthawk session for each mode, with the TLS settings of that mode. The second part reads the gateway counters from before and after a run, and adds the differences and four checks to the capacity summary. The checks are ext_proc_invariant_ok, mode_exclusivity_ok, forward_proxy_overflow_ok and gateway_counter_reset. Nothing calls the library until the runner is added. In the shared Nighthawk library, a tail_latency_slo_ms of 0 now disables the latency threshold, as the nighthawk-ingress README states.

Part of agent-substrate#3624

AI use: AI-assisted for the diff, the split and the test runs; I reviewed the diff.

  • Tests pass
  • Docs updated (none needed)

@ygao-g ygao-g self-assigned this Nov 6, 2026
@ygao-g
ygao-g force-pushed the nighthawk-egress-b1-lib branch 2 times, most recently from 188b5d1 to 8647a6d Compare November 6, 2026 17:51
@ygao-g
ygao-g changed the base branch from nighthawk-egress-base to nighthawk-ingress-rates November 6, 2026 17:51
@ygao-g
ygao-g force-pushed the nighthawk-ingress-rates branch from ff812be to c51ca95 Compare November 6, 2026 18:15
@ygao-g
ygao-g force-pushed the nighthawk-egress-b1-lib branch from 8647a6d to ae29932 Compare November 6, 2026 18:15
@ygao-g
ygao-g force-pushed the nighthawk-ingress-rates branch from c51ca95 to 3f71a82 Compare November 6, 2026 18:53
@ygao-g
ygao-g force-pushed the nighthawk-egress-b1-lib branch from ae29932 to 11ec01c Compare November 6, 2026 18:53
@ygao-g
ygao-g force-pushed the nighthawk-ingress-rates branch from 3f71a82 to 24847c9 Compare November 6, 2026 19:09
@ygao-g
ygao-g force-pushed the nighthawk-egress-b1-lib branch 2 times, most recently from 28cfbc4 to ba3e550 Compare November 6, 2026 21:53
Rename _builtin_key, _global_result and _counters to builtin_key,
global_result and counters, because nighthawk-egress/output.py imports
them in the next commit. capacity_summary() takes actors as an optional
argument and emits it only when given, since the egress benchmark has no
actor fleet. build_session_spec() treats a tail_latency_slo_ms of 0 as
disabled, as the ingress README documents.

Signed-off-by: Yuan Gao <ypgao@google.com>
@ygao-g
ygao-g force-pushed the nighthawk-ingress-rates branch from 24847c9 to af0a3db Compare November 9, 2026 23:26
@ygao-g
ygao-g force-pushed the nighthawk-egress-b1-lib branch from ba3e550 to 602805c Compare November 9, 2026 23:26
…nter parser

Define the egress-specific failure predicates and Nighthawk template
for the adaptive load search, targeting the atenet-egress gateways.
Add the prometheus output parser for the egress metrics.

Signed-off-by: Yuan Gao <ypgao@google.com>
benchmarking/nighthawk-egress/tests covers spec.py and output.py. Add
the suite to benchmarking-python.sh so it runs with the other
benchmarking suites.

Signed-off-by: Yuan Gao <ypgao@google.com>
@ygao-g
ygao-g force-pushed the nighthawk-egress-b1-lib branch from 602805c to 2f1e219 Compare November 11, 2026 22:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant