Repository navigation
Sync eng/common directory with azure-sdk-tools for PR 17084 - #5435
Merged
Helen Gao (helen229) merged 21 commits intoOct 19, 2026
Merged
Conversation
azure-sdk-automation
Bot
added
EngSys
|
Azure Pipelines: Successfully started running 1 pipeline(s). 2 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Redirect retry handling is incorrect, and the repository’s 1ES redirect lacks the required Azure Storage parameter.
3 open findings
What changed in this PR
Syncs shared evaluation infrastructure from azure-sdk-tools PR 17084, adding complete result collection and optional dashboard publication.
Changes:
- Stages and validates latest shard attempts for complete summaries.
- Builds immutable result bundles and publishes them to Azure Blob Storage.
- Adds pipeline gating, documentation, dependencies, and comprehensive tests.
| File | Description |
|---|---|
eng/common/scripts/eval/test/shard-results.test.ts |
Tests shard staging and selection. |
eng/common/scripts/eval/README.md |
Documents publication and recovery contracts. |
eng/common/scripts/eval/publisher/test/publisher.test.ts |
Tests bundling, publication, and pipeline gates. |
eng/common/scripts/eval/publisher/storage.ts |
Implements Blob upload and dashboard notification. |
eng/common/scripts/eval/publisher/publish-bundle.ts |
Adds the publication CLI. |
eng/common/scripts/eval/publisher/package.json |
Defines publisher dependencies and tests. |
eng/common/scripts/eval/publisher/package-lock.json |
Locks publisher dependencies. |
eng/common/scripts/eval/publisher/bundle.ts |
Creates and validates result bundles. |
eng/common/scripts/eval/package.json |
Narrows the evaluation test pattern. |
eng/common/scripts/eval/lib/shard-results.ts |
Stages and selects authoritative shard attempts. |
eng/common/scripts/eval/build-eval-summary.ts |
Adds completeness-aware summaries. |
eng/common/pipelines/workflow-eval.yml |
Enables gated workflow publication. |
eng/common/pipelines/templates/variables/eval-dashboard.yml |
Defines trusted publication eligibility. |
eng/common/pipelines/templates/steps/eval-publish-results.yml |
Adds bundle and upload steps. |
eng/common/pipelines/templates/stages/archetype-eval.yml |
Wires publication into evaluation stages. |
eng/common/pipelines/templates/jobs/eval-summarize.yml |
Consolidates authoritative results. |
eng/common/pipelines/templates/jobs/eval-shard.yml |
Publishes complete per-shard artifacts. |
eng/common/pipelines/skill-eval.yml |
Enables gated skill-evaluation publication. |
eng/common/pipelines/live-eval.yml |
Enables gated live-evaluation publication. |
Files not reviewed (1)
- eng/common/scripts/eval/publisher/package-lock.json: Generated file
🧠 Review effort: Balanced
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
azure-sdk-automation
Bot
force-pushed
the
sync-eng-common-feat/eval-blob-publishing-17084
branch
4 times, most recently
from
October 16, 2026 18:36
9f7fe59 to
99bd65a
Compare
azure-sdk-automation
Bot
force-pushed
the
sync-eng-common-feat/eval-blob-publishing-17084
branch
from
October 16, 2026 19:16
99bd65a to
d6d4c23
Compare
Helen Gao (helen229)
approved these changes
Oct 19, 2026
Helen Gao (helen229)
deleted the
sync-eng-common-feat/eval-blob-publishing-17084
branch
October 19, 2026 21:01
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sync eng/common directory with azure-sdk-tools for PR Azure/azure-sdk-tools#17084 See eng/common workflow