Verify CLI stack binaries match runtime images

[osmman]

Why

A release handover can contain CLI stack archives and runtime images from different builds. Archive format checks alone do not detect this desynchronization.

What

Add optional runtime image/path mappings to product CLI inventories and compare SHA-256 hashes of the extracted executable bytes. Runtime extraction explicitly selects and verifies each Linux platform without starting containers. Missing artifacts and mismatched binaries fail with image, path, platform, and checksum diagnostics.

Enable mappings for RHTAS 1.5 tools and the 1.4 inventory, including single-architecture tuftool. Preserve archive-only inventories, YAML replacement/disabling, and legacy client-server behavior. Document the new configuration.

Support tests, acceptance compilation, and repository lint pass. CLI checks pass against the supplied 1.5.0 snapshot and the 1.4.3 snapshot; an intentionally mismatched runtime executable fails as expected. The releases repository is unchanged.

[qodo-for-securesign]

PR Summary by Qodo

Verify CLI stack binaries match runtime images

✨ Enhancement 🧪 Tests 📝 Documentation ⚙️ Configuration changes 🕐 40+ Minutes

AI Description

• Compare packaged Linux executables with platform-specific runtime image binaries using SHA-256.
• Validate optional runtime mappings while preserving archive-only and inventory override behavior.
• Enable parity checks for RHTAS 1.5 and 1.4 inventories with diagnostic tests.

Diagram

graph TD
  CFG["CLI Config"] --> TEST["CLI Stack Test"] --> STACK["Stack Image"] --> ARCHIVE["Archive Binary"] --> MATCH{"Hashes Match"}
  TEST --> SNAP["Snapshot Images"] --> RUNTIME["Runtime Image"] --> EXTRACT["Platform Extractor"] --> MATCH

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Registry-native layer extraction

• ➕ Avoids creating temporary Docker container objects.
• ➕ Could operate without a Docker daemon using OCI registry APIs.
• ➖ Requires layer and whiteout reconstruction or an additional image library.
• ➖ Introduces more implementation complexity and dependency surface.

2. Build provenance comparison

• ➕ Can establish artifact origin without downloading every runtime executable.
• ➕ May reduce repeated multi-architecture image extraction work.
• ➖ Requires trustworthy provenance linking archives and runtime images.
• ➖ Does not directly prove that distributed executable bytes are identical.

Recommendation: Retain the PR's byte-level SHA-256 comparison through the existing Docker support layer. It directly verifies the required invariant, reuses established extraction behavior, supports foreign architectures without execution, and keeps runtime parity optional. Registry-native extraction could be considered later if daemon dependence or extraction cost becomes problematic.

Files changed (12) +494 / -91

Enhancement (4) +127 / -7

config.goAdd validated runtime mappings to CLI inventory entries +18/-4

Add validated runtime mappings to CLI inventory entries

• Extends stack binaries with optional runtime image and path metadata. Validation restricts mappings to Linux and requires a nonempty image key plus a canonical absolute executable path.

test/support/cli/config.go

helper.goRun runtime parity during CLI stack validation +5/-0

Run runtime parity during CLI stack validation

• Retains the complete snapshot image map and invokes runtime comparison after archive extraction and executable format validation whenever a mapping is configured.

test/support/cli/helper.go

parity.goCompare archive and runtime executable hashes +50/-0

Compare archive and runtime executable hashes

• Adds platform-aware runtime executable extraction and SHA-256 comparison. Errors include stack and runtime image references, paths, platform, and both checksums when bytes differ.

test/support/cli/parity.go

image_utils.goExtract files from explicit image platforms +54/-3

Extract files from explicit image platforms

• Adds platform-aware image inspection and pulling, verifies the selected OS and architecture, and copies files through unstarted temporary containers. Existing extraction now returns initialization errors instead of panicking and closes Docker clients.

test/support/image_utils.go

Refactor (2) +24 / -25

client_server_test.goReuse the shared SHA-256 file helper +7/-25

Reuse the shared SHA-256 file helper

• Replaces the acceptance test's private checksum implementation with the shared support utility. Removes duplicate hashing imports and code while preserving legacy client-server comparisons.

test/acceptance/rhtas/client_server_test.go

file_utils.goProvide a shared SHA-256 file checksum helper +17/-0

Provide a shared SHA-256 file checksum helper

• Moves file hashing into the common support package so CLI parity and existing acceptance checks use one error-reporting implementation.

test/support/file_utils.go

Tests (3) +269 / -5

config_test.goTest runtime mapping validation and inventory behavior +65/-5

Test runtime mapping validation and inventory behavior

• Verifies expected parity mapping counts for release inventories and covers valid, malformed, inherited, replaced, product-scoped, and disabled configurations.

test/support/cli/config_test.go

parity_test.goCover parity matches and diagnostic failures +60/-0

Cover parity matches and diagnostic failures

• Tests identical and mismatched binaries, checksum read failures, missing snapshot image mappings, and extraction errors containing actionable image, path, and platform context.

test/support/cli/parity_test.go

image_utils_test.goTest multi-architecture image extraction +144/-0

Test multi-architecture image extraction

• Uses a mock Docker API to cover cached, foreign, replaced, unavailable, and incorrectly resolved platforms plus missing files. It also verifies explicit platform propagation and temporary container cleanup.

test/support/image_utils_test.go

Documentation (1) +21 / -1

README.mdDocument runtime binary parity mappings +21/-1

Document runtime binary parity mappings

• Documents the optional Linux-only runtime image and executable path mapping, platform selection, checksum behavior, and failure diagnostics. Clarifies enabled RHTAS defaults and preserved archive-only configuration semantics.

README.md

Other (2) +53 / -53

defaults.yamlEnable RHTAS 1.5 runtime parity checks +28/-28

Enable RHTAS 1.5 runtime parity checks

• Maps 28 Linux CLI archives across four architectures to their corresponding runtime images and executable paths. Non-Linux and unsupported distribution channels remain archive-only.

test/acceptance/rhtas/defaults.yaml

testconfig-1.4.yamlEnable RHTAS 1.4 runtime parity mappings +25/-25

Enable RHTAS 1.4 runtime parity mappings

• Adds runtime image mappings for 24 multi-architecture Linux binaries and the amd64-only tuftool executable in the RHTAS 1.4 inventory.

testdata/testconfig-1.4.yaml

[qodo-for-securesign]

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 🔗 Cross-repo conflicts (1) 📜 Skill insights (0)

1. CLI-only releases fail validation 🔗 Cross-repo conflict ≡ Correctness

Description

DescribeCLIStackImageTests now applies the default runtime mappings even when a snapshot contains
only CLI-stack images, so compareRuntimeBinary cannot resolve the first runtime image key. The
releases repository’s CLI-only 1.4 snapshot triggers the CLI-stack suite without a test override but
contains none of the required runtime images, causing its validation to fail.

Code

test/acceptance/rhtas/defaults.yaml[81]

+        - {path: /binaries/cosign_linux_amd64.tar.gz, os: linux, arch: amd64, runtime: {imageKey: cosign-cli-image, path: /usr/local/bin/cosign}}

Relevance

●●● Strong

Concrete release-validation failure from defaults applying runtime mappings to CLI-only snapshots.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The PR adds a runtime dependency to each default Linux archive and attempts to resolve it from the
same snapshot. The releases workflow detects any cli-stack section, runs the CLI-stack suite, and
supplies no override when a testing resource is absent, while the standalone 1.4 snapshot contains
only stack-image references.

test/acceptance/rhtas/defaults.yaml[79-102]
test/support/cli/helper.go[36-60]
test/support/cli/parity.go[15-20]
External repo: securesign/releases, rhtas/cli-stack/1.4.1/tech-preview/snapshot.json [4-14]
External repo: securesign/releases, .github/workflows/structural.yml [170-181]
External repo: securesign/releases, .github/workflows/structural.yml [211-224]
External repo: securesign/releases, .github/workflows/structural.yml [267-272]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The default CLI-stack inventory now requires runtime image keys, but CLI-only snapshots in the releases repository contain only stack images and are still selected for CLI-stack validation.

## Fix Focus Areas
- test/acceptance/rhtas/defaults.yaml[79-136]
- test/support/cli/helper.go[36-60]

## Recommended Fix
Coordinate with the releases repository so CLI-only snapshots include digest-pinned runtime image entries for every enabled parity mapping. If those snapshots must remain archive-only, provide a release-specific `testingResources/structural-tests.yaml` inventory that replaces the defaults with mappings that omit `runtime`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Directory mappings verify unrelated files 🐞 Bug ≡ Correctness

Description

StackConfig.validate accepts any canonical non-root absolute runtime path, including a directory
such as /usr/local/bin, without confirming that it denotes an executable file. When such a mapping
is used, the new runtime comparison reaches fileFromImage, whose tar extraction writes the first
regular entry it sees rather than verifying that the requested path itself was copied.

Code

test/support/cli/config.go[R96-99]

+				if binary.OS != osLinux || strings.TrimSpace(runtime.ImageKey) == "" ||
+					!path.IsAbs(runtime.Path) || path.Clean(runtime.Path) != runtime.Path || runtime.Path == "/" {
+					return fmt.Errorf("CLI stack %q archive %q: runtime requires Linux, imageKey and a canonical absolute executable path", image.ImageKey, binary.Path)
+				}

Relevance

●● Moderate

Plausible extraction correctness issue, but no closely matching precedent for validating executable
file types.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The added validation only requires Linux, a nonblank image key, and a canonical absolute non-root
path; it has no file-type check. The new parity path calls the image extraction helper, and that
helper iterates the copied tar stream until it finds any regular entry, without matching the entry
name to the configured source path.

test/support/cli/config.go[95-99]
test/support/cli/parity.go[28-32]
test/support/image_utils.go[271-325]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

Issue description
A runtime mapping can name a directory even though the configuration and documentation require an executable path. The extraction helper then accepts the first regular tar entry, so a misconfigured directory mapping can compare the archive executable with an unrelated file inside that directory.

Fix Focus Areas
- test/support/cli/config.go[95-99]
- test/support/image_utils.go[271-325]

Recommended Fix
After copying the configured runtime path, inspect the returned container path metadata and reject directories and non-regular entries before extracting or hashing. Keep the existing canonical-path validation, and add a test showing that a directory runtime mapping fails rather than comparing a descendant file.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Context sources

✅ Cross-repo context — repo relationships

  Explored: repo: securesign/gitsign (sha: 9b580f08)

  Explored: repo: securesign/cosign (sha: d493674f)

  Explored: repo: securesign/releases (sha: e5fe10fb)

  Explored: repo: securesign/trillian (sha: f0514f44)

  Explored: repo: securesign/rekor (sha: df4a220a)

  Explored: repo: securesign/timestamp-authority (sha: 3736f2d6)

Review mode: 🧠 Deep: This is a broad, behavior-changing parity feature spanning configuration, multi-platform image extraction, checksum validation, inventories, and acceptance/test paths, with many independent logic sites where subtle defects could be missed in one pass.

Tip of the day

💡 Did you know, you can route each action level your way: inline, summary, both, or drop

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗