Make CLI and Ansible release tests configurable

README.md

@@ -60,6 +60,78 @@ To run model validation operator tests use:
 go test -v ./test/acceptance/model_transparency/... --ginkgo.v
 ```
 
+## CLI stack configuration
+
+CLI stack checks run inside the `rhtas`, `model_transparency`, and
+`policy_controller` acceptance suites. Each product owns its expected inventory:
+
+```yaml
+rhtas:
+  cliStack:
+    images:
+      - imageKey: tufcli-cli-stack-image
+        binaries:
+          - {path: /binaries/tufcli_linux_amd64.tar.gz, os: linux, arch: amd64}
+```
+
+Set `TEST_CONFIG` to a local or remote YAML file, as for operator and FBC tests.
+The example above replaces the complete default inventory with one archive;
+list every image and archive the release must publish. Missing configured images
+or archives fail the tests. Invalid CLI configuration also fails. Archive paths
+must be under `/binaries/`; both `.tar.gz` and `.exe.gz` packages are read as tar
+archives. Executable format and architecture are checked without running binaries.
+
+Omitting `cliStack` or its `images` field inherits the product defaults.
+`cliStack: {images: []}` explicitly disables CLI stack checks. An unwrapped
+`cliStack` section applies only to RHTAS. Product sections remain independent.
+
+RHTAS defaults describe 1.5.x (including tufcli). The 1.4.x override retains tuftool
+and model-transparency archives. Older example configurations disable CLI stack
+checks. Model Transparency defaults include its CLI stack with standard, PKCS11,
+and OTel variants. Policy Controller defaults to an empty inventory. Each product’s
+release YAML can replace its inventory or explicitly disable it.
+
+```sh
+VERSION=1.5.0 SNAPSHOT=/path/to/1.5.0/snapshot.json \
+  go test -v ./test/acceptance/rhtas --ginkgo.focus='CLI Stack Images'
+
+VERSION=1.4.3 SNAPSHOT=/path/to/1.4.3/snapshot.json \
+  TEST_CONFIG=testdata/testconfig-1.4.yaml \
+  go test -v ./test/acceptance/rhtas --ginkgo.focus='CLI Stack Images'
+```
+
+Relative `TEST_CONFIG` paths resolve from this repository's root. The 1.4.x
+example configures CLI checks and enables Ansible; combine these sections
+with the release's operator, Ansible image lists, and FBC overrides for a full run.
+Client-server checks retain the legacy tuftool comparisons and skip RHTAS 1.5+.
+Set `VERSION` when the snapshot path does not contain a release version; existing
+version resolution treats an unspecified version as the latest release.
+
+Release CI migration (in `securesign/releases`, separately): remove the standalone
+`cli_stack` package selection/`CLI_STACK` flag, run the appropriate product suite,
+and supply that release's inventory in `testingResources/structural-tests.yaml`.
+Use explicit empty inventories for releases without CLI stacks. Existing FBC suite
+registration reads the snapshot even when CLI checks are focused, so these examples
+use complete product snapshots.
+
+## Optional Ansible checks
+
+RHTAS 1.5 defaults disable Ansible because the release no longer publishes it.
+Control the suite in each release's `TEST_CONFIG`, independently of its version:
+
+```yaml
+rhtas:
+  ansible:
+    enabled: false
+```
+
+Set `enabled: true` for older releases that ship Ansible, or a future release
+that restores it. The older-stream example configurations enable it explicitly.
+Partial Ansible overrides preserve the default flag and image lists; setting
+only `enabled` does not discard those lists. When enabled, the collection image
+is still required for releases from 1.2 onward. Disabled checks skip before
+loading the collection, snapshot, or repository metadata.
+
 ## Repository List
 The [repositories.json](testdata/repositories.json) file is used to check of all images are published correctly. To pull the list of repositories from Pyxis API:
 

test/acceptance/model_transparency/cli_stack_images_test.go

@@ -0,0 +1,5 @@
+package acceptance
+
+import "github.com/securesign/structural-tests/test/support/cli"
+
+var _ = cli.DescribeCLIStackImageTests(product, defaults)

test/acceptance/model_transparency/defaults.yaml

@@ -0,0 +1,19 @@
+cliStack:
+  images:
+    - imageKey: model-transparency-cli-stack-image
+      binaries:
+        - {path: /binaries/model_transparency_cli_linux_amd64.tar.gz, os: linux, arch: amd64}
+        - {path: /binaries/model_transparency_cli_linux_arm64.tar.gz, os: linux, arch: arm64}
+        - {path: /binaries/model_transparency_cli_linux_ppc64le.tar.gz, os: linux, arch: ppc64le}
+        - {path: /binaries/model_transparency_cli_linux_s390x.tar.gz, os: linux, arch: s390x}
+        - {path: /binaries/model_transparency_cli_pkcs11_linux_amd64.tar.gz, os: linux, arch: amd64}
+        - {path: /binaries/model_transparency_cli_pkcs11_linux_arm64.tar.gz, os: linux, arch: arm64}
+        - {path: /binaries/model_transparency_cli_pkcs11_linux_ppc64le.tar.gz, os: linux, arch: ppc64le}
+        - {path: /binaries/model_transparency_cli_pkcs11_linux_s390x.tar.gz, os: linux, arch: s390x}
+        - {path: /binaries/model_transparency_cli_otel_linux_amd64.tar.gz, os: linux, arch: amd64}
+        - {path: /binaries/model_transparency_cli_otel_linux_arm64.tar.gz, os: linux, arch: arm64}
+        - {path: /binaries/model_transparency_cli_otel_linux_ppc64le.tar.gz, os: linux, arch: ppc64le}
+        - {path: /binaries/model_transparency_cli_otel_linux_s390x.tar.gz, os: linux, arch: s390x}
+        - {path: /binaries/model_transparency_cli_darwin_amd64.tar.gz, os: darwin, arch: amd64}
+        - {path: /binaries/model_transparency_cli_darwin_arm64.tar.gz, os: darwin, arch: arm64}
+        - {path: /binaries/model_transparency_cli_windows_amd64.exe.gz, os: windows, arch: amd64}