Repository navigation
feat(vagrant): publish the boxes in the AlmaLinux Vagrant box catalog on S3 - #379
Merged
Merged
Conversation
… on S3 HCP Vagrant stopped accepting new box versions on 2026-10-01 and shuts down on 2026-12-31. The almalinux/* boxes are served from our own catalog on S3 instead: one Vagrant metadata JSON per box at s3://<AWS_S3_BUCKET>/vagrant/almalinux/<box>[.json], public, served as application/json, pointing at the .box files the build uploads to S3 - users set VAGRANT_SERVER_URL to https://<AWS_S3_BUCKET>.s3.us-east-1.amazonaws.com/vagrant. "Vagrant: Build, Test and Publish" now publishes there instead of HCP: - each build leg whose box is on S3 and passed its test (the vagrant up test, or the offline validation for hyperv and the aarch64 legs) records it (file name, S3 key, sha256) as a catalog-entry artifact; - a new publish-catalog job, after all builds, adds them all at once with .github/scripts/vagrant-catalog-add.py: one new version per catalog box with all its providers (a re-run replaces, never duplicates), the previous file kept under vagrant/.history/<time stamp>/, the new <box> and <box>.json uploaded and checked over HTTPS. A vagrant-catalog concurrency group keeps runs from interleaving their updates; - release_to_catalog (default true, so the scheduled runs publish) replaces release_to_hcp; publishing happens in almalinux/cloud-images only, as forks and ci-dev share the bucket; - the HCP publish steps and the publish-aarch64 job are gone (the catalog needs no Vagrant CLI). tools/vagrant_catalog.py, the generator the catalog was seeded with (from a snapshot of the HCP registry metadata, plus the releases built after HCP closed), is committed, to rebuild the catalog from scratch if ever needed. Docs updated.
yuravk
force-pushed
the
vagrant-s3-catalog
branch
from
October 19, 2026 10:54
cb0ed6d to
8e49863
Compare
andrewlukoshko
approved these changes
Oct 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
HCP Vagrant stopped accepting new box versions on 2026-10-01 and shuts down on 2026-12-31 (#363). Since 2026-10-01, new
almalinux/*box versions are built, tested and uploaded to S3, but nothing publishes them forvagrant box add/vagrant box update.The replacement is a Vagrant box catalog on our S3 bucket, already live and verified with Vagrant:
s3://almalinux-cloud/vagrant/almalinux/<box>[.json], public, served asapplication/json;.boxfiles the builds upload to S3;.sha256sumfiles were generated. It also has the releases built since HCP closed (9.8.20261005, 10.2.20261012, the VirtualBox arm64 boxes).Users switch with one variable:
This PR makes "Vagrant: Build, Test and Publish" add new versions to that catalog, instead of publishing to HCP.
What changes
vagrant-build-test-publish.ymlcatalog-entry-<variant>-<leg>artifact (file name, S3 key, sha256). It does so only when the box is on S3 and passed its test: thevagrant uptest, or the offline validation on the Hyper-V and aarch64 legs, which have no live test.publish-catalogjob runs after all builds and adds the recorded boxes at once with.github/scripts/vagrant-catalog-add.py:default_architecture(amd64 where present);vagrant/.history/<time stamp>/;<box>(forVAGRANT_SERVER_URL) and<box>.json, taggedpublic=yes, withmax-age=300;vagrant-catalogconcurrency group makes catalog updates wait for each other, across runs too, so two runs can't interleave download-change-upload on the same file. Users never see a partly published version.release_to_catalogreplacesrelease_to_hcp. It defaults totrue, so the scheduled runs publish, and it needsupload_to_s3.AlmaLinux/cloud-imagesonly. Forks and ci-dev use the same bucket, and must not change the production catalog.publish-aarch64job are removed. The arm64 legs no longer need an x86_64 runner, because the catalog needs no Vagrant CLI..github/scripts/vagrant-catalog-add.py(new): adds the recorded boxes to the catalog, as described above. It reuses the box-name parsing oftools/vagrant_catalog.py, e.g.AlmaLinux-10-Vagrant-vmware-10.2-20261012.0.aarch64.box->almalinux/1010.2.20261012vmware_desktop/arm64.tools/vagrant_catalog.py: the generator the live catalog was seeded with, from a snapshot of the HCP registry metadata plus the releases built after HCP closed (--extra). It is kept to rebuild the whole catalog from scratch, if ever needed.Docs:
VAGRANT_BUILD_TEST_PUBLISH.mdgets a new "The Vagrant box catalog" section.SCHEDULED_BUILDS.md,BUILD_VAGRANT.md,README.mdand thescheduled-builds.ymlcomment are updated.The standalone
vagrant-publish.ymland itsvagrant-publish-stepscomposite still publish to HCP only, and are left as they are.Requirements
The CI bot's
almalinux_cloud_s3policy allowss3:PutObjectands3:PutObjectTaggingonalmalinux-cloud/vagrant/*(already added).