From 1e287fe573546fd8ebab38e987604e33afeb0a07 Mon Sep 17 00:00:00 2001 From: Isaac Karrer Date: Tue, 29 Sep 2026 10:49:39 -0500 Subject: [PATCH] docs(floxhub-onprem): document object storage (ENT-352) Document per-catalog object storage, publish credentials, signing, and client trust. Link to the shared signing-key guidance and explain verification with independent stores. Co-Authored-By: Morgan Helton Co-Authored-By: Claude Opus 5 --- .../administration/object-storage.mdx | 195 +++++++++++++++++- llms.txt | 2 +- styles/config/vocabularies/Flox/accept.txt | 2 + 3 files changed, 189 insertions(+), 10 deletions(-) diff --git a/floxhub-onprem/administration/object-storage.mdx b/floxhub-onprem/administration/object-storage.mdx index e83b59d..70b9f3d 100644 --- a/floxhub-onprem/administration/object-storage.mdx +++ b/floxhub-onprem/administration/object-storage.mdx @@ -1,15 +1,192 @@ --- title: "Object storage" -description: "Configuring an S3-compatible store for published packages." +description: "Configure an S3-compatible store so published packages carry binaries, sign them, and let consumers trust and substitute them." --- - - **This page is under construction.** FloxHub on-prem documentation is - being written section by section. This page is a placeholder so the - surrounding navigation and links are complete. - +Publishing a package to your deployment records two different things: the +package's metadata in the catalog, and the built binaries themselves. The +catalog always takes the metadata. Binaries need somewhere to go, and that +somewhere is an S3-compatible object store you supply. -## Planned contents +## The default: metadata only -Ingress and egress URLs, publisher credentials, signing keys, consumer read -access and trust, and how to verify that publishing and installation work. +A new deployment has no object store, and catalogs default to **metadata +only**. `flox publish` succeeds against such a catalog, and records everything +the catalog needs to resolve the package — but uploads no binaries, so nothing +can substitute the build. A consumer resolving that package has metadata and no +artifact to fetch. + +That default is deliberate: publishing works out of the box without making you +provision a bucket first. Configure a store when you want published packages to +be installable without rebuilding. + +## Store types + +| Type | Behavior | +| --- | --- | +| `meta-only` | Records metadata, uploads nothing | +| `nix-copy` | Copies the build closure to an S3-compatible store you own | +| `null` | Unconfigured. Publishing is refused until you choose a type | +| `publisher` | A credential-minting service. Not available on-prem | + +`nix-copy` is the type to use. The `publisher` type belongs to the hosted +service, which runs a component your deployment does not. + + + A catalog set to `null` refuses publishing with a `422`. The error text names an + internal Flox tool; configure the store with the API call below instead. + + +## Configure a catalog's store + +Store configuration is per catalog, applied over the catalog API. Create the +catalog if it does not exist, then set its store: + +```bash +curl -fsS -X POST --oauth2-bearer "$TOKEN" \ + "${FLOXHUB_SITE_URL}/api/v1/catalog/catalogs/?name=${CATALOG}" +``` + +```bash +curl -fsS -X PUT --oauth2-bearer "$TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "store_type": "nix-copy", + "ingress_uri": "s3://?endpoint=https://", + "egress_uri": "https:///" + }' \ + "${FLOXHUB_SITE_URL}/api/v1/catalog/catalogs/${CATALOG}/store/config" +``` + +Read a catalog's current store back with the same path and `GET`. Setting a +store needs write access to that catalog. + +### Ingress and egress are different URLs + +| Field | Direction | Shape | +| --- | --- | --- | +| `ingress_uri` | Writes, during publish | An `s3://` URI naming the bucket, with the endpoint as a query parameter | +| `egress_uri` | Reads, during install | A plain HTTP URL serving the same bucket as a binary cache | + +They address one bucket by two routes. Publishing uses the S3 protocol and +needs credentials; installing is an ordinary cache read over HTTP. Add +`&scheme=http` to the ingress URI when your endpoint is not TLS-terminated. + +## Credentials belong to whoever publishes + +This is the part that differs most from the hosted service. + +With `nix-copy`, FloxHub hands the publishing client the bucket URI and nothing +else. It does not mint, hold, or proxy object-store credentials. **Whoever +publishes must already be able to write to the bucket**, authenticating with +standard AWS environment variables in their own environment: + +```bash +export AWS_ACCESS_KEY_ID= +export AWS_SECRET_ACCESS_KEY= +``` + + + Distributing write credentials for the store is yours to arrange, and there is + no per-user scoping inside FloxHub. Anyone who can publish to a catalog needs + bucket write access, and holds it directly. + + +Read access is separate and needs no credential if the egress URL serves the +bucket anonymously, which is the simplest arrangement. A private bucket means +every consumer needs read credentials too. + +## Sign what you publish + +Consumers will not substitute an unsigned artifact. Generate a binary cache key +pair once, and keep the private half with your deployment's other secrets: + +```bash +nix-store --generate-binary-cache-key \ + ./catalog-signing-private-key ./catalog-signing-public-key +``` + +```bash +chmod 600 ./catalog-signing-private-key +``` + +Pick a stable ``: it is recorded in every signature, and consumers +trust the name. Changing it invalidates trust for everything already published +under the old one. + +Point publishing at the private key, either once for an account: + +```bash +flox config --set publish.signing_private_key /absolute/path/to/private-key +``` + +or per publish: + +```bash +flox publish --signing-private-key /absolute/path/to/private-key +``` + +The first form is what a deployment's own build machinery uses, so its +publishes are signed without passing the flag each time. + + + The private key path must be absolute. A relative path is rejected. + + + + `nix-store --generate-binary-cache-key` needs neither the `nix-command` nor + the `flakes` feature, so it works on a host where those are unavailable. + [Signing keys](/customer/signing-keys) shows the equivalent `nix key` + commands, which do need them. + + +## Let consumers trust the key + +A client substitutes from your store only once it trusts the signing key. Add +the **public** key to the client's trusted keys, and your store as a +substituter: + +```text +extra-trusted-public-keys = : +extra-substituters = https:/// +``` + +The value is the contents of the public key file, and it is not secret — +distribute it however you distribute client configuration. + +Where those settings live differs by how the client installed Nix, and applying +them needs the Nix daemon restarted. +[Signing keys](/customer/signing-keys) covers both in full, for standalone Nix, +NixOS, nix-darwin, and home-manager, along with how to verify a key is trusted. +Everything there applies unchanged to an on-prem store. + +## Verify it works + +Publishing succeeding is not proof that the store works; a metadata-only +publish also succeeds. Prove the round trip instead, using **two machines** with +independent Nix stores: + +1. On the first, publish a package to the configured catalog with a signing key. +2. On the second — one that has never built the package — install it. + +If the second machine installs without building, the closure reached the store +and came back out of it. Doing this on one machine proves nothing: a local store +already holds the build, so the install would succeed with no object store at +all. + +Check along the way: + +```bash +curl -fsI "https:////nix-cache-info" +``` + +A readable `nix-cache-info` confirms the egress URL is serving as a binary +cache. If publishing fails, the cause is almost always bucket credentials or an +ingress endpoint that does not match your store. + +## Deployment-wide defaults + +Store configuration is per catalog today, so each catalog that should carry +binaries is configured on its own. A deployment-wide default store, so new +catalogs inherit one rather than being configured individually, is being added +and will be documented here. diff --git a/llms.txt b/llms.txt index e62ee95..01bc712 100644 --- a/llms.txt +++ b/llms.txt @@ -143,7 +143,7 @@ Key terms: - [Configure FloxHub](https://flox.dev/docs/floxhub-onprem/administration/configure.md): Where site configuration lives and how to apply it. - [Environment variables](https://flox.dev/docs/floxhub-onprem/administration/environment-variables.md): The site configuration contract and its defaults. - [Log system](https://flox.dev/docs/floxhub-onprem/administration/logs.md): Where each service writes its logs and how to read them. -- [Object storage](https://flox.dev/docs/floxhub-onprem/administration/object-storage.md): Configuring an S3-compatible store for published packages. +- [Object storage](https://flox.dev/docs/floxhub-onprem/administration/object-storage.md): Configure an S3-compatible store so published packages carry binaries, sign them, and let consumers trust and substitute them. - [PostgreSQL](https://flox.dev/docs/floxhub-onprem/administration/postgresql.md): The databases FloxHub uses and how they are configured. - [Base catalog](https://flox.dev/docs/floxhub-onprem/administration/base-catalog.md): Populating and refreshing the package catalog. - [Secure FloxHub](https://flox.dev/docs/floxhub-onprem/administration/security.md): The security model of a FloxHub deployment. diff --git a/styles/config/vocabularies/Flox/accept.txt b/styles/config/vocabularies/Flox/accept.txt index 0fb68ae..d2e7aa3 100644 --- a/styles/config/vocabularies/Flox/accept.txt +++ b/styles/config/vocabularies/Flox/accept.txt @@ -105,6 +105,7 @@ subproblems subprojects subshell subshells +substituter systemd tcsh toolchain @@ -113,6 +114,7 @@ toplevel transactionally tzdata UIDs +unconfigured uninstallation unintuitive unmerged