feat(runtime): add platform-owned native build and Compose profiles

.github/actions/python-distributions/action.yml

@@ -60,6 +60,12 @@ runs:
 
         import adx_environment
         import akernel_sdk
+        from akernel_sdk import ComposeProject, DockerRuntime, ServiceHandle, ServiceProcessHandle
+        from importlib.resources import files
+
+        assert all(callable(value) for value in (ComposeProject, DockerRuntime, ServiceHandle, ServiceProcessHandle))
+        for helper in ("_image_bridge_guest.py", "_docker_network_guest.py", "_service_process_guest.py"):
+            assert files("akernel_sdk").joinpath(helper).read_text()
 
         assert version("akernel-sdk") == os.environ["EXPECTED_VERSION"]
         assert any(

.github/workflows/ci.yml

@@ -15,6 +15,34 @@ concurrency:
   cancel-in-progress: ${{ github.event_name == 'pull_request' }}
 
 jobs:
+  image-bridge:
+    name: Managed Docker image bridge
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - name: Set up runtime adapter tests
+        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        with:
+          python-version: "3.12"
+      - name: Test managed runtime adapters
+        run: python -m unittest discover -s builder/managed-docker -p 'test_*.py'
+      - name: Set up Go
+        uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6
+        with:
+          go-version: '1.25.5'
+          cache-dependency-path: builder/image-bridge/go.sum
+      - name: Test and build the host/guest helper
+        working-directory: builder/image-bridge
+        env:
+          CGO_ENABLED: '0'
+        run: |
+          test -z "$(gofmt -l .)"
+          go vet ./...
+          go test ./...
+          GOOS=linux GOARCH=amd64 go build -trimpath -o "$RUNNER_TEMP/akernel-image-bridge" .
+
   sdk-unit-tests:
     name: Python SDK unit tests (${{ matrix.python-version }})
     runs-on: ubuntu-latest

AGENTS.md

@@ -196,6 +196,13 @@ covers the archive, not a bare runsc binary. Keep CI's final-image payload
 checks synchronized; do not mix helper versions or fetch them at runtime.
 
 The sandboxd submodule supplies the source compiled by `make build`.
+For native Kubernetes snapshots spanning control/worker upgrades, preserve the
+qualified EROFS runtime bytes independently of the platform image. The opt-in
+`node.runtimeRootfsPin` stages a digest-pinned source runtime on the existing
+node data disk and verifies the source/current static Execd hashes before a
+read-only mount. Keep the pin unchanged during control-only upgrades and retain
+referenced cache files; do not relax runsc superblock checks or claim arbitrary
+guest-runtime migration. See `deploy/docs/runtime-rootfs-pinning.md`.
 Helm and standalone enable `runsc.background_restore` and `checkpoint_sync.sync_output`;
 runtime files and private writable layers remain on node-local storage.
 AKernel's `builder/distill-fs-versions.env` fixes the distill-fs release URL
@@ -963,3 +970,50 @@ ADX profile, applying patches only to present roles. Resolve profile/role
 selectors with ADX's single-pass environment/default/escape rules while
 preserving source expressions for the authoritative loader. Severity queries consume
 Collector-normalized log metadata and plain bodies.
+
+
+## Managed Docker runtime
+
+`build_image()` accepts optional preparation CPU/memory scheduling requests while retaining its existing 1000 millicore/2048 MiB limits. Validate requests before creation and retain cleanup on all submission outcomes. Reservation overrides are explicit load-test inputs, not additional runtime capacity; ordinary callers keep the existing defaults.
+
+Managed profile storage metadata is derived by `builder/managed-docker/profile_metadata.py` from the same Sandboxd profile and advertised through ordinary ADXlet node labels. The ADX backend reserves the profile floor and constrains placement before creation; node-side authorization and storage checks remain authoritative. Managed YAML must reach official Compose unchanged, including native merge tags. Service handles become available after runtime service discovery. Keep authorization allowlists outside the snapshot compatibility fingerprint, while checking current authorization on every restore.
+
+Coalesce managed resource probes within the authenticated backend and bound
+successful advisory views to one second. Do not cache failures or share views
+across backend credentials; discard inherited process state before locking.
+This does not limit concurrent creates or replace fresh server authorization.
+
+Foreground command deadlines raise TimeoutException with an explicit timeout
+label and available partial command_result. Preserve its native error code and
+retry/outcome guidance; application stderr is output, not the timeout reason.
+Build/Compose logs retain that partial output without replaying commands or
+replacing the primary error when local log persistence fails.
+
+Publish native builds under fresh invocation tags. Reconcile only the known
+post-publication Buildx history error, using official Docker inspection and
+verified immutable manifests for every planned output. Preserve the original
+error and fail on partial publication; do not replay an unknown build outcome.
+The exact post-publication internal deadline additionally requires every fresh
+target's completed push acknowledgement to match its inspected manifest digest;
+explicit attestations require native completion. Exit 100 alone proves neither
+history failure nor publication. Default optional metadata provenance to Docker's
+supported disabled mode while preserving operator overrides; do not claim this
+repairs the BuildKit server's history or storage paths.
+
+The experimental remote-build path in `builder/managed-docker` delegates Dockerfile and Compose parsing/builds to the official Docker CLI, Compose, and Buildx remote driver. Connect to an operator-provided BuildKit service; its native Nydus exporter publishes to the configured internal Distribution Registry. Single-image builds use the broker's metadata-only `/describe` result and leave final rootfs acquisition to the final sandbox; never mount a task image on a disposable build helper just to read its process defaults. Compose image preparation uses sandboxd's existing image manager and Dragonfly/ROFS cache, with a private, operator-authorized image collection. Keep the broker's explicit owner and collection-flock protocol synchronized with Sandboxd's node-local live/checkpoint leases; missing/copied lease markers fail closed, and unknown legacy collections require an operator drain. Neither local mount release nor an expired capability authorizes Registry artifact deletion. Cross-node reconstruction, per-tenant Registry authorization and artifact GC remain separate release requirements. Do not add Dockerfile instructions to the legacy SDK interpreter for this path. `ComposeProject.bind_runtime()` accepts a fresh runtime-prepared opaque descriptor without remote startup; checkpoint recovery must continue using pure `attach()` and captured tunnel descriptors.
+
+Client-managed DockerRuntime owns image discovery/OCI metadata validation, ROFS mount plans and guest daemon startup. With an opt-in platform-managed profile, the node supplies the pinned tool/bootstrap image, authorized collection, credentials and startup parameters; the SDK transfers task files and invokes runtime operations. Sources live in `builder/image-bridge`, `builder/dind`, and `sdk/python/akernel_sdk/_docker_runtime.py`. Callers supply original project files and keep service orchestration in their adapter. Snapshot restore must rebind mounts, runtime extra_config and local tunnels without rerunning startup. Reuse the existing sandboxd image provider; do not add a parallel layer cache or modify deployment/runtime pins for client refactoring. See the SDK and builder guides for managed-upper prerequisites and supported scope.
+
+Managed builds export native targets without adding a Python helper layer or changing task PATH. Whole-sandbox C/R is independent of guest interpreters and application quiescence. `ServiceHandle.start_command()` runs native Compose exec as a reconnectable sandbox command, with an outer Docker CLI PID and stable command ID; restore reattaches without replay or signals. Keep the explicit Python-based inner `start_process()` API optional.
+
+`BuiltImage.has_startup_command` reflects the effective OCI ENTRYPOINT/CMD
+returned by the runtime. Consumers that inherit image startup must disable
+inheritance for empty commands; such images remain valid interactive roots.
+
+Compose project lifecycle and Docker service/process operations are SDK extensions in `_compose.py` and `_service_process.py`; the dependency-free `_service_process_guest.py` runs in the selected service PID namespace. Keep project descriptors JSON-compatible and restore attachment free of fresh-boot operations. Preserve explicit build/network-policy/up boundaries, unknown-outcome identities, source fencing, and container/process start-time checks before signals. Callers retain task/harness semantics. Run the real-process helper unit tests with owned-process cleanup and the complete SDK quality gate when changing these contracts. This feature does not add Docker semantics to Agent DX or change runtime pins.
+
+Automatic Compose image preparation covers image-only services as well as Dockerfile bases. Verify pinned Docker references and committed ROFS snapshots instead of running native pulls, including during up. Keep digest-reference regression coverage and fail closed on missing registrations; never relax bridge extraction protection or populate the private upper with replacement lower layers.
+
+Managed discovery requires helper schema 2 and probes capabilities before sending project input. Confine client Dockerfile/build-context and `.env` reads to the frozen project; never inherit unspecified host environment variables. External include/extends files remain unsupported in client-managed discovery. Validate manifest/config/chain relationships and allow legitimate same-chain roots with different target digests. Keep the empty automatic-plan marker for scratch-only builds. Unknown command exits and failed cancellation retain their identities and cleanup errors without replay.
+
+Managed Compose defaults preserve native IPv4 bridge definitions. Keep forwarding/NAT and per-service tunnel exposure in the SDK/runtime, with daemon-only legacy iptables selection for Docker DNS. Restore attachment stays pure; explicit reconnect_network refreshes only owned chains for the new outer TAP before the harness resumes. Do not claim external TCP address migration. Keep the opt-in Compose network integration regression, including existing inner and published-port sockets, synchronized with these APIs. Docker 28 operator flags are prerequisites; client work must not change production or runtime bundle pins.

builder/dind/Dockerfile

@@ -0,0 +1,12 @@
+# Build from the repository root: docker build -f builder/dind/Dockerfile .
+ARG GO_BUILD_IMAGE=golang:1.25.5-bookworm
+FROM ${GO_BUILD_IMAGE} AS bridge
+WORKDIR /src
+COPY builder/image-bridge/go.mod builder/image-bridge/go.sum ./
+RUN go mod download
+COPY builder/image-bridge/ ./
+RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -o /out/akernel-image-bridge .
+
+FROM docker:28.3.3-dind
+RUN apk add --no-cache bash python3 coreutils util-linux shadow tar xz
+COPY --from=bridge /out/akernel-image-bridge /opt/akernel-image-bridge

builder/image-bridge/NOTICE

@@ -0,0 +1,5 @@
+Managed Docker image bridge source attribution
+
+The snapshotter derives from tianyuzhou95's agent-worker implementation at
+commit 9b3434d36086d920b6bb7f3322da642381a254b8. Retain this attribution when
+redistributing the derived helper. Repository license terms apply.

builder/image-bridge/README.md

@@ -0,0 +1,29 @@
+# Managed Docker image bridge
+
+This helper exposes sandboxd-provided immutable ROFS image mounts as committed containerd snapshots, with private upper/work directories owned by the sandbox. It does not introduce another registry layer cache. Metadata-only image registration and source Dockerfile/Compose discovery use containerd, compose-go and BuildKit parsers.
+
+Source attribution is retained in [NOTICE](NOTICE).
+
+The operator packages this helper in the [managed runtime](../managed-docker/README.md). Platform-managed clients do not need a local helper, Go toolchain, Registry credentials or node paths. The following discovery/startup details apply to client-managed mode.
+
+Build a Linux/amd64 helper from a host with the Go version required by [go.mod](go.mod):
+
+```sh
+cd builder/image-bridge
+CGO_ENABLED=0 go test ./...
+CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -o /path/to/akernel-image-bridge .
+```
+
+In client-managed mode, the Python SDK's `prepare_compose_images()` invokes `--mode discover`, resolves tags and verifies OCI manifest/config bytes. Credentials stay on the client; the runtime image manager must independently be authorized to fetch the pinned images. Only metadata is uploaded. Layer bytes are mounted by the existing sandboxd OCI/ROFS provider, which owns node-wide caching and distribution.
+
+`DockerRuntime.sandbox_options()` supplies the ROFS mounts and `fileBackedTmpfsMounts` to fresh creation and snapshot restore. `DockerRuntime.start()` uploads the helper when configured, starts its private containerd and bridge, registers original image aliases, then starts dockerd. Call start only once on a fresh sandbox; never on a restored sandbox. Failed startup is cleaned up by the owner deleting the newly created sandbox, not by touching shared node state.
+
+Automatic Compose uses these mounted roots for image-only services as well as build bases. After registration, the SDK verifies aliases against their pinned target digests and checks the committed ROFS snapshots. It does not use native Compose pull for those inputs: Docker Compose can pull digest references even with `--policy missing`, which would incorrectly enter the ordinary layer extraction path. Automatic `up` also disables repeat pulls. An absent registration fails instead of bypassing the mount plan or silently copying image layers into private storage. Different image manifests/configurations can legitimately share one chain ID; the validated configuration and chain bind their ROFS roots before verification allows that sharing.
+
+Client-managed discovery requires **discovery schema 2**. Check with `akernel-image-bridge --mode capabilities` before uploading project data. The probe sends no project data. Discovery rejects Dockerfile/build-context paths or `.env` links escaping the frozen project and does not inherit unspecified client environment variables. External `include` / `extends.file`, remote contexts and platform emulation are outside client-managed discovery; pass explicit supported Compose files instead. Use matching SDK and helper capability contracts.
+
+Use `DockerRuntime.default_image`, the public Docker Hub `docker.io/library/docker:28.3.3-dind` image. Fresh startup installs missing Alpine support packages, so no custom runtime image build or push is required. An explicitly supplied outer image must provide Docker 28.3.3, containerd/ctr, Python 3, findmnt/mount, Bash, iproute tools, iptables-legacy and iptables-legacy-restore. `builder/dind/Dockerfile` optionally packages these and the guest helper; use repository root as build context. The SDK can upload the matching helper from the client. No helper binary is downloaded automatically.
+
+The default Linux/amd64 runsc profile preserves default/custom IPv4 Compose bridge networks, DNS and TCP/UDP userland port mappings. The operator must enable `extra_args = ["--net-raw=true", "--allow-packet-socket-write=true"]` under `[plugin.runtime.runsc]` for Docker 28. The SDK owns in-sandbox legacy forwarding/SNAT and per-service tunnel entries; Docker owns bridges/IPAM/DNS. After restore call `ComposeProject.reconnect_network()` to rebind the new outer TAP without restarting Docker. Task Dockerfiles and Compose files remain unchanged. Host networking is an explicit compatibility option. Remote contexts, runtime-created unknown image references and platform emulation remain outside this profile. Client-managed native builds run inside the sandbox; the platform-managed profile delegates builds to remote BuildKit. This is not a complete shared Docker content store: exporting/saving/pushing an image may still fetch layers.
+
+The file-backed upper requires sandboxd managed tmpfs support and runsc checkpoint support for those mounts. It consumes the sandbox storage limit. Reflink must be configured by the deployment, on a compatible filesystem and same mount; the SDK does not change deployment durability policy or replace runtime bundles. Memory mode is an explicit alternative and counts file contents toward memory snapshots.