Skip to content

Allow checking out other repos in CI - #1780

Merged
jpwilliams merged 22 commits into
mainfrom
jack/ci-checkout-repo-v3
Oct 14, 2026
Merged

jpwilliams merged 22 commits into
mainfrom
jack/ci-checkout-repo-v3

Conversation

@jpwilliams

Copy link
Copy Markdown
Member

Summary

Allows checking out any repository, branch, tag, or commit from a ci.job(), as well as key a cache on another repository's files.

  • checkout() now accepts checkout({ repo, ref })
  • files("...") now accepts files("...", { repo, ref })
  • Means that jobs with no natural repository such as cron runs can specify a repo too
  • The chosen ref is resolved to a single commit and memoised per run
await checkout({ repo: "example/platform" });
await checkout({ repo: "example/platform", ref: "v2.1.0" });
await checkout({ ref: "main" });
const image = ci.job({
  id: "image",
  cache: {
    key: files("images/node-base/**", { repo: "example/platform", ref: "main" }),
  },
}, async () => {
  await checkout({ repo: "example/platform", ref: "main" });
  await $`docker build images/node-base`;
});

Checklist

  • Added a docs PR that references this PR N/A Later bump
  • Added unit/integration tests
  • Added changesets if applicable

x and others added 20 commits October 9, 2026 23:02
Jobs, commands and extra machines become spans, CI's steps are named for what
they do and marked as CI's work. A job's `name` sets its span name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every span is an entry in the `spans` catalog with a required kind, step IDs
are built by `joinId`, and starting a machine is a flat staged function in
`machine/start.ts`. Folds the CI half of ci2-span-kinds: commands, sandboxes,
saves and attempts carry kinds, each statement on a background process is a
row of its own, and a snapshot's span is passed to an SDK that takes it
(feature-detected by version) and opened by CI otherwise. Names the checkout
steps. Adds a `ciTest` harness.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Moving it to its own file cost more diff than it saved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The span API and a snapshot's own span are always there, so the stub, the
version gate and the dual snapshot-span path are gone. The shared test
harness had one user, so it is local to that test again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he lookups

CI's step IDs and names move to one catalog in pipeline/names.ts, run through
a thin ciRun helper that tags the step as before. Sandbox paging and snapshot
deletes (machine/admin.ts), snapshot names (cache/names.ts), the cache lookup
and the check-complete payload each have one implementation. No step ID, name
or wire format changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ef })

Adds repo and ref to checkout() and files(), resolved to a commit once per run, with the CheckoutSource list, once() and localRepo().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… calls

A later direct call has the path `id (n)`, which is not a registry key, so
its span lost the configured name. The span now takes the name from the
job's config and keeps the call number.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adapt the job span name fix into the steps catalog refactor: names.ts keeps
the catalog and takes traceName.job(config, path) from the base.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…al to main's helpers)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The first lookup mints a token narrowed to the repository under the run's own
installation. For a repository outside that installation GitHub answers 422,
which wasn't swallowed, so the installation lookup never ran.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A malformed `repo` in `files()` only failed when the run resolved it. It now
throws CiUsageError where the pipeline is defined, so it fails at boot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Enterprise Managed User accounts (`name_shortcode`) own repositories, and
`repo` accepted them before the owner pattern was added.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
checkout() has two ways to fill the machine, the local working tree and a
GitHub clone. The CheckoutSource/CheckoutPlan abstraction existed for a
speculative third. No behaviour change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The commit lookup accepts HEAD, so a repository with no ref resolves in its
one ref step. This drops the separate default-branch step, its run-scope map
and its step name. Step IDs for a repo without a ref are now
`github › ref:owner/name@HEAD`; the branch is unreleased, so no in-flight run
depends on the old IDs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Turn the repeated checkout, installation, local-run and files() cases in
source.test.ts into test.each tables and merge near-duplicate tests. The
same behaviours are covered. fakeGitHub.handle stays: the installation
fallback test replies by Authorization header.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jpwilliams jpwilliams self-assigned this Oct 14, 2026
Copilot AI balanced review requested due to automatic review settings October 14, 2026 01:23
@changeset-bot

changeset-bot Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 9c9fedc

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@inngest/ci Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

External refs are not pinned across invoked cache-build runs, and empty source options can be silently discarded.

2 open findings
What changed in this PR

Adds cross-repository and ref-aware checkout and cache-file resolution to @inngest/ci.

Changes:

  • Adds memoized repository/ref resolution with installation-aware authentication.
  • Extends checkout() and files() with { repo, ref }.
  • Adds tests, documentation, and a changeset.
File Description
.changeset/​ci-checkout-repo.md Records the minor release.
packages/​ci/​README.md Documents cross-repository usage.
packages/​ci/​src/​cache/​cache.ts Resolves and hashes files from selected sources.
packages/​ci/​src/​checkout/​checkout.ts Checks out resolved external sources.
packages/​ci/​src/​github/​README.md Lists the new source module.
packages/​ci/​src/​github/​auth.ts Adds repository-scoped installation authentication.
packages/​ci/​src/​github/​source.test.ts Tests source resolution and authentication.
packages/​ci/​src/​github/​source.ts Implements memoized source resolution.
packages/​ci/​src/​index.ts Exports FilesOptions.
packages/​ci/​src/​pipeline/​names.ts Names ref-resolution steps.
packages/​ci/​src/​pipeline/​pipeline.ts Initializes source memoization.
packages/​ci/​src/​pipeline/​scope.ts Stores resolved sources per run.
packages/​ci/​src/​testing/​fakeGitHub.ts Supports request-aware fake responses.
packages/​ci/​src/​types.ts Defines file source options.
packages/​ci/​src/​util.ts Adds repository parsing and memoization helpers.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread packages/ci/src/cache/cache.ts
Comment thread packages/ci/src/cache/cache.ts
@greptile-apps

greptile-apps Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 4/5

[High impact] Adds repository and ref resolution to the CI checkout system.

Pin remote commits across cached builds before merging so moving branches cannot break valid jobs.

Findings

  1. P1 Moving branches break cached builds ▶
  2. P2 Narrower patterns cannot fix truncation ▶
Fix with agent prompt
### Issue 1
packages/ci/src/cache/cache.ts:353
If a remote branch moves before a cached job's build starts, the build can fail permanently. `resolveKeySources()` pins the commit in the calling run, but `invokeBuild()` sends only the cache key, not the resolved sources. The build starts with an empty `sources` map and resolves the branch again. If matched files changed, `verifyBuildTarget()` rejects the build because its key differs.

Pass the pinned sources to the build and reuse them when checking the key and running `checkout()`.

### Issue 2
packages/ci/src/cache/cache.ts:221
The new error tells users to use narrower patterns, but both GitHub paths fetch the entire recursive tree. `hashTree()` rejects a truncated tree before applying patterns, so users following that advice get the same failure.

Remove that advice, or fetch only the requested subtrees so narrower patterns actually work.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds repository and ref options to checkout() and files(), with shared commit lookup within each run.

  • CI jobs can check out another repository at a chosen ref.
  • Cache keys can follow files in another repository and ref.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A["checkout or files with repo/ref"] --> B["Resolve commit once in calling run"]
  B --> C["Checkout pinned commit"]
  B --> D["Hash matched files"]
  D --> E["Invoke cache build with computed key"]
  E --> F["Build starts with empty sources map"]
  F --> G["Resolve ref again"]
  G --> H{"Files still match caller's key?"}
  H -->|Yes| I["Reuse or build snapshot"]
  H -->|No| J["Reject build"]
Loading

Reviews (1) · Last reviewed commit: "refactor(ci): table the repeated source ..." · Reviewed by Greptile

Comment thread packages/ci/src/cache/cache.ts
Comment thread packages/ci/src/cache/cache.ts Outdated
…o/ref in files()

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he invoker's commits

Narrower files() patterns cannot help: the whole recursive tree is fetched
and rejected when GitHub truncates it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jpwilliams
jpwilliams merged commit 4491a64 into main Oct 14, 2026
46 checks passed
@jpwilliams
jpwilliams deleted the jack/ci-checkout-repo-v3 branch October 14, 2026 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants