Cask on Artifacts: how it works

The first write-up described a Git host whose authoritative copy of every repository is a log in object storage, with the bare repositories on the container's disk treated as caches. That design still exists, and it is still the rollback path. But Cask no longer stores Git that way. Every repository on cask.build now lives in Cloudflare Artifacts, a managed service that stores Git repositories and speaks the Git protocol itself.

This is a write-up of what moved, what did not, and the handful of things I learned by moving it carefully. As before, it is not a product pitch. Most of the work was deciding which component is allowed to be the source of truth for what, and then making sure nothing else quietly disagreed.

Why move at all

The log design is honest, but it is also a Git storage engine that I maintain. Every push path rule in the first post — spool the body, classify acceptance from the ref store, write the log before replying — exists because I own the durability of every object. That is a reasonable thing to own when nothing else will. Once a service exists whose whole job is to store Git repositories and serve the protocol, owning it myself stops being a design choice and becomes a maintenance cost.

What I did not want to give up was everything that sits around storage: the browser UI, pull requests, review, merges, webhooks, and the control plane that knows which repositories exist and who may touch them. Artifacts stores and serves Git. It does not render a file view or decide whether a branch is protected. So the shape of the change was narrow: replace the storage layer, keep everything above it.

Three pieces

There are now three components, and each has one job.

The Worker is the front door. It terminates every request to cask.build, checks Cask's existing credentials, and decides where the request goes. It is also the only component that can talk to Artifacts with authority: it holds the Artifacts binding and mints short-lived repository tokens from it.

Artifacts holds every object and every ref. It is the one durable copy of each repository, and it serves the fetch side of the Git protocol directly.

The container runs everything that is not storage: the UI, pull requests, the catalog, settings, webhooks, and the receive side of the protocol, which is where Cask's own rules about branches are enforced. It still keeps bare repositories on disk, and they are still projections — but now they are projections of Artifacts rather than of the log.

URLs did not change. A repository is still https://cask.build/{owner}/{repo}.git, and existing clones kept working without a remote update.

The Worker as an auth gateway

Clients never see Artifacts credentials. A clone or fetch arrives at the Worker with the same Basic or token credentials Cask has always accepted. The Worker checks them, confirms the repository exists in Cask's catalog, mints a repository-scoped token, and forwards the request to Artifacts with that token attached. The response streams straight back to the client.

The catalog check matters more than it looks. Artifacts knows about repositories; it does not know about Cask's catalog. Without the check, a repository deleted in Cask but not yet in Artifacts — or a name that only ever existed in Artifacts — would be reachable. With it, the Worker answers 404 for anything Cask does not list, and 401 for anything without valid credentials, before Artifacts is involved at all.

Tokens are minted for fifteen minutes and cached for most of that. Long enough that a busy morning does not mint one per request, short enough that a leaked token is a small problem. The container does not hold a token at any point. When it needs to read or write Git, it makes an ordinary request to an internal hostname, and the Worker intercepts that outbound request, attaches a fresh token, and forwards it. The process with the most code in it has the least authority.

Fetch at the edge, push through Cask

Fetches go from the Worker straight to Artifacts. Pushes do not.

A push still enters the container, runs git receive-pack against the local projection, and passes through Cask's hooks. Only if every rule passes does the container push the accepted refs on to Artifacts, and only after Artifacts accepts them does the client see ok. If Artifacts refuses, the local refs are rolled back and the client gets an error. The ordering rule from the first post survives intact, with Artifacts in the place the log used to occupy: the client's belief about durability is created after the durable write, never before.

It is easy to confirm which component answered. The fetch advertisement now comes from Artifacts, which identifies itself differently from the stock Git server running in the container. I used that difference as a test: during the preview, every fetch advertisement had to come from Artifacts and every push advertisement had to come from the container, or the routing was wrong.

Branch protection is Cask's job

Artifacts accepts force pushes. That is correct behaviour for a Git store — a store should store what it is told — but it means protection cannot be delegated to it. In the pilot, a force push sent directly to an Artifacts repository rewrote the branch without complaint.

So Cask enforces protection itself, in a pre-receive hook it installs in every projection. The default branch, plus any branch named in configuration, refuses non-fast-forward updates and deletion. Topic branches remain freely rewritable. Because the hook runs in the container and every push goes through the container, there is no path from a Cask credential to a rewritten main. The hook applies under both storage backends, so the guarantee does not change if I roll back.

The same reasoning decided that pushes stay in the container at all. Forwarding receive-pack straight to Artifacts from the Worker would have been fewer hops, and it would have removed the one place protection can live.

No atomic pushes, so leases instead

Artifacts does not support git push --atomic. A push touching three refs can, in principle, land two of them. Cask's own contract is all-or-nothing, so the container cannot simply forward the client's push and hope.

Instead, each ref is pushed to Artifacts with an explicit lease: update this ref only if it is still at the object ID I last saw. Leases do work in Artifacts. If any ref in the set is refused, the refs that did land are put back to their previous values under the same kind of lease, the local projection is rolled back, and the client sees a failure. It is more machinery than a single atomic call, but every step is a plain Git operation whose result is checked rather than assumed.

Merges use the same path. A pull request merge is computed in the container's projection, then pushed to Artifacts with a lease on the target branch. The pull request is only marked merged once Artifacts reports the branch at the merge commit.

The container's copy is disposable

I had two options for the container: keep a synced mirror that is pushed to Artifacts in the background, or treat the local repository as a cache that is rebuilt from Artifacts on demand. A synced mirror is two durable copies, and two durable copies means a reconciliation problem the first time they disagree.

So the container keeps a projection, exactly as before. Before work that reads Git, it checks a marker; if the projection is older than a short window, it fetches every ref from Artifacts with pruning, so the local copy matches Artifacts exactly, including deletions. Writes go to Artifacts first, as above. If the container is replaced, the next request rebuilds the projection and nothing is lost, because the projection never held anything Artifacts did not.

A few operations are refused under Artifacts rather than half-implemented. Changing a repository's default branch is one: Artifacts sets the default at creation, and I would rather return a clear 409 than have the UI and the store disagree about which branch HEAD names.

The default branch has to arrive first

This is the lesson I would most like to have read before starting.

An Artifacts repository's HEAD follows the first branch that is pushed into it. In the first pilot I pushed every ref of an existing repository in one go, and HEAD ended up pointing at whichever branch happened to arrive first, which was not main. Clones checked out the wrong branch. Re-pushing did not move it. The fix was to delete the repository, recreate it with the default branch set explicitly, and push main on its own before anything else.

That became a rule of the migration script, and of Cask itself: a repository is created with its default branch named, the default branch is pushed alone first, and only then are the other branches, tags and notes pushed. In Artifacts mode, Cask also refuses a first push to an empty repository that does not include the default branch, so a new repository cannot end up in the same state.

Moving eight repositories

The migration ran one repository at a time, with Cask's own repository last. For each one, the script:

• mirror-clones it from cask.build and checks the source with git fsck;
• refuses to continue if any blob is over 32 MB, because Artifacts rejects objects above that size during receive-pack with an artifacts_git_receive_pack_object_too_large error, and it is far better to find that before the push than halfway through it;
• creates the Artifacts repository with the default branch set, pushes the default branch, then pushes every other head, tag and note with pruning;
• compares every ref and object ID between cask.build and Artifacts, checks that Artifacts' HEAD names the right branch, and runs git fsck --full on a fresh clone from Artifacts.

None of the repositories came close to the object limit. The largest is about 27 MiB packed in total, and its biggest single file is under 4 MB. One repository was empty and simply had to exist with the right default branch. Every other repository matched on every ref and object ID, and every fresh clone from Artifacts passed fsck.

Two smaller findings from the pilots are worth recording. Pushes over a megabyte, which trigger Git's probe request, went through the Worker to Artifacts without trouble; that was the size-dependent bug from the first post, and I checked for it on purpose. Partial clone also worked, but Cask does not depend on it, and nothing here assumes it will keep working.

Cutover

The switch is a single setting, STORAGE_BACKEND, with two values: wal and artifacts. Everything else is the same build.

Before switching, the new code ran as a separate preview Worker with its own container and its own Artifacts namespace, and had to pass the whole suite there: clone, push, fetch, tags, notes, branch creation and deletion, refused credentials, refused force pushes, the repository list, file view, history, pull requests, merges and webhook delivery. Only then did production change.

The production change itself was short. Cask went read-only, the migration ran once more to pick up anything that had changed since the first copy, every ref was compared again, the setting was flipped, and Cask came back read-write on Artifacts. Then the checks ran again against cask.build itself: a fresh clone, a test branch pushed and deleted, and the UI pages.

Rollback

The write-ahead log was not touched by the migration and is not touched now. While Cask runs on Artifacts, nothing writes to it. That makes rollback the same size as cutover: set STORAGE_BACKEND back to wal and deploy. Cask comes back serving the log, exactly as it was at the moment of the switch.

Anything pushed while on Artifacts would be missing from the log after a rollback, so there is a script that pushes Artifacts' state back through Cask into the log, ref by ref, under the same branch protection as any other push. I would rather have a rollback that loses nothing and takes one extra command than one that is instant and quietly forgets a week of commits.

Per-agent repositories and forks

The part of this I find most interesting is not the migration but what it makes cheap.

A Git store that is an API, rather than a directory on a disk I manage, makes repositories disposable in a useful way. An automated process that needs to make changes — a build job, a scheduled task, any agent that writes code — can have a repository of its own, created on demand and deleted when the work is done, with a token scoped to exactly that repository and expiring in minutes. Artifacts can also fork a repository, so the natural shape is a fork per piece of work: the agent writes into its own fork, and the result comes back to the real repository as a pull request, through the same container, the same review and the same branch protection as everything else.

That is not built yet. The pieces it needs — scoped short-lived tokens, a gateway that keeps credentials out of the processes doing the work, and protection enforced by Cask rather than by the store — are the pieces this migration put in place.

What I would tell someone doing the same thing

Decide which component owns each fact, and refuse to let a second copy become authoritative by accident. The container's repositories are useful precisely because they are not allowed to be the source of truth.

Do not assume the new store enforces the rules the old one did. Artifacts is a store; it accepts force pushes and does not offer atomic multi-ref pushes. Both were found by testing, not by reading, and both turned into code in Cask.

Keep the old path intact until you no longer need it, and make returning to it a single setting. The best part of this migration is that I never had to trust it before I could undo it.