Cache server

@vzn/vx-cache-server is a remote cache for CI jobs. It keeps task artifacts in one bucket and nothing else: no database and no state on the node. It speaks the vx-cache protocol (/v1). vxCache() from @vzn/vx-cache points vx at it.

vx asks its local cache first and the server on a local miss. Every miss’s artifact is uploaded in the background. A down server or a refused token reads as a miss, never as a failure.

Deploy

Cloudflare Workers

The artifacts go in R2. From packages/vx-cache-server in a checkout of the vx repository, with TRUST filled in wrangler.jsonc:

bun install
bunx wrangler login
bunx wrangler deploy

On the first deploy, wrangler creates the R2 bucket wrangler.jsonc names (vx-cache). By default a check makes 6 lookups, one round of the calls a Worker request has waiting at once, and the client sends its checks at once: the fastest way. Each lookup is one subrequest, so a higher MAX_CHECK_HASHES must fit your plan, as MAX_UPLOAD_BYTES must fit its request body. A download is read through the data center’s cache: after R2 serves an artifact once, that data center answers it without R2.

Bun

The artifacts go in any S3-compatible bucket:

TRUST='[…]' \
S3_BUCKET=vx-cache S3_ENDPOINT=https://s3.example.com \
S3_ACCESS_KEY_ID=… S3_SECRET_ACCESS_KEY=… \
bun packages/vx-cache-server/src/bun.ts --port 8080

--port (default 8080) and --host (default 0.0.0.0) set where it listens. A node keeps no state of its own, so any number of nodes can share one bucket behind a load balancer.

Docker

From the root of the vx repository:

docker build -f packages/vx-cache-server/Dockerfile -t vx-cache-server .
docker run -p 8080:8080 -e TRUST='[…]' -e S3_BUCKET=… -e S3_ENDPOINT=… \
  -e S3_ACCESS_KEY_ID=… -e S3_SECRET_ACCESS_KEY=… vx-cache-server

Or run it with the other two servers using Docker Compose: the cache service, on port 8081.

Put TLS in front of it: a CI token is a bearer token.

Settings

Setting	Default	What it sets
TRUST	empty: refuse all	the repositories it serves and the jobs that write protected refs (TRUST)
AUDIENCE	the origin it is reached at	the aud a CI token must name
READ_ACCESS	unset: a token for every request	public: a request with no token reads a repository TRUST names, and writes nothing
S3_BUCKET, S3_ENDPOINT, S3_REGION, S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY		the bucket (Bun and Docker), as Bun’s S3 client reads them
MAX_CHECK_HASHES	500 (6 on Workers)	the lookups (hashes × scopes) one check may make
MAX_UPLOAD_BYTES	100 MB	the largest upload, one artifact

On Workers, each setting is a var in wrangler.jsonc. Every check answer names the limits (vx-max-* headers), and so does GET /v1/capabilities. Clients check hashes in chunks within those limits. Downloads and uploads go one artifact at a time, many at once.

Scopes and keys

The server answers by the token’s rules (TRUST): a job reads its repository and writes only its own branch’s scope. A key is { scope, hash }, where scope is a ref’s hash: the first 128 bits of the ref’s SHA-256, as 32 lowercase hex characters (refs/heads/main is f921bd05e68b03740c450e565e0e6173). A plain ref is refused with 400, so no ref crosses the wire. A write names no scope: the server hashes the token’s ref the same way. A read is exact: it names each scope it wants. A check names vxCache()’s branch, then the branch it merges into, then the default branch, and the first scope that holds the hash answers; the download names that scope (vx-scope).

An artifact’s object name in the bucket is:

<hash(issuer)>/<hash(repository name)>/<hash(scope)>/<hash>

Each of the first three parts is the first 128 bits of its SHA-256, in hex (32 characters), with no secret key, so the bucket names no repository or branch. One repository never reads another’s artifacts. An artifact is never rewritten: a second upload of a key that is already held keeps the first bytes.

Expiry

Artifacts expire by age since they were written, through the bucket’s own lifecycle rule. The server runs no sweep. One rule covers the whole bucket, for example 7 days on R2:

bunx wrangler r2 bucket lifecycle add vx-cache all '' --expire-days 7

On S3 the same rule goes in the bucket’s lifecycle configuration. A run that misses an expired artifact runs the task and stores it again.

Point vx at it: vxCache()

.vx/workspace.ts

import { defineWorkspace } from '@vzn/vx/config'
import { vxCache } from '/path/to/vx/packages/vx-cache/src/index.ts'

export default defineWorkspace({
  plugins: [
    vxCache({
      url: 'https://cache.example.com', // or VX_CACHE_URL
      issuer: 'https://token.actions.githubusercontent.com',
      repository: 'acme/app',
    }),
  ],
})

Not on npm yet

Until @vzn/vx-cache is on npm, import it from a checkout of the vx repository, as above.

vxCache({ url?, issuer, repository, timeoutMs?, uploadTimeoutMs?, headers? }):

• url is the server’s base URL, or VX_CACHE_URL.
• issuer (the CI issuer’s URL, as its tokens name it; on GitLab https://gitlab.com or your instance) and repository (org/name there) are required. Every request names both by their hashes, and the server refuses a token of another repository.
• timeoutMs (default 30 000) bounds each read and check, and uploadTimeoutMs (default 60 000) bounds each upload.
• headers are sent on every request to the server, for a proxy in front of it, such as a Cloudflare Access service token: headers: { 'CF-Access-Client-Id': process.env.CF_ACCESS_CLIENT_ID, 'CF-Access-Client-Secret': process.env.CF_ACCESS_CLIENT_SECRET }. An unset value is left out; authorization stays the CI token’s. A DTE worker sends them too, so give its job the same secrets.

Each CI job needs a token for the server’s origin:

GitHub Actions

.github/workflows/ci.yml

permissions:
  contents: read
  id-token: write

The plugin asks GitHub for the token itself, and asks again before it expires. A job without id-token: write warns and reads only.

GitLab CI

.gitlab-ci.yml

vx:
  id_tokens:
    VX_CACHE_TOKEN:
      aud: https://cache.example.com

Off CI there is no token: the run reads only, from a server with READ_ACCESS=public, and uploads nothing; any other server refuses the first read, said once, and the run stays local. A refused token is said once and turns the remote off for the run. A refused write (a job that only reads) turns off writes alone. In a distributed run, the run lends its token to its workers, so they read and write the scopes the run reads.

API

Turbo’s routes under /v1, each read naming its scope (PROTOCOL.md):

Route	Does
GET /v1/artifacts/<hash>	one artifact’s bytes, its scope in vx-scope
HEAD /v1/artifacts/<hash>	whether the scope holds it, and its size
PUT /v1/artifacts/<hash>	one artifact’s bytes, to the token’s scope
POST /v1/artifacts	{ hashes, scopes }: each hash’s size and the first scope holding it
GET /v1/capabilities	the limits

Any server that speaks the protocol can stand in for this one. To mount this one in a server of your own, createCacheServer({ blobs, authorize, limits? }) returns a fetch handler over a bucket of your own. The README lists the whole API.