Benchmarks
Empirical overhead numbers vs. Turborepo, Nx and Vite Task on synthetic workspaces. The head-to-head is re-measured on every change to vx and filled in when the site is built (§ Head to head).
Every harness (compare.ts, run.ts, ab.ts, real/turbo-repo.sh,
real/nx-repo.sh) runs vx from a vx lock snapshot, the lock taken
once per workspace before the timed reps, as every run reads one
(2026-09-29). A section dated before that ran vx without a lock,
evaluating every config per run. The harness never re-locks: a config edited after the
lock is re-locked by running vx lock again, and vx lock --check is
what fails loudly on a stale lock (a run trusts it, 2026-06-13,
docs/design/config-lock-2026-06.md); only a project the lock lacks
fails the run itself.
Only the native-config runs (each runner on the same graph from its own
config: the head-to-head, the scaling table) measure
vx. The real-repo runs went through turbo() or nx() on the repo’s
own turbo.json or Nx graph, a migration bridge rather than a claim
(2026-10-02), and are removed (§ Real repos). Those repos are to
be rerun on the native config bunx @vzn/vx-migrate writes.
Warm-run overhead (2026-09-02)
Section titled “Warm-run overhead (2026-09-02)”The number that matters most to a developer is the warm no-op run: every
task a cache hit, nothing to restore. private/vx-bench/generate.ts workspaces,
vx run build --all, this machine (macOS arm64, Bun 1.4.0), best of 5:
The table
| Projects | Before (2026-09-02 morning) | Wave 1 | Wave 2 | Wave 5 | Wave 6 | Wave 7 | What changed |
|---|---|---|---|---|---|---|---|
| 100 | 105 ms | 92 ms | 79 ms | 78 ms | 77 ms | 74 ms | git overlaps config load; .git/HEAD read replaces a git spawn; batched probe; no output walk; git first |
| 1000 | 380–450 ms | 270 ms | 242 ms | 237 ms | 204 ms | 172 ms | + cached pure-config evals; one worktree walk; readdir discovery; batched probe; no output walk; git first |
Wave 7 (2026-09-03) starts the unscoped git enumeration the moment the
workspace root is known — ahead of the workspace config, discovery and
the cache open, ~25 ms earlier than before — so the status walk that is
the warm run’s wall floor overlaps everything. Interleaved A/B against a
worktree at the previous commit, three rounds of three: 1000 projects
min 195 → 172 ms.
Wave 6 (2026-09-03) removed the per-hit output walk: for dist/**-shaped
globs, the mtimes of every directory under the prefix, recorded at the
last save or restore, prove the output set unchanged (docs/caching.md
§ A current tree). Interleaved A/B against a worktree at the previous
commit, three rounds of three: 1000 projects min 224 → 204 ms.
Wave 3 (batched short-circuit probe, output rows carried on the entry,
memoised Bun.Globs) measured on the graph WITH dependencies —
vx run test --all, 2000 tasks, interleaved arms against an immutable
worktree of the previous commit: 327–329 ms → 308–314 ms.
Where Wave 2’s 242 ms at 1000 projects went (VX_DEBUG_TIMING=1, see
below; Waves 6 and 7 took 70 ms off it, mostly the run-graph phase and
the git overlap): discovery 22 ms, config load 31 ms (all cache hits) overlapped
with git’s one worktree walk (status -uall, ~57 ms, the critical
path), the run-graph phase 78 ms (1000 hits: probe, output glob, stat
check), history recording 12 ms, cache open 9 ms, and ~50 ms of process
start + module load + exit outside the table. The 1.7 s cold run is the
1000 cp commands.
Reproduce: bun private/vx-bench/run.ts 1000 5 (it prints the median
and every rep; the best is the min). Its last row, one edited, changes
one package’s source per rep: one miss, every other task a hit.
A second machine, same shape (2026-09-20)
Section titled “A second machine, same shape (2026-09-20)”The table above is one machine’s. A four-core Linux container — sharing nothing with it — reads, medians with the full spread:
The table
| Projects | Warm | Restore | Cold |
|---|---|---|---|
| 1000 | 271 ms (243–315) | 1031 ms (1023–1051) | 3147 ms (2905–3388) |
| 5000 | 807 ms (760–809) | 3931 ms (3462–4172) | 14 181 ms (13 798–14 986) |
Slower in absolute terms, as a shared container should be, and those numbers are not comparable to the ones above — different hardware. What does compare is the SCALING: 5× the projects costs 2.98× the warm run here against 2.97× on the other machine (restore 3.81× against 3.97×, cold 4.51× against 4.99×). The sub-linear warm curve is the code’s, not one box’s.
One number to take from this before optimising against the harness: the warm arm spreads ±13 % about its median on identical code, because every rep is a whole CLI invocation. An A/B here needs an effect bigger than that, and a control arm beside it.
The same container class on 2026-10-29 (Bun 1.4.4, vx at 38ac3be9d), medians with the full spread, five reps at 1,000 and three at 5,000:
The table
| Projects | Warm | Restore | Cold |
|---|---|---|---|
| 1000 | 355 ms (298–410) | 938 ms (891–1091) | 2905 ms (2760–2960) |
| 5000 | 808 ms (679–836) | 4025 ms (3501–4472) | 11 400 ms (11 132–11 423) |
These rows replace the 2026-09-23 ones (1,000: warm 239, restore 906, cold 2634 ms; 5,000: warm 711, restore 3253, cold 10 950 ms).
The warm row at 1,000 reads above that run’s spread. Runs on a shared container are not an A/B: whether the warm path slowed needs one against a worktree of the old commit.
5× the projects costs warm 2.28×, restore 4.29×, cold 3.92×.
Profiling a run
Section titled “Profiling a run”Two tools, and they answer different questions:
VX_DEBUG_TIMING=1 vx run …prints a stage table to stderr at the end of the run —startup,workspace config,discover projects,package graph,open cache,load configs,build graph,git enumeration,plugin stages,classify + probe,schedule,run graph,record history,output dir snapshots,close, and in a dry runplan— with each stage’s own and cumulative time, plus accumulated per-task spans (cache.get,output glob,output statandtask hashamong them;modules/timing.mdlists every one). This is the first thing to read: it says WHICH stage moved. The per-task spans run under the scheduler’s concurrency, so they over-count (a span’s wall includes time yielded to other tasks); compare them to each other, not to the stage total.bun --cpu-prof --cpu-prof-dir=/tmp/prof packages/vx/src/bin.ts run …thenbun private/vx-bench/profile-summary.ts /tmp/prof/*.cpuprofilegives self time by function and by file. Good for finding a hot loop; unreliable about where anawaitwaited (it attributes the wait to whatever frame was on the stack).strace -f -o <file>around a run, thenbun private/vx-bench/strace-vx.ts <file> [<before-file>]: the count of vx’s OWN syscalls (the tasks’ shells are told apart by PID), as a diff against the same run on agit worktreeof the base. A syscall count is deterministic where this container’s wall time is not: the restore path’s three spare round trips per artifact (item 627) were three rows of that table —readlink1,003 → 3,mkdir2,002 → 1,002,newfstatat−2,000 — and the save’s twomkdirs of the cache directory (630) one row, before either was a number. Awriteper round trip is the thread pool’s wake, so that row counts round trips.bun --preload ./private/vx-bench/sqlite-tally.ts …is the same idea for SQLite statements;restore-bench.tsandsave-bench.tsin the same package restore or save every artifact sequentially, where a per-artifact change of tens of microseconds shows above the four-worker run’s noise (and one of a few microseconds does not: 630).
Three measurement lessons from this wave, recorded so they are not
re-learned. A compiled Bun 1.4.0 binary resolves on-disk packages by
<pkg>/index.ts only and ignores exports, so private/vx-bench/compare.ts
measured nothing (“vx skipped”) until the packages gained root shims —
if the vx row ever reads n/a again, read the skip line first. A micro-benchmark of a sync call in isolation (statSync
2 µs vs stat 13 µs) does not predict the run — the async forms run in
parallel on the thread pool under the scheduler’s concurrency, and
switching the warm-hit path to sync calls made the 1000-project run
40 ms SLOWER. And Bun 1.4.0’s --compile binaries carry a signature this
macOS rejects (SIGKILL on launch); an ad-hoc codesign -s - --force
repairs it, which the release workflow now does on a macOS runner.
A/B two builds
Section titled “A/B two builds”private/vx-bench/ab.ts is the method the numbers here use: one
workspace copy per arm, warmed by that arm, rounds that run every arm
once in a rotated order, min and median per arm, and an A/A arm (the
same vx twice) as the noise floor. Every arm after the first prints its
min against the first arm’s as a signed percent. An arm is a compiled binary or a
checkout; the runs see git’s defaults and no BUN_OPTIONS. A run is
timed from a vx lock snapshot: each arm runs vx lock in its own
copy with its own vx once, before the warm-up.
bun private/vx-bench/ab.ts 15 base=/tmp/vx-old@/tmp/w1 main=/tmp/vx-new@/tmp/w2 \ aa=/tmp/vx-new2@/tmp/w3 -- run build --allA real repo’s copies each link node_modules/@vzn/vx-migrate (or the
plugin under test) to their arm’s checkout, so a plugin change is
measured with its own core.
The per-PR guard
Section titled “The per-PR guard”private/vx-bench/perf-guard.ts runs in every PR’s gate as
@vzn/vx-bench#check.perf, in CI’s Perf job: a runner of its own
beside the DTE run, whose workers are packed for the run’s wall
time, not a task’s (ci.yml perf). It builds a 25- and a 100-package workspace
and drives core in-process through five phases, each from a vx lock
taken first: cold run, up-to-date run, restore run, plan, one-edit run. Two kinds of number, against
private/vx-bench/perf-baseline.json:
- Counts: processes spawned, xxh3 calls, blob-hash updates, SQLite statements. The same code does the same work on any box, so a count that grows fails; one that shrinks passes. Statements gated by the 50 ms racy-mtime windows (the directory snapshots) are left out; they move with load.
- Time, printed only: each timed phase’s min over 7 reps at 100 packages, divided by a calibration loop run between reps, flagged past 1.5× the baseline. It fails nothing: beside the test shards it read 1.97× on a PR that slowed no phase.
Each measure runs in a child with a hermetic environment, since vx
hashes BUN_OPTIONS, a HOME bunfig and its parent run’s variables.
A change meant to grow a count records it (on a conflict in the file,
take main’s and run it again):
vx run @vzn/vx-bench#perf.update # then commit perf-baseline.jsonRun it through vx run.
The per-PR bench against every rival and the base
Section titled “The per-PR bench against every rival and the base”private/vx-bench/pr-bench.ts runs in every PR’s gate after
perf-guard.ts, in the same task (@vzn/vx-bench#check.perf), so the
two benchmarks never share a machine. It times this checkout’s vx,
compiled as released, against the base commit’s vx (VX_BENCH_BASE: the
PR’s base, or main before a push, built in the same job from git archive
of packages/vx), Turborepo, Nx and Vite Task (pinned in
private/vx-bench/package.json). The task runs outside the sandbox, as
the site’s bench does: the sandbox’s tracer stops every runner at each
traced call and would be timed. Traced, vx’s cold run took 5.6 s against
Vite Task’s 5.3 s; untraced, 2.1 s against 2.8 s.
Every runner runs the site’s own workspace as the site configures it
(site-workspace.ts), with no task sleeping. Restore and nothing-changed
run on the whole of it (1,601 packages, 9,603 tasks): on the old
100-package workspace a per-task cost the site pays 9,603 times was
noise, and #4892 added 17-35 ms to every site run and passed. Cold runs on
its first two levels (100 packages): the site’s cold runs tasks for two
minutes. Each tool runs in its own copy: Turbo rewrites every output on a
warm hit, so in one shared tree vx’s nothing-changed run restored them all
(2026-10-28). The bench fails if a vx nothing-changed run rewrites an
output.
vx, the base and Turbo run in pairs, vx and the base each first in turn.
Other processes slow vx more than Turbo (four busy loops moved the
nothing-changed ratio from 0.90 to 1.28), so each pair reads the cores
other processes kept busy while it ran (/proc/stat, less the bench’s
own CPU and its children’s). A case reps until 7
pairs ran with at most half a core busy beside them, and takes each
tool’s min over its 7 quietest; with none that quiet it says BUSY and
holds anyway. Nx and Vite Task run every case on the 100-package
workspace, against vx timed there: at the site’s size their restore and
nothing-changed take 20-40 s a run, many times vx’s, and filling Nx’s
cache took minutes. Each runs once, three times when within a quarter of
vx.
vx aims to be the best of every runner in every case. The check reports
and never blocks a PR. A BEHIND line names a case where vx is slower
than a rival, or uses more CPU than Turbo or Nx. Vite Task may use less
CPU: it runs one process where vx runs a shell per task.
A SLOWER line names a case where vx ran slower than the base in more of its quiet
pairs than chance explains (a sign test, p < 0.002). There is no slack: a
4% tolerance let a regression land 3% at a time. A case whose pairs lean
slower without proving it keeps adding pairs, 40 at most. Simulated, an
unchanged vx is flagged in 0.7% of cases, and one slower in 3 of 4 pairs
in 63%.
The base is main, the code whose numbers the site shows, timed on the same machine in the same minute. The site’s own figures cannot hold a PR: five commits of one day whose vx timed level side by side read nothing-changed at 0.87 to 1.33 of Turbo on the CI machines they drew.
private/vx-bench/real/nightly.ts re-runs the repos on
Real repos each night against the tool they ship with,
on this checkout’s vx, and exits 1 with a LOSS line for any case vx
loses. --json <file> also writes each repo’s row into
private/vx-bench/real-repos.json’s shape.
Head to head
Section titled “Head to head”Every figure in this section is filled in when the site is built, from main’s latest measured run (2026-10-29: no benchmark figure is typed by hand).
On GitHub the page shows the %bench.…% tokens instead.
The shape (owner’s spec, 2026-10-09): 29 levels of 50 libraries and a last level of 100 apps; 50 more libraries at level 15 that nothing uses.
One e2e project depends on every app and on those 50.
Every project depends on 2–5 others in the five levels below, and five core libraries at level 1 are each used by about a quarter of the projects. 1,601 projects, 20 source files each.
Tasks: installDeps (no command, after the dependencies’ build),
build, lint and test (after installDeps), publish (after build).
typecheck runs after the dependencies’ build; e2e has lint and
test. 9,603 task nodes.
Each task sleeps: build 300 ms, test and typecheck 150 ms, lint 75 ms, publish 30 ms.
That is long enough that the tasks, not any tool’s own work, set the pace.
No two of a project’s commands are the same text: each task spells its seconds to its own decimals. Vite Task keys a task on its command and inputs, so test and typecheck were one key and it ran one of the pair.
Vite Task installs the workspace’s package manager before a run. The bench seeds its data directory with the bun it runs on, so no run downloads.
build writes 200 KB of incompressible random bytes plus a file made
from its dependencies’ outputs. The bench times what CI runs: a cold
build, a run with nothing changed, and a restore from cache.
No edit row: an edit needs Nx’s daemon, which CI does not run, so no edit row compares fairly (2026-10-09).
Same repo, same commands, every tool at concurrency 10, each in its own native config.
Turborepo reads turbo.json and runs no daemon (it uses none for
turbo run since 2.8.11). Nx runs its own graph (nx:run-commands targets, ^ inputs,
.env loading off).
Vite Task runs vp run over each vite.config.ts.
vx runs as the compiled binary from a vx lock snapshot, as every vx
run does.
It orders tasks with vxHistory({ file: 'vx-timings.json' }); the file
holds each task’s own duration, as CI would carry it from an earlier run.
Time each tool adds over the ideal run:
The table
| vx | Turborepo | Nx | Vite Task | |
|---|---|---|---|---|
| Nothing changed | 412 ms | 484 ms | 7.90 s | 6.46 s |
| Nothing changed, outputs restored | 817 ms | 1.04 s | 7.72 s | 6.33 s |
| Cold build | 6.26 s | 10.34 s | 2 min 44 s | 6.49 s |
| Cold build: CPU the runner burns | 40.63 s | 1 min 16 s | 5 min 4 s | 21.09 s |
Time each tool adds over the ideal run; lower is better.
Measured: vx 0.0.0 @ c2adbdd66 on 2026-11-12, Turborepo 2.11.10 on 2026-11-12, Nx 24.0.2 on 2026-11-12, Vite Task 1.4.0 on 2026-11-12; linux x64, 4 cores (AMD EPYC 9V45 96-Core Processor), 16 GB.
The ideal run is the theoretical best case, so every row is overhead. Nothing changed and outputs restored: 0, so all a tool spends there is overhead.
Cold: every task list-scheduled critical-path first (critical path 9.15 s, work ÷ workers 1 min 53 s, the ideal run 1 min 53 s).
CPU is user + system of the invocation and the children it waited
for, less what the task commands burn under xargs -P 10
(22.58 s).
A daemon that outlives the invocation (Turborepo’s, Nx’s) is not counted, so their CPU is a lower bound.
Methodology note: a synthetic graph with
sleep-based tasks isolates runner overhead from real compilation. All four runners are configured identically: same commands, the samesrc/**inputs anddist/**outputs, the same concurrency, and each sees a dependency’s change (Turborepo and vx through upstream keys; Nx through^inputs; Vite Task through each dependency’s output as an input).
Where the figures come from
Section titled “Where the figures come from”Each runner is one task, @vzn/vx-bench#bench.<runner> (vx, turbo,
nx, vite-task), which runs compare.ts for that runner alone and
writes its rows to private/vx-bench/site/<runner>.json.
@vzn/vx-docs#bench.data gathers the four files into
measured/bench.json. The site’s build reads its committed copy,
src/data/bench.json. A build with no data fails; so does a page naming
an unknown token.
The line under the table names each runner’s version, date and machine.
Only CI’s Bench workflow (bench.yml) runs them, on main and on a dispatch: vx run @vzn/vx-docs#bench.data --concurrency 1 --force on a 4-core Linux
runner of its own, one runner at a time. --force measures all four on
that one machine, never a row cached from another runner’s CPU. A
cold figure is the median of three cold runs (COLD_REPS=3): one let a
single run move it by seconds. Nx’s is one run: its overhead is minutes,
so seconds move no comparison. It
uploads measured/bench.json as the bench-data artifact.
Main’s DTE run deploys the site (@vzn/vx-docs#deploy.pages, part of
release). build.live fetches the newest bench-data from main and
builds the site with it while the tests run, so it never waits on Bench
and never ships the committed copy, older than the live one; the upload
waits only on ci. New numbers go live with the next main push.
No build depends on the
benchmarks, so neither the DTE run, where a worker is packed beside
other tasks and would time its neighbours, nor a fresh clone’s vx run build ever runs one. A pull request’s build reads the committed copy;
refresh it with a commit of the job’s file now and then.
The tasks run outside the sandbox, which on Linux traces every process they start: under it vx’s cold overhead read 55.61 s against 7.01 s outside (2026-10-29).
A rival measured once keeps its row until its key changes, so the rows in one table can come from different runners of one machine class.
Why Nx is slower
Section titled “Why Nx is slower”Two things, both per task, and both measured on the 46-package workspace (cold, 138 tasks, min of 3 interleaved arms, 2026-09-24, item 735):
The table
| Arm | Over the 10.00 s ideal | CPU |
|---|---|---|
the old harness: nx:run-script, Nx picked npm |
18.82 s | 69.80 s |
nx:run-script with bun (the fixed harness) |
11.37 s | 41.18 s |
nx:run-commands: the same commands, no fork |
1.65 s | 3.05 s |
- A Node process per task.
nx:run-scriptruns every task in a freshly forked Node process (nx/bin/run-executor.js: 138 of them in one cold run, counted withstrace -f -e execve) that loads Nx before it runs the script. That is ~270 ms of CPU per task. On four cores at concurrency 10 it saturates the CPU, so every task on the 20-task critical path waits for a core: 9.7 s of wall and 38 s of CPU, the gap between the last two rows.nx:run-commandsruns its command from the Nx process itself (NX_RUN_COMMANDS_DIRECTLY), and with it Nx runs 1.65 s over the ideal schedule. A package-based Nx repo getsnx:run-scriptfor every inferredpackage.jsonscript. Since 2026-10-04 (owner: “use run commands”) the harness gives Nxnx:run-commandstargets, the command line vx runs; the 46-package shape then read Nx 1.17 s over the ideal cold and 3.26 s of CPU against vx’s 0.40 s and 0.91 s (median of 3).installDeps, vx’s group task, is a cachednx:noopfor Nx and a script-less task for Turbo: on one 1,090-package workspace (no sleeps, min of 3) an uncachednx:noop, or a target with no executor (Nx makes itnx:noop), held Nx’s warm run at 16.5 s against 5.9 s cached. - npm, which the harness chose by accident.
nx:run-scriptruns the script through the package manager Nx detects from a lockfile. The generated workspace had none, so Nx fell back to npm, while Turbo readpackageManagerand ranbun run.npm runcosts 202 ms of CPU per task againstbun run’s 4 ms (min of 10, one task alone): 7.5 s of wall and 29 s of CPU, the gap between the first two rows. This one was the harness’s fault, not Nx’s.nx.jsonnow sayscli.packageManager: 'bun'.
Warm is spread thin, with no single cause. With no daemon (CI), each
invocation starts three plugin workers to build the project graph
(~280 ms each, in parallel; nx show projects alone is 0.56 s), then
hashes and replays the cached tasks. Turning the daemon on moves the
graph into it and saves 40–160 ms (0.81 s against 0.85, min of 5, in a
probe; 682 ms against 844, median of 3, in the harness), a fifth of the
warm gap at most.
At the 9,603-task shape (2026-10-13: 1,601 projects, Nx 23.3.0,
nx:run-commands, CI=1, daemon and .env loading off, four-core
Linux, 173 environment variables) warm has a cause: Nx’s fully cached
run took 21.2 to 23.6 s. A CPU profile of the main process (24.8 s
sampled) puts 6.10 s self in hashTasksUpfront, which hands each
task’s environment object to the native hasher; 5.55 s in garbage
collection; 3.03 s in getTaskSpecificEnv, which copies process.env
for every task even with .env loading off; 2.89 s in
copyFilesFromCache, which rewrites every cached output on each warm
run; and 1.56 s in the native planner. Most of it is a per-task copy
of the environment, so it grows with the environment’s size, and no
setting turns it off. Loading .env files cost a further 2.7 s.
The npm share grows with the graph. At 1,090 packages (3,270 tasks,
compare.ts 100 11, same box, one cold run each) Nx burned 76 min of
CPU with npm and 23 min with bun: npm was two thirds of the old
harness’s Nx number there.
Refuted, each within ±0.3 s of the fixed harness’s cold run: the
per-task pseudo-terminal (NX_NATIVE_COMMAND_RUNNER=false) and the
output style (NX_TUI=false, --outputStyle=stream). So is the daemon:
the fixed harness read Nx cold at 12.25 s over the ideal with
NX_DAEMON=true and 12.08 s without. Parallelism is honoured: with cheap tasks, the
nx:run-commands arm sits 1.65 s over the ideal schedule. The ~4 git
probes each forked task runs cost ~8 ms of it. These tables used to say
Nx’s daemon was on; CI=1 had always turned it off, and the harness
keeps it off on purpose: it simulates CI.
Why Turborepo was slower cold on 100 layers (2026-10-04)
Section titled “Why Turborepo was slower cold on 100 layers (2026-10-04)”On the earlier 1,090-package, 100-layer shape: the order it starts
ready tasks in, not its CPU (21 s cold against vx’s 17 s at 3,270
tasks). Turborepo has no ranking: its walker hands out
tasks as they become ready and each waits for a semaphore slot
(crates/turborepo-engine/src/execute.rs). When a layer’s builds finish,
the next layer’s builds become ready together with that layer’s tests,
and in ready order the tests go first, so every build on the 100-layer
critical path waits about a second. listSchedule(nodes, 10, 'fifo')
replays that order on the benchmark’s graph: 1m 20s over the ideal
schedule, against Turborepo’s measured 1m 21s and nothing ranked (private/vx-bench/tests/ideal.test.ts).
No turbo.json key changes the order. vx ranks ready tasks by remaining
critical path, Nx by how many tasks wait on each.
Reproducible head-to-head (vx vs Turborepo vs Nx vs Vite Task)
Section titled “Reproducible head-to-head (vx vs Turborepo vs Nx vs Vite Task)”private/vx-bench/compare.ts scaffolds one monorepo, the shape in
§ Head to head (private/vx-bench/shape.ts), with the identical shell
commands, src/** inputs and dist/** outputs for every runner.
It times each runner through the three states CI sees: cold, nothing changed and outputs restored.
Fairness is deliberate: vx runs as the compiled binary real users
install (not TS source), from a vx lock taken once before the reps.
The workspace is git-committed with node_modules/.turbo/.nx ignored.
Every runner is pinned to the same concurrency, and runners are measured strictly one at a time, daemons stopped between them, so they never fight for CPU.
Turborepo, Nx and vite-plus are private/vx-bench’s pinned
devDependencies, linked into the workspace, so a run needs no registry.
Cold is one run; nothing changed and restore are the median of 3.
bun private/vx-bench/compare.ts # every runner, 3 reps (about 20 min)RUNNERS=vx,turbo bun private/vx-bench/compare.ts # a subset of runnersBENCH_BUILD_MS=0 bun private/vx-bench/compare.ts 1 # zero-length tasks: each tool's own cost per taskvx run @vzn/vx-bench#site.numbers --concurrency 1 # the site's rows, one runner at a timeIt writes the runners’ rows to OUT (default
private/vx-bench/site/bench.json) and prints them as a table.
The table carries CPU columns and the ideal run, the theoretical best case, so each row reads as overhead above it.
vx lock is what the vx row runs (since 2026-09-29), as every
run does: it executes the .vx/projects.lock.json graph with zero
per-run config evaluation.
Measured 2026-09-12 on the 1,000-project bench, compiled binary, 12 interleaved reps: no lock min 154 / median 177 ms, lock 148 / 165, a tie within the noise.
The load configs stage read 20–25 ms with no lock against 6–10 ms of
lock read plus 12–14 ms of load from the lock.
The lock is for what it guarantees — what runs is what was locked,
whatever a config would read from the environment. In your repo:
vx lock, then commit .vx/projects.lock.json.
How the overhead scales with the workspace (2026-09-10)
Section titled “How the overhead scales with the workspace (2026-09-10)”The per-package figure above is one size. This is vx alone at three
sizes of the same synthetic shape (private/vx-bench/run.ts, the
generator’s workspace, one build per package whose command is a
mkdir and a cp, so the clock is the runner and almost nothing else),
the compiled Linux binary at commit 1a35ec3, median of 3 on a 4-core
Intel Xeon container:
The table
| Packages | Cold (nothing cached) | per package | Warm, nothing to rebuild | per package | Warm, restore outputs | per package |
|---|---|---|---|---|---|---|
| 100 | 310 ms | 3.1 ms | 56 ms | 0.56 ms | 126 ms | 1.26 ms |
| 300 | 762 ms | 2.5 ms | 100 ms | 0.33 ms | 269 ms | 0.90 ms |
| 1,000 | 2,091 ms | 2.1 ms | 178 ms | 0.18 ms | 808 ms | 0.81 ms |
Ten times the packages costs 6.7× the cold time and 3.2× the warm time: the per-package cost falls as the fixed cost (process start, the workspace read, the cache open) is spread over more of them, and nothing in the run grows faster than the graph. A cold run at 1,000 packages is two seconds; a warm one is under two hundred milliseconds. These are the runner’s own costs on a trivial task; § Head to head, where each task sleeps and every runner gets the same concurrency, is where vx is compared against Turborepo, Nx and Vite Task.
Scheduling order (2026-10-09)
Section titled “Scheduling order (2026-10-09)”Which ready task starts first, measured inside vx: the default order
(most transitive dependents), the @vzn/vx-history plugin, and
three other rules written as measurement-only schedule plugins. One
cold run per row (--force), linux x64, 4 cores, Bun 1.4.2, vx from
source at main a85223a. Every task is a sleep. Each history arm
starts from an empty history in its own copy of the workspace. The
post: Which task runs first?.
A slow task with nothing after it: one 60 s task, two chains of a 29.75 s parent and a 0.25 s child, 2 workers.
The table
| Rule | Wall time |
|---|---|
| Ready order | 60.2 s |
| Most transitive dependents (vx’s default) | 90.0 s |
| Critical path by step count | 89.9 s |
| Recorded time, run 1 (no history yet) | 90.0 s |
Most direct dependents ties and falls back to ready order (60.2 s); recorded time runs 2 and 3 take 60.2 s each.
The mirror: 100 independent tasks and a chain A then B, 2 s each, 10 workers.
The table
| Rule | Wall time |
|---|---|
| Ready order | 24.3 s |
| Most transitive dependents (vx’s default) | 22.3 s |
| Critical path by step count | 22.4 s |
Most direct dependents ties and falls back to ready order (24.3 s); recorded time runs 1 to 3 take 22.3 s, 22.2 s and 22.3 s.
The headline graph: 1,090 packages, 3,270 tasks, 100 layers, 1 s tasks, 10 workers. The best possible run is 218 s.
The table
| Rule | Wall time |
|---|---|
| Ready order | 301.7 s |
| Most direct dependents | 220.5 s |
| Most transitive dependents (vx’s default) | 220.4 s |
| Critical path by step count | 220.6 s |
Recorded time runs 1 to 3 take 220.6 s, 220.5 s and 220.5 s.
Time to first task, cold (2026-10-09)
Section titled “Time to first task, cold (2026-10-09)”From vx run to the first task starting, cold, from a vx lock, on the 1,090-package bench, min of interleaved runs, linux
x64, 4 cores, vx from source, Turborepo 2.11.7:
| vx at | First task |
|---|---|
| Before #3378 | ~380 ms |
#3378: a lock vx lock wrote is not re-validated |
~330 ms |
| #3382: an empty cache skips the up-front probe | 225–232 ms |
| Turborepo 2.11.7 | 276 ms |
vx now starts its first task about 20% sooner than Turborepo; the rest
of the cold run is unchanged. History: docs/history/ws-x.md X-212, X-216.
CI timings file (2026-10-09)
Section titled “CI timings file (2026-10-09)”A fresh runner with no run history, with the history plugin’s file
carried from the last run, and with this machine’s full history.
bun private/vx-bench/timings-file-bench.ts: eight packages whose
build, check and pack chain (0.5 s each) unlocks more work, three
10 s e2e tasks nothing depends on, 4 workers, every task an uncached
sleep, min of 3, linux x64, 4 cores, Bun 1.4.2, vx from source at
main d790970. The lower bound is 10.5 s. The post:
Carry the scheduler’s memory between CI runners.
The table
| Runner | Wall time |
|---|---|
| Cold, no file | 12.09 s |
| Cold, with the file | 10.68 s |
| Full local history | 10.64 s |
Step by step against Turborepo and Nx (2026-10-13)
Section titled “Step by step against Turborepo and Nx (2026-10-13)”Where each tool’s run goes, cold and warm. The synthetic bench, 1,601
projects, 8,002 run tasks (9,603 graph nodes), 4 cores, concurrency 10,
CI=1, local cache only, Linux container. Turborepo 2.11.8 with
--profile; vx from 9388bad8f with VX_DEBUG_TIMING=1; Nx 23.3.0 with
nx:run-commands, daemon off, .env loading off, CPU profile of the
main process. The post:
Where a task run spends its time.
The table
| Run | vx | Turborepo 2.11.8 |
|---|---|---|
| Cold | 120.2 s | 127.3 s |
| Warm | 892 ms | 2,496 ms first, 909 ms second |
| One source file changed | 100.1 s (6,607 run) | 105.8 s (6,607 run) |
| That edit reverted | 3.6 s (1,321 tasks restored, the rest left in place) | 5.2 s (all 8,002 replayed) |
Nx 23.3.0, warm (21 to 24 s, 9,603 of 9,603 hit), self time:
| Self time | Where |
|---|---|
| 6.10 s | hashTasksUpfront: hands each task’s environment to the native hasher |
| 5.55 s | garbage collection |
| 3.03 s | getTaskSpecificEnv: copies process.env for each task |
| 2.89 s | copyFilesFromCache: writes every cached output back |
| 1.56 s | plan, the native planner |
Real repos
Section titled “Real repos”Public repos, each moved to native vx config and timed against the
tool it ships with, through private/vx-bench/real/nightly.ts, which
drives nx-repo.sh (Nx) or turbo-repo.sh (Turbo): both tools run the
same tasks with the same worker count, three interleaved reps per
case, best of three. cold wipes the outputs and both caches;
restore wipes the outputs and keeps both caches; noop wipes
nothing. Nx runs with NX_DAEMON=false and no Nx Cloud.
The native config is either written by vx init --native on the
night, or kept in this repo as an overlay: private/vx-bench/real/<name>/
mirrors the repo’s tree with a hand-written .vx/workspace.ts and a
vx.config.ts per project that follow its turbo.json, copied over a
clone pinned to a full SHA (solid, vueuse, create-t3-turbo;
real/REPOS.md names each repo’s mapped tasks and adjustments).
Real repos on native config
Section titled “Real repos on native config”Each repo on its overlay’s native config, 10 workers each, best of five
interleaved runs; vx at main with #5206, pnpm per repo, Node 22.23, linux x64, 4 cores, 2026-11-11.
real-repos.json holds the rows and the site’s
real repos page renders
them; the nightly waits up to two minutes for a 1-minute load under 1
before each workspace, prints every rep and records the load in the row.
| Repo | Case | vx | Turbo |
|---|---|---|---|
solid b25c557, 4 tasks, 2.10.10 |
Cold build | 30.45 s | 36.01 s |
| solid | Restore | 0.075 s | 0.075 s |
| solid | Nothing new | 0.064 s | 0.066 s |
vueuse efdd69a, 12 tasks, 2.10.12 |
Cold build | 16.70 s | 18.57 s |
| vueuse | Restore | 0.140 s | 0.106 s |
| vueuse | Nothing new | 0.097 s | 0.098 s |
create-t3-turbo 8f945b7, 3 tasks, 2.5.8 |
Cold build | 5.37 s | 6.47 s |
| create-t3-turbo | Restore | 0.078 s | 0.408 s |
| create-t3-turbo | Nothing new | 0.071 s | 0.407 s |
The harness’s restore follows a cold run and its “nothing new” that
restore, and vx paid two stalls there that the run after did not. A
cold run had no .vx/cache while its workspace config loaded, so it
built no config bundle (project-loader.md), and the restore built it
inside workspace config (~30 ms); now the build runs off the run’s
path. The next run, the first on the bundle, missed the pnpm lockfile
memo, whose claimant was its functions’ printed text, which the bundle
prints unlike the source (throw new Error( against throw Error(),
and parsed pnpm-lock.yaml again (~25 ms in discover); now
@vzn/vx-lockfile names itself with its source hash (identity).
Solid, interleaved against main, min of five: restore 93 → 66 ms,
nothing new 75 → 58 ms, the run after 50 → 51 ms.
Vueuse’s restore is the first run after the cold build, and it derives
every key: the cold run took its keys after @vueuse/metadata#update
wrote, so it left no key memo. Its .gitattributes sets * text=auto eol=lf, which drops every index OID, so each key hashes its files from
the worktree, and each of its 1,300 files was stat’ed and looked up in
the hash memo once per task that reads it (12,200 statx, 13,600
SQLite lock calls). Now the run holds each file’s identity until a
write it marks (git-inputs.md) and asks the memo once for all the files
it lacks: 5,000 statx, 275 lock calls; restore 162 → 141 ms against
the two fixes above, interleaved, min of five, nothing new unchanged.
A restore marks the files it writes, and each mark dropped every
identity the run held: metadata’s index.json sent the late keys back
to stat and look up all 900 of their files. Only the marked paths’
identities go now (git-inputs.md): 137 → 133 ms. Those late keys were
core’s and nuxt’s builds, whose workspaceFiles read
packages/metadata/**. A reader was keyed late whenever its globs
reached a producer’s directory, so the gitignored index.json kept
both off the restore tier and derived them again after the restore, on
a synchronous git listing, ~13 ms of main thread each. A producer now
reaches a reader only through the outputs it can see (stable-keys.md):
126 → 114 ms, nothing new 93 → 89 ms, interleaved, min of five. Solid
and t3 take no late key and did not move: solid’s restore against the
base, 30 pairs half in each order, was slower in 17 and faster in 11,
medians 75 / 77 ms. Extraction was not the cost: one artifact at a
time it is 1.2 ms each, 15 ms for the twelve, and it overlaps.
discover looked like the pnpm lockfile parsed every run (~18 ms). It
was not: the memo is served each run (a 680 KB read, its hash and one
row, ~1 ms). The stage’s wall was vx-lockfile’s own hook (~7 ms) and the
git enumeration’s attribute check landing in it. Asked for five
attributes by name, check-attr printed all five for every path,
unspecified included: 473 KB for vueuse’s 1,583 files, ~5 ms of main
thread to decode and split every run. --all prints only what a rule
sets (160 KB here; util-git-attributes.md). Against the C2 base,
interleaved, min of five: restore 140 → 120 ms, nothing new 98 → 83 ms, cold 15.43 →
14.97 s. The hook also read package.json5 and package.yaml in each
package where neither is, two rejected reads per package (~65 µs each);
a stat skips them now: restore 91 → 87 ms, nothing new 77 → 75 ms, min
of 20, medians 1 to 2 ms, within noise.
Nothing new is a tie: 84 against 83 ms. Over 10 warm runs each both
reach 85 ms; vx makes 10,500 syscalls to Turbo’s 16,000 and spawns five
gits (var, ls-files, status, check-attr, check-ignore) to
Turbo’s two. Of vx’s 84 ms, ~12 are the process (exec, Bun’s boot,
exit) and the rest its stages, median of 12: startup 5, workspace
config 7, discover 14 (vx-lockfile’s hook and the overlapping git
enumeration), open cache and load configs 6, git enumeration 6, plugin
stages 2.5, classify 16, of it stable keys 13 (each key’s files token
stats the 1,300 files the twelve keys read: no index OID holds under
text=auto), run graph 4, history and close 4.
Restore lost by ~18 ms in the run before this one: the up-front derivation the cold run
left no memo for (stable keys, ~15 ms) and the extraction’s wall in
run graph. Remembering the keys at the end of the cold run, after its
last write, is safe (the memo’s token is checked when it is read) but
moves the cost instead of saving it: +43 ms on vueuse’s cold run for
~18 ms off its restore, +700 ms on the 8,002-task bench’s cold run for
540 ms off its. Not shipped. The bench’s derivation was dear because a
save into a project, even into its ignored dist/, ended the
project’s files digest, so a cold run stored a fold row for each
project’s first task only (1,603 of 8,002 keys). A mark now ends it only
when the digest lists the marked path (git-inputs.md): the bench’s
restore after cold 1,864 → 1,564 ms, median 2,390 → 1,655, cold within
noise (min 16,524 → 16,597 ms, median 17,423 → 16,965), interleaved,
min of five. The three repos’ cold runs did not slow (min of five:
vueuse 16.43 → 15.94 s, solid 30.13 → 29.74, t3 5.04 → 4.98). Their
restores barely moved: twelve tasks or fewer, one or two per project.
The tests written for it turned up a stale hit: an output glob’s head
linked into the project (out -> gen for out/**) wrote gen/x, but
only out/x was marked, so a reader of gen/** kept the old index OID.
The write is now marked where it landed too. Turbo without its daemon extracts
every hit on every run, the no-op too, on parallel goroutines: its
restore and no-op make the same syscalls.
Vueuse’s restore still loses, 140 against 106 ms. About 7 ms is
stable keys statting 1,124 files: under text=auto the index’s stat
cannot stand in for them, since git calls a same-size line-ending swap
clean and keeps the old stat (stale-hit-git.test.ts). Turbo keys on the
index there and replays stale output: the conformance case
inputs/same-size-line-ending-swap-under-text-attribute. The rest is
the restore’s own work, 25 ms on 4 cores: trees alone 10 ms, the five
inline artifacts alone 9 ms, all twelve 26 ms. Copying is 1.9 ms of it;
fs.promises.cp (14.5 ms) and a Worker (1.5–2.4 ms to start) lost.
Load average before each workspace: vueuse 0.3, solid 1.0 and t3 0.2, the last two rerun once the box settled (solid began at 14 after vueuse’s cold builds and lost its restore tie by one rival rep).
TanStack/query (2026-10-09)
Section titled “TanStack/query (2026-10-09)”Commit 817bd02, scope build without examples and integrations: 25
tasks each, 5 workers. vx 0.0.633, Nx 23.2.1, pnpm 12.4.2, linux x64,
4 cores.
The table
| Case | vx | Nx |
|---|---|---|
| Cold build | 34.6 s | 39.4 s |
| Restore | 0.39 s | 1.26 s |
| Nothing new | 0.22 s | 1.43 s |
Cold is the total wall time of the build: a real repo has no ideal schedule to subtract, so its total is shown (2026-10-09). Restore and nothing new are runner overhead: every task is a cache hit. Each rep in order: vx cold 40.16 / 34.99 / 34.61 s, Nx 40.95 / 39.38 / 39.71 s; vx restore 0.39 / 0.50 / 0.50 s, Nx 1.26 / 1.49 / 1.57 s; vx noop 0.22 / 0.24 / 0.23 s, Nx 1.43 / 1.59 / 1.49 s.
create-t3-turbo and react-email, against Turbo (2026-10-09)
Section titled “create-t3-turbo and react-email, against Turbo (2026-10-09)”t3-oss/create-t3-turbo at
8f945b7 (Turbo 2.5.8, 3 tasks) and
resend/react-email at
1531a39 (Turbo 2.9.14, 7 tasks), filter ./packages/*, 10 workers
each. vx 0.0.634 configured by vx init --native, plus
vxHistory({ file: '.vx-timings.json' }) fed timings from
one earlier untimed run (the file is the only thing kept across the
cache wipe). Turbo is each repo’s own, --no-daemon. pnpm 12.4.2,
Node 22.22, linux x64, 4 cores, 16 GB.
| Repo | Case | vx | Turbo |
|---|---|---|---|
| create-t3-turbo | Cold build | 6.23 s | 7.14 s |
| create-t3-turbo | Restore | 0.087 s | 0.519 s |
| create-t3-turbo | Nothing new | 0.081 s | 0.476 s |
| react-email | Cold build | 30.21 s | 33.22 s |
| react-email | Restore | 0.283 s | 0.382 s |
| react-email | Nothing new | 0.136 s | 0.252 s |
Cold is the total wall time of the build, as on every real repo; restore and nothing new are runner overhead: every task is a cache hit.
Performance history
Section titled “Performance history”Where vx’s own headroom went, on the same 1090-package / 3,270-node graph,
fully cached (vx run build test --all):
The table
| Milestone | No-restore | Restore |
|---|---|---|
| Set-closure scheduler priority (before) | 10.2 s | — |
| + bitset scheduler closure | 1.27 s | 1.59 s |
| + discovery / package-graph fixes | 1.03 s | 1.34 s |
+ frontier ^task expansion (v19, 8.5× fewer edges) |
0.62 s | 0.87 s |
Input hashing then moved to git blob OIDs (v20, git ls-files -s): clean
files cost zero reads/stats, dropping the warm run-phase from ~245 ms to
~76 ms (3.2×) at 500 projects × 30 files, and cold runs never read
committed file contents at all. The decision history lives in git (the log was retired 2026-09-02);
the shipped-optimization catalog with invariants is
optimizations.md, and the engineering tour is
comparison.md § Where vx is ahead.
Known headroom
Section titled “Known headroom”Config evaluation was the largest fixed cost of a warm run
(loadProjectConfig ~199 ms of a ~517 ms warm wall at 1,000 projects,
2026-09-02, before the day’s work). A config evaluation cache behind a
static purity gate took load configs to 16–25 ms per 1,000 configs
the same day; it was removed on 2026-10-14, when every run moved to the
lock and only vx lock evaluates.
Source vs binary. The runner invokes bun packages/vx/src/bin.ts by default, which pays ~40 ms of transpile per run that the --bytecode release binary does not (2026-09-09: 114 vs 71 ms on a two-package workspace; 20 projects warm 109 vs 64 ms). Set VX_BIN=<path> to time the shipped binary instead.