Skip to content
GitHubRSS

Two cached tasks, one output path (2026-09-20)

Status: IMPLEMENTED 2026-09-22 (item 588), on the design below, after the survey met its own gate (three repositories with the addition shape). docs/caching.md § “Additive outputs” is the user-facing contract; tests/overlapping-outputs.test.ts is the matrix, run over both shapes in the wild (a subdirectory, and one dist for both). The rewrite-in-place shape is no longer refused but stays out of scope: it works, and costs the upstream a restore per warm run. What this note adds to the sketch is the part that decided whether it was buildable: the two places vx had to change, and the one invariant the sketch as written would have broken.

Two cached tasks, B depends on A, and both declare the same directory as their output. Nx caches both; vx leaves the dependent one uncached, and @vzn/vx-migrate resolves the overlap at migration time (item 146).

Two of the five real Nx repositories have it:

  • strapi — build fills dist, build:types runs tsc into the same dist. B adds files A never wrote.
  • refine — build is tsup && node ../shared/generate-declarations.js and types is that second half again, so B rewrites A’s .d.ts files with identical bytes and new mtimes.

Only the first is in scope. The second is why the scope line exists, and § Why the rewrite stays refused says what it would cost to admit it.

execute-task.ts removes a task’s declared outputs before it runs and before it restores. That is not incidental: a restore that merged into a dirty tree would replay a mixture of two runs, which is the stale-hit class — the worst failure class this repository has. So B’s clean would delete the dist A just filled, and B’s artifact would carry A’s files as if they were B’s.

When B’s declared outputs overlap a transitive upstream A’s, and B depends on A:

  1. B’s own output set is what its run ADDED or CHANGED. Snapshot the overlap before B runs, diff it after. The proof is size + mtime — the same proof the hit path already trusts for a current tree (docs/caching.md § A current tree), so no new trust is introduced.
  2. B’s clean removes only that set, leaving A’s files where they are.
  3. B’s artifact holds only that set, so restoring B into a tree A has already filled reproduces exactly what B’s run produced.
  4. The restore order follows the edge: B is restored only after A is on disk.

Cost: one stat walk of the overlap per B miss, none on a hit.

Point 4 is not free, and the sketch does not say what it costs. A confirmed stable-key local hit is restore-tier (src/graph/scheduler.ts): it becomes ready immediately, because “a stable hit’s restore needs none of its deps’ output”. An overlap-narrowed artifact breaks that premise by construction — it is exactly an artifact that needs its dep’s output already on disk.

So the design needs a second stability axis. Today’s gate (src/orchestrator/stable-keys.ts, dependsOnSiblingOutputs) asks whether an upstream writes where this task READS. The overlap case is about where this task WRITES. The two do not coincide:

  • Same project (strapi, refine): covered by the conservative gate — upstreamOutputProjects.has(node.projectName) makes any same-project dependent of an output-declaring task unstable, so B is not restore-tier and not prefetched.
  • Cross project: B’s project-relative outputs land in B’s own directory, so an overlap requires root-anchored (workspaceFiles) outputs on one side — and local-shortcircuit.ts keeps every task whose project directory such an output can REACH out of the restore tier, and every dependant of one (restoreTierExclusions, item 584; until then the tier was disabled graph-wide the moment any task declared cache.outputs.workspaceFiles). In the fixture below B’s output reaches A’s directory, so A is kept out, and B — A’s dependant — with it. Probe reuse still applies, so those tasks are not probed twice; they simply stay dep-gated.

Measured, because an earlier draft of this note got it wrong. The first version claimed the cross-project case was unguarded and that an implementation would have to exclude it. It is guarded. The fixture: a#build writes dist/**, b#build declares outputs.workspaceFiles: ['pkgs/a/dist/b.txt'], depends on a#build, and detaches its key with cache.inputs.tasks: [] so it can HIT while A misses — the only arrangement in which B could restore into a directory A is about to clean. Polling the directory through the run put b.txt at 1142 ms and a.txt at 1131 ms: B restored AFTER A, because with a workspace output in the graph nothing was restore-tier at all (then; now because B’s output reaches A’s directory and B depends on A, so both are kept out — the same order by a narrower rule, pinned as § “the design note’s cross-project overlap stays dep-gated on both sides”). Ten further reps with A’s artifact at 2000 files left a correct tree every time.

So the constraint for an implementation is not “add an exclusion” but “do not remove the one that exists”. The blanket rule was expensive — one workspaceFiles output anywhere cost the whole graph its restore tier — and item 584 narrowed it to a reach test on the output’s static prefix, propagated down the edges; the pins that own this case are tests/local-shortcircuit.test.ts § “an unrelated project whose DIRECTORY the writer’s output reaches stays OUT” (item 425’s claim, path-based) and the fixture row above. The older row held a project with no edge to the writer at all, with the same workspace and a project-relative output as its control. It fails against the obvious narrowing (exclude the declaring node) while the older dependent-of-a-producer row stays green — which is why it is a row of its own (item 425).

refine’s types ADDS nothing: it rewrites A’s .d.ts files with the same bytes and new mtimes. Under the design above B’s own set is empty (same size, same content), but the proof is size + mtime, so the next run finds A’s outputs moved and restores them — a restore where there was nothing to restore, every run.

Admitting it needs one of:

  • Content comparison for files a downstream task touched — a hash per overlapped file, which is the cost the mtime proof exists to avoid. Bounded by the overlap rather than the tree, so it is not obviously unaffordable; it would need a measurement before anyone believes it.
  • Or the status quo: only additions are admitted, a rewrite-in-place keeps the dependent uncached, and the migrator keeps saying so.

The second is what ships until a repository makes the first worth paying for.

Twelve more real monorepos, cloned at HEAD and scanned statically for two cached targets of one project whose declared outputs overlap where the second depends on the first (turbo.json tasks × package scripts; project.json targets with nx.json targetDefaults; item 587):

RepositoryRunnerPairsShape
twentyhq/twentyNx2 (twenty-ui, twenty-shared)ADDITION: build → dist (emptyOutDir: false), build:individual depends on it and writes dist/individual (emptyOutDir: true on its own subtree)
storybookjs/storybookNx44 (every code/sandbox/*)ADDITION: sandbox generates sandbox/<dir>, build depends on it and writes sandbox/<dir>/storybook-static inside
novu, ngrx/platform, TanStack/queryNx0—
trpc, shadcn-ui/ui, formbricks, trigger.dev, dub, cal.comTurbo0—
supabaseneither0—

So with strapi that is three repositories showing the ADDITION shape and still one (refine) showing the rewrite; condition (1) below holds, and the rewrite stays refused. The shape in the wild is a SUBDIRECTORY (dist/individual, <dir>/storybook-static) or a sibling file set, never an interleaving of the same files — which is what makes the size + mtime diff of the design sufficient.

What had to be true to build it (all four held, item 588)

Section titled “What had to be true to build it (all four held, item 588)”
  1. A third repository shows the ADDITION shape (Next 16’s own gate; the two known repositories are one of each, which is not a pattern yet).
  2. The restore-tier exclusion above still holds — via the reach test (item 584): an overlap-narrowed artifact’s output reaches the upstream’s directory by construction, so the dependant is kept out; a narrowed artifact is only safe while it cannot be restored early, so anyone changing that rule pins this case first.
  3. The snapshot/diff lands in execute-task.ts’s clean + save path, where cleanOutputs already returns the paths it removed and already marks them in the git files cache; the narrowed set flows to the same two places.
  4. A stale-hit test proves the composed case: A hit + B miss, A miss + B hit, both hit, both miss, and a B whose run adds nothing — each leaving a tree byte-identical to a cold run of both.

Until (1), this note is the record of what the sketch costs, so the next reader does not have to re-derive the restore-tier conflict.