Task execution lifecycle

This document traces what happens between vx run build typed at the terminal and a task succeeding or failing. Read it alongside architecture.md (module map) and caching.md (cache mechanics).

End-to-end timeline

 ┌─ CLI dispatch (src/bin.ts → src/cli/index.ts → src/cli/run.ts)
 │    1. bin.ts spawns; forwards process.argv to cli.run().
 │    2. cli/index.ts dispatches by subcommand (run / watch / cache /
 │       lock / init / upgrade / show / info / why / last / docs /
 │       completions / help / version); any other verb is asked of the
 │       workspace's plugins (`commands` seam).
 │    3. cli/run.ts:parseRunArgs(argv) → RunArgs (validated; resolves
 │       the 4-axis cache policy from --cache-policy / --no-cache / --force).
 │    4. cli/run.ts:runCmd resolves the project scope:
 │         - bare positionals respect --all / --filter / --affected / cwd
 │         - anchored positionals (pkg#task) target directly
 │         - no positionals + TTY → interactive picker → pkg#task
 │    5. The options map to RunOptions and the run executes IN THIS
 │       PROCESS — always. (A whole-run `backend` seam existed until
 │       2026-08; it moved the scheduler server-side, which is why it
 │       went. Per-task placement is the `executor` capability.)
 │       --dry short-circuits into planRun instead.
 │
 ├─ Prepare (src/orchestrator/prepare.ts:prepareRun — shared with planRun)
 │    1. findWorkspaceRoot(cwd) — the directory of the nearest
 │       .vx/workspace.* up to the git top-level. An UNSCOPED run (no
 │       explicit project scope, at least one bare task name) then
 │       starts step 12's git enumeration over the whole tree,
 │       overlapping steps 2–11; a scoped run waits, since its
 │       pathspecs depend on which projects the task graph holds.
 │    2. Read .vx/projects.lock.json: the project directories `vx lock`
 │       found (every directory holding a vx.config.*, git's ignore
 │       rules applied, none under node_modules) and their configs. Each locked config file is
 │       stat'ed; one gone is refused ("run `vx lock`").
 │    3. loadWorkspacePlugins — loadWorkspaceConfig reads the
 │       .vx/workspace.{ts,mts,js,mjs,cts,cjs} the root lookup found (baseBranch /
 │       run / cache / inputs / outputs / watch / plugins / lock),
 │       then the plugin `config` stage runs on it.
 │    4. The plugin `discover` stage: more directories, names and
 │       edges. A project is named by its config's `name`, else a
 │       plugin's, else its path (`//` for the root); two of one name
 │       are refused naming both directories.
 │    5. buildPackageGraph — project edges the `discover` plugins gave.
 │    6. Local cache open: new Cache(cacheDir, { read, write }, root)
 │       with the policy's local slice. Then
 │       readClaimedFiles: one read of each root file a `fingerprint`
 │       plugin claims (`@vzn/vx-lockfile`: its lockfile and install
 │       settings; nothing without one), the bytes for the mid-run
 │       check. Core folds none of them into a key.
 │    7. SCOPED config loading — only in-scope projects plus their
 │       transitive dependency closure load ('^task' frontier
 │       expansion never escapes the closure); one staged load,
 │       loadProjects, shared with every reading verb. A run's configs
 │       load from .vx/projects.lock.json — no evaluation, no staleness check
 │       of its own (vx lock --check is the audit), frozen-env
 │       semantics; a missing lock or entry is a hard UserError. The
 │       verbs that evaluate (vx lock, the readers) use
 │       loadProjectConfigs in one batch: native Bun await import()
 │       with a content-hash query-string bust, every config
 │       evaluated; the loader validates each
 │       TaskConfig shape. vx lock runs the plugin `project` stage on
 │       each config with a re-validation after every plugin and
 │       records the result; a run reads that record, no hook runs.
 │    8. computeNestedProjectDirs — set of projects rooted inside each
 │       project, computed over EVERY config-bearing project (loaded
 │       or not) for boundary enforcement.
 │    9. expandRequested (see Task selection below).
 │   10. Cache layer: an injected RunOptions.remoteCache is composed
 │       with the local cache into a LayeredCache (it wins); else
 │       resolveCache layers each plugin's remote over the local
 │       cache, in declared order; else bare local.
 │   11. buildTaskGraph (see below).
 │   12. Bulk git populate — the enumeration step 1 started is
 │       awaited, or a scoped run starts it here over the projects
 │       that own a task (not the dependency closure step 7 loaded,
 │       which a `lint` of one package does not key). THREE spawns at
 │       the root, concurrent, and the repository's facts asked while
 │       they run (`ls-files -s -v -z` for the index: every tracked
 │       path's OID and its cache-state flag;
 │       `status --porcelain -z -uall` for
 │       the dirty AND untracked sets, the one worktree walk; `var -l`
 │       for core.autocrlf and the attributes files git reads outside
 │       the tree; the facts — prefix, common dir, object format, index
 │       file, git dir — read off a plain `.git` directory, else one
 │       `rev-parse --show-prefix --git-common-dir --show-object-format
 │       --git-path index --absolute-git-dir`,
 │       memoized per process and shared with the file hasher)
 │       fill the per-project GitFilesCache with file lists + index
 │       OIDs. `ls-files --others` is NOT among them — status's
 │       `-uall` already answers untracked, and asking git twice
 │       walked the same tree again. `core.ignorecase=true` on a disk
 │       that tells case apart runs status again with
 │       `-c core.ignorecase=false`. A fifth, `check-attr`, runs only
 │       when an attributes file could rewrite bytes. An index the
 │       cache holds no blob-size verdict for (keyed by the index
 │       file's hash, A-60) adds `ls-files --debug` for the recorded
 │       sizes and `cat-file --batch-check` (one per core) for the
 │       blob sizes `blob_sizes` lacks: an OID whose blob is not the
 │       recorded size is not trusted. The run's HashCache is created
 │       after it.
 ├─ Task selection (graph/task-graph.ts:expandRequested)
 │    Bare task names fan out across the resolved candidate projects
 │    (every project that declares the task). Anchored entries
 │    (`pkg#task`) resolve directly. Duplicates are deduped.
 │    Empty result → run returns `{ ok: false, outcomes: [] }` —
 │    no task is treated as a CI footgun. The exception: a diff-scoped
 │    run (--affected) whose task names are all declared by projects
 │    outside the selection is `none-affected` and returns
 │    `{ ok: true, outcomes: [] }`.
 │
 ├─ Task graph (src/graph/task-graph.ts:buildTaskGraph)
 │    Starting from the resolved {project, task}[] pairs, walk
 │    dependsOn:
 │      - 'name'     → same-project task
 │      - '^name'    → task in the nearest deps declaring it
 │                     (frontier walk; non-holders passed through)
 │      - 'pkg#name' → specific package's task
 │    Detect cycles — throws with the path. After the graph and key
 │    stages, excluded edges (per --exclude-dependencies[=names]) leave
 │    the SCHEDULE; each dropped task is still keyed and folded into
 │    its dependant (orchestrator/excluded-keys.ts).
 │    Each node carries: id (`${project}#${task}`), projectName,
 │    projectDir, taskName, config, sorted deps, `requested: boolean`.
 │    markSurfacedDeps then flags the display-only `surfaced` tasks a
 │    requested GROUP stands for (transparent folders).
 │
 ├─ Plugins + telemetry (src/orchestrator/run.ts)
 │      1. installPlugins — always called; each plugin's optional
 │         setup() hook runs; a throw aborts the run with a clean
 │         UserError naming it. With no plugins it runs nothing.
 │      2. Run context capture — always, for the `invocations` row:
 │         commit + branch read from `.git` directly, `git rev-parse`
 │         only when that reader does not understand the layout; dirty
 │         reuses the GitFilesCache's status; CI-env detection,
 │         host/os/arch. Never fails a run.
 │      3. Telemetry — only when a plugin has a `telemetry` hook or
 │         RunOptions.telemetrySinks is passed: the run-context record
 │         (adding captureWorkspaceIdentity) is built and
 │         subscribeTelemetry collects the sinks; with ZERO sinks it
 │         returns undefined and NOTHING subscribes (the no-telemetry
 │         hot path is byte-identical).
 │
 ├─ Run-level state
 │    • runId   — UUIDv7 stamped once per `vx run` invocation; every
 │                task in the resulting graph carries it.
 │    • runStartHrTimeNs — hrtime.bigint() anchor; per-task wallclock
 │                spans are stored relative to it.
 │    • persistentRegistry — Map<taskId, Subprocess> of long-running
 │                children.
 │    • liveChildren — Set<Subprocess> of in-flight children; the
 │                runner adds/removes each around its spawn.
 │    • SIGINT/SIGTERM/SIGHUP handlers (removed in a finally): on
 │                signal, forward it (SIGHUP as SIGTERM) to everything
 │                in liveChildren + persistentRegistry (and kill any
 │                running inputs.runtime probe, C-65), wait
 │                2 s for their groups, SIGKILL
 │                what is still there, let the run finish its own
 │                end (flush, teardown, cache close), then
 │                die of the signal (SIGINT → 130, SIGTERM → 143,
 │                SIGHUP → 129), so a shell script running vx stops
 │                too; a run that handed a task the terminal exits
 │                with that code instead, so Bun restores it. SIGHUP is registered because a task
 │                runs in its own session, so a closing terminal
 │                reaches vx and nothing else. A second signal
 │                SIGKILLs and exits at once.
 │    • SIGTSTP/SIGCONT handlers: Ctrl-Z SIGSTOPs every task group,
 │                then vx; SIGCONT (fg, bg) continues them.
 │
 ├─ Cache acceleration (before scheduling)
 │    • REMOTE PREFETCH (a cache layer with a remote only) — derive every
 │      stable-key cacheable task's key up front (reusing the run's
 │      hashCache memo) and fire the remote GETs in the background
 │      under a bounded pool. Not awaited before scheduling (the
 │      overlap is the point); drained before cache.close(). At most
 │      one remote GET per key (shared in-flight map with the lazy
 │      read-through).
 │    • LOCAL SHORT-CIRCUIT (local reads on, ≥1 task) — derive the
 │      same stable keys and probe the LOCAL store in ONE batched
 │      getMany (a layer without getMany: get once per task under a
 │      bounded pool) → a `preProbed` map (hits AND stable misses;
 │      with a remote declared, hits only: a local miss may be a
 │      remote hit) + a `restoreTier` set (confirmed hits). Awaited
 │      before scheduling; never asks a remote; never throws
 │      (degrades to the normal schedule).
 │
 ├─ Scheduling (src/graph/scheduler.ts:runGraph — two-tier)
 │    Two ready queues, each on its own lane:
 │      - EXEC tier — dep-gated; ready when every dep completed; up
 │        to N (concurrency) at once.
 │        Priority: transitive-reverse-dependent count (bitset
 │        closure), optionally overridden by a `priorities` map.
 │      - RESTORE tier — confirmed stable local hits; ready
 │        IMMEDIATELY (no dep gate, no failed-dep→skip check — their
 │        key is dep-independent) at LOW priority, on a restore lane
 │        of 2×N. The drain rule: exec-tier first; a restore never
 │        takes an exec slot and the two lanes never wait on each
 │        other — except at N = 1, where they share the one slot.
 │    On failure: exec-tier dependents are marked `skipped` (exit 1,
 │    durationMs 0, no spawn); independent siblings keep running.
 │    The scheduler doesn't know about caching; the execute callback
 │    is the seam. (An embedder running concurrent runs in one process
 │    may pass a shared `inflight` map; admission.ts then dedupes
 │    identical-key tasks across them. A plain `vx run` passes none.)
 │
 ├─ Per-task execution (src/orchestrator/execute-task.ts:executeTask)
 │    Each task takes one of three paths:
 │
 │    A. GROUP — no `exec`.
 │       Return success with a derived hash rolled up from upstream
 │       outcomes. No spawn, no I/O. Wallclock = 0.
 │
 │    B. PERSISTENT — `exec.persistent` set.
 │       1. Build isolated env (essentials + passThrough + define +
 │          <projectDir>/node_modules/.bin, then
 │          <workspaceRoot>/node_modules/.bin, prepended to PATH).
 │       2. runPersistent — Bun.spawn the command; subscribe to
 │          stdout/stderr chunk-by-chunk.
 │       3. Resolve `ready` when:
 │            - no readyWhen → immediately on spawn
 │            - readyWhen matches the output (complete lines OR the
 │              trailing partial line) → on that match
 │            - child exits 0 before the match → resolve
 │            - child exits otherwise before the match → reject with
 │              PersistentReadyError('persistent task exited before
 │              becoming ready (exit N)'); its output already streamed
 │              through the logger
 │            - readyWhen set and the timeout (exec.timeout, else the
 │              run default) elapses first → fail, SIGTERM, SIGKILL
 │              after the grace
 │       4. On ready: stash child in persistentRegistry; return
 │          success. Downstream tasks unblock.
 │       Note: cache + persistent is a config error (rejected by the
 │       project loader). Persistent tasks never write to cache.
 │
 │    C. NORMAL — `exec.command` only.
 │       1. resolveInputs(files, env, runtime)
 │            - glob inputs.files (git-backed, declared-outputs-
 │              excluded, nested-projects-excluded)
 │            - glob inputs.workspaceFiles from the WORKSPACE ROOT
 │              (git-aware, NO boundary rule — the documented escape
 │              hatch); resolved paths join the same input list
 │            - read host process.env values for inputs.env names
 │            - resolve inputs.runtime / workspaceRuntime command
 │              output (deduped per run; a non-zero exit throws a
 │              UserError that fails this task — its dependents skip,
 │              independent tasks continue)
 │       2. hashTaskConfig (memoized per run via HashCache)
 │       3. filterUpstreamHashes (apply inputs.tasks filter)
 │       4. cache.key({ taskId, taskConfigHash,
 │                      forwardArgs, envValues, runtimeValues,
 │                      workspaceRuntimeValues, upstreamHashes,
 │                      pluginParts, platform, inputFiles }) → 16-hex xxh3
 │          (pluginParts is whatever the `key` stage contributed,
 │          folded between the upstream keys and the input files —
 │          see caching.md § Cache key derivation for the order.)
 │          (Skipped when the local short-circuit pre-derived it.)
 │       5. If willRead (`cache: true` + a read axis on):
 │            consume the up-front probe when present, else
 │            cache.get(hash)
 │              · hit → up-to-date short-circuit when the on-disk tree
 │                       already matches; else cleanOutputs →
 │                       restoreOutputs → replay stored stdout →
 │                       return cache-hit / cache-hit-remote
 │              · miss → fall through; cleanOutputs first (when
 │                       willWrite) so a stale prior build can't
 │                       survive the fresh exec
 │       6. Build isolated env. For a cacheable task,
 │          describeTaskInputs({ captureInto }) then describes the input
 │          set ONCE, before the spawn: the executor gets the input
 │          values and the save reuses the captured per-component input
 │          fingerprint (a re-fold over the memos, no I/O).
 │       7. runCommand (or runSandboxed when `sandbox` is declared) →
 │          Bun.spawn shell with `command` + forwardArgs
 │          (shell-quoted). Buffer chunks via onStdout/onStderr.
 │          exec.timeout SIGTERMs an overrun → real `failed`, never
 │          cached. An attempt that ends while the run is STOPPING
 │          (Ctrl-C teardown) classifies as `aborted` instead, whatever
 │          its exit — not recorded, not retried.
 │       8. On exit 0 + willSave (willWrite, no tainted upstream):
 │            keyStillTrue() — if a claimed root file or an
 │              input moved since the key was taken, the result stands
 │              but the save is withheld (said on the status line).
 │            serverDiedUnder() — a server it depends on died while
 │              it ran: the save is withheld, and its dependants'.
 │            saveMiss — in the slot: resolveOutputs(outputs) +
 │              resolveWorkspaceOutputs → file lists, and the
 │              project's git snapshot notes the written output paths
 │              so a same-project downstream task doesn't re-spawn git
 │              unless its globs overlap. Deferred onto the run's save
 │              lane (2×concurrency wide): cache.save(...) — pack
 │              <hash>.tar.zst with stdout + outputs/<rel>
 │              (+ workspace-outputs/<rel-to-root>), upsert entries +
 │              output_files + entry_inputs rows in one transaction.
 │              The slot frees at once; dependents wait on the save's
 │              `landed` promise (the scheduler's settledOf). Under a
 │              LayeredCache with remote writes on, the remote upload
 │              fires in the BACKGROUND (drained at end of run).
 │       9. Return TaskOutcome { node, status, exitCode, durationMs,
 │            hash, cpuMs?, peakRssBytes?, restored?,
 │            wallclockStartNs, wallclockEndNs }.
 │
 └─ End-of-run
    1. SIGTERM dependency-only persistent children still up (each is
       stopped earlier, once every task using it finished); persistent tasks
       the user REQUESTED, those a requested group stands for (through
       nested groups, not through a one-shot), and the persistent tasks
       they depend on, are kept alive (see below).
    2. Run summary footer (projects / tasks / cache meters + info +
       time + result) — counts only real tasks (with `exec`); group tasks
       don't pollute the totals; failure frames replay just above it.
    3. Optional --profile Chrome-trace JSON (default profile.json).
    4. recordRunBundle — one transaction writing a `runs` row per real
       task plus the `invocations` header row (command, git/CI/host
       context, counts). Group + aborted tasks skipped; a run
       a signal stopped records only the tasks that finished, with no
       header row, so `vx last` does not list it. A stop in the
       keep-alive wait (step 8), the usual end of `vx run dev`, is no
       such stop: the run is recorded ok.
    5. Telemetry: emit the RunSummaryRecord to every active sink +
       await their flush (crash-isolated; skipped when no sink).
    6. Drain background remote prefetches/uploads; cache.close();
       sandbox teardown when any task was sandboxed.
    7. Return { ok, outcomes }; ok = every task ended success or a
       cache hit or a platform skip (any other failed/skipped/aborted →
       ok = false → exit 1),
       and no persistent child exited on its own before the run
       stopped it.
    8. FOREGROUND ONLY: if the user requested persistent tasks (dev
       servers) and nothing else failed (or under `--continue=always`),
       the process now blocks until ONE of them, or a persistent task they
       depend on, exits — the
       summary is already printed, `▸ <id> running` rows list what's
       alive, Ctrl-C reaps the process group. That first exit ends the
       session: the others are torn down (SIGTERM, grace, SIGKILL) and
       a non-zero exit makes the run exit 1, so a script's `vx run dev`
       fails when the server it started fell over. Under
       `holdPersistent` (the watch loop) run() instead returns them, and
       every dependency-only server with `inputs.files` too, on
       `RunSummary.persistent`, still running, for the caller to stop;
       one that dies on its own after that is said, its `stop()` is not.

One command per task

exec.command is a single shell command — there’s no multi-step sequence. Three ways to chain:

• Shell composition — &&, ;, pipes. The shell is the API.

  exec: {
    command: 'gen && tsc && cp -r assets dist/'
  }

• Separate tasks linked by dependsOn:

  codegen: { exec: { command: 'gen' }, ... },
  build:   { exec: { command: 'tsc' }, dependsOn: ['codegen'], ... },

• Group task that fans out:

  release: {
    dependsOn: ['build', 'test', 'package']
  }

Per-task caching is the right granularity for invalidation. Splitting gives you per-step caching naturally; combining with && gives you one cache slot for the whole chain.

A task is over when its command’s own process exits. What a descendant writes after that — one the command detached with setsid … &, or daemonized — is not the task’s output: the save has already run, and a later hit’s restore removes it (caching). wait for what the command backgrounds.

Env isolation

The child process gets, in priority order (lowest first):

1. Essential allowlist (PATH, HOME, SHELL, USER, LOGNAME, TMPDIR, TEMP, TMP, LANG, LC_ALL, LC_CTYPE, TERM, COLORTERM, FORCE_COLOR, NO_COLOR, CI, NODE_OPTIONS, COREPACK_HOME, PNPM_HOME, RUSTUP_HOME, CARGO_HOME, GOPATH, GOMODCACHE, GOCACHE, XDG_CONFIG_HOME, XDG_DATA_HOME, XDG_CACHE_HOME, XDG_STATE_HOME, XDG_RUNTIME_DIR — the list is ESSENTIAL_ENV in src/util/essential-env.ts).
2. exec.env.passThrough names → values from host process.env.
3. exec.env.define literal name/value pairs.
4. PATH augmentation — <projectDir>/node_modules/.bin, then <workspaceRoot>/node_modules/.bin, are prepended so installed tools (oxlint, vite, etc.) run by name. Never a sibling project’s bin; those stay invisible, even when vx itself runs from npm run in a sibling: the PATH vx inherits loses each node_modules/.bin inside the workspace first (by spelling, or by canonical path for one spelled through a symlink), and so does an inputs.runtime probe’s. A bin directory whose path holds PATH’s delimiter (:) is left out: PATH cannot name it, and split it became two entries, one relative to the task’s cwd. A task whose shell exits 127 or 126 gets one more frame line (orchestrator/shell-verdict.ts): for a bare word, that 127 is the shell’s “command not found”, the word (when the command is a plain word args…), the two bin directories vx puts first, and that a sibling project’s bin is never visible (or, when a bin directory was left out for its :, that directory and the fix: move the workspace), and on 126 chmod +x; for a word with a slash, what the file says — missing (the resolved path), a directory, not executable by this user, a #! interpreter that does not exist (a CRLF line ending is named as such), or no #! line. An exit above 128 gets the same line for its signal: which one, and what sends it (the OOM killer, a crash in native code, an abort, a reader that left a pipe, a ulimit, a seccomp refusal, a SIGTERM or SIGINT from outside vx); vx’s own timeout keeps its own line, and a task the run’s stop killed (the SIGKILL past the grace too) gets none.

Anything not in these four layers is absent from the child’s environment, except the two vx sets itself — VX_RUN_WORKSPACE and VX_RUN_TASK — so a task that shells out to vx run in its own workspace is refused before it forks without bound, and npm_execpath, the workspace’s package manager (unkeyed; schema.md § env). A sandboxed task also gets the sandbox’s own proxy, CA and TMPDIR values (modules/sandbox-runtime.md). This prevents incidental env leakage between machines and gives reproducible runs. It is hygiene, not a boundary: an unsandboxed task runs as you and can still read your home (vx hides its own environment, modules/util-hide-env.md). exec.sandbox is the boundary (security.md).

The allowlist + isolation contract lives in modules/env.md and is the only field the contract assumes for “what every command needs to function” on Linux and macOS (Windows is WSL, which is Linux). Adding to the allowlist would be a deliberate schema-extending change (consumer code expects a particular set; broader access has cache-stability implications).

Failure handling

Failure	Behavior
Exec exits non-zero	Task is failed; cache NOT written; output streamed live + the failure frame replays at run end
exec.timeout overrun	SIGTERM to the task’s group, SIGKILL for whoever is left after the grace; task is failed (timed out), exit 143 (137 when the SIGKILL took it), never cached; the line names the SIGKILL when it took anyone
Child killed by Ctrl-C teardown (SIGINT/SIGTERM/SIGHUP), or by an embedder’s RunOptions.signal abort — which also completes every never-started task aborted	Task is aborted — not recorded in history. An attempt that ends while the run is stopping is aborted whatever its exit: a trap that exits 0 is not cached, and a failure is not retried (item 962). What it printed still shows in its frame. A child killed by a signal while the run is NOT stopping (a supervisor’s SIGTERM, a kill from another shell) is failed (exit 143, 128 + SIGTERM): retried, shown, and fail-fast trips (item 1100)
execute() throws (internal error)	Task marked failed; stderr written [vx] internal error in <id> (a UserError reports plainly); a plugin executor’s throw reports plainly as plugin '<p>' (executor '<e>') failed in execute: <reason> (C-63, C-85)
Persistent task exits non-zero before ready	Task marked failed with not ready: exited (exit N) before a line matched readyWhen /…/, and a 127, 126 or signal exit gets the one-shot task’s verdict line; its output already streamed live; aborted when the run’s stop killed it (C-62)
Upstream task fails	Under the default --continue=deps-ok, dependents marked skipped (exit 1, durationMs 0); no command runs — EXCEPT a restore-tier task, whose confirmed cache hit still restores (its key is dep-independent). --continue=never stops dispatch; --continue=always runs dependents, never saved
Sandbox violation (macOS monitor / Linux structural)	Task is failed; violations render in the frame; nothing cached
Remote-cache error (500, timeout, corrupt artifact)	Degrades to a cache miss; never fails the run
No .vx/workspace.* in cwd or any parent up to the git top-level	findWorkspaceRoot throws (UserError); vx run exits 1
Same-project task referenced in dependsOn not declared	buildTaskGraph throws with the offending edge
Two projects of one name	checkProjectNames throws with both paths
Cycle in task graph	detectCycle throws with the cycle path
Malformed config	loadProjectConfigs throws (UserError) with file + field
inputs.runtime / workspaceRuntime probe exits non-zero	cache key not derived: inputs.runtime probe exited N: <command>; that task is failed (cache key not derived) with no command run, dependents skip

By default (--continue=deps-ok) failures don’t kill the scheduler — independent tasks already in flight finish, and unrelated tasks not yet started still run. --continue=never stops dispatch at the first failure; --continue=always runs dependents too (cli.md § Failure propagation). The overall exit code is 1 if any task ended in failed, skipped (a platform skip, and what it blocks, aside) or aborted status, or a persistent child exited before the run stopped it. A task’s own exitCode agrees: a platform skip and what it blocks record 0, a skip behind a failure 1.

Output capture and rendering

The orchestrator emits RunEvents through an in-process bus; the terminal renderer is the always-on subscriber (plugins, telemetry, and wire forwarders attach beside it). What renders:

• Flow-aware policy. What gets rendered depends on the run’s intent: FOCUSED (no selection flag) streams the requested task’s output raw and live and silences successful dependencies; BROAD (--all / --filter / --affected) prints news only — one grid one-liner per executed task, failure frames replayed at run end, silence for cache hits; truthy CI env (and the programmatic default) keeps full grouped output. --output-logs overrides everything. Full table in cli.md.
• The task grid. Reported task lines share one column grid: &lt;status&gt; &lt;cache&gt; <±usual> &lt;time&gt; &lt;cpu&gt; &lt;mem&gt; &lt;outputs&gt; &lt;blocks&gt; &lt;worker&gt; &lt;mark&gt; &lt;task&gt;, with no glyph. The status word, at column 12, is the task axis and its colour the outcome (success green, failed red, skipped and running yellow). The cache word (miss, no-cache, fresh, local, remote) is the cache axis. The mark is a yellow ◆ on the critical path. An empty cell shows —.
• Buffered, framed (non-focused paths). runCommand listens to the child’s stdout/stderr and calls onStdout / onStderr per chunk, a line at a time across the two: a chunk on one stream waits while the other’s line is open (dash writes its /dev/tty error in pieces, and a stdout write landed inside it), at most 50 ms and 64 KiB, then goes as written: a newline vx added would be the stream’s own text when a hit replays it. Outside focused streaming, the default logger buffers the chunks per-task and dumps the full body as a framed block on task completion. No per-line prefix, no interleaving between concurrent tasks.
• Cache write. The captured stdout — its first and last 8 Mi characters (8 MiB of ASCII), the cut middle named — is stored in the entry; replay is stdout-only.
• Cache hit replay. The stored stdout is fed through the same logger path, so it renders per the active flow (streamed raw for a focused requested task, framed in full mode, silent in broad).
• Live stream for failures. Even though cached output is not written for failures, the live stream means the user sees the failure as it happens; the full frames replay together at run end, right above the summary.
• Status region. On TTY stdout outside CI, a multi-row region tracks the run live: a blank separator, a pinned running row per persistent task, one row per worker slot (the ticking elapsed time IS the motion — no spinner), and the live summary section (the same meters as the final footer). Redrawn in place; forced redraws coalesce under a 30 ms floor; erased before the summary prints. All default-logger writes serialize through one writer so content and the region never interleave.
• GitHub Actions. With GITHUB_ACTIONS truthy in full mode, task blocks collapse under ::group:: commands; failures stay open and emit ::error annotations.

A task’s stdout and stderr are pipes, so echo x > /dev/stdout and cmd | tee /dev/stderr work as in a terminal or under Turbo. Bun’s own 'pipe' is a socketpair, which Linux will not open by path, and they failed with “No such device or address” until X-113 (tests/runner.test.ts › “a task’s stdout and stderr are pipes it can open by path”; sandboxed in tests/sandbox-runtime.unsafe.test.ts).

There is no special handling for binary output. Stdin is never the terminal unless the task declares exec.interactive and vx’s stdin is a TTY (schema.md § interactive): then the task gets vx’s stdin, stdout and stderr, runs alone, and nothing of it passes through vx. Otherwise the rule splits on exec.persistent:

• A one-shot task gets 'ignore': it sees EOF at once, so a task that reads stdin finishes instead of hanging CI (tests/runner.test.ts › “a task that reads stdin sees EOF at once, never a hang”). A task that needs TTY input declares exec.interactive.
• A persistent task gets a pipe vx holds and never writes: it stays open while vx lives and ends when vx does. A dev server that exits on stdin EOF — esbuild --watch, the Vite case in turborepo#8915 — became ready and exited 0 under 'ignore', and vx run dev ended green (fixed 2026-09-24, tests/keep-alive.test.ts › “a server that exits on stdin EOF stays up while vx runs”). Each task has its own pipe, so several persistent tasks never compete for keystrokes; none receives any, so a server’s keyboard shortcuts (Vite’s h, r) do not reach it unless it declares exec.interactive. Not the terminal, because several servers would steal each other’s input, and a CI’s /dev/null stdin is the same EOF. Turbo’s stream mode draws the same line: its pin is named nonpersistent_task_sees_eof_on_stdin.

A one-shot task ends when its shell exits, not when its pipes close. A process the task left running (server & echo up) keeps stdout and stderr open, so vx reads on for 250 ms after the shell exits and then stops. That bound is what keeps a leftover server from hanging the run. What the leftover writes after it is not captured — not in the frame, not in the cache entry — and the frame says so with one line on stderr: [vx] output after the task's shell exited was cut: a process it left running still held its stdout/stderr 250 ms later. Until 2026-09-24 the cut was silent (nx#35302 reproduced on vx; tests/runner.test.ts › “output a backgrounded child writes after the drain bound is cut, and the frame says so”). A task the run stopped (Ctrl-C, a vx watch restart) gets no such line: vx killed it, and what is left is vx’s to sweep. A cached task cut this way is not saved: the leftover may still write its outputs, and a hit would replay a tree the run never leaves (tsc & vite build saved vite’s output without tsc’s; tests/left-running-save.test.ts). So is one whose leftover sent its stdio elsewhere (writer >/dev/null 2>&1 &): it holds no pipe, so nothing is cut, but it is still in the task’s process group when the shell exits, and the same warning names it. A leftover of a failed attempt withholds its retry’s save too: it may write into what the retry produces. Nor is what is built on it saved: a dependent read the half-written tree, and its key folds the leftover task’s, so the next run rebuilt that task and hit the dependent’s partial bytes. Under a Linux sandbox the leftover dies with the task’s pid namespace instead, setsid or not. Unsandboxed, or under macOS’s sandbox, a leftover that also left the group (setsid, a daemon that double-forks) and holds no pipe is not seen: the task is saved with its outputs as they stood at exit, and a hit replays that. vx does not chase it: macOS keeps nothing that ties such a process to the task that started it, Linux’s ways (a child subreaper, a cgroup) work only there, and either would also take a shared daemon a task starts (sccache, Gradle’s, Bazel’s server) for a leftover and stop that task’s saves. Turbo waits for the pipe instead, which a leftover server holds forever. A task whose output matters waits for what it starts (wait).

Every surface uses one outcome vocabulary: task axis success / failed / skipped / aborted (+ running live), cache axis miss / up-to-date (fresh) / local / remote. The --verbosity 1 table spells the combinations out as success / restored-local / restored-remote / up-to-date.

The colors / framing modules:

• ui/theme.ts — ANSI truecolor (ansi-16m), gated by NO_COLOR / FORCE_COLOR / isTTY. Programmatic-logger callers always see plain text.
• report/framed-output.ts — the ┌─ frames, the task grid (formatTaskRow), one-liners, persistent markers.
• report/status-line.ts — the serialized writer + the worker region.
• report/logger.ts — composes them; resolves the output view and applies the per-flow visibility policy.
• report/summary.ts — the closing footer (underlined title line, projects/tasks/cache meters, info and time rows).

Concurrency

• Default — the cores this process may use: navigator.hardwareConcurrency capped by the cgroup CPU quota a container runs under (cpu.max on v2, cpu.cfs_quota_us on v1, an ancestor’s quota binding too; a 1.5-core quota is two workers, never below one — util/cgroup.ts), or .vx/workspace.ts’s run.concurrency field when set. Inside a container the raw count is the HOST’s, and eight workers on a two-core quota is oversubscription by four; --concurrency <n>% is a percentage of the same capped count.
• Override — --concurrency N (CLI). CLI wins over workspace config.
• concurrency: 1 serializes everything while still respecting topo order: execs and restores share the one slot.
• The scheduler never exceeds the cap; tasks queue. Above 1, restore-tier tasks run on their own lane, 2 × concurrency wide, and never take an exec slot.
• Failure of a task doesn’t pause the scheduler — independent siblings continue running and starting (unless --continue=never).

Executor pools

concurrency counts LOCAL worker slots. An executor that runs its tasks somewhere else declares its own capacity, and the tasks placed on it are admitted against that number instead — so a 64-wide worker pool is not throttled by a 10-core laptop, and a --concurrency 1 run still keeps the pool full. A capacity that is not a positive integer is refused: plugin '<p>' returned executor '<e>' with capacity <v>: it must be a positive integer. So is a flag (remote, sandbox, servers, runtime, exclusive, timeouts) that is not true or false, which placement read as false (remote: 'yes' was offered sandboxed tasks), and an optional hook (accepts, demand, …) that is not a function.

• Placement is decided once per task, before scheduling. run() asks the declared executors, in order, which one takes each task (see schema.md § remote for what pins a task here); the scheduler then knows which pool every task will occupy.
• A pooled task holds nothing on this machine. The admit stage is asked only for tasks about to run here, with the tasks running here; a task on an executor pool is never asked and never counted, so it can never park behind a local reservation.
• Restore-tier tasks always use the restore lane, even when placed on an executor with a capacity: a restore is local disk work.
• No pooled executor declared = the legacy path. With every task on the local pool the admission gate is byte-identical to before pools existed.

Cache control flags

--no-cache turns all four cache axes off: every task runs, nothing is read or written, and cleanOutputs is skipped — the user is debugging and managing the tree themselves; silently wiping dist/ mid-debug would be hostile.

--force turns only the READ axes off: every task re-executes, but fresh artifacts are still written (outputs ARE cleaned so the saved snapshot is clean). --cache-policy=<spec> gives per-layer control. Full semantics: cli.md § Cache control and caching.md § Cache policy.

Planning path (--dry)

--dry short-circuits execution. The planner (orchestrator/plan.ts) runs the same setup steps — workspace discovery, config load, package graph, task graph — and the same per-task hash derivation, then probes the cache. The probe is read-only; against a remote layer it is an existence check only (no artifact download, no ingest). The one deliberate side effect: inputs.runtime / workspaceRuntime probe commands DO run, because predicting a key requires resolving them — keep runtime inputs side-effect-free.

• --dry[=pretty|json] → pretty (default) or JSON list of predicted outcomes per task: hit-local, hit-remote, miss, no-cache, group. Formatter: cli/plan-format.ts:formatPlanText / formatPlanJson. When the workspace declares more than one executor, each line also names the executor the task would be PLACED on — the only surface that makes exec.remote checkable before the run. Resolving the executors is a plugin-factory call, the same class the cache capability already makes at plan time; a plugin that throws costs the label, never the plan. Combining it with --profile is a parse error (the profile needs a real run).

Run artifacts

• --profile[=<path>] — Chrome-trace JSON of every task’s wallclock span. Default path: profile.json (cwd-relative). One tid per project so concurrent tasks render on distinct lanes. Open with chrome://tracing or https://ui.perfetto.dev.

The writer is orchestrator/run-artifacts.ts:writeRunProfile. A write error is surfaced via log.status and does not change the run’s exit code — the run already happened. A CI job summary is @vzn/vx-ci’s.

Why each rule

• Cache failures are never cached. Caching a failure prevents retry flows. The next run gets the same failure even after the user fixes the cause.
• Group tasks are silent in the summary. Including them in the count is confusing — “3 of 4 cached” when one was a group that ran nothing isn’t informative.
• Project scope defaults to cwd. Most invocations are “build/ test the thing I’m working on”. --all / --filter exist for the workspace-wide case.
• The scheduler doesn’t bail on first failure by default. A flaky test failing shouldn’t stop an unrelated build. Independent siblings continue; only dependents are skipped. --continue=never opts into fail-fast.
• Misses own the worker pool. A miss is the critical path, so the exec lane is concurrency wide and only executions take it. Restores run on their own lane, so warm work never queues behind ordering it doesn’t need and never takes a slot from real work.
• Forwarded args don’t reach upstream tasks. Otherwise vx run build -- --watch would set --watch on every upstream’s build, and upstream cache keys would partition by CLI args that don’t change their behavior.
• recordRunBundle skips group tasks. They aren’t real runs; analytics queries that sum duration would double-count without it.
• A requested persistent task keeps the run alive. The dev server IS the point of the run; tearing it down the instant it became ready would make vx run dev useless.