A DTE relay that stores no message

The relay behind vx DTE keeps no message. It stores only who takes part, one row each, so a restart keeps the run. A quiet run of four sockets sends 60 frames through it in ten heartbeats.

flowchart LR
  O[orchestrator] <-->|"socket: numbered, acked"| R((relay))
  R <-->|"socket: numbered, acked"| W1["worker small#1"]
  R <-->|"socket: numbered, acked"| W2["worker small#2"]

One route, one socket each

Each side opens one websocket and joins on it, with its CI OIDC token. There is no other route. A refused join gets its socket closed, with the reason in the last frame and in the close.

The two ends make sure a message lands

sequenceDiagram
  participant O as orchestrator
  participant R as relay
  participant W as worker
  O->>R: message 7, an order
  R--xW: socket gone, dropped
  Note over O: no ack after a heartbeat
  O->>R: message 7, again
  R->>W: message 7
  W->>R: ack 7
  R->>O: ack 7

The relay pushes a message down the receiver’s socket, or drops it. Each side numbers what it sends and sends it again until the other acks it. The receiver takes each number once, in order.

A closed socket is a lost worker

xychart-beta
  title "Frames, quiet run, 4 sockets, 10 heartbeats"
  x-axis ["relay pings every socket", "peers probe only the quiet"]
  y-axis "frames" 0 --> 100
  bar [92, 60]

1 orchestrator, 3 idle workers · 88-96 frames before, 24 up and 36 down after (#4309)

When a socket closes, the relay drops its worker at once. A worker that goes quiet on an open socket is asked after 1.5 heartbeats. No answer, and it is lost in under 10 s. When the orchestrator’s socket closes, every worker’s closes too.

Free on Cloudflare, or one container

On Cloudflare the run is one Durable Object, there only to hold its sockets. What a woken object needs rides on each socket, and who takes part is one row each, so a deploy keeps the run. A join costs one request, and every 20 websocket messages count as one more.

docker build -t vx-dte-server packages/vx-dte-server
docker run -p 8080:8080 vx-dte-server --allow https://gitlab.com:group/app:.gitlab-ci.yml --audience https://relay.example.com

The container is the same relay on Bun. With --state on a volume, a restart keeps its runs too. CI builds the image and carries a run through it and across docker restart (test.live).

Relay README · more at vznjs.github.io/vx.