Streaming Firecracker builds over WebSockets

The hardest part of building a deployment platform is not running the build. It is telling the user what is happening while it runs.

A build inside a Firecracker microVM is, by design, cut off from everything. That isolation is the entire security story — and it is also the reason a naive implementation gives you a spinner that sits there for ninety seconds and then either succeeds or does not.

The shape of the problem

There are three independent streams you need to reconcile:

1. Process output — stdout and stderr from the build command inside the guest.
2. Lifecycle events — VM booted, snapshot restored, dependencies restored from cache, artifact uploaded.
3. Health signals — is the guest still alive, is the socket still warm, did the agent die mid-step.

If you multiplex these into a single unordered channel, the log looks correct most of the time and lies to you exactly when you need it most: during a failure.

What Runway actually does

Every VM boots with a small agent. The agent owns the pty, tags each chunk with a monotonic sequence number and a stream identifier, and pushes it over a vsock connection to the host. The host relays into a WebSocket fan-out keyed by deployment id.

type BuildFrame =
  | { type: 'log'; seq: number; stream: 'stdout' | 'stderr'; data: string }
  | { type: 'lifecycle'; seq: number; step: string; status: 'start' | 'ok' | 'fail' }
  | { type: 'heartbeat'; seq: number; at: number }

function applyFrame(state: BuildState, frame: BuildFrame): BuildState {
  if (frame.seq <= state.lastSeq) return state // idempotent replay
  // ...
}

The sequence number is the whole trick. Frames are idempotent, so a client that reconnects can say "I have everything through 4,182" and get a replay of the gap instead of a duplicated log or a hole in the middle.

Cached installs change the numbers, not the design

Reusing a dependency cache across builds is the single biggest win on wall-clock time — but it introduces a step that can partially succeed. A cache restore that half-populates node_modules is worse than no cache at all.

So the restore is atomic: hydrate into a scratch directory, verify the lockfile hash, then swap. If the hash does not match, the frame goes out as { step: 'cache', status: 'fail' } and the build falls through to a cold install. The user sees exactly that, in order, as it happens.

What I would tell anyone building this

Design the transport before the feature. Ordering, replay and backpressure are not things you bolt on once the logs look pretty — they are the reason the logs can be trusted at all.