We ran the same broken app on three cloud sandboxes

October 11, 2026 · Andrew Arrow · Updated October 12

Move "Join the waitlist" to the top of our little editor, hit Save, and everything looks fine. The button is still there. The editor says "Saved." Reload the page and the button disappears. Three blocks went in; two came back.

We planted this bug, then wrote a correct patch and a convincing fake. To check them independently, we wanted three copies of the prepared app, with dependencies already installed. That raises a question a Git checkout doesn't answer: are we copying the files, or the running application? And if the connection drops halfway through, how do we find the work again?

We tried Mainbrella, E2B, and Daytona. We build Mainbrella, so one of these is ours. All three caught the fake repair. The differences showed up in the state their copies preserved and the evidence we needed to recover and clean up afterward. No coding agent attempted these repairs.

After Save

After reload

A fix has to survive reload

You can try the broken editor without an account. Its serializer assigns positions from the array index, then filters them:

blocks
  .map((block, position) => ({ ...block, position }))
  .filter(block => block.position);

Zero is falsy. Goodbye, first block. The storage write succeeds, so the UI announces "Saved" and keeps showing the untouched in-memory draft. JavaScript has done exactly what we asked, which is rude but fair.

The correct patch removes the filter. The misleading patch calls render() after saving. It redraws the same healthy-looking draft without reading the stored document back. A video that ends at Save can look great with either patch. Reloading makes the fix earn its keep.

Our browser test checks ordered block IDs and text before Save, after Save, and after reload. Searching for "Join the waitlist" somewhere on the page would accept help text while the document had already lost the button.

The verifier reports passed, reproduced, or unverified. Reproducing this bug requires the intact document before reload and exactly that document minus its first block afterward, with the remaining order and text unchanged. An HTTP error, timeout, skipped test, or unrelated assertion is unverified. If the prepared baseline doesn't reproduce the loss, we stop before trying patches.

The passing baseline check tests Reset: it must restore the demo without deleting an unrelated browser-storage key. The other checks cover reordered blocks, the original order, edited text, both reload paths, and six shuffled documents generated from a recorded seed. Each holdout document goes through two reloads. These are controls with known answers, not an agent benchmark.

The verifier runs outside the writable app copies. A proposed repair can't fix its score by editing the test. This is the first workload for Repo Rodeo, our harness for giving repair attempts independent environments. Before handing it to agents, we wanted to know whether it could reject a patch we already knew was wrong.

What came out of the copies

The editor runs used the same pinned application archive, patch hashes, browser checks, and seed. Each provider prepared one source and made three branches. The checkout and installed dependencies survived in all nine branches; none needed another npm ci. After the controls finished, each branch kept its own marker and expected source hashes, while the source had no branch marker.

Getting to that point exposed mistakes in our setup. Mainbrella forwarded the guest Host header as container, which Vite rejected with 403 until the launcher allowed that exact host. E2B's base template selected an older Node through its login shell; our launcher needed an explicit PATH to the pinned binary. Then our browser helper mishandled authenticated redirects. A ready server and a usable browser preview were separate things to check.

We kept those failures in the setup notes. Six red tests on a forbidden page tell us nothing about the button. Once the preview worked, the more interesting question was whether a copied server was already listening.

A PID alone can't establish that: a fresh process can reuse a number. Our launcher generates a random nonce once at startup and serves it from an uncached endpoint. It never loads the nonce from disk. A branch answering with the source's nonce and PID before we launch anything has inherited that process state.

Here are the observations before any branch patch or server launch. Each file check covers 417 tracked files, the installed dependency lockfile, a prepared marker, its permissions, and a symlink target. Nonces are abbreviated here; the full values and receipt hashes are available together.

Mainbrella kept the source server running through capture, but each restored branch needed a fresh server. E2B brought the source's listener, PID, and nonce into all three forks. Its fork documentation also says capture pauses the source and drops active connections. A process remembering its work doesn't mean the terminal watching it stays connected.

Daytona's container snapshot required a stopped source. After capture, the source and all three branches needed fresh servers. Its successful rerun preserved files, dependencies, and branch isolation and produced the same repair outcomes as the other two. This tests Daytona's container snapshots; its VM memory snapshots are a separate mode we haven't exercised.

For the editor, restarting was fine. The document lives in the external browser's localStorage, outside the server container. Each browser test reconstructs it. We even stopped E2B's inherited servers before checking the patches. That experiment demonstrated process preservation, but the repair never used it.

A worker you can't restart into correctness

To make the process state matter, we added a small worker with five accepted jobs held only in its JavaScript heap. Two jobs represent free items: their unit price is zero, but they still need receipts. The worker pauses before processing the batch. That paused instance, including its queue, is what we copy.

The faulty processor filters jobs by unitCents before calculating totals. It drops both free items and returns three receipts with a completed flag. The correct patch removes the filter and returns all five receipts. A misleading patch takes the shortcut of returning an empty output list and claiming completion.

There's a deliberate constraint here. The worker imports processor.mjs when we release the checkpoint, so we can replace that file without restarting the process holding the queue. We aren't hot-patching a function halfway through execution. Without that explicit reload point, copying a live process wouldn't by itself make changed source code run.

The outside verifier submits the jobs once and keeps its own ledger of IDs and expected amounts. It never sends them again to a branch. Before resuming, we check the preparation ID, queue fingerprint, pending count, PID, and nonce. Afterward, a completed badge isn't enough: every expected receipt must appear exactly once with the right amount, and the same process must have handled the same prepared queue.

That last control matters as much as the successful patch. Restarting the source and applying the correct processor leaves it with no jobs to process. A worker saying "completed" over an empty queue can look healthy while all five accepted jobs have vanished. The verifier reports reproduction-lost, which is neither a successful repair nor evidence against the patch.

We also give each branch a different in-memory marker and check the other branches and source. Sharing an unchanged nonce is evidence of copied state, not permission to share subsequent writes. Where capture preserves the source process, it must still hold its original pending batch after the branch repairs finish.

This is a small, constructed workload, and the checkpoint makes patch application unusually convenient. It establishes a use for live cloning in this repair: three candidates can consume independent copies of the already-prepared queue. It doesn't establish that a real service needs memory snapshots. A durable queue or a replayable input log could rebuild this batch after restart. We deliberately left that recovery path out so we could observe what copying the running process contributes.

The two fixtures therefore ask different things of the cloud. The browser editor can recreate its document outside the sandbox, so filesystem restores suffice. The worker has already accepted state that the branch runner isn't allowed to reconstruct. For that fixture, an empty restarted process has lost the thing we meant to test.

Finding work after losing the connection

A preserved process still needs a way for a new client to find it. We started an eight-second job, detached after its first output, then connected with a new client. All three providers recovered the same job identity, exit status zero, and remaining output. The mechanisms differed: Mainbrella replayed an SSE event cursor, E2B recovered command output using a byte offset, and Daytona recovered retained session output. Daytona's check detached a polling client; it didn't test a broken live stream.

Losing a capture reply leaves a different problem. The cloud may have saved a workspace while the client never learned its ID. Blindly retrying can create another copy and leave the first unclaimed.

On Mainbrella, we deliberately discarded the body of a successful capture response. Before sending it, the runner saved the exact request body and idempotency key. Retrying with both recovered the same workspace; a further replay returned it again. Three requests produced one workspace and one capture reservation.

The lost reply came after successful HTTP headers. This establishes recovery of a completed save, not what happens if capture is interrupted earlier. We didn't run keyed replay on E2B or Daytona: those adapters don't implement it. That is a gap in this comparison, not a demonstrated failure of their services. Recovering a command and deduplicating a mutation need separate rows in a capability table.

Checking what deletion actually did

Our first Daytona snapshot exceeded the harness's three-minute deadline while the provider kept working. A later delete returned 403: our key could manage sandboxes but couldn't delete persistent snapshots. We had permission to make the mess and insufficient permission to clean it up.

The adapter now checks both sets of permissions before provisioning, saves the requested snapshot name before submission, and polls separately with a ten-minute deadline. The rerun completed its snapshot in about 34 seconds and deleted all four test sandboxes and the snapshot, preserving thirteen pre-existing snapshots.

Yet its overall result said failed. The API confirmed deletion, but several old previews returned 401 instead of the 404 or 410 the adapter required. Meanwhile, E2B passed a special rule allowing repeated 502s after API-confirmed absence. It also got more probes. Those labels reflected different acceptance rules in our code. Using them to rank providers was a mistake.

None of these checks establishes when physical storage is reclaimed. Our earlier local Mainbrella experiment left two unattached Docker snapshot images after API deletion; we haven't established their collection time. Production also reported roughly 259 MB captured while reserving the requested 8 GB disk. Deleting the workspace released the live reservation but left monthly capture counters intact. The accounting and run notes keep those quantities separate. Neither is a measurement of unique physical bytes or the bill.

Which capability the workflow needs

For our unattended workflow, I prefer having a persisted request key that can recover a completed capture after the reply disappears. That's the Mainbrella behavior we demonstrated. If the reproduction depends on a running worker's memory, E2B's live fork supplies something our saved-workspace restore doesn't. Daytona's container snapshots handled the editor's filesystem requirements. The choice depends on which state the repair needs and which interruptions the client must recover from.

The runs used different allocations and guest runtimes, with too few samples for a reliability, speed, or price ranking. Mainbrella requested half a vCPU and 4 GiB; E2B reported two vCPUs and 512 MiB; Daytona reported one vCPU and 1 GiB. We didn't measure resource enforcement or actual charges. The results qualify these fixtures and adapters, not arbitrary repositories or autonomous agents.

The public editor bundles include structured document observations and the extracted process values. Raw browser traces and API receipts stay private because they can contain bearer preview credentials. Readers can inspect the reported findings, but the public bundle cannot independently replay the full original verification decision.

You can run the standalone editor controls from the repository with its pinned Node 22.22.2:

npm ci
npx playwright install chromium
npm run experiment -- --seed 12345

The baseline should lose a block, the misleading repair should still lose it, and the correct repair should preserve the document through reload. Those are the answers the harness has to get right before an agent's proposed fix deserves a look.