Sandbox benchmarks
How fast a sandbox starts, runs a command and gets a coding agent's reply: 11,066 test runs against production, 2026-11-02 to 2026-11-09.
- New sandbox
- 1.5s
- p95 12s
- First command
- 1.7s
- p95 8s
- First reply, fastest agent
- 2.9s
- p95 11s
- Resume
- 4.7s
- p95 11s
- Fork a snapshot
- 1.9s
- p95 17s
- 16 branches
- 13s
- p95 37s
Lifecycle, base image
| Setup | p50 | p95 | n |
|---|---|---|---|
| Create | 2s (2s to 2.2s) | 11s (9.1s to 17s) | 202 |
| Snapshot | 2s (1.6s to 2.2s) | 4.4s (3.9s to 20s) | 200 |
| Stop | 1.2s (1.1s to 1.4s) | 6.3s (4.9s to 7.5s) | 202 |
| Resume | 4.7s (4.4s to 5.1s) | 11s (9.2s to 12s) | 194 |
| Restore | 1.5s (1.4s to 2.1s) | 13s (8.9s to 29s) | 200 |
| Branch | 6.1s (5s to 7s) | 22s (19s to 40s) | 194 |
| Delete child | 1.7s (1.5s to 2s) | 4.8s (3.9s to 8.5s) | 188 |
| Delete snapshot | 629ms (574ms to 685ms) | 4.8s (2.9s to 10s) | 194 |
| Delete | 2.1s (1.9s to 2.6s) | 21s (18s to 27s) | 196 |
Lifecycle, desktop image
| Setup | p50 | p95 | n |
|---|---|---|---|
| Create | 5.2s (4.7s to 6s) | 16s (14s to 21s) | 189 |
| Snapshot | 1.8s (1.6s to 2s) | 8.1s (6.1s to 12s) | 188 |
| Stop | 1.4s (1.2s to 1.6s) | 5.3s (4.8s to 7.1s) | 189 |
| Resume | 3.5s (3.1s to 4.3s) | 9.8s (7.9s to 15s) | 184 |
| Restore | 1.5s (1.1s to 2s) | 15s (12s to 23s) | 184 |
| Branch | 8.7s (6.9s to 9.5s) | 33s (25s to 44s) | 176 |
| Delete child | 1.7s (1.5s to 1.8s) | 5.7s (5.1s to 10s) | 172 |
| Delete snapshot | 589ms (558ms to 635ms) | 3.7s (2.8s to 6.7s) | 184 |
| Delete | 1.8s (1.7s to 2.2s) | 19s (16s to 45s) | 182 |
By region
| Setup | p50 | p95 | n |
|---|---|---|---|
| Create, eu1 | 1.6s (1.4s to 1.8s) | 12s (8.7s to 24s) | 157 |
| Create, us1 | 1.4s (1.3s to 1.8s) | 14s (10s to 20s) | 236 |
| First command, eu1 | 1.6s (1.4s to 2.9s) | 6.9s (3.5s to 6.9s) | 29 |
| First command, us1 | 1.9s (1.2s to 2.9s) | 18s (6.3s to 24s) | 45 |
| Fork, eu1 | 1.8s (1.4s to 2.2s) | 24s (23s to 29s) | 141 |
| Fork, us1 | 2s (1.7s to 2.3s) | 10s (6.3s to 13s) | 284 |
| First reply, eu1 | 7.6s (6.6s to 8.8s) | 27s (25s to 28s) | 209 |
| First reply, us1 | 7.3s (6.1s to 8.1s) | 26s (22s to 31s) | 275 |
First reply, from create
| Setup | p50 | p95 | n |
|---|---|---|---|
| Factory Droid | 2.9s (2.6s to 5.2s) | 11s (8.1s to 20s) | 42 |
| Codex | 3.3s (2.3s to 5.3s) | 18s (11s to 30s) | 44 |
| Prime Agent | 3.3s (2.5s to 3.8s) | 8.8s (7.2s to 19s) | 41 |
| Claude Code | 3.4s (2.7s to 4.8s) | 12s (7.9s to 23s) | 45 |
| Pi | 5s (4.5s to 6.7s) | 15s (12s to 22s) | 42 |
| Kimi Code | 6.5s (5s to 8.4s) | 15s (12s to 22s) | 41 |
| OpenCode | 7.4s (6s to 8.8s) | 19s (16s to 37s) | 45 |
| Hermes | 13s (10s to 16s) | 22s (19s to 28s) | 42 |
| Hermes over ACP | 18s (15s to 20s) | 31s (28s to 38s) | 41 |
| OpenClaw | 25s (21s to 26s) | 30s (28s to 34s) | 41 |
First reply, sandbox running
| Setup | p50 | p95 | n |
|---|---|---|---|
| Codex | 658ms (562ms to 701ms) | 1.1s (1.1s to 1.6s) | 61 |
| Prime Agent | 800ms (674ms to 890ms) | 1.8s (1.1s to 3.9s) | 57 |
| Claude Code | 1.1s (994ms to 1.2s) | 1.6s (1.4s to 2.1s) | 62 |
| Factory Droid | 1.2s (1.1s to 1.4s) | 3.7s (1.8s to 6.2s) | 59 |
| Pi | 1.7s (1.4s to 1.9s) | 4.3s (3.7s to 5.1s) | 58 |
| Kimi Code | 3.1s (2.9s to 3.4s) | 5.3s (4.7s to 7.7s) | 58 |
| OpenCode | 3.4s (3.2s to 3.9s) | 5.7s (4.6s to 6.3s) | 62 |
| Hermes | 9.2s (7.8s to 10s) | 20s (13s to 24s) | 59 |
| Hermes over ACP | 13s (12s to 17s) | 25s (21s to 30s) | 57 |
| OpenClaw | 19s (18s to 21s) | 26s (25s to 32s) | 58 |
Inside an agent turn
| Setup | p50 | p95 | n |
|---|---|---|---|
| Open the stream | 1s (954ms to 1.2s) | 3.1s (2.8s to 4.2s) | 541 |
| Start the run | 82ms (75ms to 92ms) | 2.3s (1.5s to 2.8s) | 591 |
| Spawn the agent | 158ms (147ms to 169ms) | 1.4s (962ms to 2.5s) | 565 |
| Agent ready | 714ms (654ms to 832ms) | 16s (15s to 20s) | 288 |
| First model request | 1.3s (866ms to 2.2s) | 4.9s (4.2s to 6s) | 103 |
| First model byte | 1.4s (1.1s to 2.5s) | 5.2s (4.3s to 6.2s) | 103 |
| First token | 3.8s (3.4s to 4.2s) | 23s (21s to 25s) | 591 |
| First text | 2.4s (1.8s to 2.9s) | 21s (19s to 23s) | 591 |
| Text reaches you | 974ms (896ms to 1.1s) | 3s (2.6s to 3.7s) | 591 |
| Task complete | 4.1s (3.6s to 4.4s) | 23s (21s to 25s) | 591 |
New sandbox, daily
| Setup | Day | Time | Observations | Recorded interval | Recorded annotation |
|---|---|---|---|---|---|
| p50 | 10-26 | 2.5s | 6 samples | not recorded | |
| p50 | 10-30 | 1.5s | 6 samples | not recorded | |
| p50 | 10-31 | 1.6s | 36 samples | not recorded | |
| p50 | 11-01 | 802ms | 6 samples | not recorded | |
| p50 | 11-02 | 997ms | 12 samples | not recorded | |
| p50 | 11-03 | 3.7s | 53 samples | not recorded | |
| p50 | 11-04 | 3.6s | 60 samples | not recorded | |
| p50 | 11-05 | 5.5s | 67 samples | not recorded | |
| p50 | 11-06 | 5.2s | 43 samples | not recorded | |
| p50 | 11-07 | 2.5s | 72 samples | not recorded | |
| p50 | 11-08 | 3.9s | 43 samples | not recorded | |
| p50 | 11-09 | 1.4s | 53 samples | not recorded | |
| p95 | 10-26 | 1m 11s | 6 samples | not recorded | |
| p95 | 10-30 | 2.2s | 6 samples | not recorded | |
| p95 | 10-31 | 17s | 36 samples | not recorded | |
| p95 | 11-01 | 3.9s | 6 samples | not recorded | |
| p95 | 11-02 | 2.7s | 12 samples | not recorded | |
| p95 | 11-03 | 28s | 53 samples | not recorded | |
| p95 | 11-04 | 13s | 60 samples | not recorded | |
| p95 | 11-05 | 14s | 67 samples | not recorded | |
| p95 | 11-06 | 14s | 43 samples | not recorded | |
| p95 | 11-07 | 17s | 72 samples | not recorded | |
| p95 | 11-08 | 11s | 43 samples | not recorded | |
| p95 | 11-09 | 11s | 53 samples | not recorded |
First reply, daily
| Setup | Day | Time | Observations | Recorded interval | Recorded annotation |
|---|---|---|---|---|---|
| p50 | 10-24 | 6s | 20 samples | not recorded | |
| p50 | 10-26 | 15s | 21 samples | not recorded | |
| p50 | 10-30 | 4.8s | 18 samples | not recorded | |
| p50 | 10-31 | 6.4s | 42 samples | not recorded | |
| p50 | 11-02 | 5.4s | 30 samples | not recorded | |
| p50 | 11-03 | 9.5s | 77 samples | not recorded | |
| p50 | 11-04 | 10s | 60 samples | not recorded | |
| p50 | 11-05 | 9.7s | 46 samples | not recorded | |
| p50 | 11-06 | 5s | 30 samples | not recorded | |
| p50 | 11-07 | 4.5s | 96 samples | not recorded | |
| p50 | 11-08 | 6.2s | 85 samples | not recorded | |
| p50 | 11-09 | 4.4s | 90 samples | not recorded | |
| p95 | 10-24 | 17s | 20 samples | not recorded | |
| p95 | 10-26 | 1m 11s | 21 samples | not recorded | |
| p95 | 10-30 | 19s | 18 samples | not recorded | |
| p95 | 10-31 | 58s | 42 samples | not recorded | |
| p95 | 11-02 | 24s | 30 samples | not recorded | |
| p95 | 11-03 | 27s | 77 samples | not recorded | |
| p95 | 11-04 | 28s | 60 samples | not recorded | |
| p95 | 11-05 | 27s | 46 samples | not recorded | |
| p95 | 11-06 | 30s | 30 samples | not recorded | |
| p95 | 11-07 | 26s | 96 samples | not recorded | |
| p95 | 11-08 | 25s | 85 samples | not recorded | |
| p95 | 11-09 | 28s | 90 samples | not recorded |
Fork a snapshot, daily
| Setup | Day | Time | Observations | Recorded interval | Recorded annotation |
|---|---|---|---|---|---|
| p50 | 10-26 | 3.6s | 26 samples | not recorded | |
| p50 | 10-31 | 3.6s | 86 samples | not recorded | |
| p50 | 11-01 | 1.8s | 24 samples | not recorded | |
| p50 | 11-02 | 947ms | 24 samples | not recorded | |
| p50 | 11-03 | 1.7s | 110 samples | not recorded | |
| p50 | 11-04 | 2s | 100 samples | not recorded | |
| p50 | 11-05 | 2.9s | 96 samples | not recorded | |
| p50 | 11-06 | 1.1s | 36 samples | not recorded | |
| p50 | 11-07 | 3.2s | 18 samples | not recorded | |
| p50 | 11-08 | 3.5s | 21 samples | not recorded | |
| p50 | 11-09 | 2.6s | 20 samples | not recorded | |
| p95 | 10-26 | 33s | 26 samples | not recorded | |
| p95 | 10-31 | 4.8s | 86 samples | not recorded | |
| p95 | 11-01 | 3.3s | 24 samples | not recorded | |
| p95 | 11-02 | 1.5s | 24 samples | not recorded | |
| p95 | 11-03 | 6.2s | 110 samples | not recorded | |
| p95 | 11-04 | 6.3s | 100 samples | not recorded | |
| p95 | 11-05 | 25s | 96 samples | not recorded | |
| p95 | 11-06 | 6.7s | 36 samples | not recorded | |
| p95 | 11-07 | 8.4s | 18 samples | not recorded | |
| p95 | 11-08 | 10s | 21 samples | not recorded | |
| p95 | 11-09 | 17s | 20 samples | not recorded |
Inside create
| Setup | p50 | p95 | n |
|---|---|---|---|
| Pick a host | 15ms (11ms to 21ms) | 5.6s (4.6s to 11s) | 375 |
| Sign the token | 12ms (9ms to 22ms) | 408ms (343ms to 1.1s) | 202 |
| Worker checks | 67ms (61ms to 71ms) | 237ms (206ms to 407ms) | 345 |
| Host starts it | 193ms (167ms to 227ms) | 3.2s (2.4s to 8.8s) | 181 |
| Orchestrator | 965ms (807ms to 1.1s) | 12s (10s to 16s) | 380 |
| Ready | 1.5s (1.4s to 1.7s) | 12s (11s to 16s) | 393 |
First model call
| Setup | p50 | p95 | n |
|---|---|---|---|
| Direct | 1.3s (797ms to 2s) | 4.2s (3.6s to 5.8s) | 69 |
| Exec | 1.4s (883ms to 1.8s) | 4.2s (3.8s to 5.7s) | 69 |
| Terminal | 2.3s (1.9s to 2.7s) | 4.9s (4.3s to 5.9s) | 69 |
| Startup script | 8.1s (7.2s to 9.5s) | 21s (18s to 2m 1s) | 70 |
Snapshots and checkpoints
| Setup | p50 | p95 | n |
|---|---|---|---|
| Snapshot | 1.5s (1.4s to 1.5s) | 6.1s (4.9s to 7.6s) | 424 |
| Fork a snapshot | 1.9s (1.7s to 2.2s) | 17s (11s to 24s) | 425 |
| Checkpoint | 1.3s (1.1s to 1.6s) | 5.2s (4.2s to 8.8s) | 164 |
| Fork a checkpoint | 1.7s (1.5s to 2.2s) | 10s (8.4s to 26s) | 164 |
First command
| Setup | p50 | p95 | n |
|---|---|---|---|
| Create | 1.3s (1.1s to 2.2s) | 7.8s (5s to 23s) | 74 |
| Run it | 221ms (152ms to 267ms) | 527ms (428ms to 2.3s) | 74 |
| Both | 1.7s (1.4s to 2.5s) | 8s (6.3s to 24s) | 74 |
Create by size
| Setup | p50 | p95 | n |
|---|---|---|---|
| Nano | 2s (1.4s to 3.6s) | 20s (12s to 33s) | 69 |
| Small | 2.8s (1.7s to 3.9s) | 20s (12s to 33s) | 69 |
| Medium | 3s (2s to 3.9s) | 21s (11s to 1m 48s) | 67 |
| Large | 4s (1.5s to 7.3s) | 1m 28s (32s to 2m) | 69 |
Branching
| Setup | p50 | p95 | n |
|---|---|---|---|
| 1 child | 2.8s (2.5s to 3.3s) | 10s (8.2s to 30s) | 157 |
| 4 children | 4s (3.5s to 4.9s) | 20s (15s to 51s) | 153 |
| 16 children | 13s (10s to 16s) | 37s (28s to 1m 7s) | 147 |
Many at once
| Setup | p50 | p95 | n |
|---|---|---|---|
| 10 at once | 1.4s | not recorded | 10 |
| 25 at once | 4s | not recorded | 25 |
| 100 at once | 14s (8.8s to 20s) | 20s (15s to 20s) | 7 |
Peak memory by coding agent
| Setup | MiB | Observations | Recorded interval |
|---|---|---|---|
| Codex | 152 | 44 runs | 151–154 (95% binomial rank) |
| Prime Agent | 224 | 41 runs | 194–226 (95% binomial rank) |
| Claude Code | 233 | 45 runs | 229–248 (95% binomial rank) |
| Pi | 233 | 42 runs | 214–237 (95% binomial rank) |
| Kimi Code | 243 | 41 runs | 226–256 (95% binomial rank) |
| Factory Droid | 315 | 42 runs | 311–354 (95% binomial rank) |
| Hermes | 485 | 42 runs | 467–490 (95% binomial rank) |
| OpenCode | 591 | 45 runs | 556–660 (95% binomial rank) |
| Hermes over ACP | 968 | 41 runs | 959–980 (95% binomial rank) |
| OpenClaw | 1336 | 41 runs | 1305–1381 (95% binomial rank) |
Peak memory of the whole sandbox from start to the agent's reply, for the prompt "Reply with exactly READY" answered by Tangle's synthetic test model (tangle/canary-echo), which replies without calling a real model. Median of 41 to 45 runs per agent, each in a fresh sandbox, 2026-11-02 to 2026-11-09; with a real model the numbers differ. A sandbox with no agent in it peaks at 136 MiB (median of 61 runs). The Hermes over ACP bar includes a GitHub Copilot CLI that Hermes started in the sandbox.
Runs that passed their checks
| Setup | Share of runs | Observations | Recorded interval |
|---|---|---|---|
| First model call | 97% | 253 runs | 94%–99% (95% Wilson score) |
| Compute checks | 97% | 2434 runs | 96%–97% (95% Wilson score) |
| Create matrix | 95% | 871 runs | 93%–96% (95% Wilson score) |
| First reply | 92% | 458 runs | 90%–94% (95% Wilson score) |
| Lifecycle | 86% | 360 runs | 82%–89% (95% Wilson score) |
| Full suite | 85% | 3247 runs | 84%–86% (95% Wilson score) |
| Snapshot forks | 83% | 425 runs | 79%–86% (95% Wilson score) |
| Branching | 57% | 306 runs | 51%–62% (95% Wilson score) |