VM Compute Cost vs Token Cost in Cloud Agent Runs

A cloud agent run generates four separate bills, not one, and the team that optimizes the wrong one ends up fixing a cost line that was never the problem. One founder working on PandaStack spent two weeks tuning sandbox compute, then noticed his model bill ran roughly thirty times larger than the infrastructure he'd been optimizing. That gap between what's visible and what's dominant is the subject of this piece: it maps every cost component of a cloud agent run, shows why tokens structurally outweigh compute, and identifies the levers that actually move the total bill.

Four separate bills for a cloud agent run

Model tokens cover every input and output token across every turn of the agent loop, billed by whichever model provider is running the session. Sandbox compute covers the seconds a VM or container stays alive, billed by the compute provider on its own separate clock. Egress and storage cover bytes moved in and out, plus snapshots, artifacts, and logs, often split across three different providers. Failed runs are the first three lines paid again for work that produced nothing, because a run that dies at turn 10 still consumed tokens and VM seconds on the way there.

No single vendor holds the total. The model provider doesn't know which run a given request belonged to. The compute provider doesn't know which agent a given sandbox served. Without a layer that joins these fragments back together, nobody can answer the only question that matters: what did this run actually cost, and did it succeed? Cost per successful run is the unit worth tracking. It's the one number that goes down when a team gets more efficient and up when something's gone wrong. Every other number on an invoice moves simply because usage grew, which tells a team nothing about whether that growth was earned.

Why input tokens grow quadratically while compute grows linearly

The agent loop is stateless. A model doesn't remember turn three when a team sends turn four. It has to resend the entire prior transcript as input tokens, every single turn, for the full duration of the session. That single fact is the mechanical root of why token cost behaves so differently from compute cost.

The arithmetic works out like this: total input tokens equal turns times a constant prefix, plus average tokens per turn times turns times (turns minus one), divided by two. The first term is linear. The second term is quadratic, and it dominates once a session runs past a handful of turns. Cut the same task down to 8 turns and input volume drops by roughly half, despite the turn count falling by only a third. That's the quadratic term at work: small reductions in turn count produce disproportionate reductions in token spend.

Most engineers instinctively reason about cost in terms of output, the length of the answer. But output tokens make up roughly 2% of token volume in a typical agentic run. Compute, meanwhile, scales with wall-clock session time, and that grows close to linearly with task complexity. It doesn't compound the way token volume does. Letting an agent "think a bit longer" sounds like a harmless default. In practice, every additional turn resends everything that came before it, which makes turn count one of the most expensive levers a team can leave unmanaged.

The Token-to-Compute Ratio in Real Agent Runs

The quadratic argument isn't theoretical. The quadratic argument appears directly in production numbers. The PandaStack founder's bill, mentioned above, ran roughly thirty times higher on tokens than on compute, a ratio that makes clear that optimizing compute alone solves a problem that barely exists at that scale.

Anthropic's own worked example for Claude Managed Agents backs this up from the vendor side. A one-hour Opus 5 session consuming 50,000 input tokens and 15,000 output tokens costs $0.705 total: $0.25 for input, $0.375 for output, and $0.08 for session runtime. Tokens account for nearly 8 times the runtime charge, and that's before prompt caching enters the picture, which would widen the gap further still. Even in a product explicitly priced on two dimensions, tokens and session-hours, runtime comes out as a rounding line.

Scale changes the stakes but not the ratio. A single non-trivial task can push 400,000 to 2,000,000 cumulative input tokens through the API, and that's not an edge case: it's what agent mode looks like on any real codebase. When automation runs heavy, per-engineer monthly costs can dwarf compute line items. The Requesty 12-month production dataset shows P95 users spending materially more per month than the average user, and that long tail is where budget surprises actually live, not in compute overruns. A high failure rate compounds this problem rather than softening it: a run that fails at turn 10 still burned input tokens quadratically on the way there, so failures don't cut the token bill, they add to it.

Prompt caching and the effective cost of input tokens

Prompt caching is the single largest lever a team has for controlling token cost, because it turns the quadratic input problem into something closer to linear. It does this by making repeated prefix tokens, the system prompt, tool schemas, and repo map resent on every turn, dramatically cheaper than new tokens.

The Requesty 12-month dataset shows platform-wide cache hit rates climbing from 52% to 86% over the period ending April 2026. Teams that adopted caching early kept per-call costs compressed even as their context windows grew larger. At an 86% hit rate, the effective cost of input tokens runs substantially lower than list price, which amounts to a different cost regime entirely for large-context agentic work, not a marginal discount. On Sonnet 4.6, cached input tokens bill at $0.30 per million tokens versus $3.00 per million for uncached tokens, a 90% discount that compounds across every turn resending the same constant prefix.

This produces a counterintuitive result: a long interactive session that keeps its prefix cached ends up cheaper per token than a cold API call. So measured Claude Code costs in practice land well below what naive token-times-base-rate math would predict. In the Claude Managed Agents example, prompt caching covering 40,000 of the 50,000 input tokens in that one-hour session drops total cost from $0.705 to about $0.53. The runtime component barely moves at $0.08. Every dollar of savings comes from the token side.

Caching optimizes the prefix, the fixed, repeated part of every request. It does nothing for the part of the context that keeps growing: the conversation history that accumulates turn over turn and gets resent in full regardless of whether it's still relevant to the task at hand.

Why context discipline matters as much as caching

Unbounded context accumulation is a quadratic cost driver on its own, and caching alone doesn't touch it. Long-running agents build up stale tool results, prior reasoning steps, and decisions that have already been superseded, and all of it gets resent on every subsequent turn even after it stops informing the task.

Claude Code carries roughly 27,000 tokens of fixed overhead on every request before any task-specific history even starts accumulating, so per-turn cost runs higher than task complexity alone would suggest. Clearing or compacting context between discrete tasks reduces per-message token cost materially, because the quadratic term resets to zero at the start of each new task rather than continuing to climb across tasks that have nothing to do with each other.

Context discipline also protects against a different kind of cost: the failed run. The dominant failure mode in long-horizon agent work isn't a reasoning failure, it's knowledge loss through compaction, where an agent that has outlived its context window discards information that was load-bearing for the task. Those errors are expensive to diagnose and expensive to retry, since each retry resends the same bloated history that caused the problem. Context discipline functions as both an economic lever and a reliability lever at once, and the two reinforce each other: tighter context means fewer tokens resent and fewer failures caused by lost information.

Not every harness handles this the same way. Some compact stale history automatically. Others leave the decision entirely to the team running the agent, turning context management into a manual discipline.

Harness choice, model selection, and the total bill

Harness and model choice don't sit next to each other on an invoice as additive cost factors. They're multiplicative: they determine how many tokens get generated in the first place, what price gets paid per token, and whether caching and context compaction happen automatically or get left to the team to manage by hand.

Model selection is the single largest per-token lever available. Token prices across current coding models span more than two orders of magnitude, from Claude Fable 5 at $10/$50 per million input/output tokens down to DeepSeek V4 Flash at $0.14/$0.28 per million as of June 2026 (pricing changed August 16, 2026). The performance gap between mid-tier and top-tier models doesn't track that price gap linearly. An orchestrator-and-worker architecture, one high-capability model directing several cheaper worker models, costs materially less than running every agent at top-tier rates. Worker turns are high-volume and routine, and the quality difference on routine subtasks is small enough that the savings are close to free.

Harness behavior shapes the bill just as much as model choice does. Harnesses differ in how aggressively they populate the system prompt, how much tool schema they attach per turn, and whether they compact context automatically or let it grow without limit. That consolidation tracks capability, but the per-harness cost profiles behind it differed meaningfully. Harness lock-in functions as cost lock-in too: a team committed to one harness can't route tasks to whichever model is most cost-efficient for that task class as pricing shifts.

Parallelism multiplies the bill along the same lines. Running agents concurrently consumes token quota at the same multiple, so ten parallel agents burn through token budget ten times faster, and each one carries its own quadratic context growth independently. Subscription tiers cap total spend but rate-limit how much parallelism a team can run; API billing removes that cap and allows unlimited parallel agents at variable cost. Which of those two constraints a team is actually hitting changes which fix applies, because the fix looks completely different depending on the answer.

Teams running more than one harness or model in production need per-harness, per-model visibility to know which combination is actually saving money, rather than assuming consolidation around one tool is automatically the cheaper path.

What compute cost is worth managing

Compute cost deserves a place in the accounting, but it responds to levers that are different from the ones that move token spend, and it's smaller by design in most modern pricing structures. Claude Managed Agents meters runtime to the millisecond and only charges while a session is actively running. Idle time spent waiting on the next message or a tool confirmation, along with rescheduling and terminated states, accrues no compute charges. A genuinely 24/7 agent on that pricing model costs about $58 a month in runtime before any tokens are counted.

Providers like Fly.io Sprites and Blaxel charge zero idle compute cost, which makes standby time close to free for bursty workloads that spend most of their time waiting. Modern provider pricing design has largely already solved the idle-compute problem for teams that pick the right infrastructure.

Compute optimization does matter more at scale. If a team runs very large numbers of simultaneous sandboxes, compute can become a meaningful second line item, and it's worth monitoring even though it stays subordinate to token spend at almost any realistic workload size. Self-hosted or BYOC architectures change the equation further: the same sandbox fleet costs materially less on BYOC than on a managed platform, which makes compute a larger share of a smaller total bill. The most common compute inefficiency in practice is simpler than any of this: idle sandboxes left running between tasks, a waste that session lifecycle management fixes without requiring any infrastructure re-architecture.

Compute optimization is a one-time infrastructure decision with a bounded upside. Token optimization, caching, context discipline, and model routing, is an ongoing operational practice with a return that compounds every month a team runs it. The right sequence fixes token economics first and tunes the compute layer second, not the other way around.

Attribution across all four cost lines as the prerequisite

None of the levers described above, caching, context discipline, model routing, harness selection, can be verified without per-run attribution across all four cost lines. The structural invisibility of the total bill is the actual root cause behind teams optimizing the wrong line item, because the model provider, the compute provider, and the egress provider each hold only a fragment of the picture, and no single console shows the join.

Cost per successful run is the only metric that captures both the efficiency of the runs that work and the waste generated by the ones that fail, and it can't be calculated without attribution down to the individual run. If a team can see per-harness, per-model cost profiles, it can route low-complexity work to cheaper models and then confirm, with real numbers, that the routing actually saved money. Teams without that visibility are guessing, even when their instincts happen to be right. The same attribution that exposes token economics also exposes compute waste: idle sandboxes, failed runs that burned VM time without producing output, and parallelism that outran what the token budget could sustain.

Teams running agents on platforms like Replicas, which execute each task in its own isolated VM pre-configured with the team's codebase and tooling, can instrument these runs end to end and measure the actual split between token and compute costs in production. Without that visibility across the full run lifecycle, teams default to optimizing whichever cost they can see first, which is almost always the wrong one. Replicas gives engineering teams attribution down to the source: a person, a harness, a model, the credential that paid for it, and the skills and MCP servers the agent reached for, for every minute an agent runs, making all four cost lines visible in one place instead of scattered across three vendor consoles and a spreadsheet someone updates by hand. The stateless loop that forces token resubmission on every turn is a hard constraint of the model API itself, not something any agent platform can route around. But platform choice still shapes whether a team can measure token volume per run, attribute it to the harness and model that produced it, and test caching or context strategies against real data instead of guessing blindly. Every agent run in Replicas is sandboxed in its own VM, where the agent installs the team's own dependencies and tooling, which keeps the compute environment consistent and attributable, a prerequisite for comparing cost across runs in any way that means something.

The four bills were never going to reconcile themselves. Seeing them in one place is what turns the levers in this piece from guesses into decisions.