Working papers

Running large language models across a mix of tiers promises lower cost and latency by resolving easy queries locally and escalating hard ones upstream. That escalation hinges on a confidence signal the small model emits about its own output. This paper shows the binding constraint inverts: where compute is billed, cost decides; where it is sunk on owned hardware, signal quality decides.

Agentic workloads run for a long time but sit idle in tool calls for most of it, which makes reserving capacity for them expensive. This paper puts that reservation on a pre-registered test bench, measuring across 108 configurations against six equally tuned baselines whether per-flow completion guarantees buy more completed work per provisioned GPU-hour than they strand. Throughput alone does not carry the reservation, but the guarantees do deliver interactive tail-latency isolation and bounded containment of runaway flows.