Continuous batching, and where the throughput actually goes
A scheduler-level look at continuous batching: why naive static batching wastes most of the GPU, what the arrival-rate math says about queue depth, and the three places real implementations lose the throughput they just won.