Inference Batching

Continuous Depth Batching for Depth-Adaptive Inference in Looped Language Models

CDB regroups tokens between loop steps, letting looped LLMs use adaptive depth without breaking batch efficien…