LLM inference
2 stories tagged “LLM inference”.
-
Decoupled Quantization: Tailoring LLM Prefill and Decode Stages
Disaggregated quantization splits LLM prefill and decode weights for speed and accuracy; Qwen3.8 provides NVFP…
-
Adaptive Parallel Reasoning: A New Paradigm for Scaling Inference Efficiency
Sequential LLM reasoning faces context limits, context-rot, and long latency. Parallel reasoning offers a fix,…