miniTE identifies and closes the execution gap in native NVFP4 reinforcement learning

miniTE, an open-source framework described at infix.io/research/miniTE, reports stable and efficient native NVFP4 reinforcement learning for large language models using 4-bit weights and activations (W4A4), and treats training-inference mismatch as a numerical and systems problem. The result matters because low-bit reinforcement learning has often been discussed as a throughput question, while the reported finding shows that identical weights and identical precision can still lead to different policies if the learner and rollout engine execute numbers differently.

Background: Same Weights Did Not Guarantee Same Policy

The reported starting point is a warning rather than a speed metric: same weights + same precision ≠ same policy. In a reinforcement learning pipeline where a learner updates a model and a rollout engine generates behavior, both sides may use NVFP4 weights and activations. Even then, subtle differences in numerical execution can change the probabilities the learner observes. Over many update steps, those differences can grow into training-inference mismatch and eventually destabilize learning.

The team describes the issue as subtle but consequential. The parameters may appear equivalent, yet different engines can follow different paths through kernels, scaling, and grouped general matrix multiplications. For native four-bit floating point workloads, this becomes more than a rounding error: it can affect which policy is learned.

Development: Aligning Numerics, Then Making NVFP4 Fast

miniTE addresses the problem in two directions. First, it diagnoses cross-engine execution differences and aligns them in a strict setting. The reported alignment brings training-inference log-probability gaps down to ~1e-8. That figure is the core numerical checkpoint: before speed matters, the learner and rollout engine must agree on the policy they are evaluating.

Second, the framework optimizes the low-bit execution stack itself. The work targets native NVFP4 kernels, scaling, and grouped general matrix multiplications. On 8×B300 GPUs, the reported learner throughput reaches up to 1.39× BF16. This places four-bit floating point not only as a way to reduce memory and compute, but also as a path to higher throughput when the kernels are built for the format.

The stability claim extends to long-horizon reinforcement learning on Qwen3-30B-A3B. The reported downstream performance is close to BF16. The result suggests that, once the numerical gap is controlled, native NVFP4 can preserve the quality needed for longer reinforcement learning runs instead of collapsing under accumulated mismatch.

Latest Progress: Framework, Kernels, Diagnostics, and Recipes

Beyond the benchmark-style findings, miniTE is presented as a reproducible research package. The release includes the framework, kernels, diagnostics, and training recipes. That package matters because the reported mismatch is engine-dependent: a result is hard to reuse if only final scores are shown. Diagnostics and recipes give other teams a way to inspect alignment, not just compare loss curves.

The project links to an arXiv paper at https://arxiv.org/abs/2610.23928 and research details at https://infix.io/research/miniTE. The team invites feedback for pushing low-precision reinforcement learning further and frames the goal as making every bit work, not merely using fewer bits.

Current stage: the project is no longer only a hypothesis about four-bit floating point. It is an open-source attempt to make native NVFP4 reinforcement learning practical by pairing numerical alignment with optimized kernels. The next signal to watch is how these diagnostics and recipes behave outside the reported strict-alignment setting, especially as models, engines, and low-bit formats diversify.

Meta Description: miniTE framework stabilizes native NVFP4 W4A4 reinforcement learning for Qwen3-30B-A3B, cutting gaps to 1e-8 and reaching 1.39× BF16 throughput on 8×B300 GPUs.

Tags: miniTE, NVFP4, reinforcement learning, Qwen3-30B-A3B, low-precision training

Featured Image: abstract low-bit matrix grid with glowing blue and orange lines, no people, logos, or text.