QwenGyre: A Dynamically Scalable Reinforcement Learning Framework for Training xLong-Horizon Agents

Meta description: QwenGyre targets long-horizon online RL, elastically reallocating GPUs and deduplicating trajectories while lifting NL2RepoBench from 52.5% to 58.5%.

Tags: QwenGyre, online reinforcement learning, long-horizon agents, GPU scheduling, NL2RepoBench

Featured image: abstract technology-style graphic with flowing GPU allocation curves, branching trajectory trees, and benchmark progress lines, no people, logos, or text.

Large language model (LLM) agents increasingly undertake extreme-long (xlong) horizon tasks, where a single execution can span hours, hundreds of model-environment interactions, and nearly 1M tokens per rollout. QwenGyre is an end-to-end framework for xlong-horizon online reinforcement learning (RL), introduced to reduce the GPU waste and training redundancy created by those executions. The framework is scaled to the flagship model Qwen 3.8 2.4T, with 700K tokens per rollout, and yields a 6.0% absolute gain on NL2RepoBench (52.5% → 58.5%) in 48 steps.

Background: Where long-horizon RL breaks down

The shift from short prompts to agent workflows changes the training bottleneck. A single rollout is no longer a brief exchange; it can span hours and hundreds of model-environment interactions. That scale makes online reinforcement learning difficult in two fundamental ways.

First, severe execution variance and prolonged rollout delays cause massive GPU idling. When some trajectories take much longer than others, compute resources can spend time waiting rather than advancing model updates. Second, complex non-linear branching generates massive trajectory redundancy, crippling training efficiency. As agents explore many possible paths through an environment, duplicated or overlapping histories can inflate the amount of work required for each training step.

Development: Elastic GPU control and trajectory reconstruction

QwenGyre responds by treating rollout and training as resources that can be rebalanced during live execution. Its core mechanism elastically reallocates GPUs between rollout and training without interrupting live executions. This means the system does not need to stop a long trajectory in order to shift capacity toward the side of the workload that is currently constrained.

The second part of the framework is a trajectory processor. It reconstructs branching histories, scores partial progress, and deduplicates redundant paths to bound training costs. In practice, this turns the messy output of long-horizon exploration into a more structured training signal. Rather than carrying every duplicated branch into the update loop, the processor can identify which partial paths matter and which are redundant.

Latest progress: 700K-token rollouts and benchmark gains

The framework is reported at scale on the flagship model Qwen 3.8 2.4T, using 700K tokens per rollout. On NL2RepoBench, QwenGyre yields a 6.0% absolute gain, moving the score from 52.5% to 58.5% in 48 steps.

Across evaluations on diverse domains of training datasets, QwenGyre delivers up to 1.85x and 1.78x speedups over Colocate and Async, respectively. Those comparisons suggest that the framework's value is not limited to a single benchmark result; it also aims to reduce the time and resource cost of the training loop itself.

Current stage: What the numbers suggest

The results position QwenGyre as a response to a specific infrastructure problem in agent training: long executions can stall GPU utilization, and branching trajectories can make updates expensive. The reported 700K-token rollout setting and 48-step NL2RepoBench gain show that the approach can be applied to a large model while producing a measurable improvement in benchmark performance.

What remains to be seen is how the same elastic GPU reallocation and trajectory reconstruction behave as tasks move closer to the nearly 1M-token per rollout ceiling described in the background. The speedups over Colocate and Async also point to a broader training-system question: whether QwenGyre can maintain efficiency gains across more domains and longer horizons without interrupting live executions.