RayOrch: Programming and Running Lineage-Aware Multi-Granular Data Pipelines for Foundation Model Data Preparation

Meta description: RayOrch is a distributed data pipeline system for multimodal foundation models, cutting MinerU end-to-end time by 13.1% versus Ray Data and 29.0% versus Daft.

Tags: RayOrch, data pipelines, multimodal AI, MinerU, distributed systems

Featured image: abstract technology-style distributed data pipeline network with nodes, arrows, and processing blocks, no real people, no company logos, no text.

RayOrch has been introduced as a distributed data pipeline system for multimodal foundation model data preparation. The introduction is notable because data preparation is one of the operational bottlenecks in building models that consume text, images, video, and other mixed inputs. The material positions RayOrch alongside existing large-scale processing tools such as Ray Data and Daft, and reports end-to-end time reductions on the MinerU workload of 13.1% compared with Ray Data and 29.0% compared with Daft.

Background: Why multimodal data pipelines matter

Multimodal foundation models do not train on a single clean table of text. They consume heterogeneous material: documents, scanned pages, images, video clips, captions, and other related records. Before a model can learn from this material, the data must be parsed, split, filtered, transformed, and delivered to accelerators such as GPUs. That work often runs across many machines, so a data pipeline becomes a scheduling system as much as a storage or parsing system.

Large-scale tools such as Ray Data and Daft address the general problem of moving and processing distributed datasets. Their common challenge is keeping expensive GPU stages fed without wasting resources on poorly aligned batches, repeated work, or output reassembly. In a multimodal pipeline, the unit of work may not match the unit of output. A document may contain many pages, and a video may contain many frames, but each page or frame still belongs to a specific parent input. If the pipeline loses that relationship, downstream training or evaluation can become difficult.

Development process: RayOrch targets granular, mixed-stage pipelines

RayOrch is described as a system built for pipelines with multiple stages and levels of granularity. The examples given are document processing at page level and video processing at frame level. This design recognizes that many useful model datasets are not monolithic records; they are collections of smaller pieces that need to be processed, mixed, and later reassembled.

The system’s core scheduling behavior is to track which input each piece belongs to, batch ready work across inputs for GPU execution, and gather results in the correct order even when tasks finish out of order. In practical terms, that means the pipeline can send available pages, frames, or other chunks to GPUs as soon as they are ready, while still preserving the structure needed for later use. Ordered output matters because many training and evaluation workflows assume that examples, labels, or derived records remain aligned.

This approach is less about adding one new operator and more about managing dependencies across a distributed workflow. If a GPU batch waits for all inputs in a group to finish, the accelerator may sit idle. If it emits pieces too early without tracking parent inputs, the final dataset may be incomplete or shuffled. RayOrch’s reported design aims to sit between those two failure modes by keeping work ready for execution and results traceable for assembly.

Latest progress: MinerU workload shows reported gains

The most concrete performance signal in the introduction is the MinerU workload. On that workload, RayOrch is reported to reduce end-to-end time by 13.1% compared with Ray Data and 29.0% compared with Daft. These figures describe total pipeline time rather than isolated stage speed, which is relevant because distributed data preparation can be slowed by scheduling, dependency management, and output collection as much as by raw transformation stages.

The introduction states that it welcomes feedback and questions, which suggests the material is being presented for review and early adoption discussion rather than as a closed production case study. The current public information centers on the system’s intended use, its scheduling model, and the reported MinerU comparison.

What to watch next

At this stage, RayOrch is best understood as a newly introduced distributed data pipeline approach for multimodal foundation model preparation, with a specific benchmark claim tied to MinerU and comparisons against Ray Data and Daft. The next useful signals would be additional workloads beyond MinerU, clearer deployment details, and feedback on how the ordered-output and cross-input batching mechanisms perform in larger heterogeneous pipelines.