Rho-1, Reka AI’s omni-model, unifies text, images, video, and robot control in one system

Meta description: Reka AI released Rho-1, a 19-billion-parameter omni-model that unifies text, images, video, and robot control actions in one neural network.

Tags: Rho-1, Reka AI, multimodal AI, world models, robot control

Featured image: Abstract neural network visual connecting text, video frames, and robotic arm glyphs, no real people, no company logos, no text.

Background: The Push Toward Unified Multimodal Systems

Most AI systems split work across specialized parts: one model for language, another for vision, another for planning, and so on. That approach can be reliable, but it often requires routing, tool calls, and separate outputs. Reka AI's research preview of Rho-1 shows another path. The company describes it as a 19-billion-parameter omni-model, meaning one neural network both takes in and produces text, images, video, and robot control actions.

The broader context is the interest in world models, a term used for AI systems that learn from continuous streams of observations and actions, similar to how agents interact with physical environments. Reka AI is not entering this area from scratch. In April 2024, the company shipped Reka Core, a multimodal language model that competed with GPT-4, Claude 3, and Gemini Ultra on benchmarks. That earlier release positioned the company within a research trend that wants models to understand more than one mode of information at once.

Development Process: From Reka Core to a Single Model

Rho-1 is built around one shared context window. In ordinary terms, a context window is the span of information the model can consider together. Reka AI says text, images, video, and robot control actions are all handled as tokens inside that window. Tokens are the pieces of data a model reads and generates, so using one context for all modalities reduces the need to translate between separate systems.

The model does not use tool calls or external models for the tasks described in the preview. Instead, the same weights that predict camera images also drive robot movements. This design links perception and action more directly than systems that first see, then call a separate controller. It also raises a practical problem: robots need large amounts of training data showing how actions change the world, and that data is scarce.

Reka AI addressed the shortage by building an inverse dynamics model. In this setting, inverse dynamics means inferring the control signals that likely produced observed changes in video. The company says the model pulls control signals from ordinary internet videos, giving the system a broader training source than robot-only datasets. The training run used 320 H100 GPUs over about three months.

Latest Progress: Real-Time Video and Robot Control

The preview suggests Rho-1 can generate continuous video in real time. More importantly, it can respond to new instructions while the stream is running, without restarting the model. That behavior matters for interactive systems, because restarts add latency and can break the flow of perception and action.

For robotics, the key claim is not simply that the model sees or predicts, but that the same learned representation supports control. If visual prediction and motor output share weights, the model may learn a tighter coupling between what happens in the scene and what the machine should do. The preview does not present a finished product, but it illustrates the kind of unified architecture that world-model research is aiming for.

Current Stage: A Research Preview Still Needs Wider Testing

Rho-1 is currently a research preview, not a commercial deployment. The facts Reka AI provided point to a clear stage: a 19-billion-parameter single-network multimodal system, trained with 320 H100 GPUs over about three months, using an inverse dynamics model to stretch scarce robot data, and following the company's April 2024 Reka Core release. The next question is how well this shared context holds up when compared with specialized systems under real-world demands such as reliability, safety, and long-horizon robot control.