D-JEPA: A Latent World Model Aligned with Decision-Making
D-JEPA aligns latent futures with decision value, improving action selection in control, robotics, manipulation, and autonomous driving.
Key Takeaways
- D-JEPA introduces a decision-aligned latent world model that improves action selection by focusing on which future-state relationships matter for successful execution.
- It addresses a key planning failure where a predicted future may look closer to the goal but still imply a worse action choice.
- Instead of relying only on latent distance, D-JEPA aligns compressed future representations with decision value and learns relational corrections from real outcomes.
- It combines evidence across complementary predictive geometries and supports native latent-distance planning within a JEPA-compatible framework.
- D-JEPA shows measurable gains in control, robotic manipulation, physical robots, and autonomous driving, with resources and comparisons publicly available.
Meta description: D-JEPA uses decision-aligned latent world models to improve action selection in control, robotic manipulation, and autonomous driving, with measurable gains.
Tags: D-JEPA, latent world models, robotics, autonomous driving, decision alignment
Featured image: Abstract technology style visual of latent future trajectories aligning with decision pathways, no real people, no company logos, no text.
D-JEPA introduces a decision-aligned latent world model that targets a practical planning failure: a predicted future can look closer to a goal yet still suggest a worse action. It matters because action selection in control, robotic manipulation, physical robots, and autonomous driving depends on how candidate futures are compared.
Background: From Distance-to-Goal to Execution Value
D-JEPA is built to identify which relations among candidate futures matter for successful execution. In a latent world model, future states are represented in compressed form; D-JEPA aligns that representation with decision value rather than treating latent distance alone as proof of a good action.
Development: Corrections, Geometries, and Planning
Built on pretrained predictive models, D-JEPA learns bounded relational corrections from executed outcomes. It combines evidence across complementary predictive geometries and realizes the aligned decision structure in JEPA-compatible future representations for native latent-distance planning.
Latest evidence: Measured gains across tasks
Across control, robotic manipulation, physical robots, and autonomous driving, D-JEPA improves action selection. It reports 87.89% success on the PushT confirmation evaluation, a 15.04-percentage-point average gain on RoboTwin, and a 17-point gain on physical robot tasks.
Current stage: Public resources and comparisons
The release is currently accessible through code, checkpoints, and decision-supervision data. A narrated interactive explainer and recorded comparisons can be explored at https://nebulis-lab.com/D-JEPA.
Comments
No comments yet. Be the first to comment.