InfiniHand: Real-Time World-Space Hand Motion Estimation from Egocentric Video

Meta description: InfiniHand estimates world-space hand motion, MANO parameters, camera trajectories, and hand locations from uncalibrated egocentric video, reaching 11.19 FPS.

Tags: InfiniHand, egocentric video, hand pose estimation, world-space tracking, ARCTIC benchmark

Featured image: Abstract technology image showing streaming point clouds, hand skeleton lines, and camera trajectory paths, no real people, no company logos, no text.

InfiniHand is an end-to-end streaming feed-forward framework for estimating hand motion in world space directly from uncalibrated egocentric video. The system jointly recovers MANO parameters, a parametric representation of hand pose and shape, camera trajectories, and hand locations. Its value lies in replacing separate hand pose estimation and camera motion tracking modules with one architecture, reducing error buildup and computational overhead.

Background: The Hand-Camera Estimation Gap

Before InfiniHand, the task of world-space hand motion estimation from egocentric video had two tightly linked problems: recovering 3D articulated hand geometry and tracking camera egomotion. Many prior systems addressed them as separate stages. They often cascaded independent hand pose estimators with SLAM, or simultaneous localization and mapping, components. This design made the pipeline complex and created a clear weakness. Mistakes in one stage could pass to the next, causing error accumulation. It also brought severe computational overhead because each module ran as its own process rather than as one stream.

The practical consequence was limited robustness. A first-person video stream usually lacks camera calibration, moves quickly, and shows hands at close range. When the camera trajectory and hand geometry are estimated separately, the final world-space hand motion can drift, especially over longer sequences.

Development: A Unified Streaming Framework Trained in Two Stages

InfiniHand responds with a single feed-forward framework designed for streaming use. It integrates persistent spatiotemporal memory with hand-centered visual features. The architecture explicitly couples camera motion with local hand geometry instead of treating them as independent tasks. This coupling is central to the timeline of the method: first, the system learns how a hand appears relative to the camera; later, it extends that camera-space knowledge into world-space reconstruction.

Training follows two progressive stages. In the first stage, InfiniHand learns robust camera-space hand priors. These priors help it understand hand shape and pose from nearby first-person views. In the second stage, the model is extended to streaming world-space reconstruction, where it must estimate hand locations and camera trajectories together. To support this staged development, the authors aggregated a pretraining corpus of approximately 5,000 hours of egocentric data across multiple public datasets. That corpus gives the model broad exposure to hand-camera interactions before the world-space streaming stage.

Latest Progress: Benchmark Gains, 11.19 FPS, and World-Space Stability

The latest reported results place InfiniHand ahead of state-of-the-art baselines on in-domain benchmarks. On the ARCTIC benchmark, measured by PA-p, InfiniHand achieves a 21.4% reduction compared with ViDiHand. That number shows the gain is not only qualitative: the system reduces the specific error metric used in that evaluation. The framework also substantially mitigates world-space drift, which was one of the main failure modes of earlier cascaded pipelines.

Speed is the other headline. InfiniHand operates at 11.19 FPS, delivering more than twice the throughput of HaWoR. For streaming egocentric video, throughput matters because a model must keep pace with incoming frames while maintaining hand geometry and camera trajectory estimates. A feed-forward streaming design is therefore more useful for real-time pipelines than a heavier multi-stage system, even when accuracy improves.

In addition to benchmark performance, the evaluation notes robust generalization to in-the-wild videos. This matters because egocentric recordings vary widely in viewpoint, hand placement, lighting, and camera movement. A system trained on approximately 5,000 hours of public egocentric data and structured around persistent memory can transfer beyond controlled benchmark clips.

At this stage, InfiniHand represents a move from split hand-pose and camera-tracking pipelines toward a unified, streaming world-space estimator. The reported 21.4% ARCTIC PA-p reduction, 11.19 FPS throughput, and 5,000-hour pretraining corpus define its current position: a method that jointly estimates MANO parameters, camera trajectories, and hand locations while reducing drift in uncalibrated egocentric video.