CARE: Verifying Acceleration for Vision-Language-Action Inference
CARE addresses hidden failures in accelerated vision-language-action inference by comparing reference and accelerated rollouts. It uses calibration episodes to certify that speedups keep failure risk below a budget, deploying the fastest certified model or falling back. On LIBERO and other benchmarks, it achieves large speedups while preserving most reference successes, offering a deployable way to quantify acceleration risk.
Key Takeaways
- CARE targets a hidden risk in accelerating vision-language-action robot policies: faster inference can silently cause episode failures that average metrics miss.
- It defines failure through paired rollouts from identical initial conditions, counting cases where the reference policy succeeds but the accelerated policy fails.
- CARE provides finite-sample guarantees that acceleration-induced failure risk stays within a user-specified budget, selecting the fastest certified candidate and falling back when needed.
- Experiments on LIBERO show 9.0–10.8x speedups with OpenVLA-OFT while preserving at least 85.8% of reference-solved episodes with 95% confidence.
- The method generalizes across different acceleration mechanisms and agents, while sequential testing reduces the number of costly calibration rollouts.
Vision-language-action (VLA) models have become a cornerstone of modern robot learning, but running a full inference pass at every control step is costly. A new method called CARE (Certified Accelerator selection for Robot Execution) addresses a gap that latency benchmarks and average success rates have long obscured: acceleration can silently destroy tasks that the original policy would have solved.
Background: Why Average Metrics Hide Failures
Prior acceleration work leans on action chunking and visual-token pruning, judged mainly by latency and mean task success. The problem is that action deviations compound across closed-loop trajectories. A small error at one step shifts the state distribution, and failure may only become visible across an entire episode. Average success rates can therefore stay high while specific episodes break.
Defining Acceleration-Induced Failure
The authors formalize this with paired rollouts from identical initial conditions. A failure is counted when the reference policy succeeds but the accelerated policy fails on the same episode. This episode-level, paired definition captures the compounding nature of closed-loop error that step-level metrics miss.
How CARE Certifies Speedups
CARE performs paired rollouts on a calibration set to produce finite-sample guarantees that acceleration-induced failure risk stays below a user-specified budget. It deploys the fastest certified candidate and falls back to the reference when none qualify. Because CARE relies only on terminal outcomes and measured compute, it applies unchanged across different acceleration mechanisms. Sequential testing and failure-triggered reference rollouts keep certification affordable.
Results on LIBERO and Beyond
On four LIBERO suites with OpenVLA-OFT, CARE certifies 9.0–10.8 times speedups while guaranteeing, at 95% confidence, that at least 85.8% of reference-solved episodes are preserved. Under tight budgets, selectors without guarantees exceed the budget in up to 75% of trials, whereas CARE stays within budget. Its sequential form uses 78.9% fewer rollouts than exhaustive evaluation. CARE also generalizes to flow-step reduction for pi_{0.5} and to Qwen3.5-9B and Llama-3.1-8B agents in Crafter.
What to Watch
The results suggest that certified selection could become a standard layer between VLA policies and deployment hardware, letting teams pick acceleration levels with quantified risk rather than post-hoc averages. The open question is how these guarantees scale to longer horizons and real-world manipulation, where calibration episodes are costlier to collect.
Comments
No comments yet. Be the first to comment.