We evaluate KAI in simulation and the real world, focusing on sample efficiency, generalization to unseen backgrounds and distractors, and the benefit of human-video co-training.
Articulated object manipulation requires an understanding of kinematic structure that is difficult and costly to learn from robot demonstrations alone. We introduce the Kinematic-Aware Articulation Interface (KAI), a structured intermediate representation that captures the kinematic structure of articulated objects. By embedding interpretable geometric and kinematic priors into policy learning, KAI provides a strong inductive bias aligned with the underlying structure of articulated motion.
This design effectively improves sample efficiency, with gains particularly pronounced in low-data regimes: across six simulation tasks, our method achieves an average success rate of 82.9%, matching or surpassing baseline performance while using only half the demonstration data. Our method also exhibits robust generalization to unseen backgrounds and visual distractors, transferring from a single clean training environment to cluttered real-world scenes. KAI’s action-agnostic design further enables co-training with human interaction videos: under diverse visual distractions, our method with video co-training achieves over 70% average success rate.
End-to-end imitation learning maps observations to actions without explicitly modeling kinematic structure, forcing the policy to infer joint mechanics from data alone. Under limited demonstrations, this often leads to brittle generalization. We introduce the Kinematic-Aware Articulation Interface (KAI), a structured intermediate representation that encodes articulated kinematics by localizing interaction keypoints on the moving part and predicting their future displacement trajectories—providing an inductive bias aligned with how articulated objects actually move.
Concretely, KAI represents an articulated object with K keypoints on the moving part. At each timestep, the model predicts future keypoint locations and corresponding displacements over the next N steps. A geometric loss regularizes these predictions: isometric constraints preserve pairwise keypoint structure over time, while joint-type motion priors encourage coherent translation for prismatic joints and axis-anchored rotation for revolute joints.
We adopt a phased-reasoning architecture with a Perception–KAI–Action workflow. Unidirectional attention masks ensure KAI tokens attend to perceptual tokens but not action tokens, decoupling structural understanding from control. Because KAI is action-agnostic, it can also be supervised from human interaction videos without robot action labels, enabling co-training that improves real-world robustness under visual distractions.
We evaluate KAI in simulation and the real world, focusing on sample efficiency, generalization to unseen backgrounds and distractors, and the benefit of human-video co-training.
Six articulated tasks in Isaac Sim (open/close drawer, door, and laptop), evaluated with 200 rollouts per task across 8 unseen scenes.
| Method | Avg. SR | Open Drawer | Open Door | Open Laptop | Close Drawer | Close Door | Close Laptop |
|---|---|---|---|---|---|---|---|
| ACT | 58.5 | 44.5 | 52.5 | 67.5 | 39.0 | 54.0 | 93.5 |
| Seer | 73.4 | 84.8 | 67.2 | 67.7 | 68.1 | 63.7 | 89.2 |
| DP3 | 37.3 | 25.0 | 12.5 | 25.5 | 2.0 | 97.0 | 62.0 |
| RISE-2 | 22.3 | 21.0 | 9.0 | 7.5 | 21.5 | 19.0 | 56.0 |
| ArticuBot | 38.5 | 4.0 | 22.5 | 24.0 | 28.0 | 71.0 | 81.5 |
| Ours | 82.9 | 86.7 | 54.5 | 81.7 | 82.3 | 92.5 | 100.0 |
| Ours (w/ video) | 84.6 | 86.8 | 66.0 | 85.0 | 81.8 | 88.8 | 99.2 |
Ours reaches 82.9% average success without video co-training, already surpassing all baselines. Video co-training yields only a modest gain (+1.7%) in simulation, largely due to the perceptual domain gap between human videos and synthetic images; this gap narrows in the real world.
Data efficiency. Success rates of our method, Seer, and DP3 across different training data volumes. Under 100 demonstrations, KAI outperforms the strongest baseline by ~22 percentage points; at larger scales it matches or exceeds the best baseline with only half the training data.
Franka Research 3 + Robotiq 2F-85 with two RealSense D455 cameras. Three tasks (open drawer, close laptop, close microwave), 100 demos each collected in a single clean setting. Evaluated under no distraction, background distractions, and object distractors.
| Distractions | Method | Avg. SR / Loc-SR | Open Drawer | Close Laptop | Close Microwave |
|---|---|---|---|---|---|
| None | DP3 | 33.3 / 55.5 | 20.0 / 40.0 | 40.0 / 53.3 | 40.0 / 73.3 |
| Seer | 37.8 / 64.4 | 26.7 / 53.3 | 46.7 / 73.3 | 40.0 / 66.7 | |
| Ours | 66.7 / 80.0 | 46.7 / 60.0 | 73.3 / 86.7 | 80.0 / 93.3 | |
| Ours (w/ video) | 73.3 / 82.2 | 60.0 / 66.7 | 80.0 / 86.7 | 80.0 / 93.3 | |
| Background | DP3 | 28.9 / 57.8 | 20.0 / 46.7 | 40.0 / 46.7 | 26.7 / 80.0 |
| Seer | 26.7 / 51.1 | 20.0 / 40.0 | 33.3 / 53.3 | 26.7 / 60.0 | |
| Ours | 62.2 / 77.8 | 40.0 / 53.3 | 66.7 / 86.7 | 80.0 / 93.3 | |
| Ours (w/ video) | 73.3 / 84.5 | 53.3 / 66.7 | 80.0 / 86.7 | 86.7 / 100.0 | |
| Object | DP3 | 24.4 / 42.2 | 20.0 / 33.3 | 20.0 / 53.3 | 33.3 / 40.0 |
| Seer | 31.1 / 53.3 | 20.0 / 40.0 | 40.0 / 66.7 | 33.3 / 53.3 | |
| Ours | 62.2 / 71.1 | 40.0 / 60.0 | 80.0 / 80.0 | 66.7 / 73.3 | |
| Ours (w/ video) | 75.6 / 77.8 | 60.0 / 60.0 | 86.7 / 86.7 | 80.0 / 86.7 |
Our method remains robust when transferring from a single clean training setting to cluttered real-world scenes with background and object distractors. Video co-training further improves real-world generalization substantially.
Open Drawer
Close Laptop
Close Microwave
@misc{li2026kaikinematicawareinterfacedataefficient,
title={KAI: A Kinematic-Aware Interface for Data-Efficient Articulated Object Manipulation},
author={Yaping Li and Zhaxizhuoma and Qiaojun Yu and Jia Zeng and Dahua Lin and Jiangmiao Pang},
year={2026},
eprint={2607.24493},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2607.24493},
}