RoboDojo

depth · video · **RoboDojo** is a sim-and-real evaluation benchmark built to answer a blunt question: how good are generalist manipulation policies really? The answer it reports is sobering — the best of **30 policies evaluated in simulation** reached only **8.80%** success against **76.03%** for human teleoperation. - **60 tasks total**: **42 simulation tasks** (NVIDIA Isaac Sim 5.1 / Isaac Lab 2.3) spanning five capability axes — Generalization (12), Memory (6), Precision (8), Long-Horizon (8), Open-Vocabulary (8) — plus **18 real-world tasks** across three embodiments (ARX X5, Piper, Piper X). - Ships a **training dataset**: **3,500 simulated trajectories** (1,859,602 frames, **20.66 h**) and **1,800 real-world trajectories** (1,611,841 frames, **17.91 h**), all bimanual and recorded at **25 Hz**. - Distributed in several formats — **LeRobot v3.0** (120 GB), LeRobot v2.1 (64 GB), **HDF5** (523 GB), depth HDF5 (~4.5 TB), and real-world data (273 GB). - Real-world rigs use **3 synchronised cameras**: one head camera (Gemini 335L) and **two wrist cameras** (Gemini 305). Evaluation is **10 trials × 18 tasks = 180 real trials** per policy, scored double-blind by three independent evaluators. - **RoboDojo-RealEval** is a cloud-accessible real-robot evaluation service with standardised hardware and scene reset, so a policy can be benchmarked on physical robots remotely without owning any. - Policy integration runs through **XPolicyLab**, which unifies **40+ policies** behind one interface. - Simulation demonstrations come from two sources: **automated trajectory synthesis** composed from motion-planning primitives (grasp, place, handover, insert, open, close, stack, push_up), and **VR teleoperation** for tasks too complex to synthesise. Real-world demos are **leader-follower teleoperation**, 100 per task from four operators. Assembled by a 43-author consortium led by MMLab@HKU, with contributors from UC Berkeley, Tsinghua, Peking University, Stanford and MIT.

ee_pose · **RoboDojo** is a sim-and-real evaluation benchmark built to answer a blunt question: how good are generalist manipulation policies really? The answer it reports is sobering — the best of **30 policies evaluated in simulation** reached only **8.80%** success against **76.03%** for human teleoperation. - **60 tasks total**: **42 simulation tasks** (NVIDIA Isaac Sim 5.1 / Isaac Lab 2.3) spanning five capability axes — Generalization (12), Memory (6), Precision (8), Long-Horizon (8), Open-Vocabulary (8) — plus **18 real-world tasks** across three embodiments (ARX X5, Piper, Piper X). - Ships a **training dataset**: **3,500 simulated trajectories** (1,859,602 frames, **20.66 h**) and **1,800 real-world trajectories** (1,611,841 frames, **17.91 h**), all bimanual and recorded at **25 Hz**. - Distributed in several formats — **LeRobot v3.0** (120 GB), LeRobot v2.1 (64 GB), **HDF5** (523 GB), depth HDF5 (~4.5 TB), and real-world data (273 GB). - Real-world rigs use **3 synchronised cameras**: one head camera (Gemini 335L) and **two wrist cameras** (Gemini 305). Evaluation is **10 trials × 18 tasks = 180 real trials** per policy, scored double-blind by three independent evaluators. - **RoboDojo-RealEval** is a cloud-accessible real-robot evaluation service with standardised hardware and scene reset, so a policy can be benchmarked on physical robots remotely without owning any. - Policy integration runs through **XPolicyLab**, which unifies **40+ policies** behind one interface. - Simulation demonstrations come from two sources: **automated trajectory synthesis** composed from motion-planning primitives (grasp, place, handover, insert, open, close, stack, push_up), and **VR teleoperation** for tasks too complex to synthesise. Real-world demos are **leader-follower teleoperation**, 100 per task from four operators. Assembled by a 43-author consortium led by MMLab@HKU, with contributors from UC Berkeley, Tsinghua, Peking University, Stanford and MIT.

language · has-success-labels · **RoboDojo** is a sim-and-real evaluation benchmark built to answer a blunt question: how good are generalist manipulation policies really? The answer it reports is sobering — the best of **30 policies evaluated in simulation** reached only **8.80%** success against **76.03%** for human teleoperation. - **60 tasks total**: **42 simulation tasks** (NVIDIA Isaac Sim 5.1 / Isaac Lab 2.3) spanning five capability axes — Generalization (12), Memory (6), Precision (8), Long-Horizon (8), Open-Vocabulary (8) — plus **18 real-world tasks** across three embodiments (ARX X5, Piper, Piper X). - Ships a **training dataset**: **3,500 simulated trajectories** (1,859,602 frames, **20.66 h**) and **1,800 real-world trajectories** (1,611,841 frames, **17.91 h**), all bimanual and recorded at **25 Hz**. - Distributed in several formats — **LeRobot v3.0** (120 GB), LeRobot v2.1 (64 GB), **HDF5** (523 GB), depth HDF5 (~4.5 TB), and real-world data (273 GB). - Real-world rigs use **3 synchronised cameras**: one head camera (Gemini 335L) and **two wrist cameras** (Gemini 305). Evaluation is **10 trials × 18 tasks = 180 real trials** per policy, scored double-blind by three independent evaluators. - **RoboDojo-RealEval** is a cloud-accessible real-robot evaluation service with standardised hardware and scene reset, so a policy can be benchmarked on physical robots remotely without owning any. - Policy integration runs through **XPolicyLab**, which unifies **40+ policies** behind one interface. - Simulation demonstrations come from two sources: **automated trajectory synthesis** composed from motion-planning primitives (grasp, place, handover, insert, open, close, stack, push_up), and **VR teleoperation** for tasks too complex to synthesise. Real-world demos are **leader-follower teleoperation**, 100 per task from four operators. Assembled by a 43-author consortium led by MMLab@HKU, with contributors from UC Berkeley, Tsinghua, Peking University, Stanford and MIT.

**RoboDojo** is a sim-and-real evaluation benchmark built to answer a blunt question: how good are generalist manipulation policies really? The answer it reports is sobering — the best of **30 policies evaluated in simulation** reached only **8.80%** success against **76.03%** for human teleoperation. - **60 tasks total**: **42 simulation tasks** (NVIDIA Isaac Sim 5.1 / Isaac Lab 2.3) spanning five capability axes — Generalization (12), Memory (6), Precision (8), Long-Horizon (8), Open-Vocabulary (8) — plus **18 real-world tasks** across three embodiments (ARX X5, Piper, Piper X). - Ships a **training dataset**: **3,500 simulated trajectories** (1,859,602 frames, **20.66 h**) and **1,800 real-world trajectories** (1,611,841 frames, **17.91 h**), all bimanual and recorded at **25 Hz**. - Distributed in several formats — **LeRobot v3.0** (120 GB), LeRobot v2.1 (64 GB), **HDF5** (523 GB), depth HDF5 (~4.5 TB), and real-world data (273 GB). - Real-world rigs use **3 synchronised cameras**: one head camera (Gemini 335L) and **two wrist cameras** (Gemini 305). Evaluation is **10 trials × 18 tasks = 180 real trials** per policy, scored double-blind by three independent evaluators. - **RoboDojo-RealEval** is a cloud-accessible real-robot evaluation service with standardised hardware and scene reset, so a policy can be benchmarked on physical robots remotely without owning any. - Policy integration runs through **XPolicyLab**, which unifies **40+ policies** behind one interface. - Simulation demonstrations come from two sources: **automated trajectory synthesis** composed from motion-planning primitives (grasp, place, handover, insert, open, close, stack, push_up), and **VR teleoperation** for tasks too complex to synthesise. Real-world demos are **leader-follower teleoperation**, 100 per task from four operators. Assembled by a 43-author consortium led by MMLab@HKU, with contributors from UC Berkeley, Tsinghua, Peking University, Stanford and MIT.

depth · simulation · **RoboDojo** is a sim-and-real evaluation benchmark built to answer a blunt question: how good are generalist manipulation policies really? The answer it reports is sobering — the best of **30 policies evaluated in simulation** reached only **8.80%** success against **76.03%** for human teleoperation. - **60 tasks total**: **42 simulation tasks** (NVIDIA Isaac Sim 5.1 / Isaac Lab 2.3) spanning five capability axes — Generalization (12), Memory (6), Precision (8), Long-Horizon (8), Open-Vocabulary (8) — plus **18 real-world tasks** across three embodiments (ARX X5, Piper, Piper X). - Ships a **training dataset**: **3,500 simulated trajectories** (1,859,602 frames, **20.66 h**) and **1,800 real-world trajectories** (1,611,841 frames, **17.91 h**), all bimanual and recorded at **25 Hz**. - Distributed in several formats — **LeRobot v3.0** (120 GB), LeRobot v2.1 (64 GB), **HDF5** (523 GB), depth HDF5 (~4.5 TB), and real-world data (273 GB). - Real-world rigs use **3 synchronised cameras**: one head camera (Gemini 335L) and **two wrist cameras** (Gemini 305). Evaluation is **10 trials × 18 tasks = 180 real trials** per policy, scored double-blind by three independent evaluators. - **RoboDojo-RealEval** is a cloud-accessible real-robot evaluation service with standardised hardware and scene reset, so a policy can be benchmarked on physical robots remotely without owning any. - Policy integration runs through **XPolicyLab**, which unifies **40+ policies** behind one interface. - Simulation demonstrations come from two sources: **automated trajectory synthesis** composed from motion-planning primitives (grasp, place, handover, insert, open, close, stack, push_up), and **VR teleoperation** for tasks too complex to synthesise. Real-world demos are **leader-follower teleoperation**, 100 per task from four operators. Assembled by a 43-author consortium led by MMLab@HKU, with contributors from UC Berkeley, Tsinghua, Peking University, Stanford and MIT.

Open · Apache-2.0 · hdf5 · lerobot