RoboArena

video · **RoboArena** is a living, distributed real-robot benchmark: Chatbot Arena, but the contestants are robot policies and the arena is a network of physical Franka arms. Crucially, **every episode is an autonomous policy rollout, not a teleoperated demonstration** — a human is present only to reset the scene and score the result. - **10,783 policy episodes** across **3,883 evaluation sessions** in the latest snapshot (**21.7 GB**, 27,148 MP4 videos), up from 4,613 episodes in the first public dump. - Every site runs the **DROID platform**: a **Franka Panda** 7-DoF arm with a Robotiq 2F-85 gripper, a ZED-mini stereo **wrist camera**, and ZED 2 stereo **shoulder cameras** (left and right) on a mobile height-adjustable table. - Evaluators at **7+ academic institutions** — Berkeley, Stanford, UW, Montréal, NVIDIA, Penn, UT Austin, Yonsei — each run **pairs of competing policies** on tasks of their own choosing, then rate which did better. There is no fixed task list and no central lab. - **21 policies** appear in the latest index, including pi0, pi0-fast, pi0.5, several PaliGemma variants, MolmoAct2-DROID and Cosmos3-Nano-Policy, spanning both joint-velocity and joint-position action spaces. - Each session records a natural-language task instruction, a **binary A/B/TIE preference**, a **0–100 continuous partial-success score per policy**, a binary success flag, episode duration, and free-form evaluator feedback. - Released as periodic **DataDump** snapshots on Hugging Face (Aug 2025, Feb 2026, Jul 2026), so the corpus grows over time rather than freezing. - Published at **CoRL 2025**. Because it captures how policies actually fail in the wild, it is useful well beyond leaderboards — for failure analysis, reward modelling, and preference learning.

**RoboArena** is a living, distributed real-robot benchmark: Chatbot Arena, but the contestants are robot policies and the arena is a network of physical Franka arms. Crucially, **every episode is an autonomous policy rollout, not a teleoperated demonstration** — a human is present only to reset the scene and score the result. - **10,783 policy episodes** across **3,883 evaluation sessions** in the latest snapshot (**21.7 GB**, 27,148 MP4 videos), up from 4,613 episodes in the first public dump. - Every site runs the **DROID platform**: a **Franka Panda** 7-DoF arm with a Robotiq 2F-85 gripper, a ZED-mini stereo **wrist camera**, and ZED 2 stereo **shoulder cameras** (left and right) on a mobile height-adjustable table. - Evaluators at **7+ academic institutions** — Berkeley, Stanford, UW, Montréal, NVIDIA, Penn, UT Austin, Yonsei — each run **pairs of competing policies** on tasks of their own choosing, then rate which did better. There is no fixed task list and no central lab. - **21 policies** appear in the latest index, including pi0, pi0-fast, pi0.5, several PaliGemma variants, MolmoAct2-DROID and Cosmos3-Nano-Policy, spanning both joint-velocity and joint-position action spaces. - Each session records a natural-language task instruction, a **binary A/B/TIE preference**, a **0–100 continuous partial-success score per policy**, a binary success flag, episode duration, and free-form evaluator feedback. - Released as periodic **DataDump** snapshots on Hugging Face (Aug 2025, Feb 2026, Jul 2026), so the corpus grows over time rather than freezing. - Published at **CoRL 2025**. Because it captures how policies actually fail in the wild, it is useful well beyond leaderboards — for failure analysis, reward modelling, and preference learning.

language · has-success-labels · **RoboArena** is a living, distributed real-robot benchmark: Chatbot Arena, but the contestants are robot policies and the arena is a network of physical Franka arms. Crucially, **every episode is an autonomous policy rollout, not a teleoperated demonstration** — a human is present only to reset the scene and score the result. - **10,783 policy episodes** across **3,883 evaluation sessions** in the latest snapshot (**21.7 GB**, 27,148 MP4 videos), up from 4,613 episodes in the first public dump. - Every site runs the **DROID platform**: a **Franka Panda** 7-DoF arm with a Robotiq 2F-85 gripper, a ZED-mini stereo **wrist camera**, and ZED 2 stereo **shoulder cameras** (left and right) on a mobile height-adjustable table. - Evaluators at **7+ academic institutions** — Berkeley, Stanford, UW, Montréal, NVIDIA, Penn, UT Austin, Yonsei — each run **pairs of competing policies** on tasks of their own choosing, then rate which did better. There is no fixed task list and no central lab. - **21 policies** appear in the latest index, including pi0, pi0-fast, pi0.5, several PaliGemma variants, MolmoAct2-DROID and Cosmos3-Nano-Policy, spanning both joint-velocity and joint-position action spaces. - Each session records a natural-language task instruction, a **binary A/B/TIE preference**, a **0–100 continuous partial-success score per policy**, a binary success flag, episode duration, and free-form evaluator feedback. - Released as periodic **DataDump** snapshots on Hugging Face (Aug 2025, Feb 2026, Jul 2026), so the corpus grows over time rather than freezing. - Published at **CoRL 2025**. Because it captures how policies actually fail in the wild, it is useful well beyond leaderboards — for failure analysis, reward modelling, and preference learning.

**RoboArena** is a living, distributed real-robot benchmark: Chatbot Arena, but the contestants are robot policies and the arena is a network of physical Franka arms. Crucially, **every episode is an autonomous policy rollout, not a teleoperated demonstration** — a human is present only to reset the scene and score the result. - **10,783 policy episodes** across **3,883 evaluation sessions** in the latest snapshot (**21.7 GB**, 27,148 MP4 videos), up from 4,613 episodes in the first public dump. - Every site runs the **DROID platform**: a **Franka Panda** 7-DoF arm with a Robotiq 2F-85 gripper, a ZED-mini stereo **wrist camera**, and ZED 2 stereo **shoulder cameras** (left and right) on a mobile height-adjustable table. - Evaluators at **7+ academic institutions** — Berkeley, Stanford, UW, Montréal, NVIDIA, Penn, UT Austin, Yonsei — each run **pairs of competing policies** on tasks of their own choosing, then rate which did better. There is no fixed task list and no central lab. - **21 policies** appear in the latest index, including pi0, pi0-fast, pi0.5, several PaliGemma variants, MolmoAct2-DROID and Cosmos3-Nano-Policy, spanning both joint-velocity and joint-position action spaces. - Each session records a natural-language task instruction, a **binary A/B/TIE preference**, a **0–100 continuous partial-success score per policy**, a binary success flag, episode duration, and free-form evaluator feedback. - Released as periodic **DataDump** snapshots on Hugging Face (Aug 2025, Feb 2026, Jul 2026), so the corpus grows over time rather than freezing. - Published at **CoRL 2025**. Because it captures how policies actually fail in the wild, it is useful well beyond leaderboards — for failure analysis, reward modelling, and preference learning.

**RoboArena** is a living, distributed real-robot benchmark: Chatbot Arena, but the contestants are robot policies and the arena is a network of physical Franka arms. Crucially, **every episode is an autonomous policy rollout, not a teleoperated demonstration** — a human is present only to reset the scene and score the result. - **10,783 policy episodes** across **3,883 evaluation sessions** in the latest snapshot (**21.7 GB**, 27,148 MP4 videos), up from 4,613 episodes in the first public dump. - Every site runs the **DROID platform**: a **Franka Panda** 7-DoF arm with a Robotiq 2F-85 gripper, a ZED-mini stereo **wrist camera**, and ZED 2 stereo **shoulder cameras** (left and right) on a mobile height-adjustable table. - Evaluators at **7+ academic institutions** — Berkeley, Stanford, UW, Montréal, NVIDIA, Penn, UT Austin, Yonsei — each run **pairs of competing policies** on tasks of their own choosing, then rate which did better. There is no fixed task list and no central lab. - **21 policies** appear in the latest index, including pi0, pi0-fast, pi0.5, several PaliGemma variants, MolmoAct2-DROID and Cosmos3-Nano-Policy, spanning both joint-velocity and joint-position action spaces. - Each session records a natural-language task instruction, a **binary A/B/TIE preference**, a **0–100 continuous partial-success score per policy**, a binary success flag, episode duration, and free-form evaluator feedback. - Released as periodic **DataDump** snapshots on Hugging Face (Aug 2025, Feb 2026, Jul 2026), so the corpus grows over time rather than freezing. - Published at **CoRL 2025**. Because it captures how policies actually fail in the wild, it is useful well beyond leaderboards — for failure analysis, reward modelling, and preference learning.

Open · MIT · custom