RoboSet (RoboAgent)
RoboSet is a large-scale real-world multi-task robot manipulation dataset collected on Franka-Emika Panda arms with Robotiq grippers across everyday kitchen scenes, combining teleoperated (9,500) and kinesthetic-playback (19,000) demonstrations. It spans 12 manipulation skills expressed across 38 tasks, recording RGB and depth frames from 4 camera views (top, left, right fixed plus a wrist camera) along with Franka joint positions/velocities, end-effector/gripper pose/velocities, and applied controls. It was released alongside the RoboAgent / MT-ACT system (CMU and Meta AI/FAIR). Data is stored as HDF5 trials containing image, proprioception, and action fields.
- Scale
- 28500 episodes
- Formats
- hdf5
- License
- MIT
- Published
- 2023-09-05
Decision summary
teleoperation
Metadata requires review against the official source before publication.
Read the dataset manifest and feature schema.
Catalog assessment
Selection evidence
77/100
Provisional · confidence 48 · 4/6 evaluated
Access and governance
Access and license are declared by the source.
Schema and signal coverage
No machine-readable schema has been verified yet.
Policy training readiness
Observation and action/state signals are declared; alignment quality still depends on sample verification.
View 3 more dimensions
World-model readiness
Temporal observations plus geometry or semantic context are declared; sample alignment remains to be audited.
Failure and recovery readiness
No verified failure/recovery annotation evidence is available yet.
Download and processing readiness
Scale is declared; transfer and processing estimates are not measured.
Review unresolved evidence and next checks
Signal gaps
- Gaze / attention. Not enough evidence is available to classify this signal.
- Feedback / correction / failure. Not enough evidence is available to classify this signal.
Next checks
- Read the dataset manifest and feature schema.
- Run a bounded sample audit before assigning Strong readiness.
Record specifics
Dataset facts
- Source
- Carnegie Mellon University and Meta AI
- Evidence
- secondary claim
- Formats
- hdf5
- episodes
- 28500
- tasks
- 38
Metadata coverage
Loop signals
5/7 present or partial
No decision-grade evidence captured yet.
unknownNo decision-grade evidence captured yet.
unknownView 5 more signal categories
depth · video · RoboSet is a large-scale real-world multi-task robot manipulation dataset collected on Franka-Emika Panda arms with Robotiq grippers across everyday kitchen scenes, combining teleoperated (9,500) and kinesthetic-playback (19,000) demonstrations. It spans 12 manipulation skills expressed across 38 tasks, recording RGB and depth frames from 4 camera views (top, left, right fixed plus a wrist camera) along with Franka joint positions/velocities, end-effector/gripper pose/velocities, and applied controls. It was released alongside the RoboAgent / MT-ACT system (CMU and Meta AI/FAIR). Data is stored as HDF5 trials containing image, proprioception, and action fields.
presentee_pose · RoboSet is a large-scale real-world multi-task robot manipulation dataset collected on Franka-Emika Panda arms with Robotiq grippers across everyday kitchen scenes, combining teleoperated (9,500) and kinesthetic-playback (19,000) demonstrations. It spans 12 manipulation skills expressed across 38 tasks, recording RGB and depth frames from 4 camera views (top, left, right fixed plus a wrist camera) along with Franka joint positions/velocities, end-effector/gripper pose/velocities, and applied controls. It was released alongside the RoboAgent / MT-ACT system (CMU and Meta AI/FAIR). Data is stored as HDF5 trials containing image, proprioception, and action fields.
partiallanguage · RoboSet is a large-scale real-world multi-task robot manipulation dataset collected on Franka-Emika Panda arms with Robotiq grippers across everyday kitchen scenes, combining teleoperated (9,500) and kinesthetic-playback (19,000) demonstrations. It spans 12 manipulation skills expressed across 38 tasks, recording RGB and depth frames from 4 camera views (top, left, right fixed plus a wrist camera) along with Franka joint positions/velocities, end-effector/gripper pose/velocities, and applied controls. It was released alongside the RoboAgent / MT-ACT system (CMU and Meta AI/FAIR). Data is stored as HDF5 trials containing image, proprioception, and action fields.
presentdepth · RoboSet is a large-scale real-world multi-task robot manipulation dataset collected on Franka-Emika Panda arms with Robotiq grippers across everyday kitchen scenes, combining teleoperated (9,500) and kinesthetic-playback (19,000) demonstrations. It spans 12 manipulation skills expressed across 38 tasks, recording RGB and depth frames from 4 camera views (top, left, right fixed plus a wrist camera) along with Franka joint positions/velocities, end-effector/gripper pose/velocities, and applied controls. It was released alongside the RoboAgent / MT-ACT system (CMU and Meta AI/FAIR). Data is stored as HDF5 trials containing image, proprioception, and action fields.
presentOpen · MIT · hdf5
partialEvidence details and provenance
Official signal claims
Schema and annotations
No machine-readable schema facts are captured.
Sample verification
Pending. Metadata does not prove sample coverage, alignment, or file integrity.
curated official source
Integration notes
- Metadata requires review against the official source before publication.
Catalog links