Description
Dobb·E is an open-source framework that enables teaching robots to perform household tasks using imitation learning. It aims to overcome the limitations of current home robotics by offering an affordable and user-friendly way to gather demonstrations.
This is accomplished using a tool called the Stick, which consists of a $25 Reacher-grabber stick, 3D printed components, and an iPhone. Dobb·E utilizes the Stick to gather data from the Homes of New York (HoNY) dataset, comprising 13 hours of interactions across 22 homes in New York City.
The dataset features RGB and depth videos, along with action annotations for the gripper's 6D pose and opening angle. Utilizing this collected data, Dobb·E trains a representation learning model known as Home Pretrained Representations (HPR).
This model, which uses a ResNet-34 architecture and is trained with self-supervised learning objectives, initializes a robot policy for executing new tasks in different environments. Dobb·E has shown an average success rate of 81% in completing new tasks within 15 minutes, based on five minutes of data collected in a new home. Access to pre-trained models, code, and documentation is available through GitHub.
Furthermore, the open-access paper On Bringing Robots Home offers additional insights into Dobb·E's methodology and results.
