DeepMind, the renowned AI research lab, has made significant strides in the field of robotics with the development of RoboCat, an advanced AI model. What sets RoboCat apart is its remarkable ability to solve and adapt to multiple tasks using various real-world robots—a breakthrough achievement in the realm of robotics.
According to Alex Lee, a research scientist at DeepMind and a key contributor to the RoboCat project, “We demonstrate that a single large model can solve a diverse set of tasks on multiple real robotic embodiments and can quickly adapt to new tasks and embodiments.”
Inspired by Gato, another successful AI model by DeepMind that excels in text, image, and event analysis, RoboCat underwent extensive training on a combination of simulated and real-world robotics data. The dataset comprised images and actions collected from different robot-controlled models within virtual environments, as well as data from humans controlling robots and previous versions of RoboCat itself.
The training process began by collecting 100 to 1,000 demonstrations of specific tasks performed by a robotic arm controlled by a human. Subsequently, RoboCat was fine-tuned on each task, leading to the creation of specialized “spin-off” models that underwent an average of 10,000 practice iterations for further refinement.
The researchers leveraged both the data generated by the spin-off models and the demonstration data to continually expand RoboCat’s training dataset. Multiple iterations of RoboCat were trained using this approach.
In its final form, RoboCat was trained on a remarkable total of 253 tasks and benchmarked on 141 variations of these tasks, both in simulation and the real world. DeepMind claims that after observing 1,000 human-controlled demonstrations, RoboCat successfully acquired the ability to operate different types of robotic arms.
Despite its impressive capabilities, RoboCat’s success rate in various tasks during DeepMind’s testing exhibited a wide range, ranging from 13% to 99%. Notably, the number of demonstrations had a predictable impact on the success rate, with fewer demonstrations resulting in less frequent successes.
Nevertheless, DeepMind reports that in some scenarios, RoboCat achieved proficiency in new tasks with as few as 100 demonstrations.
Looking ahead, Alex Lee envisions a future where RoboCat could significantly reduce the number of demonstrations required to teach it new tasks, aiming for a groundbreaking milestone of fewer than 10 demonstrations.
The research team’s ongoing efforts to refine RoboCat hold promise for revolutionizing the robotics field, potentially lowering the barriers to solving new tasks and empowering robots to adapt swiftly to diverse challenges.
