In one popular robot training technique, called #imitation #learning, models learn to perform tasks by, for example, imitating the actions of a human teleoperating a robot or using a VR headset to collect data on a robot.
It’s a technique that has gone in and out of fashion over decades but has recently become more popular with robots that do manipulation tasks, says Russ Tedrake, vice president of robotics research at the Toyota Research Institute and an MIT professor.
By pairing this technique with #generative #AI, researchers at the Toyota Research Institute, Columbia University, and MIT have been able to quickly teach robots to do many new tasks.
They believe they have found a way to extend the technology propelling generative AI from the realm of text, images, and videos into the domain of robot movements.
The idea is to start with a human, who manually controls the robot to #demonstrate #behaviors such as whisking eggs or picking up plates.
Using a technique called #diffusion #policy, the robot is then able to use the data fed into it to learn skills.
The researchers have taught robots more than 200 skills, such as peeling vegetables and pouring liquids, and say they are working toward teaching 1,000 skills by the end of the year.
Many others have taken advantage of generative AI as well. #Covariant, a robotics startup that spun off from OpenAI’s now-shuttered robotics research unit, has built a multimodal model called RFM-1.
It can accept prompts in the form of text, image, video, robot instructions, or measurements.
Generative AI allows the robot to both understand instructions and generate images or videos relating to those tasks.
The Toyota Research Institute team hopes this will one day lead to
🔸 “large behavior models,” 🔸which are analogous to large language models, says Tedrake
“A lot of people think behavior cloning is going to get us to a ChatGPT moment for robotics,” he says.
In a similar demonstration, earlier this year a team at Stanford managed to use a relatively cheap off-the-shelf robot costing $32,000 to do complex manipulation tasks such as cooking shrimp and cleaning stains. It learned those new skills quickly with AI.
Called Mobile ALOHA (a loose acronym for “a low-cost open-source hardware teleoperation system”), the robot learned to cook shrimp with the help of just 20 human demonstrations and data from other tasks, such as tearing off a paper towel or piece of tape.
The Stanford researchers found that AI can help robots acquire transferable skills: training on one task can improve its performance for others
https://www.technologyreview.com/2024/04/11/1090718/household-robots-ai-data-robotics/
