Jim Fan, a researcher at Nvidia who has collaborated with Goldberg, says the way AI can now control robots mirrors how software agents function. LLMS can use models trained to control physical actions—visual language action (VLA) models—just as they access different software on a computer.
This week, Fan, Goldberg, and others open sourced an agentic framework for robots called Aspire, which allows a robot to discover and use new skills. One example: GPT-6 Astra figured out how to use a pair of robot arms to solve a Rubik’s cube.
“It picks it up, tries to twist it, but has the wrong angle,” Fan says. “Then it reasons for a very, very long time, but it eventually is able to solve it because it learns the skills over time.”
At the Conference on Robot Learning (CoRL), held in Austin, Texas, this week, Fan gave a keynote speech in which he talked about the incredible realization that AI models can automate a lot of the work needed to have a robot do sophisticated things. Fan showed off a demo in which Astra was put in control of a robot arm and told to paint a picture of the Golden Gate Bridge. Using some paint and brushes nearby, it duly completed the task.
Of course, getting robots to work reliably and efficiently is still extremely difficult, requiring significant engineering expertise. The vision system on my robot duck does not work well enough to be deployed in a real product.
Still, the ease and speed of training robots with AI could soon have an enormous impact on manufacturing and e-commerce, and Goldberg says a number of industrial robotics and automation firms have shown an interest in the approach.
Much like vibe coding a few years ago, we may well see breakthroughs in “vibe robotics” as hobbyists and engineers realize how easy it is to program and train robots. And, assuming I can perfect my robot helper, Leona may never have to go without treats again.
This is an edition of Will Knight’s AI Lab newsletter. Read previous newsletters here.