Imagine a robot helping you wash dishes. You ask it to grab a soapy bowl, but it misses slightly. Researchers from MIT and NVIDIA found a way to fix this. You can point to the bowl on a screen, trace a path for the robot, or nudge its arm. This method helps the robot understand what you want without needing to retrain its brain.
Normally, robots use a machine-learning model, which is a system that learns from examples. Old methods need new examples and retraining if the robot fails. This takes time and skill. The new framework skips that. It uses real-time feedback, like a nudge, to guide the robot. The robot then picks a valid action, or movement, that matches your goal.
Tests showed this method works better. It beat another approach by 21 percent. The framework could help factory robots do home tasks, even in new places. People shouldn’t need to train robots with data, says Felix Yanwei Wang, the lead researcher. He’s a graduate student in electrical engineering and computer science. His team includes experts from MIT and NVIDIA. They’ll share this at a big robot conference.
Real time feedback
Robots often learn a policy, or set of rules, from a generative AI model. This model makes valid paths for the robot. But these paths don’t always fit what users want. Retraining fixes this, but it’s slow. The new method lets users adjust the robot right away. It offers three ways: pointing, tracing, or nudging. Nudging is best because it keeps all the details clear.
To avoid mistakes, like knocking things over, the method uses sampling. This picks a safe action close to what you want. It blends your input with the robot’s learned rules. Tests in a toy kitchen proved it works well. If the robot picks the wrong bowl, you can fix it instantly. Over time, it could learn from these fixes for next time. The team wants to make it faster and test it in new places. This could make robots easier to use at home.