MIT Masked IRL: LLMs help robots clarify and ignore cues

MIT’s Computer Science and Artificial Intelligence Laboratory introduced Masked Inverse Reinforcement Learning, or Masked IRL, on June 26, 2026, a two-LLM pipeline that clarifies ambiguous user prompts and masks irrelevant environmental details so robots can learn chores using nearly five times less demonstration data. The team says the system identified unstated user preferences up to 15 percent more often than comparable baselines.

What is Masked IRL and how does it work?

Masked IRL is a two-stage system that first uses one large language model to elaborate unclear prompts and compare a recorded trajectory to the shortest path, then uses a second LLM to mask irrelevant environment details and pass only important factors to a motion planner. The pipeline starts from kinesthetic demonstrations, records trajectories and sensor data, has LLM1 expand vague instructions into concrete constraints, and has LLM2 score environment elements as "1" (important) or "0" (not so much) before an algorithm incorporates the chosen details into an action plan.

The system treats demonstrations as trajectories and asks the first LLM to turn vague phrases such as "stay close" into specific constraints like "stay close to the surface of the table." The second LLM inspects obstacle positions and object shapes and effectively ignores items deemed irrelevant, for example marking whether a user leaning on a table is relevant with a "0." The result feeds into an inverse reinforcement learning approach that produces robot motion plans.

How well did it perform in tests?

Masked IRL required fewer demonstrations than comparable baselines and improved correctness on unstated preferences by up to 15 percent, while showing strong real-world transfer after training on 50 kinesthetic demonstrations. In simulation and real-world demos the approach helped virtual and physical robots maneuver objects around obstacles, for example moving a coffee mug around a laptop to different spots on a table, and correctly identified users' preferences more often than the baselines by up to 15 percent.

Researchers write that the robots learned faster: the system used nearly five times less demonstration data to reach similar task understanding compared with prior methods. In a real robotic-arm evaluation, after 50 kinesthetic demonstrations the arm moved a cup toward a human while avoiding a user's computer, wiped a table while "staying close" to it, and handed a bag of chips while "staying away" from both a human and a table. The team attributes these gains to the LLMs' ability to clarify vague instructions and filter irrelevant scene details before planning.

Why it matters

Masked IRL reduces the burden on humans who would otherwise need to provide many demonstrations or detailed instructions, making robot teaching more practical for homes, offices and factories. By separating clarification and masking tasks across two LLMs, the system focuses the planner on the details that change behavior, which improves safety when robots must navigate near humans, laptops or shelving. The method also points toward using multimodal sensing, since the researchers plan to add cameras so a robot can highlight and ignore objects it sees.

One of the lead authors, MIT PhD student Minyoung Hwang, framed the goal succinctly: "We’re minimizing human effort by enabling machines to get to the bottom of what users really want." The paper lists coauthors Alexandra Forsey-Smerek, Nathaniel Dennler, and Andreea Bobu, and notes support from the Tata Group via the MIT Generative AI Impact Consortium Award and from the Department of Defense.

What to watch

The team will present Masked IRL at the 2026 IEEE International Conference on Robotics and Automation in June, and they plan to extend the system with cameras so a robot can visually select which nearby elements to mask. Watch for demonstrations that combine language clarification with image-based attention to see whether visual cues further reduce required demonstrations and improve real-world robustness.