MIT ultrasound wristband: robot hand mimics human dexterity

MIT researchers have built a wristband that images the wrist and uses AI to translate those images into real-time positions for all five fingers and the palm. The device combines a miniaturized ultrasound "sticker," a hydrogel that sticks to skin, and an algorithm trained on ultrasound images labeled by humans.

How does the wristband work?

The wristband produces ultrasound images of wrist muscles, tendons, and ligaments and an AI model converts those images into finger and palm positions in real time. The hardware pairs miniaturized transducers with a hydrogel adhesive to image the wrist while the wearer moves, then a machine-learning system trained on human-labeled ultrasound frames continuously maps that imaging to the positions of the five fingers and the palm.

Mechanical engineering professor Xuanhe Zhao and colleagues at the Institute and the University of Southern California built the system. The researchers describe the biomechanics this way: "The tendons and muscles in your wrist are like strings pulling on puppets, which are your fingers," says Gengxi Lu, a former MIT postdoc and one of the lead authors.

What did the demonstrations show?

A person wearing the band can wirelessly pilot a robotic hand to mimic gestures, play a simple tune on a piano, and shoot a mini basketball into a desktop hoop. The team demonstrated multiple interaction modes: mirroring gestures on a robotic hand, pinching motion to resize virtual objects on a screen, and direct manipulation of a physical robotic hand. The band translates wrist imaging into synchronized movement of the five fingers and the palm so the robotic hand performs the same gestures in real time.

The researchers say the current wristband hardware is similar in size to a cell phone. They plan to further miniaturize that hardware and to train the AI on movements from more volunteers with a wider variety of hand sizes, finger shapes, and gestures.

Why it matters

Human hands coordinate 34 muscles, 27 joints, and over 100 tendons and ligaments to perform nuanced movements, and existing robot controllers struggle to replicate that dexterity. Imaging the wrist internal anatomy and mapping it directly to finger positions provides a compact, wearable route to high-resolution hand tracking without instrumenting each finger. That makes the approach promising for controlling humanoid robot hands and for direct, intuitive manipulation of virtual objects in design, video games, and virtual reality.

The team envisions a larger data set of hand motions that could train humanoid robots for delicate tasks such as surgical procedures. The combination of wearable imaging and AI could shift how people interact with both physical and virtual robotic hands by putting fine-grained control on the wrist rather than on complex glove systems or vision-only tracking.

What to watch

Watch for a smaller, phone-sized device to shrink to a truly wearable band and for the research team to publish results from training on a broader set of volunteers. Evidence that the approach scales across diverse hand sizes and more complex manipulations, or that the researchers release a large hand-motion data set, would confirm whether the technique can generalize to clinical or industrial robot training.