Get the Mach newsletter.
Researchers in Japan have developed a “telepresence” system that lets one person remotely control a set of robotic arms attached to a backpack worn by another person, effectively allowing two people to work together on complex tasks without being in the same place.
Developed at Keio University and the University of Tokyo, the prototype system consists of a set of human-like robotic arms and a robotic “head” equipped with stereo vision and microphones, all attached to a backpack, which contains control circuitry and a battery.
Real-time data from the robotic head is beamed to a virtual reality headset worn by the other person, who manipulates handheld controllers to operate the robotic arms.
“Thus the user can feel as being ‘fused’ with the surrogate body, and therefore both can share their actions,” MHD Yamen Saraiji, an assistant professor at Keio University and one of the researchers who developed the system, told NBC News MACH in an email.
The “Fusion” system, presented last month at a computer graphics conference in Vancouver, Canada, allows an expert in one location to guide someone in another — for example, showing how to assemble something or giving instruction in the operation of tools or instruments.
The robotic hands can be removed and the robotic wrists temporarily fixed to the surrogate’s own wrists to give the user complete control — an arrangement that could be used to teach someone to play a musical instrument, for example.
“There are also potential applications in assistive robotics, for example, by providing extra support or coordination to someone with a motor impairment like weakness or a tremor,” Henny Admoni, an assistant professor at Carnegie Mellon University’s Robotics Institute, told MACH in an email. “The potentially provocative aspect of this robot,” she added, “is that it’s designed to also allow a remote operator to physically guide the wearer.”
This isn’t the first supplemental arm system that has been developed. In 2014 researchers at the Georgia Institute of Technology demonstrated a wearable third arm that could play the drums, and last year researchers at Cornell University unveiled a third arm that could assist with a variety of tasks. But unlike the Fusion system’s arms, these other arms weren’t designed to be controlled by a remote user.
Saraiji said the researchers hope to create a commercial version of the Fusion system once more development and testing are completed. But given the complexities, Admoni said she would be surprised if it became a commercial technology “anytime soon.”