Robot swarms could revolutionize everything from search and rescue missions to mining, but figuring out how to oversee so many moving parts is tough. A new approach lets people control tiny, semiautonomous robots using virtual reality (VR)—much like a child might herd a swarm of crawling ants.
After donning VR goggles, the operator hovers above a virtual arena containing up to nine robots that can independently navigate and coordinate with each other. Three of these virtual robots also have real-world counterparts nearby. To move the robots, the user simply reaches out in the simulation and picks them up. The user can also guide them—or trap them—by building walls with a pinch-and-drag motion (above).
Those movements are picked up by Leap Motion gesture-tracking technology attached to the outside of the headset, and the instructions are sent to the real-world bots via radio signals. Simple gesture controls also allow the user to zoom in and out of the virtual environment and move about the simulation.
VR and gesture tracking are already popular tools for controlling robots, but the new approach—combining them so that users can directly interact with large numbers of robots—is uniquely intuitive, the researchers say. The new system allowed untrained operators to corral 50 free-roaming virtual robots into three different areas in about 5 minutes, they report this week on the preprint server arXiv. In surveys afterward, users were broadly positive about the system.
Real world applications still face many challenges, including capturing enough data to create realistic simulations and transmitting them quickly enough to prevent the simulation from lagging. But the researchers say forthcoming 5G mobile communication technology could be the missing ingredient.