New system lets you control robots with brain waves and hand gestures

MIT scientists have developed a system that allows humans to control robots using brainwaves and simple hand gestures, preventing machines from committing errors in real time. By monitoring brain activity, the system can detect in real time if a person notices an error as a robot does a task. Using an interface that measures muscle activity, the person can then make hand gestures to scroll through and select the correct option for the robot to execute.

The team from Massachusetts Institute of Technology (MIT)'s Computer Science and Artificial Intelligence Laboratory (CSAIL) in the US demonstrated the system on a task in which a robot moves a power drill to one of three possible targets on the body of a mock plane. They showed that the system works on users it has never interacted with before, meaning that organisations could deploy it in real-world settings without needing to train it on users.

"This work combining EEG and EMG feedback enables natural human-robot interactions for a broader set of applications than we've been able to do before using only EEG feedback," said CSAIL director Daniela Rus, who supervised the work.

"By including muscle feedback, we can use gestures to command the robot spatially, with much more nuance and specificity," said Rus. In previous research, systems could generally only recognise brain signals when people trained themselves to "think" in very specific but arbitrary ways and when the system was trained on such signals. For instance, a human operator might have to look at different light displays that correspond to different robot tasks during a training session.

Such approaches are difficult for people to handle reliably, especially if they work in fields like construction or navigation that already require intense concentration. Meanwhile, the team harnessed the power of brain signals called "error-related potentials" (ErrPs), which researchers have found to naturally occur when people notice mistakes. "What's great about this approach is that there's no need to train users to think in a prescribed way. The machine adapts to you, and not the other way around," said Joseph DelPreto, a PhD candidate at CSAIL.

For the project the team used "Baxter", a humanoid robot from Rethink Robotics. With human supervision, the robot went from choosing the correct target 70 percent of the time to more than 97 percent of the time. To create the system the team harnessed the power of electroencephalography (EEG) for brain activity and electromyography (EMG) for muscle activity, putting a series of electrodes on the users' scalp and forearm.