A device that measures the oxygen levels in different brain regions. Researchers have claimed this technology can allow locked-in patients to communicate.
Artinis Medical Systems/Brite
A research group’s claimed ability to communicate with completely paralyzed people has come under fire, prompting research misconduct investigations at a German university and at Germany’s main research agency, the German Research Foundation (DFG). Two years ago, researchers in Germany and Switzerland claimed that, by analyzing blood flow in different parts of the brain with an electronic skullcap, they could elucidate answers to yes or no questions from completely paralyzed people. The find, published in PLOS Biology in 2017, raised hopes for patients with degenerative diseases like amyotrophic lateral sclerosis that ultimately leave them without any voluntary muscle control—not even the ability to blink or move their eyes—a condition called a “completely locked-in state.” Now, a simmering controversy about the paper has erupted into public view.
As first reported by the German newspaper Süddeutsche Zeitung, PLOS Biology yesterday published a critique of the paper that claims the authors’ statistical analysis is incorrect. Martin Spüler, an informatics specialist at the Eberhard Karls University of Tübingen in Germany, says his analysis of the data shows no support for the authors’ claim that their system could allow patients to answer questions correctly 70% of the time. His critique, first raised in late 2017, has prompted investigations of possible scientific misconduct, both at the University of Tübingen and DFG.
Spüler says he originally wanted to test whether a different algorithm could make the method even more accurate, but when he analyzed the data he found that the team had averaged their data in a way that ended up always producing a statistically significant result. “With the statistical tests they use, you will always get a positive answer.” He says his attempts to get explanations from the authors were unsuccessful. “It doesn’t add up,” he says.
The paper’s first author, Ujwal Chaudhary of the University of Tübingen, says Spüler is applying his statistical tests to their data incorrectly and failing to account for biological and clinical circumstances behind their analyses. “You can’t analyze the data without knowing the way we did the experiment and the neurophysiology behind the data,” he says.
The researcher who led the work, Niels Birbaumer of the University of Tübingen and the Wyss Center for Bio and Neuroengineering, in Geneva, Switzerland, says the disagreement is over statistical methods. He is not worried by the misconduct investigations, which he says “are legally absolutely correct.” He says he was happy to answer the investigators’ questions and acknowledges that the data are difficult to interpret. “My conclusion is of course we have to document the responses of these patients much more carefully in order to avoid such a discussion at all,” he says.
Birbaumer’s earlier work has helped severely disabled patients use brainwave devices to choose letters and write messages. He says the paper was a first hint that similar methods might also work with people in a completely locked-in state. “We never said anywhere that we are confident that we can read these people’s thoughts, even their yes or no thoughts. We say this may be a useful first step.” Press coverage of the paper wildly exaggerated the claims, he says. “In the press, it’s completely ridiculous.” Still, he says, at least one patient from the study is still using the device to communicate with his wife.
Reinhold Scherer, who studies brain-computer interfaces and neural engineering at the University of Essex in Colchester, U.K., and who wrote a commentary accompanying the critique, says the team’s claim was always inherently tricky to prove, he says. Trials with locked-in patients are extremely expensive and logistically difficult, he notes, so it is hard for other groups to replicate the work. Birbaumer “is the only one in the community who has the access and the funding to do this research,” he says. The hint that there might be a way to communicate with these patients is a welcome message, he says, “but there’s just not enough evidence that we can definitely say it’s working.”