Neuroscience Moving a robot arm with the brain
Researchers study the principles of man-machine collaboration in the virtual world.
For patients suffering from tetraplegia caused by an accident or a disease, it would be a quantum leap towards independence: a robot arm that can be controlled like a real body part. The Emmy Noether research group headed by Dr Christian Klaes endeavours to make this dream come true.
“An arm like that doesn’t bear much resemblance to a real human arm,” explains Klaes. “It would not be attached directly to the user – mounted on the wheelchair, perhaps.” Utilising tried-and-tested industrial robots in such applications is a potential avenue open to science, as they boast the necessary degrees of freedom to perform fine motor movements.
A percutaneous connector links the array to a processor
The arm would receive commands for these movements, such as lifting a cup to the mouth, directly from the patient’s brain. The signals from the neurons involved in the process can be recorded via so-called Utah electrode arrays that are implanted in movement related brain areas. Those electronic components are only four times four millimetres in size and the electrodes are merely one millimetre long. A percutaneous connector links them to a processor that, in turn, controls the robot arm. The arrays had originally been developed for retinal stimulation purposes.
“In paralysed patients, it would make sense to implant three of such arrays permanently,” explains Christian Klaes. “One in the parietal cortex, one in the motor cortex and one in the somatosensory cortex.” These brain regions are responsible for planning and controlling movements as well as processing the neural feedback of movements and haptic sensations.
Having already worked with patients with implants in the USA, Klaes and his team are now awaiting the certification of the Utah arrays for applications in robot assistance systems in the EU. Once they are approved, the researchers will look for patients willing to get implants as part of the study.
Up to five patients may participate in the study. One of the criteria to participate is tetraplegia caused by spinal injury between the third and fourth cervical vertebrae. Currently, these are the only instances where the potential benefits would outweigh the risks. The researchers deliberately chose the university hospital Knappschaftskrankenhaus Bochum located in Bochum-Langendreer in the heart of the Ruhr metropolis as their headquarters, as it facilitates networking and enables them to get in touch with many patients. Moreover, electrodes can be implanted in patients’ brains on location at the hospital’s in-house neurosurgery department, headed by Prof Dr Kirsten Schmieder.
Wireless technology
Waiting for the first patients to be admitted to tests, the research group is currently working with healthy participants in virtual reality settings. The aim is to scrutinize the principles of controlling technical aids with the neural cells of the brain.
One of the aspects investigated by the researchers is the question which neural signals should be translated into movements. The test participants are equipped with virtual reality goggles, which will be wireless in the near future to ensure greater freedom of movement. Then, the participants are invited to perform various tasks with the aid of different controllers – for example handheld controllers, gloves, or a camera that records their hand movements. The researchers monitor the process by measuring the brainwaves with the aid of light wireless EEG caps, in order to ascertain which brain regions are active during a given task.
Another question that is being studied is: does haptic feedback play a significant role in controlling a robot arm? “Anyone who’s ever tried to lift a cup with a numb arm will know how complicated the simple task becomes if visual feedback is all you have when moving your arm,” says Klaes.
The group also collaborates with the epilepsy department at the hospital Knappschaftskrankenhaus, headed by Dr Jörg Wellmer. Here, the patients are temporarily implanted with electrodes in their brains that are meant to help locate the brain region responsible for their seizures. As a side effect, these electrodes can also provide information about the nerve cells that send signals when specific movements are performed.“ Since our virtual reality equipment is fully portable, we can visit the respective patients in their rooms and carry out experiments right then and there,” explains Christian Klaes. Specific electrodes are frequently located in the patients’ hippocampus, a brain region responsible for spatial memory and navigation, with the aim of finding out if and which hippocampus signals might be feasibly used for controlling technical aids.
Artificial intelligence helps to handle data flood
In order to handle the accumulated data flood, the researchers deploy artificial intelligence methods. One of the PhD students involved in the project has specialised in so-called deep learning, which is utilised for filtering useful information from large data volumes.
This year will be the year of virtual reality.
Christian Klaes
In addition to the dream of the robot arm, Klaes and his colleagues can think of another option of helping paralysed patients regain a certain degree of independence, such as an exoskeleton that moves a person’s arms and legs in lieu of muscles. “It might also be possible to move the muscles themselves by applying or implanting electrodes that cause them to contract and relax in turn,” says Christian Klaes. However, muscles tire very quickly under such conditions. The patients are moreover in danger of injuring themselves without realising it, due to particularly forceful movements or impacts.
In spite of those obstacles, Klaes and his team have an optimistic outlook, because the technology is developing at a rapid pace. “This year will be the year of virtual reality,” he reckons. "VR goggles will be hidden in many Christmas stockings.”