Introduction to Adaptive Neurotechnologies

Introduction to Neurotechnologies

The human central nervous system (CNS) is the most convoluted known neural network. The inherent complexity of CNS poses serious challenges to studying its behavior as well as treating its anomalies. The longing for understanding the intricacies of the CNS, in tandem with the ever-increasing need for assisting patients who are suffering from debilitating neurological disorders, has been the principal driving factor for collaboration among engineers and scientists from different disciplines–this collective interdisciplinary endeavor gave rise to the field that is known as neurotechnology. Neurotechnology is an avant-garde, interdisciplinary field that employs state-of-the-art technologies to study and interact with the CNS. From different imaging, recording, and monitoring modalities to innovative neuromodulation techniques, neurotechnology embodies a broad range of methodologies.

Neuroimaging

Imaging is a crucial step in investigating both the physiological structure and the function of the nervous system.

Structural Imaging

Structural imaging techniques allow for studying anatomy, configuration, and structure of the nervous system on different scales. Here we introduce two common imaging techniques for studying the macroscopic structure of the brain and the spinal cord.

Computed Tomography (CT)

Computed tomography (CT) is a noninvasive imaging technique that can produce detailed images of internal organs, bones, soft tissue, and blood vessels (Toga). Although the term "Computed tomography" might refer to a group of imaging techniques such as Positron Emission Tomography or Single-Photon Emission Computed Tomography (in the most general sense) SPECT), here we only use it to refer to X-ray CT scan. A CT of the brain (and in general any other tissue) entails two steps: in the first step, the scanner uses a rotating X-ray beam to generate cross-sectional images—or "slices—of the brain. These 2D images are called tomographic images, and they are more informative than conventional X-rays. In the next step, a computer stacks successive slices and creates a 3D image of the brain. CT scan of the brain allows for easier identification and location of basic structures and possible tumors or abnormalities (Toga).

Magnetic Resonance Imaging (MRI)

Magnetic resonance imaging (MRI) is another noninvasive imaging technique that creates a high-definition 3D image of the brain or other internal organs (Toga). The underlying mechanism of MRI is based on the intrinsic magnetic properties of fundamental particles and, in particular, the hydrogen atoms in water molecules. In the presence of a uniform external magnetic field, a secondary nonuniform magnetic field with a certain gradient can be used to evoke electromagnetic radiation, which will reveal the configuration of these atoms (Bruggen). MRI can be used in two main modes: T1-weighted modes are used to detect the water content of tissues, and T2-weighted modes are used to study the fatty tissues. MIR has a much better resolution compared to CT. While brain CT scan is generally used to diagnose acute bleeding or skull fracture, MRI is the ideal method for visualization of more chronic pathological conditions (Hurley).

Functional Imaging

Functional imaging techniques allow for recording and studying the functionality of different parts of the nervous system. There are two general approaches to functional imaging direct and indirect. The former approach is based on capturing the electrophysiological signals which are a direct indication of neuronal activities, and the latter is based on measuring the hemodynamic signals which are the secondary indication of neuronal activities. Here we briefly introduce two common modalities.

Electroencephalography (EEG)

Electroencephalography (EEG) is a noninvasive recording technique that allows capturing large-scale brain signals from the surface of the scalp. EEG signals primarily reflect the synaptic potential of cortical neuronal populations that fire in a synchronous mode. Due to the excitable nature of neurons measuring their electrical activities is considered a direct approach to study their functioning. The intracranial EEG– also known as Electrocorticography (ECoG)–uses implanted electrodes to capture the electrical activities directly from the surface of the cortex.

functional Magnetic Resonance Imaging (fMRI)

functional Magnetic Resonance Imaging (fMRI) is the most common technique for recording the hemodynamic activity of the nervous system. fMRI measures the blood-oxygen-level-dependent (BOLD) signals which is an indirect approach to identifying which area of the brain has a relatively higher activity. The idea behind these groups of imaging techniques is that measuring the contrast in BOLD signals allows for identifying the more active regions in the network that are consuming higher levels of oxygen and thus receive more blood flow. While fMRI and other recording modalities based on BOLD signals enjoy a high spatial resolution, the indirect nature gives them a lower temporal resolution compared to direct functional imaging techniques such as EEG and ECoG.

Neuromodulation

(To be completed)

Introduction to Adaptive Neurotechnologies

Adaptive neurotechnology (AN) is a well-established branch in the field that allows for developing neuro-assistive and neurorehabilitation systems. At the very core, the functionality of these systems relies on the direct, closed-loop interaction with the nervous system in real-time and requires three basic steps:

Five application area for BCI-based adaptive neurotechnologies

• Recording and processing the neuronal activities, e.g. electrophysiological or hemodynamic signals.
• Translating the biosignals into the desired output, e.g. command for robotic systems or prosthetics.
• Improving performance via neurofeedback and neuromodulation

Fulfillment of these basic steps is typically achieved by Brain-Computer Interfaces (BCI) which are the integral components in AN. Five principal application areas can be envisioned for BCI-based AN:

1. Replacing lost capabilities, such as spelling
2. Restoring lost functions, such as grasping
3. Improving functions, such as upper limb movement
4. Enhancing natural functions, such as error detection
5. Supplementing natural function, such as a future “third hand”

Replacing lost capabilities

Most adults can talk, write an email, move to different rooms, get a glass of water, or perform other everyday tasks without assistance. But millions of people in the US alone have trouble speaking, walking, using their hands, or performing other tasks that require movement. For some of these people, ANs can replace these lost capabilities by replacing the brain’s normal output pathways. Instead of relying on information sent from the brain to the muscles, ANs can detect the user’s mental activity and send those messages or commands to an artificial device. There are many different ways that ANs can replace functions that a user lost due to injury, disease, or other causes. These ways can be viewed as tools for communication or device control.

Communication

One of the most basic communication needs is simply answering questions with “YES” or “NO”. This may seem very limited, but it can make a huge difference in the lives of patients, as well as the people who love them or care for them. Someone might ask questions like “Do you have pain in your left foot?” or “Should I turn on the fan now?” or “Do you want to move to the living room now?” It’s very important to phrase questions carefully so the user can clearly respond with yes or no. Asking a patient if s/he wants to hear music could be confusing. What kind of music? Will you just play the song once, or will you put on background music continuously? Thus, several yes/no questions might be needed to fully determine what the user wants, and people need to make sure the patient can understand the different options that are available. Similarly, asking if a patient would like a chicken enchilada or wants to go swimming can be foolish if the patient can only eat through a feeding tube or has lost the motor control needed to swim. Some ANs have gone beyond yes and no by providing a third option such as “skip”, meaning that the user doesn’t want to answer that question now. Another option is “end”, meaning that the user wants to end the communication session. But, with yes, no, skip, or end, we’re already up to four possible choices that the user could communicate. With only six choices, a user could choose from any letter in the English alphabet or the numbers from 1 to ten. Of course, this requires making two selections. First, the user chooses a group of six items, such as the letters A-H, and then the system could present a second level of choices with those eight letters. Some AN-based spellers use two levels of selection like this. Many systems used for spelling don’t require two levels. For example, the user might see a monitor that has six rows and six columns. You can ask the user to focus on one of these 36 items, and maybe perform a mental task like counting each time a certain letter flashes. ANs could detect the user’s brain activity to determine which of the 36 items s/he wants to communicate. What next? The system might just present answers like yes or no, or more complex messages with spellers, on the monitor. This can be adequate if someone is in the room watching the monitor. However, once you know what someone wants to communicate, it’s relatively easy to convey that message in other ways, such as through a voice synthesizer, email, Twitter, or Facebook. Communication can be more challenging with some users. ANs have been developed for people who aren’t familiar with the English language, such as spellers for Chinese, Japanese, and Arabic speakers. Other ANs have provided communication tools that don’t require vision for people with visual disabilities. Instead of observing items on a monitor, a user might pay attention to different sounds or vibrations, such as stimuli presented to the left vs. right ear or the left vs. right hand. In another approach, people could imagine moving the left hand to convey yes or move a cursor left toward a target letter, or instead imagine moving the right hand to convey no or move a cursor toward a different letter.

Control

ANs can also replace lost functions by helping people control devices, such as a wheelchair, prosthetic arm, or a mobile robot. The mobile robot could go to the kitchen, open the refrigerator, get a glass of water, and bring it to the patient. These control applications have to rely heavily on intelligent software that can manage many complex details that we take for granted. If you want to get a glass of water, you have to figure out the best path to get to the kitchen and back. This requires detecting any obstacles, like chairs or tables, and avoiding them. This may seem easy for people, but robotic devices like wheelchairs and mobile robots need cameras and advanced software to detect and avoid obstacles. Many ANs for control don’t rely on any outside devices beyond a laptop. People could control a web browser or common software applications with AN. Some groups have developed ANs to control games. The commercial game “World of Warcraft” has been used in a few different ANs, since it was a very popular game. In one approach, a user’s character changed from an elf to a bear if the user’s brain activity conveyed stress. Other groups have let users move the game character in three directions, with a fourth option that can perform different tasks depending on the game context. A few companies have released their own games designed to work with BCIs. Ultimately, we’d like to provide ANs that can replace lost functions as completely as possible – meaning that the user can just perform the same mental activities as a healthy person, and get the same results. This is a long way off. Communication with ANs is much slower than typing or talking. Controlling all of the details of a prosthetic hand, including each finger movement, is not yet possible. Moving a wheelchair or robot to another room takes a lot more time than just walking there. However, there has been a lot of progress. The last few years have seen better sensors to detect brain activity, improved interaction with other software, immersive VR-based environments that are designed to be as user-friendly as possible, faster recognition of what the user wants to convey, and improved customization to each individual user. We hope that ongoing research will lead to new ANs that can help meet the needs and desires of the many different people who need help to replace lost functions.

Restoring lost functions

Many people have trouble doing things that were once easy. For example, some people may have intact arms or legs, but can no longer use them because of damage to the brain or spinal cord. Difficulty controlling bladder or bowel activity can also occur. Some ANs can help restore lost functions by sending information directly from the brain to devices that can help their arms, legs, or other areas work properly.

Movement: imagination to reality

For most people, moving the hand is easy. You just think about grasping with your right hand, and it moves just the way you think. But what if the brain areas that produce that movement are damaged? What if the brain is intact, but the spinal cord can no longer send information from the brain to the hand? For these users, and some others with movement disabilities, an orthosis could be the best choice to restore control of movement. An orthosis is a device that can be attached to part of the body, such as a hand, to help that body part complete a movement. Muscle control can also be restored with a device called a functional electrical stimulator, or FES. An FES is a device that sends a weak electrical current through the muscles. This electrical current can make the muscles activate. This is similar to how your nervous system naturally controls muscles. People who want to move usually send a signal from the brain, along the spinal cord, that causes electrical activity to trigger movement. With FES, this electrical activity comes from an outside device instead. FES pads must be placed over specific muscles that are responsible for a movement. For example, placing FES pads placed over the forearm could cause the hand to flex at the wrist. These types of devices can be controlled with ANs that directly read brain activity relating to movement. When you think about grasping with your right hand, the motor areas of your brain are active. In fact, millions of neurons work together for even simple movements. This activity varies a little across different people, and ANs that use motor imagery may need some time to learn each person’s unique brain signals. But, the brain activity involved in different kinds of movements is generally predictable. So, with a few hours of training, people could control an orthosis or FES that makes their right hand grasp or open just by thinking about one of those two movements. While grasping is a common task that ANs can help restore, some people need other types of ANs to control other parts of the body. Some ANs are designed to help people who have trouble walking by stimulating muscles in the feet, legs, and hips. ANs can also control exoskeletons and/or orthoses that influence these parts of the body. The basic principle is the same: when you think about walking, the movement areas of the brain are active in relatively predictable ways. Thinking about walking doesn’t produce exactly the same activity as thinking about moving the hand, so ANs could potentially be used to restore function in different parts of the body. Besides restoring the ability to walk, ANs can also help people with gait. Many people can walk, but have trouble walking smoothly. One major cause is Parkinson’s disease. People with this disease may still be able to walk, but their gait can appear jerky and erratic, because the brain areas that control movement no longer produce the chemicals needed for smooth movements. For these people, walking can be risky, with a high risk of falling. Besides, many patients are understandably uncomfortable walking or performing other movements when other people can see that they have a problem. Aside from restoring basic functionality and safety, ANs can help restore dignity to people who might be embarrassed to invite friends to their homes or go outside. ANs that help restore the ability to walk can be more challenging than ANs for grasping for two reasons. First, ANs for walking are usually more bulky and expensive, because walking requires precise control of many different muscle groups. Second, ANs for walking involve more issues with safety, since mistakes could cause the user to fall.

Bladder and bowel control

Many persons with disabilities cannot control the muscles that are responsible for bladder and bowel control. ANs could be used to activate a device that activates the bladder or bowel muscles based on brain activity. This would not require high-bandwidth communication, since these are activities that people select relatively infrequently, and can be executed with a simple switch mechanism.

Improving functions

Most ANs are designed to provide some benefit very quickly, as part of a real-time interaction. If a user wants to stand up with the help of AN, a delay of even a second between thought and action can be frustrating. ANs for communication, gaming, virtual navigation, web browsing, and many other applications typically focus on accomplishing a user’s immediate goals as quickly as possible. If the user wants some benefit from the AN, then the AN has to be active at that moment to provide that benefit. However, ANs to improve function instead aim to create a lasting improvement in the nervous system – changes that persist even when the AN is no longer being used. Hundreds of millions of people have trouble with movement, memory, attention, or other mental activities. These motor and cognitive impairments could result from stroke, injuries, dementia, Alzheimer’s disease, Parkinson’s disease, or other causes. ANs could help people retrain their nervous systems to counteract damage, leaving them with an improved capability to function even when they are not using ANs anymore. These types of ANs could help healthy people as well, along with ANs to reduce stress.

Rehabilitation

Some ANs aim to help people retrain the brain or spinal cord to work properly again. Millions of people each year participate in physiotherapy, which means that a medical expert called a physiotherapist helps them perform exercises that are designed to help them learn to move again. Often, this physiotherapy involves a device called a functional electrical stimulator, or FES, described in the “restore” section. Physiotherapists may also use a virtual avatar that moves the hand when the user is supposed to move the hand. In the last few years, several groups have described ANs that could provide better results by linking the FES system and other tools to brain activity. That is, the system only activates the FES or the avatar when AN detects the correct movement imagination in the user’s brain. This helps patients learn how to imagine (and then perform) the movement again. Tools like orthoses and exoskeletons can also be important components of a rehabilitation system. Another option for rehabilitation involves the H-reflex. The H-reflex is a natural reaction that people have after certain types of muscle stimulation. ANs can use the H-reflex to help restore spinal cord function that they lost because of an injury. This approach has already shown remarkable results in testing with rats and cats, leading to high hopes for extending this work to help people. ANs are helpful for people with trouble moving and also for people with other problems. Some groups have developed ANs to help people improve their memory. In this approach, the ANs can detect brain activity related to mental tasks like remembering cards or letters. The AN might show people a sequence of letters and then ask: is this the same as the letter that you saw earlier? Like other ANs for rehabilitation, these tools can help people restore neurons involved in memory. Related work could help people who have trouble with attention, emotion, or other mental functions.

Healthy users

These tools to improve function are also being explored for healthy users. Some groups have studied ANs with elite competitive athletes in several different sports. For these users, even tiny improvements could make the difference in winning a competition. More broadly, many people would love to be faster, stronger, nimbler, or smarter. Future ANs might be able to help with these goals, but raise some ethical issues that should be considered further. In addition to improving peak performance, ANs could also reduce the time needed to learn new skills. Research has provided the theoretical basis for learning different types of skills and facts more quickly. People could use this to improve their work productivity or performance, or to learn new games or hobbies. This is also a more futuristic direction for ANs. One application for healthy users is not especially futuristic – relaxation. For decades, people have used different systems that can help them relax. Here, the goal is to provide some improvement that persists when the AN is no longer active, but the improvement is simply feeling more peaceful. Older systems detected tension from electrical activity in the brain and muscles. Newer tools can also use methods like eye tracking and posture analysis. By helping people reduce activities like jaw clenching, teeth grinding, forehead tension, poor posture, and rapid shifts of attention, ANs can improve well-being.