2010 Horizon Report

Brain-Computer Interfaces

Perhaps the ultimate computer interface, and one that remains some way off, is mind control.

Surgical implants or electroencephalogram (EEG) sensors can be used to monitor the brain activity of people with severe forms of paralysis. With training, this technology can allow "locked in" patients to control a computer cursor to spell out messages or steer a wheelchair.

Some companies hope to bring the same kind of brain-computer interface (BCI) technology to the mainstream. Last month, Neurosky, based in San Jose, CA, announced the launch of its Bluetooth gaming headset designed to monitor simple EEG activity. The idea is that gamers can gain extra powers depending on how calm they are.

Beyond gaming, BCI technology could perhaps be used to help relieve stress and information overload. A BCI project called the Cognitive Cockpit (CogPit) uses EEG information in an attempt to reduce the information overload experienced by jet pilots.

The project, which was formerly funded by the U.S. government's Defense Advanced Research Projects Agency (DARPA), is designed to discern when the pilot is being overloaded and manage the way that information is fed to him. For example, if he is already verbally communicating with base, it may be more appropriate to warn him of an incoming threat using visual means rather than through an audible alert. "By estimating their cognitive state from one moment to the next, we should be able to optimize the flow of information to them," says Blair Dickson, a researcher on the project with U.K. defense-technology company.

 

 

Brain-computer interface (BCI) is a direct connection between computer(s) and the human brain. It is the ultimate in development of human-computer interfaces or HCI. Recently advances have been made with Brain-Machine Interfaces (BMI).

After a blood vessel popped in his brain, a patient lost his ability to move and speak. On The Learning Channel's "Understanding," see how revolutionary medical technology has restored his ability to communicate through a computer. (October, 2008)

Currently research is being conducted the fields of neuroscience and neuroengineering regarding BCI and BMI. Using chips implanted against the brain that have hundreds of pins less than the width of a human hair protruding from them and penetrating the cerebral cortex, scientists are able to read the firings of hundreds of neurons in the brain. The language of the neural firings is then sent to a computer translator that uses special algoriths to decode the neural language into computer language. This is then sent to another computer that receives the translated information and tells the machine what to do. Applications of this technology range from protheses to control of robotic UAVs to non-verbal human communication.

As far as real-world testing of this technology, the majority has been conducted using rats and monkeys Neuroprosthetics is an area of neuroscience concerned with neural prostheses—using artificial devices to replace the function of impaired nervous systems or sensory organsin laboratories. Using the rewards / punishment system researchers train animals to do a certain task with their bodies, and then, using the chip, the animal eventually figures out it doesn’t actually have to do the task, it just has to think the task, and the reward will be received.

There are other means of reading brain activity than direct neural contact via pins. The first and most common is electroencephalography (EEG) where electrodes are placed against the scalp are used to pick up brain signals. However, this approach is not nearly as accurate as direct neural contact and can only pickup blurry, weak readings. The other, much newer, and much more accurate non-invasive technology is magnetoencephalography (MEG) but is also more equipment intensive. Using MEG requires a room filled with super-conducting magnets and giant super-cooling helium tanks surrounded by shielded walls. This technology, while providing the speed and accuracy needed for a successful non-invasive BMI, will require significant improvement of technology in order to be realistic for everday use.

Gesture-based interfaces are changing the way we interact with computers,

giving us a more intuitive way to control devices. They are increasingly built into things we can already use; Logitech and Apple have brought gesture-based mice to market, and Microsoft is developing several models. Smart phones, remote controls, and touch-screen computers accept gesture input. As more of these devices are developed and released, our options for controlling a host of electronic devices are expanding. We can make music louder or softer by moving a hand, or skip a track with the flick of a finger. Apple’s Remote app for the iPhone turns the mobile device into a remote control for the Apple TV; users can search, play, pause, rewind, and so on, just by gliding a finger over the iPhone’s surface. Instead of learning where to point and click and how to type, we are beginning to be able to expect our computers to respond to natural movements that make sense to us.

Devices that can accept multiple simultaneous inputs (like using two fingers on the Apple iPhone or the Microsoft Surface to zoom in or out) and gesture-based inputs like those used on the Nintendo Wii have begun to change the way we interact with computers. We are seeing a gradual shift towards interfaces that adapt to—or are built for—humans and human gestures. The idea that natural, comfortable movements can be used to control computers is opening the way to a host of input devices that look and feel very different from the keyboard and mouse.

Gesture-based computing allows users to engage in virtual activities with motion and movement similar to what they would use in the real world. Content is manipulated intuitively, making it much easier to interact with, particularly for the very young or for those with poor motor control. The intuitive feel of gesture-based computing is leading to new kinds of teaching or training simulations, that look, feel, and operate almost exactly like their real-world counterparts. Larger multi-touch displays support collaborative work, allowing multiple users to interact with content simultaneously, unlike a single-user mouse.

Relevance for Teaching, Learning & Creative Expression

• Researchers at Georgia Tech University have developed gesture-based games designed to help deaf children learn linguistics at the critical time of language development.
• Using off-the-shelf existing technologies, the Sixth Sense project from MIT provides a gesture interface that can be used to augment information into real world spaces.
• After discovering the significant improvement in dexterity that surgeons-in-training gained from playing with the Wii (48%), researchers are developing a set of Wii-based medical training materials.

Jeff Han demos his breakthrough touchscreen