BRAIN PORT VISION TECHNOLOGY

1. The voice for the Blind

1.1 INTRODUCTION:-

Brain Port is a technology where by sensory information can be sent to one's brain via a signal from the Brain Port (and its associated sensor) that terminates in an electrode array which sits atop the tongue. It was initially developed by Paul Bach-y-Rita as an aid to people's sense of balance, particularly of stroke victims. Brain Port technology has been developed for use as a visual aid. For example, the Brain Port has demonstrated its ability to allow a blind person to see his surroundings in polygonal and pixel form. In this scenario, a camera picks up the image of the surrounding; the information is processed by a chip which converts it into impulses which are sent through an electrode array, via the tongue, to the person's brain. The human brain is able to

1|Page

interpret these impulses as visual signals and they are then redirected to the visual cortex, allowing the person to see. Brain Port technology is based on the phenomenon of sensory substitution. For the vision application, visual information is perceived via the sense of touch on the human tongue. Well, not exactly through her tongue, but the device in her mouth sent visual input through her tongue in much the same way that seeing individuals receive visual input through the eyes. All sensory information sent to the brain is carried by nerve fibers in the form of patterns of impulses, and the impulses end up in the different sensory centers of the brain for interpretation. To substitute one sensory input channel for another, you need to correctly encode the nerve signals for the sensory event and send them to the brain through the alternate channel. The brain appears to be flexible when it comes to interpreting sensory input. You can train it to read input from, say, the tactile channel, as visual or balance information, and to act on it accordingly. In JS On line's "Device may be new pathway to the brain,

What is the Brain Port vision device?

2|Page

The Brain Port vision device is an investigational non-surgical assistive visual prosthetic device that translates information from a digital video camera to your tongue, through gentle electrical stimulation. The Brain Port vision system consists of a postage-stamp-size electrode array for the top surface of the tongue (the tongue array), a base unit, a digital video camera, and a hand-held controller for zoom and contrast inversion. Visual information is collected from the user-adjustable head-mounted camera (FOV range 390 degrees) and sent to the BrainPort base unit. The base unit translates the visual information into an stimulation pattern that is displayed on the tongue. The tactile image is created by presenting white pixels from the camera as strong stimulation, black pixels as no stimulation, and gray levels as medium levels of stimulation, with the ability to invert contrast when appropriate. Users often report the sensation as pictures that are painted on the tongue with Champagne bubbles. With the current system (arrays containing 100 to 600+ electrodes), study participants have been able to recognize high-contrast objects, their location, movement, and some aspects of perspective and depth. Trained blind participants use information from the tongue display to augment understanding of the environment. Ongoing research with the Brain Port vision device demonstrates the great potential of tactile vision augmentation and we believe that these findings warrant further exploration.

Parts of brain port:

Brain Port uses the tongue instead of the fingertips, abdomen or back used by other systems. The tongue is more sensitive than other skin areas -- the nerve fibers are closer to the surface, there are more of them and there is no stratum corneum (an outer layer of dead skin cells) to act as an insulator. It requires less voltage to stimulate nerve fibers in the tongue -- 5 to 15 volts compared to 40 to 500 volts for areas like the fingertips or abdomen. Also, saliva contains electrolytes, free ions that act as electrical conductors, so it helps maintain the flow of current between the electrode and the skin tissue. And the area of the cerebral cortex that interprets touch data from the tongue is larger than the areas serving other body parts, so the tongue is a natural choice for conveying tactile-based data to the brain.

3|Page

An accelerometer is a device that measures, among other things, tilt with respect to the pull of gravity. The accelerometer on the underside of the 10-by-10 electrode array transmits data about head position to the CPU through the communication circuitry. When the head tilts right, the CPU receives the "right" data and sends a signal telling the electrode array to provide current to the right side of the wearer's tongue. When the head tilts left, the device buzzes the left side of the tongue. When the head is level, Brain Port sends a pulse to the middle of the tongue. After multiple sessions with the device, the subject's brain starts to pick up on the signals as indicating head position -balance information that normally comes from the inner information. ear -instead of just tactile

From the CPU, the signals are

sent to the tongue via a "lollipop," an electrode

4|Page

array about nine square centimeters that sits directly on the tongue. Each electrode corresponds to a set of pixels. White pixels yield a strong electrical pulse, whereas black pixels translate into no signal. Densely packed nerves at the tongue surface receive the incoming electrical signals, which feel a little like Pop Rocks or champagne bubbles to the user. A dental-retainer-like unit would house a battery, the electrode array and all of the microelectronics necessary for signal encoding and transmitting. In the case of the Brain Port vision device, the electronics might be completely embedded in a pair of glasses along with a tiny camera and radio transmitter, and the mouthpiece would house a radio receiver to receive encoded signals from the glasses. It's not exactly a system on a chip.

1.2 WORKING OF BRAIN PORT:-

To produce tactile vision, Brain Port uses a camera to capture visual data. The optical information -- light that would normally hit the retina -- that the camera picks up is in digital form, and it uses radio signals to send the ones and zeroes to the CPU for encoding. Each set of

5|Page

pixels in the camera's light sensor corresponds to an electrode in the array. The CPU runs a program that turns the camera's electrical information into a spatially encoded signal. The encoded signal represents differences in pixel data as differences in pulse characteristics such as frequency, amplitude and duration. Multidimensional image information takes the form of variances in pulse current or voltage, pulse duration, intervals between pulses and the number of pulses in a burst, among other parameters. To the extent that a trained user may simultaneously distinguish between multiple of these characteristics of amplitude, width and frequency, the pulses may convey multidimensional information in much the same way that the eye perceives color from the independent stimulation of different color receptors. The electrode array receives the resulting signal via the stimulation circuitry and applies it to the tongue. The brain eventually learns to interpret and use the information coming from the tongue as if it were coming from the eyes.

An experience (research) of a blind woman who go to see the world with the help of this evolutionary device BRAINPORT: A blind woman sits in a chair holding a video camera focused on a scientist sitting in front of her. She has a device in her mouth, touching her tongue, and there are wires running from that device to the video camera. The woman has been blind since birth and doesn't really know what a rubber ball looks like, but the scientist is holding one, and when he suddenly rolls it in her direction, she puts out a hand to stop it. The blind woman saw the ball through her tongue. Well, not exactly through her tongue, but the device in her mouth sent visual input through her tongue

6|Page

in much the same way that seeing individuals receive visual input through the eyes. In both cases, the initial sensory input mechanism the tongue or the eyes send the visual data to the brain, where that data is processed and interpreted to form images.

1.3 CONCEPT OF ELECRTICAL STIMULATION:Electro tactile stimulation for sensory augmentation or substitution, an area of study that involves using encoded electric current to represent sensory information that a person cannot receive through the traditional channel and applying that current to the skin, which sends the information to the brain. The brain then learns to interpret that sensory information as if it were being sent through the traditional channel for such data. Most of us are familiar with the augmentation or substitution of one sense for

7|Page

another. Eyeglasses are a typical example of sensory augmentation. Braille is a typical example of sensory substitution -- in this case, you're using one sense, touch, to take in information normally intended for another sense, vision. Electro tactile stimulation is a higher-tech method of receiving somewhat similar (although more surprising) results, and it's based on the idea that the brain can interpret sensory information even if it's not provided via the "natural" channel.

The idea is to communicate non-tactile information via electrical stimulation of the sense of touch. In practice, this typically means that an array of electrodes receiving input from a nontactile information source (a camera, for instance) applies small, controlled, painless currents (some subjects report it feeling something like soda bubbles) to the skin at precise locations according to an encoded pattern. The encoding of the electrical pattern essentially attempts to mimic the input that would normally be received by the non-functioning sense. So patterns of light picked up by a camera to form an image, replacing the perception of the eyes, are converted into electrical pulses that represent those patterns of light. When the encoded pulses are applied to the skin, the skin is actually receiving image data those nerve fibers forward their image-encoded touch signals to the tactile-sensory area of the cerebral cortex, the parietal lobe. Within this system, arrays of electrodes can be used to communicate non-touch information through pathways to the brain normally used for touch-related impulses. The multiple channels that carry sensory information to the brain, from the eyes, ears and skin, for instance, are set up in a similar manner to perform similar activities. All sensory information sent to the brain is carried by nerve fibers in the form of patterns of impulses, and the impulses end up in the different sensory centers of the brain for interpretation. To substitute one sensory input channel for another, you need to correctly encode the nerve signals for the sensory event and send them to the brain through the alternate channel. The brain appears to be flexible when it comes to interpreting sensory input. You can train it to read input from, say, the tactile channel, as visual or balance information, and to act on it accordingly. In JS Online's "Device may be new pathway to the brain," University of Wisconsin biomedical engineer and Brain Port technology co-inventor Mitch Tyler states, "It's a great mystery as to how that process takes place, but the brain can do it if you give it the right information." Action potentials (AP's) thus recorded had amplitudes from 0.1 to 1.0 mV and a 5 : 1 signal-to-noise ratio (SNR).A circular electrode surrounding the recording site served as the ground reference. Following pre amplification and band pass filtering (200-10 000 Hz), a differential amplitude detector identified AP's, producing an output pulse whenever the recorded signal entered a predefined 8|Page

amplitude-time window. In the first experiment, electro tactile entrainment currents (iEN) were determined by adjusting the stimulation current from near zero to the minimal value resulting in one AP for each stimulation pulse. These currents exceeded the absolute thresholds (the currents causing occasional AP's) by approximately 5%. The entrainment current was determined twice for positive- and negative-polarity stimulation pulses of ten different widths: 20, 30, 40, 50, 70, 100, 150, 200, 300, and 500 _s, delivered at a rate of 10 pulses/s. The width sequence was reversed during the second run on each of the three fibers.

Relative timing between simultaneous mechanical and electrotactile stimulation. The top trace represents the sinusoidal, 30-Hz, 50-100-_m (0-P) mechanical displacement.

1.4 WHAT BRAINPORT USERS SEE:-

9|Page

With the current prototype (arrays containing 100 to 600+ electrodes), study participants have recognized the location and movement of high-contrast objects and some aspects of perspective and depth. In most studies, participants use the device for between two and 10 hours, often achieving the following milestones:
y

Within minutes: users perceive where in space stimulation arises (up, down, left, and right) and the direction of movement

Within an hour: users can identify and reach for nearby objects, and point to and estimate the distance of objects out of reach

Within several hours: users can identify letters and numbers and can recognize landmark information.

The device provides a new sensory language with which users learn to translate the impulse patterns on the tongue to objects in space. Neuro imaging research suggests that using BrainPort stimulates the visual regions of the brain in blind individuals. At present, Brain Port is an investigational prototype and not commercially available. A number of academic and research institutions have had or will have studies using the BrainPort Vision Device with specific participation requirements. Contact Wicab for more information. While Brain Port does not replace vision, it enhances the overall sensory experience and gives users information on the size, shape, and location of objectsperception that will no doubt help blind and visually impaired persons move with greater independence and lead fuller lives. In any case, within 15 minutes of using the device, blind people can begin interpreting spatial information via the Brain Port, says William Seiple, research director at the nonprofit vision healthcare and research organization Lighthouse International. The electrodes spatially correlate with the pixels so that if the camera detects light fixtures in the middle of a dark hallway, electrical stimulations will occur along the center of the tongue.

"It becomes a task of learning, no different than learning to ride a bike," Arnoldussen says, adding that the "process is similar to how a baby learns to see. Things may be strange at first, but over time they become familiar."

10 | P a g e

1.5 RESOLUTION:The resolution of brain port camera varies according to the necessity. The images below demonstrate how information from the video camera is represented on the tongue. Today's prototypes have 400 to 600 points of information on a ~3cm x 3cm tongue display, presented at approximately 30 frames per second, yielding an information rich image stream. Our research

suggests that the tongue is capable of resolving much higher resolution information and we are currently working to develop the optimal tongue display hardware and software.

11 | P a g e

1.6 APPLICATIONS:
The Brain Port test results are somewhat astonishing and lead many to wonder about the scope of applications for the technology

CURRENT APPLICATIONS:The current or foreseeable medical applications include:

y y y y

providing elements of sight for the visually impaired providing sensory-motor training for stroke patients providing tactile information for a part of the body with nerve damage alleviating balance problems, posture-stability problems and muscle rigidity in people with balance disorders and Parkinson's disease

enhancing the integration and interpretation of sensory information in autistic people

POTENIAL APPLICATIONS:
The Brain Port electrodes would receive input from a sonar device to provide not only directional cues but also a visual sense of obstacles and terrain. Military-navigation applications could extend to soldiers in the field when radio communication is dangerous or impossible or when their eyes, ears and hands are needed to manage other things -- things that might blow up. BrainPort may also provide expanded information for military pilots, such as a pulse on the tongue to indicate approaching aircraft or to indicate that they must take immediate action. With training, that pulse on their tongue could elicit a faster reaction time than a visual cue from a light on the dashboard, since the visual cue must be processed by the retina before it's forwarded to the brain for interpretation. Other potential Brain Port applications include robotic surgeries. The surgeon would wear electro tactile gloves to receive tactile input from robotic probes inside someone's chest cavity. In this way, the surgeon could feel what he's doing as he controls the robotic equipment. Race car drivers might use a version of Brain Port to train their brains for

12 | P a g e

faster reaction times, and gamers might use electro tactile feedback gloves or controllers to feel what they're doing in a video game. A gaming Brain Port could also use a tactile-vision process to let gamers perceive additional information that isn't displayed on the screen. Brain Port is currently conducting a second round of clinical trials as it works its way through the FDA approval process for the balance device. The company estimates a commercial release in late 2006, with a roughly estimated selling price of $10,000 per unit. Brain Port envisions itself even smaller and less obtrusive in the future. In the case of the balance device, all of the electronics in the handheld part of the system might fit into a discreet mouthpiece.

1.7 DISADVANTAGES:The major disadvantage to the Brain Port is that you cant see and speak at the same time with the electric array (lollipop) in your mouth. Whatever you do, dont drive and speak on the phone while using this device at the same time.

1.8 CONCLUSION:Brain port is indeed one of the finest and useful technologies. This article offers insights and navigates the action about the pro and cons, of the brain port technology. Technology is a boon in biomedical and can work for all the field like defense, sports, robotics, spy gadgets, and is able to change the life of physically and mentally impaired persons. In comparison to biology, human-machine interface technology is in its early infancy.

13 | P a g e

REFERENCES: Bach-y-Rita, Paul et al. "Form perception with a 49-point electrotactile stimulus array on

the tongue: A technical note." Journal of Rehabilitation Research and Development, 1998. http://kaz.med.wisc.edu/Publications/1998-BachyRita-JRRD-Tongue.pdf
 Blakeslee, Sandra. "New Tools to Help Patients Reclaim Damaged Senses." New York

Times,

Nov.

23,

2004.

http://www.goupstate.com/apps/pbcs.dll/article?AID=/20041123/ZNYT05/411230391/10 51/NEWS01
 Kaczmarek, Kurt, Ph.D. "Tongue Display Technology." University of Wisconsin, Aug.

18, http://kaz.med.wisc.edu/Publicity/Synopsis.html

2005.

 Kupers, Ron et al. "Activation of visual cortex by electrotactile stimulation of the tongue

in

early-blind

subjects."

Human

Brain

Mapping

2003.

http://208.164.121.55/hbm2003/abstract/abstract1557.htm
 Manning, Joe. "Device may be new pathway to the brain." JS Online, Dec. 7, 2004.

http://www.jsonline.com/story/index.aspx?id=282145
 Phone interview with Kurt Kaczmarek, Ph.D., Senior Scientist, University of Wisconsin

Department of Orthopedics and Rehabilitation Medicine. July 7, 2006.

 Ptito, Maurice et al. "Cross-modal plasticity revealed by electrotactile stimulation of the

tongue

in

the

congenitally

blind."

Brain,

2005.

http://brain.oxfordjournals.org/cgi/content/abstract/128/3/606
 U.S. Patent #6,430,450. "Tongue placed tactile output device."  Wicab,

Inc.

http://www.wicab.com/
 "Wicab to present BrainPort at Boston conference." WTN News. Oct. 4, 2005.

http://wistechnology.com/printarticle.php?id=2319

14 | P a g e