CHAPTER 1

INTRODUCTION
1.1 Introduction
Research on braincomputer interfaces (BCIs) has approximately 40 years during which time the potential of BCI applications has been explored in many fields, including: communication, robotics, and mobility control, and neuroprosthetics. The BCI translates intentions directly into machinery commands, bypassing the conventional neural muscular conduction pathway in the human body. This enables patients with paralysis to regain independence, improve their quality of life, and potentially reduce the great economic cost caused by the continuous needs of care givers. Our group has been working on electroencephalography (EEG)-based BCIs aiming at developing 2-D control BCIs for potential applications such as wheelchair control. With such BCIs the users should be able to control the wheelchair at their own pace and the control is quickly attained. To that end, users learn how to voluntarily modulate the different oscillatory rhythms through execution of physical or imagined motor tasks and quickly develop control of the BCI. To facilitate the learning process, we rely on both machine learning techniques to train/select effective classifiers that yield high classification accuracies using subject-specific EEG features and incorporating new physiological features to essentially enhance the difference/separability between patterns generated by mental tasks.

CHAPTER 2

KEYWORDS
2.1 BRAIN-COMPUTER INTERFACE (BCI)
Brain-Computer Interface (BCI) is a communication system, which enables the user to control special computer applications by using only his or her thoughts. Different research groups have examined and used different methods to achieve this. Almost all of them are based on electroencaphalography (EEG) recorded from the scalp. The EEG is measured and sampled while the user imagines different things (for example, moving the left or the right hand). Depending on the BCI, particular preprocessing and feature extraction methods are applied to the EEG sample of certain length. It is then possible to detect the task-specific EEG signals or patterns from the EEG samples with a certain level of accuracy. First signs of BCI research can be dated back to 1960s, but it was in 1990s when the BCI research really got started. Faster computers and better EEG devices offered new possibilities. To date there have been over 20 BCI research groups. They have taken different approaches to the subject, some more successful than others. Less than half of the BCI research groups have build an online BCI, which can give feedback to the subject. None of the BCIs have yet become commercial and only a couple have been tested outside laboratory environments. Despite the technological developments numerous problems still exists in building efficient BCIs. The biggest challenges are related to accuracy, speed and usability. Other interfaces are still much more efficient. If a disabled person can move eyes or even one muscle in a controlled way, the interfaces based on eye-gaze or EMG switch technology are more efficient than any of the BCIs today. However, BCI could provide a new communication tool for people suffering from so called locked-in syndrome. They are completely paralyzed physically and unable to speak, but cognitively intact and alert. Locked-in syndrome can be caused, for example, by amyotrophic lateral sclerosis (ALS), high-level spinal cord injure or brain stem stroke. In its severest form people are not able to move any muscle in their body.

2.2 ELECTROENCEPHALOGRAPHY (EEG)

Electroencephalography (EEG) is a method used in measuring the electrical activity of the brain. This activity is generated by billions of nerve cells, called neurons. Each neuron is connected to thousands of other neurons. Some of the connections are excitatory while others are inhibitory. The signals from other neurons sum up in the receiving neuron. When this sum exceeds a certain potential level called a threshold, the neuron fires nerve impulse. The electrical activity of a single neuron cannot be measured with scalp EEG. However, EEG can measure the combined electrical activity of millions of neurons . The temporal resolution of EEG is very good: millisecond or even better. However, the spatial resolution is poor. It depends on the number of electrodes, but the maximum resolution is in centimeter range whereas, for example, in MEG, PET or fMRI it is in millimeter range . The ongoing EEG is characterized by amplitude and frequency. The amplitudes of the EEG signals typically vary between 10 and 100 _V (in adults more commonly between 10 and 50 _V). The electrical activity goes on continuously in every living humans brain. We may sleep one third of our life times, but the brain never rests. Even when one is unconscious the brain remains active. Much of the time, the brain waves are irregular and no general pattern can be observed . Allison lists four prerequisites, which must be met for the activity of any network of neurons to be visible in EEG signal: 1) The neurons must generate most of their electrical signals along a specific axis oriented perpendicular to the scalp; 2) The neuronal dendrites must be aligned in parallel so that their field potentials summate to create a signal which is detectable at a distance; 3) The neurons should fire in near synchrony; 4) The electrical activity produced by each neuron needs to have the same electrical sign. All this means that an overwhelming majority of neuronal communication is practically invisible in EEG. However, there exists various properties in EEG, which can be used as a basis for a BCI: 1. Rhythmic brain activity 2. Event-related potentials (ERPs) 3.Event-related desynchronization (ERD) and event-related

synchronization (ERS).
3

2.3 EVENT-RELATED DESYNCHRONIZATION (ERD) AND EVENT-RELATED SYNCHRONIZATION (ERS)

Event-related desynchronization (ERD) and event-related synchronization (ERS) can be defined as follows:

1. Event-related desynchronization (ERD) is an amplitude attenuation of a certain EEG rhythm.

2. Event-related synchronization (ERS) is an amplitude enhancement of a certain EEG rhythm.

In order to measure an ERD or an ERS, the power of a certain frequency band (for example, 8-12 Hz) is calculated before and after certain event over a number of EEG trials. The event can be externally-paced (such as light stimulus) or internally paced (such as voluntary finger movement). The power (averaged over a number of trials) is then measured in percentage relative to the power of the reference interval. The reference interval is defined, for example, as 1 second interval between 4.5 and 3.5 seconds before the event (i.e. during the rest). The ERS is the power increase (in percents) and the ERD is the power decrease relative to the reference interval (which is defined as 100 %). To keep the power at the reference interval at the resting level, the interval between two consecutive events should be random and not shorter than a few seconds.

ERD and ERS can be presented in time and space. Figure 2.11 presents power time courses for three different frequency ranges (10-12 Hz, 14-18 Hz and 36-40 Hz). EEG was recorded over the electrode position C3 during the right index lifting. Vertical line at t=0 presents the movement onset. Figure 2.2 displays ERD maps for the left and the right motor imagery obtained from a single subject. Using single trial-EEG data, distribution of the alpha band (9-13 Hz) ERD was calculated. The maps are shown at t=625 ms after the representation of the cue

Fig 2.1: ERD time courses computed for three different frequency bands from EEG trials recorded from electrode position C3 during right index finger lifting
.

Fig 2. 2: ERD maps for a single subject for the cortical surface of a realistic head model. The distribution of the alpha band (9-13 Hz) ERD was calculated for left and right motor imagery.
5

CHAPTER 3

MEASURING EEG

3.1 MEASURING EEG

In the scalp EEG the electrical activity of the brain is recorded non-invasively, i.e. from the surface of the scalp using normally small metal plate electrodes. While the number of the electrodes varies from study to study, they are usually arranged according to an international 10-20 system. Recordings can be made either using reference electrode(s) or bipolar linkages. The EEG signal can be affected by many artifacts coming from the equipment or the subject.

3.2 ELECTRODES
The EEG is recorded with electrodes, which are placed on the scalp. Electrodes are small plates, which conduct electricity. They provide the electrical contact between the skin and the EEG recording apparatus by transforming the ionic current on the skin to the electrical current in the wires. To improve the stability of the signal, the outer layer of the skin called stratum corneum should be at least partly removed under the electrode. Electrolyte gel is applied between the electrode and the skin in order to provide good electrical contact. Usually small metal-plate electrodes are used in the EEG recording.

3.3 ELECTRODE PLACEMENTS

In order to make patients records comparable over time and to other patients records, a specific system of electrode placement called International 10-20 system is used. The system is for 21 electrodes. The distance between the specific anatomic landmarks (nasion and inion, see Figure 3.1) is measured after which the electrodes are placed on the scalp using 10 and 20 % interelectrode distances. Each electrode position has a letter (to identify the underlying brain lobe) and a number or another letter to
6

identify the hemisphere location. Odd numbers are on the left side and even on the right side. Z (for zero) refers to electrode placements at midline. The system allows the use of additional electrodes. As can be seen in Figure 3.1 midline (or zero) electrodes are flanked up by electrodes numbered 3 on the left and 4 on the right.

Fig 3.1: The international 10-20 electrode system: side and top views.

CHAPTER 4

METHOD
4.1 SUBJECT
Five right-handed healthy subjects (S1S5, two females and three males) age 23 30 participated in this study. S2 had 6 hours BCI experience half a year ago for another irrelevant study; S1, S3S5 had no previous BCI experience. All the subjects were participating for the first time in this study, and gave informed consent. The protocol was approved by the Institutional Review Board.

4.2 DATA RECORDING

EEG was recorded from 27 (tin) surface electrodes (F3, F7, C3A, C1, C3, C5, T3, C3P, P3, T5, F4, F8, C4A, C2, C4, C6, T4, C4P, P4, T6, FPZ, FZ, FCZ, CZ, CZP, PZ, and OZ) attached on an elastic cap (Electro-Cap International, Inc., Eaton, OH) according to the international 1020 system, with reference from the right ear lobe and ground from the forehead. Two surface electromyography (EMG) electrodes were taped over the right and left wrist extensors, used to monitor the hand movement. Electrodes for bipolar electrooculogram (EOG) above left eye and below right eye were also pasted. Total duration of preparation for obtaining consent, explaining paradigm, and setting up electrodes took around 30 min to 1 h. Signals from all the channels were amplified (g.tec GmgH, Schiedlberg, Austria), filtered (0.1100 Hz) and digitized (sampling frequency was 256 Hz). All impedances were kept below 5 k . The digital signal was processed online using a custom-made MATLAB (MathWorks, Natick, MA) Toolbox: BCI to virtual reality or BCI2VR. The BCI2VR programs provided both the visual stimulus for the calibration and the 2-D wheelchair-control game, as well as online processing of the EEG signal.

4.3 EXPERIMENTAL PARADIGM 4.3.1 ONLINE WHEELCHAIR CONTROL PARADIGM

Figure 4.1 shows the small screen images which demonstrate a sequence of steps in one run. The converted real size of the simulated scenario was 20 20m . The relative sizes of the targets on all four sideswere 6x1 m2 . Each run began with the wheelchair at
8

the center, and terminated when the wheelchair either hit the target (success) or at any time hit the edge elsewhere (failure). Subjects could inform the investigator to stop the experiment at any time, and the investigator monitored the signal quality and EMG activity throughout the experimental procedure. The wheel chair moving speed was set to 0.4 m/s, with a rotating speed 27 s/360 , within common wheelchair speeds . The wheelchair could move forward only in the direction of the blue bar, which always faced upwards at the beginning of each run. The wheelchair rotated with the square in its center as the axis. The square also served as the color cue; its color changes synchronized with the external auditory cues in different frequencies, reminding subjects of different task periods. Section II-C2 explains in detail how the wheelchair was controlled by performing mental tasks and the EEG patterns that were expected to be exhibited along with the tasks.

Fig. 4.1 Screen shots of virtual wheelchair control in one run.

In figure 4.1 (a) Wheelchair started at the center of the screen-Stop State. (b) Wheelchair started to move upward along the blue bar direction after receiving Go/Stop Command-Running State. (c) Wheelchair started to make right turn after receiving Right Turn Command-Rotating State. (d) Wheelchair stopped rotating and started to move along blue bar direction after receiving Go/Stop Command-Running State. (e) Wheelchair stopped moving and started to make right turn after receiving Right Turn CommandRotating State. (f)Wheelchair stopped rotating and started to move along blue bar
9

direction after receiving Go/Stop Command-Running State. (g) Wheelchair reached the target-Stop State. (h) Simulation of wheelchair control restarted-Stop State.

Fig 4.2. Strategy of wheelchair control.

I n figure 4.2 (a)(d) Follow the time sequence. (a) In stop state, the wheelchair is keeping still. (b) In the first cue period T1, subjects start any of the motor tasks; from top to bottom are the four situations: right wrist extension-left hemisphere ERD pattern, right wrist extension-left hemisphere ERD pattern, left wrist extension-right hemisphere ERD pattern or no motor task-Idle/baseline activity. (c) In the second cue period T2, subjectscontinue with the task: continue right wrist extension-left hemisphere ERD pattern, stopright wrist extension and relax-left hemisphere ERS, stop left wrist extension and relaxright hemisphere ERS, or no motor task-baseline activity. (d) Movement intention is decoded and wheelchair is driven to move forward (or stop when moving), turn right, turn left or keep current moving status. Inter-trial interval (the end of T2 to the beginning of next T1) is 2 s.

4.3.2 WHEELCHAIR CONTROL STRATEGY

Figure 4.2 shows the EEG power changes with execution of four mental tasks separately (in four rows). The performing of each task together with expected EEGpower changes and output control command is summarized in Table I, and.Takethetoprowforexample.This is a Go/Stop command. Task began with the T1
10

time window (01.5 s), indicated by the color changes of the color cue (the square in the center of the wheelchair changing to green) which occurred simultaneously with the first auditory cue onset.When this happened, subjects performed rightwrist extension ormotor imagery (refer Table I).When the color cue changed to blue, T2 time window (1.54 s) began and subject heard the second auditory cue. In this period, subjects continued performing right wrist extension ormotor imagery until the end of T2 window, when the color cue changed to white and the third auditory cue on set.Movement intention was decoded and control action was taken. Specifically.

1) Go/Stop Command (GS): bilateral ERD in both T1 and T2 windows, i.e., when users want the wheelchair to move forward, or stop when it is moving, they perform (imagined) right wrist extension in both T1 and T2 windows. This command will make one of the following three possible state changes. a) Switch from Stop State to Running State, wheelchair will start moving. b) Switch from Running State to Stop State, wheelchair will stop moving. c) Switch from Rotating State to Running State, wheelchair will stop rotating and start to move.

2) Right Turn Command (RT): bilateral ERD in T1 window and ERS on the left hemisphere in T2 window, i.e., when users want to make a right turn, they perform (imagined) right wrist extension in T1windowand stop the (imagined) movement at the beginning of T2 window. This command will make the following state change. a) Switch from Stop State or Running State to Rotating State, wheelchair will start to rotate to right.

3) Left Turn Command (LT): bilateral ERD in T1 window and ERS on the right hemisphere in T2 window, i.e., when users want to make a left turn, they perform (imagined) left wrist extension in T1 window and stop the (imagined) movement at the beginning of T2 window. This command will make the following state change. a) Switch from Stop State or Running State to Rotating State, wheelchair will start to rotate to left. The Idle (ID)/noncontrol state is the natural state when subjects do not want to change moving state; whenever the computer detects ID state, no control
11

command will be sent out and thus the wheelchair keeps its current state. The involvement of ID state in this study renders the continuous control and makes the control process easier.

TABLE 4.1

Wheelchair control commands, associated motor tasks, detected EEG patterns, and output control actions.

12

CHAPTER 5

ADVANTAGES

Ease in handling. BCI provides an easeful platform for paralyzed or handicapped users. The ERD/ERS based BCI virtual wheelchair system is feasible, which subjects could rapidly attain a significant level of control with a short calibration period.

Reasonable control accuracy.

13

CHAPTER 6

CHALLENGES

Insufficient decoding accuracy due to the low SNR of EEG signals. Require intensive user training. EMG contamination from facial muscles may possibly cause serious problems in BCI development.

14

CHAPTER 7

CONCLUSION
The results proved the feasibility of the ERD/ERS based BCI virtual wheelchair system, which subjects could rapidly attain a significant level of control with a short calibration period. In the future research, we would consider how to develop asynchronous wheelchair control based on the current system to further increase control speed while maintaining reasonable control accuracy.

15

CHAPTER 8

REFERENCE
N. Birbaumer, N. Ghanayim, T. Hinterberger, I. Iversen, B. Kotchoubey, A. Kubler, J. Perelmouter, E. Taub, and H. Flor, A spelling device for the paralysed, Nature, vol. 398, pp. 297298, Mar. 25, 1999. B. Obermaier, G. R. Muller, and G. Pfurtscheller, Virtual keyboard controlled by spontaneous EEG activity, IEEE Trans. Neural Syst. Rehabil. Eng., vol. 11, no. 4, pp. 422426, Dec. 2003. R. M. Jdel, F. Renkens, J. Mourino, and W. Gerstner, Noninvasive brain-actuated control of a mobile robot by human EEG, IEEE Trans. Biomed. Eng., vol. 51, no. 6, pp. 10261033, Jun. 2004. J. M. Carmena, M. A. Lebedev, R. E. Crist, J. E. ODoherty, D. M. Santucci, D. F. Dimitrov, P. G. Patil, C. S. Henriquez, and M. A. Nicolelis, Learning to control a brain-machine interface for reaching and grasping by primates, PLoS Biol., vol. 1, p. E42, Nov. 2003. M. A. Nicolelis and J. K. Chapin, Controlling robots with the mind,

16