POSTER 2011, PRAGUE MAY 12

Real Time EEG Processing System

Vladimr Cern, Jaromr Doleal
Dept. of Circuit Theory, Czech Technical University, Technick 2, 166 27 Praha, Czech Republic cernyvl3@fel.cvut.cz, dolezja7@fel.cvut.cz

Abstract. Our group has been dealing with a research in the eld of movement-related EEG recognition towards developing a brain computer interface. Our previous experiments were performed ofine. The logical next step is to move towards a real-time processing. This contribution presents our advances in development of a real-time EEG processing system. Our system is easy extendable, open, distributed, modular, supports parallel processing and can be customized to perform various types of experiments. The modules are independent on used operating system and hardware as the Java language is used. The modules communicate over a local network via a well-dened interface which is independent on the real EEG recording machine. Function of the system was tested and veried in several experiments. The system pilot testing already included detection activity in EEG related to closed and open eyes and performed movements on the basis of EMG traces.

activities. We focus on motor activity because it is the most natural way to control our surroundings. A typical BCI architecture is shown in Fig. 1. Users brain waves are captured by EEG device, parametrized, classied and then the result is nally presented back to the user to give him/her feedback. This enables the user to see immediate results of classication. The user can train to control his own brain waves. This further improves classication results [2]. The delay between event in the brain and display of classication results must be as short as possible for efcient feedback. Thus classication must be done in real-time with low latency.

2. Implementation
The system consists of several standalone modules implemented in Java and connected by a computer network [3].

Keywords
EEG, BCI, real-time, distributed processing.

The network connection allows to run the system on several computers. This can be used for load balancing or for easier control (visualization of results can be on other computer than EEG device is connected to, see Fig. 2). Every module can write logs of incoming and outgoing trafc and output data. Logs can be loaded into Matlab and used for verication of proper module function. Logs are also used for storing data from experiments followed by ofine analysis and for testing the system on previously recorded data. The modules are based on a common class that implements data transfer, logging and loading of settings. The only difference between the modules is the data processing method. Compared to the design presented in [3], we extended the system by common conguration system and improved communication protocol to support optional elds. Every module reads settings from a common settings le and from an experiment settings le. Both les are plain text les with simple notation "key=value". The common le contains all settings for all modules and experiments. The experiment settings le chooses which setting from the common le is used for the given experiment. This allows us to set up different experiments only by appropriate connecting and setting of modules. In addition, we also add following modules to the design presented in [3]: Control, Feedback, Visualiza-

1. Introduction
User
EEG Recording

Raw EEG
Feature extraction

Parameterized EEG
Classification

Feedback

Visualization

EEG types

Fig. 1. Typical BCI system architecture.

A brain computer interface (BCI) is a device which can transfer information directly between a brain and a computer [1]. There are several methods how to acquire data from the brain. We use EEG because it is relatively cheap, small and easy to use for example in comparison with MRI which is neither cheap nor small. BCI can be based on various mental

V. CERN, J. DOLEAL REAL TIME EEG PROCESSING SYSTEM

EEG machine

Bridge

User

LAN Connections

Local connections

Control

Feedback

Detection Monitoring Feature Extraction Classification

resentation station

roccessing ser!er

Fig. 2. Architecture of our real time processing system.

tion, Perceptron, Detection, module supporting new device (Biopac Bridge) and a log sending module for verication and testing purposes.

2.1. Data transfer

Because the system shall work in real-time, low latency of data transfer is very important. A packet which arrives late or out of correct order can not be used. Reliable protocols (e.g. TCP) have methods for sorting packets, conrming packets delivery and methods for repeated sending of lost packets. These methods are useless for purposes of this system and they also add additional delay. Therefore an unreliable protocol is better suitable for real time applications. Thus we based our communication on Real Time Protocol (RTP) [4] which is based on UDP datagram protocol and does not contain mechanism for conrmation of packet delivery. Original purpose of the RTP was real-time video and audio streaming. The protocol was designed to allow real time performance at the price of reliability. As the protocol is standard, it is recognized by network components and prioritized. Since the original RTP protocol does not support EEG transmission, we developed an extension for EEG transmission. We used jlibRTP library [5] to implement RTP communication in Java. The RTP packets consists of header and data section. The header contains packet number, timestamp, source identication and payload (type of data). Data section consists of obligatory and optional elds. The obligatory elds are

based on data format used by EEG devices manufactured by Alien technik [6]. The obligatory elds are EEG data, sample frequency and scale. The optional elds were added to design presented in [3]. They contain information about experiment mode or results of classication. They also contain commands for example such as save, load, reset or commands for runtime changing of experiment parameters. The optional elds are dened in the separate text le and can be added without change of source code of modules which does not use them.

3. Description of the system modules

3.1. Bridge and Generators
The Bridge modules translate data from EEG devices to our system. This enables use of different devices with the minimum need of programming. Today we have bridge modules for Biopac [7] and Alien [6] EEG devices. A Generator module can be also used as a source of data. The Generator module reads data from le and sends them to the system. We have two types of the Generator. The rst type reads raw EEG and it simulates behaviour of Alien EEG device and is used for testing the whole system. The second generator reads data from log le which contains all communication including states, commands and results so it can simulate any module. This is used for verication and testing of new algorithms.

POSTER 2011, PRAGUE MAY 12

Fig.3.

The monitor module displaying four channels and one feature. Green rectangle marks detected movement. Detection times are in text window on the right.

3.2. Monitor
The Monitor module visualizes all incoming data. Signals are visualized as charts with adjustable scale. Events such as results of classication are displayed as color bars in charts (Fig. 3). Other information is printed to the text eld in form of log. The module also displays information about experiment settings (type of lter, frequency ranges, etc.). The Monitor can be easily connected to an output of all modules. The distributed nature of the system allows us to run multiple instances of Monitor module even on multiple computers. Main purpose of the module is to debug an experiment settings and control proper function during experiment.

power is estimated by a leaking integrator. The coefcients of the FIR lters and the integration constant is loaded from the conguration le.

3.5. Detection
Original purpose of Detection module was detecting movement from EMG signal in experiments with performed movements. But we found out this module is also useful for detecting strong waves in EEG such as rhythm. The module computes a power in the same way as feature extraction module. The module compares the power with threshold and sends out result of comparison. The threshold is loaded from conguration les and can be also changed in real-time by command from the Control module. This is useful for EMG detection because strength of signal can vary with quality of contact to skin.

3.3. Control
The Control module provides a graphical user interface for controlling experiments. The module can for example change system mode from training to testing, send reset command or set module parameters during runtime. The module is placed at the start of processing chain (immediately after the Bridge) so all following modules can read control commands inserted to the data owing through the system.

3.6. Perceptron
A one layer Perceptron [8] is one of the simplest self learning structures. Thus we use it as a classication module for rst experiments. It computes weighted sum of inputs and makes decision according to the sign of result:
N

3.4. Feature extraction

The Feature Extraction module computes power in frequency bands. Each channel is ltered by FIR lters and

y=
i=1

wi xi + w0 ,

where y is value used for decision, w is vector of weights and x is input vector.

V. CERN, J. DOLEAL REAL TIME EEG PROCESSING SYSTEM

f [Hz]

The Perceptron module operates in two modes: learning and testing. An information about correct result must be provided to Perceptron for learning. Learning is performed when bad decision occurs simply by adding of scaled input states. w0 [n + 1] = w0 [n] + q, wi [n + 1] = wi [n] + qxi , where w[n+1] is new weights vector, w[n] is current weights vector, x is input vector which was wrongly classied and q is learning factor. The learning factor determines speed of learning and is loaded by the Perceptron module from conguration le. Current mode can be changed by the control module. Values of the weight vector can be stored to le by command sent from the control module. So once learned Perceptron can be used for other experiments.

18 16 14 12 10 8 6

max

min 0 2 4 6 8 t [s] 10 12 14 16

Fig. 5. Short time power spectrum time development of signal recorded during closed eyes detection. Increased power in 1012 Hz band is marked by white ellipse.

3.7. Visualization
The Visualization displays classication results in intuitive way. Currently we use visualization of a ball which is affected by gravity force and a force controlled by classication results.

4. Selected experiments
4.1. Closed and open eyes

A power in 812 Hz band ( waves band) computed by the Feature Extraction module was the only Perceptron input. Training data for Perceptron was provided by detection module which used the same lter as Feature Extraction module and compared output power with threshold. The Perceptron was trained to do the same job as the detection. The detection threshold was set online by the Control module. The Control module also changes Perceptron from training to testing mode. Perceptron output was connected to the Visualization module which visualize results as a moving ball (Fig. 4). All modules outputs were connected to the Monitor module so we could see all signals. The detection of waves was successful after setting of appropriate threshold. The Perceptron was fully trained after while in training state. Experiment showed that all used modules are working correctly. Running experiment can be seen on demonstration video [9].

4.2. Finger movements

13 12

max

f [Hz]

Fig. 4. Closed eyes detection experiment (and the rst author of this paper). One can see the Monitor module with marked detection and the Visualization module on the right monitor.

11 10 9 8 7 6 5 -3 -2 -1 0 t [s] 1 2 3 min

One of the simplest things one can see in EEG data is the presence of some spectral component. For example waves. waves are present when eyes are closed (see Fig. 5) so it is easy to control them by opening and closing eyes. We used detection of closed eyes for testing basic modules of the system. Standard 10-10 EEG cap was used for capturing EEG. Only one symmetric channel was used. Positive lead was connected to O2 electrode and negative lead was connected to Afz. Linked ears were used as a reference.

Fig. 6. Short time power spectrum time development averaged over all realizations. Black line marks time of movement. Event related synchronization is marked by ellipse.

POSTER 2011, PRAGUE MAY 12

Since we wanted to base our BCI system on motor activity we started with performed movements. Performed movements were selected rather than imagined because imagined movement requires more controlled environment using a trigger to instruct experimental subject. In contrast, experimental subject need not to be instructed because we can detect the performed movements. In this experiment, the subject was performing extending of right index nger. EEG was recorded on C3 and C1 electrodes, Cz electrode was used as reference. EMG was captured by the same device and used to tag movements by the Detection module. As the movement related activity in EEG is weaker then the activity related to closed eyes it can not be observed in single realization. After we used the detected movement times for averaging the spectra the event related synchronization (ERS) [10] become visible, see Fig. 6. Thus we have proven EMG detection was successful and we have found movement related activity which can be later used for classication.

The system was tested and correct function was veried under various conditions and settings. The system was carefully designed to support parallel processing within each module, moreover the distributed design enable to balance the load by running the modules on different processor cores of modern CPUs as well as on different computers in local network. The open design also enables to extend the system to any device with network interface and capability of running Java code, for example thin client visualization on a smart phone connected by a wireless network. The experimental results showed that the reached latency is feasible and Java implementation is able to handle data ow even on an obsolete hardware as demonstrated at the faculty open day. Our next steps will be to implement real-time algorithm for left/right hand movements classication frequently addressed in BCI design and nally to investigate the inuence of a feedback.

Acknowledgements
Jaromr Doleal has been supported by the grant GACR 102/08/H008: Biological and Speech Signal Modeling and along with the rest of the team by Grant Agency of the Czech Technical University in Prague, grant No. SGS10/178/OHK3/2T/13.

4.3. Open day demonstration

The system was demonstrated during faculty open day. We used connection of modules similar with the closed/open eyes experiment. EEG, EMG and EOG were recorded. The system was equally capable of detecting waves, EMG activity during hand movements and EOG during movement of eyes for all volunteers participating in the demonstration. Seven low end PCs were used for the Monitor and the Visualization modules so all visitors could have a clear view. All PCs were connected by the 100Mb metallic cable to the faculty network. This demonstration was useful as a stress test. We have proven that the system is capable to work in distributed manner even on slow, obsolete hardware. We did not observe any performance issues.

References
[1] WOLPAW, J. R. et al. Brain-computer interfaces for communication and control. Clin Neurophysiol. Jun 2002, 113, 6, p. 767791. [2] NEUPER, C. Feedback-Regulated Mental Imagery in BCI Applications: Using Non Invasive EEG and NIRS Signals. In BBCI Workshop 2009, Advances in Neurotechnology, Berlin, 2009. [3] TASTN, J. et al. Design Of A Modular Brain-Computer Interface. Applied Electronics. 2010, p. 319322. [4] SCHULZRINNE, H. et al. RTP: A Transport Protocol for Real-Time Applications. RFC 3550, IETF, 2005. http://www.ietf.org/rfc/rfc3550.txt. [5] JlibRTP, RTP implementation in Java [online]. [cit. 2010-05-12]. http://sourceforge.net/projects/jlibrtp/. [6] Alien technik, EEG device manufacturer [online]. [cit. 2011-25-02]. http://www.alien.cz/. [7] BIOPAC systems, inc. EEG device manufacturer [online]. [cit. 201102-25]. http://www.biopac.com/. [8] GALLANT, S. I. Perceptron-based learning algorithms. Neural Networks, IEEE Transactions on. 1990, 1, 2, p. 179191. doi: 10.1109/72.80230. [9] Demonstration video of our working real-time processing system [online]. [cit. 2011-03-006]. http://amber.feld.cvut.cz/fpga/publications/ Oci_3x.avi. [10] PFURTSCHELLER, G. NEUPER, C. Motor imagery and direct brain-computer communication. Proceedings of the IEEE. July 2001, 89, 7, p. 1123 1134. ISSN 0018-9219. doi: 10.1109/5.939829.

5. Conclusion and next steps

Our system is ready to use for performing various experiments with EEG or similar signals in real-time. The system is easily extendable as new features are added to the system by implementing new modules and extending the plain text based denition of communication protocol and shared modules settings. The system is independent on used operating system, hardware and used EEG device. Support for a new EEG device can by added by implementing appropriate Bridge module.

V. CERN, J. DOLEAL REAL TIME EEG PROCESSING SYSTEM

About Authors. . .
Vladimr CERN studied electronics, telecommunication and information technology at SP a VO Psek (high school). Then he studied Electronics and Communication Technology at Czech Technical University in Prague where he graduated with honored bachelor thesis "Real time EEG signal processing". Currently he is studying Individual course of CME (Communication, Multimedia and Electronics) at CTU today. Jaromr DOLEAL was born in Prague, the Czech Republic in 1982. He received his M.Sc. degree in biomedical engineering from the Faculty of Electrical Engineering of the Czech Technical University (FEE CTU) in 2008. Currently he is a Ph.D. student of the Circuit Theory Department, FEE CTU.