23rd Mediterranean Conference on Control and Automation (MED)

June 16-19, 2015. Torremolinos, Spain

Active Learning for Adaptive Brain Machine Interface Based on

Software Agent
Javier Castillo-Garcia1 , Enrique Hortal2 , Teodiano Bastos3 , Eduardo Ian ez2 , Eduardo Caicedo1 and Jose Azorin2

A previous work presented an adaptive brain-computer

interface based on Software Agent (SA) to discriminate
two motor imagery tasks [5]. The software agents are code
structures which try to emulate the interaction between
a human and the users environment. In that work, the
adaptability is obtained in the optimization of the feature
selection for each particular user, and the best classier
is selected. In the literature are presented works about coadaptive techniques to machine learning process applied to
BCI, where do not approach the active learning process [16].
Other works are included reinforcement training to improve
performance the BCI, without increase mutal adaptation
between user-Interface [15], [13], [14].
In this paper the vector features adaptation and active
learning features for a BMI using a low cost computational
classier based on software agent are presented. The software
agent uses the concept of exploration to dene feature
space, and explotation for dened threshold criteria, using
additionally the silhouettes coefcient for an active learning
process. The active learning classier is a K-Nearest neighbor (KNN) with weights matrix, and the online learning is
actualized the training dataset.

Abstract Brain Machine Interface (BMI) and Software

Agent (SA) can provide some new adaptive strategies for robust
BMI implementations. In this work, a non-invasive Adaptive
BMI is introduced, which has been designed to discriminate
four mental tasks. The SA allows tracking features to contribute
for an adaptive process, while the users engagement state
provides a feedback between BMI and the environment. The
Silhouettes width is the performance measurement used for the
active learning process. The results show that the implemented
system allows high accuracy (75%) in the classication process.
Index Terms Active learning, adaptive, BMI, software agent

I. INTRODUCTION
A Brain Machine Interface (BMI) is an articial communication channel to restore and augment the capabilities of
the subjects [1]. There are different paradigms based on the
use of EEG data to translate brain signals into movements
or other output commands [2]. The main challenges in developing these systems arises from the large intra-individual
differences between the neural/physiological responses. As a
result, many traditional BMIs use individually-trained recognition algorithms to process these data with low performance.
New researches present strategies seeking to track evolution
features to improve the adaptation process and increase the
performance [3].
However, the research is not just about nding solutions
to the technical difculties, due the kind of people who
will use the systems are equally or even more important.
In order to operate a BMI system, a user must be able to
produce brain activity that can be detected and classied
with a high degree of reliability and reproducibility. There
is a general consensus, based on experimental results, that a
user must learn to operate a BMI system. This will require
the development of suitable training mechanisms. There
is already evidence that subjects exhibit a high degree of
variability in terms of their ability or otherwise to master
the necessary control of their thought processes [4].

II. SOFTWARE AGENT

SA research is a domain situated at the forefront of articial intelligence [6]. It is a highly interdisciplinary research
area, connecting results from theoretical cognitive science,
neural networks, evolutionary computation, neuroscience,
and engineering [7].
The SA is a piece of code that interacts with the surrounding environment [8]. Through feedback mechanism an user
and SA can both be inuenced by the environment and act
on it. Some of its actions can change the environment, thus
changing the inuence of the environment over it, in a closed
loop structural coupling.
This paper addresses the principles used in the design of
SAs. Evaluation considerations are also discussed.

1 Javier Castillo-Garcia and Eduardo Caicedo are with School

of Electrical and Electronic Engineering, University of Valle, Cali,
Colombia, javier.castillo@correounivalle.edu.co;

A. Disambiguation of Software Agent

ecaicedo@correounivalle.edu.co

2 Enrique Hortal, Eduardo I

an ez and Jose M.Azorin are with the
Department of Bio-Engineering, University of Miguel Hernandez Elche
University, Elche, Spain, ehortal@umh.es; eianez@umh.es;

jm.azorin@umh.es

3 Teodiano Bastos is with the Postgraduate Program in Electrical Engineering, Federal University of Espirito Santo, Vitoria, Brazil,

tbastos@ele.ufes.br

978-1-4799-9935-4/15/$31.00 2015 IEEE

56

Agent architecture: Blueprint for SAs and intelligent

control systems, depicting the arrangement of components.
Agent-based model: Computational model for simulation of the actions and interactions of autonomous
individuals with a view to assess their effects on the
system as a whole.

can be seen as the cohesion and separation in the classes of

the system. Figure 2 shows the structure of BMI for active
learning process using the SA concept.
The SA is composed of the BMI, a set of rules, actions,
performance, initial settings, explotation/exploration blocks
and learning blocks.
1) Performance: Variables are dened and setups are
made in this block.
2) Explotation/exploration: The exploration step evaluates the users engagement state, and the explotation step
chooses elements from the train dataset to be eliminated.
3) Learning: In the setting of supervised learning, it is
important to nd a learning function f : X Y , where
X is conceived as a space of inputs, and Y as a space of
outputs, which predict well on instances that are drawn from
a joint probability distribution p(x, y) on X Y . In an online
approach, the learning algorithm should update a sequence
of functions f1 , f2 , fn in such way that the functions ft+1
depend only on the previous function ft , and the next data
point (xt , yt ). The rules that dene which data need to be
learned are included in the learning process. The Silhouettes
width is the a performance measurement used in the learning
process [11].

BMI
EEG

EEG

FEATURES

SOFTWARE
AGENTS

GA

COST
FUN.

CLASSIFIER

Fig. 1.

Adaptive BMI based on SA for feature and classier selection.

Intelligent agent: Autonomous entity which observes

and acts upon an environment and directs its activity
towards achieving goals.
Software agent: Piece of software that acts for a user
or other program in a relationship of agency.
Reinforcement learning: An area of machine learning
inspired by behaviorist psychology, concerned with how
SAs ought to take actions in an environment so as to
maximize some notion of cumulative reward.

III. A PPROACHES OF SOFTWARE AGENT RESEARCH IN

B RAIN M ACHINE I NTERFACE

IV. MATERIALS AND METHODS

The EEG biosignals were acquired using the equipment
g.USBamp (g.tec medical engineering, Austria). This amplier has 16 channels and the signals are registered with
a sampling frequency of 256 Hz. The electrodes used are
g.LADYbird model, sintered Ag/AgCl crown with a 2-pin
safety connector.

A classical scheme BMI is not very exible, and the

inclusion of SA allows designing new approaches of systems
using the actual pattern recognition as model to implement
a BMI system. Several aproaches will be presented to show
how BMI uses SA.
A. Feature Selection

A. Experimental Protocol

The simplest SAs act only on the basis of the current

perception, ignoring the rest of the percept history. The agent
function is based on the condition-action rule: if condition
is true, then an action is produced. This agent function only
succeeds when the environment is fully observable. When
the BMI begins the training phase, it is possible to adapt the
feature selection using genetic algorithms and dene some
criteria for the classier selection [5]. Figure 1 shows the
architecture for a BMI based on SA for feature and classier
selection.

The BMI interface distinguishes between four mental

states. The data consists of one dataset from inexperienced
users in BMI use (4 males; age 26.75 2.63; all users are
right handed). In order to facilitate the differentiation of the
tasks, two kinds of mental activity are selected:
Motor tasks related to the imagination of movement of
both hands separately (open/close movement).
Mental tasks based on alphabetical and numerical activities. The rst one is the mental recitation of the alphabet
backwards. The second one is a mental countdown from
20 to 0.
The electrodes location includes the motor function area
and close to the Wernickes area. The electrodes are placed
in the following positions (according to the International
System 10/10): Fz, FC5, FC1, FCz, FC2, FC6, C3, Cz, C4,
CP5, CP1, CP2, CP6, P3, Pz and P4. The system used a
mono-auricular reference placed on the right earlobe, and
the ground sensor is placed on the AFz position.

B. Active Learning
The interaction between SA and user is a condition
for learning and adaptability. A genuine intelligent system
should be adaptive, exible and robust; it should adjust
its operation for unexpected changes that inuence it, and
should be creative in nding solutions for completing its
tasks [8]. The feedback way allows the active learning for the
user, but the machine learning provides a static adaptation
from the feature space. For learning, and thus interaction
with the environment to be possible, the articial cognitive
system has to be able to perceive it and inuence it through
effectors, in order to get it done. It is required to know the
users mental state for the adaptive process. The environment
modication affects the feature space and this modication

B. Processing Signal
The signals related to the 16 EEG channels are processed
as follows.
Signal/noise: Firstly, the data are windowed in pieces
of one second. A CAR (Common Average Reference)

57

Software Agent
Performance

BMI

Kappa=0.61 pe=1/N : % a priori

AK=kappa*(1-pe)+pe; %Accuracy accepted
Aux=Traindata; % Copy of TrainDAta

Acquisition

Percepts

Explor / explotation

Es=Engagement_state(New_data): %Es= ----------( + )

[Label Prob]=KNN_pred(mdl, New_data)
TH= Prob: % probabili a posteriori

A/D

Sensors

if Es <= TH
AuxLabel (Df)=New_data;
else
Df=TraindataLabel (Random)
AuxLabel (Df)=New_data;
end
if Es>AK Es=AK
elseif Es<pe Es=pe;
end

Enviroment

Processing

Df=Distance_indx(TraindataLabel , New_data)

Preprocessing

Feature Extration

Learning

Classication

mdl2=KNN_Classf : % model include New_data

% Sw Silhouette width
Sw=1-intra_class/inter_class (New_data)
Sw_Df=1-intra_class/inter_class (Df)

Application

if Sw < Sw_Df
mdl=mdl2 : %model is actualized (learn)
Traindata=Aux : % TrainData is actualized
else

Actions

Device Control

Aux=Traindata : % null

end

Fig. 2.

Adaptive BMI based on software agent for the active learning process.

cessing window for the channels independently. This

method obtains more stable signals over time because
they are less afected by noise. To obtain the feature, the
statistical feature of times series method is applied. The
features used were the absolute mean of the rst difference and the absolute mean of the second difference, as
shown in Equation 3.

lter is applied in order to to enhance the signal quality

[1] lter. The CAR lter is calculated as Equation 1
shows.
= xREF

xCAR
i
i

xREF
/C
j

(1)

j=1

Engagement state: Engagement includes estimations of

cognitive activities, such as information gathering, visual scanning and sustained attention, and the workload
is a measure of effortful cognitive activity [9]. The use
of spontaneously occurring brain signals to implicitly
infer information about the cognitive or affective state
of the user. In this work, the FCz electrode was chosen,
due to the fact that the frontal region executes mental
tasks. Equation 2 is used to evaluate the engagement
state.

E=
,
(2)
( + )

F1 =

where , and are brain rhythms.

Feature Extraction: The signals are normalized using
median criteria regarding the variance in each pro-

58

N 1
n=1

|xn+1 xn |
N 1

F2 =

N 2
n=1

|xn+2 xn |
N 2

(3)

Classier: Finally, KNN (K-Nearest Neighbors) is the

classier used. It is an non parametric lazy learning
algorithm. KNN makes predictions based on the outcome of the K neighbors closest to that point. The
probability of this prediction is called probability a
posteriori. Therefore, to make predictions with a KNN,
it is necessary to dene a metric for measuring the
distance between the query point and cases from the
example samples. One of the most popular choices to
measure this distance is known as Euclidean. Other
measures are presented in Equation 4.

TABLE I
P ERFORMANCE BETWEEN THE ADAPTIVE BMI BASED ON SOFTWARE AGENT AND TRADITIONAL BMI.
Volunteers
Vol. 1
Vol. 2
Vol. 3
Vol. 4
Average

Sensitivity
0.48 (0.48)
0.68 (0.68)
0.84 (0.75)
0.95 (0.90)
0.77 (0.70)

(x p)

2
(x p)
D (x, p) =

|(x p)|

M ax(|x p|)

30
25
20

PC2

15
10

Specicity
0.83 (0.83)
0.89 (0.89)
0.95 (0.92)
0.99 (0.97)
0.91 (0.90)

ITR
37.67 (65.76)
79.07 (78.64)
89.09 (85.13)
99.68 (102.93)
76.38 (82.11)

Accuracy
0.50 (0.48)
0.68 (0.68)
0.85 (0.75)
0.96 (0.90)
0.75 (0.70)

Kappa
0.33 (0.31)
0.57 (0.57)
0.79 (0.67)
0.95 (0.87)
0.66 (0.60)

Silhouette
0.14 (0.19)
0.07 (0.07)
0.63 (0.53)
0.61 (0.56)
0.36 (0.34)

V. R ESULTS AND D ISCUSSION

Table I shows the performance for the adaptive BMI, for

four volunteers. The performance obtained with the adaptive
,

BMI is higher than the performance usually obtained with

a traditional BMI. The ITR was the lowest performance

(4) measurement obtained in this study. This happens because
where x and p are the query point and a case from the there is a relationship with the time without classication;
when the a posteriory classier probability is lower than the
example samples, respectively.
Performance Measurement: Sensitivity, Specicity, threshold value for the new data, the BMI does not classify
Accuracy, Kappas coefcient and ITR are obtained the output (class -1).
from a confusion Matrix [10]. The Silhouettes width
Figure 3 shows the principal components analysis (PCA)
(SW) refers to a method of interpretation and validation for the features of the fourth volunteer. In this gure can
of data clusters [11]. It is a composite index reecting be observed how the clusters have higher cohesion and
the cohesion and separation of the clusters, and can be separation after the active learning process. The volunteer
applied to different distance metrics. For each trial i, its V ol.1 presents the lowest performance (Specicity, SensitivSW (i) is dened as:
ity, Accuracy, etc) and its SW is as lower than the volunteers

three and four.

1 a(i)/b(i), if a(i) < b(i)
The removed data are chosen depending on the distance
0,
if a(i) = b(i)
SW (i) =
(5) respect to the new data and the users engagement state. If the

b(i)/a(i) 1, if a(i) > b(i),

engagement state is as higher than the probability a posteriori
where a(i) is the average distance of trial i to other trials from the classier, then the farther data of the same class is
in the same cluster, and b(i) is the average distance chosen. If engagement state is lower, the removed data are
of trial i to trials in its nearest neighbor cluster. The chosen randomly (within the same class). This improves the
average of s(i) across all samples reects the overall exploration mechanism from the dataset.
The index SW was included in the learning process to
quality of the clustering result. A larger averaged SW
allow
the update of the classier model. If the new data
indicates a better overall quality of the clustering result.
reduces the performance system, the old model is kept; in
other cases, it would be updated.
a)
The techniques here implemented can be used in evaluate
Task 1
Task 2
of
evolution for diagnostic disease [12], this is possible,
Task 3
Task 4
because the variations in the performance system can be
tracking.

Euclidean
Euclidean square
Cityblock
Chebyshev

5
0

VI. CONCLUSIONS

5
10
20

10

10

20
PC1

30

30

40

50

b)

25

Task
Task
Task
Task

20
15

PC2

60

Software agent research is a complex, interdisciplinary domain. The combination of BMI and software agent provides
a sound foundation upon which new adaptive strategies and
architectures can be implemented for online applications.
The active learning process helps in the feature tracking
process. The farther features are eliminated from the training
dataset and replaced for new data that increase the SW. The
implementation does not use old elements to try to adjust
the possible new tendence at new data. Future works aim
online application of this tool, as well as to optimize the
specication of the eletrodes from the frontal area, in order
to calculate the users engegement state.

1
2
3
4

10
5
0
5
10
20

10

10

20
PC1

30

40

50

60

Fig. 3. PCA dataset train representation of volunteer four, a) before active

learning, b) after active learning.

59

ACKNOWLEDGMENT

[8] F. Stan and G. Art. Is It an Agent, or Just a Program?: A Taxonomy

for Autonomous Agents. Proceedings of the Workshop on Intelligent
Agents III, Agent Theories, Architectures, and Languages. ECAI 96,
Springer-Verlag, pp. 21-35, 1997.
[9] A. Pope, E. Bogart, D. Bartolome. Biocybernetic system evaluates
indices of operator engagement in automated task, Biological Psychology, Vol. 40, pp. 187-195, 1995.
[10] N. Japkowicz, M. Shah. Evaluation Learning Algorithms a Classication Perspective vol. 1, 1st edn., pp. 110-125. Cambridge University
Press, 32 Avenue of the Americas. Chap. Machine Learning and
Statistics Overview, 2011.
[11] P. Rousseeuw. Silhouettes: A graphical aid to the interpretation and
validation of cluster analysis, Journal of Computational and Applied
Mathematics, Vol. 20, pp. 53-65, 1987.
[12] G. Farias, M. Santos and V. Lpez: Making decisions on brain tumor
diagnosis by soft computing techniques, Journal Soft Computing, No.
12, pp. 1287-1296, 2010.
[13] K. Nomoto, T. Tsubone and Y. Wada:Possibility of reinforcement
learning using event-related potential toward an adaptive BCI,Systems,
Man and Cybernetics, 2009. SMC 2009. IEEE International Conference on, pp.1720-1725, 2009.
[14] L. Bougrain, M. Duvinage and E. Klein: Inverse reinforcement
learning to control a robotic arm using a Brain-Computer Interface.
Enterface summer Workshop, 2014.
[15] Z. Zongtan, L. Yang and H. Dewen:Classication of MovementRelated Potentials for Brain-Computer Interface: A Reinforcement
Training Approach,Advances in Neural Networks, pp. 620-628, 2006.
[16] M. Bryan, S. Martin, W. Cheung and R. Rao: Probabilistic co-adaptive
brain-computer interfacing, Journal of Neural Engineering, No. 10, pp.
1-15, 2013.

The authors would like to thank IBERADA (Red

Iberoamericana para el estudio y desarrollo de aplicaciones TIC basadas en interfaces adaptadas a personas con
discapacidad, 512TR0466), funded by CYTED (Programa
Iberoamericano de Ciencia y Tecnologa para el Desarrollo)
for the nancial support.
R EFERENCES
[1] J. Wolpaw, N. Birbaumer, D. McFarland, G. Pfurtscheller and T.
Vaughan. Brain-computer interfaces for communication and control.
Clinical neurophysiology : ofcial journal of the International Federation of Clinical Neurophysiology, vol. 113, No. 6, pp. 767-791,
2002.
[2] L. Nicolas-Alonso and J. Gomez-Gil. Brain Computer Interfaces, a
Review. Sensors, Vol. 12, No. 2, pp 1211-1279, 2012.
[3] J. Faller, C. Vidaurre, T. Solis, C. Neuper and R. Scherer. Autocalibration and Recurrent Adaptation: Towards a plug and play online
ERD-BCI. IEEE Transactions on Neural Systems and Rehabilitation
Engineering, Vol. 20, No. 3, pp 313-319, 2012.
[4] R. Adams, R.Comley and M. Ghoreyshi.The Potential of the BCI for
Accessible and Smart e-Learning. Lecture Notes in Computer Science
in : Universal Access in Human-Computer Interaction Intelligent and
Ubiquitous Interaction Environments,Springer Berlin Heidelberg, Vol
5615, pp. 467-476, 2009.
[5] J. Castillo-Garcia, A. Cotrina, A. Benevides, D. Delisle-Rodriguez, B.
Longo, E. Caicedo, and T. Bastos. Adaptive BCI based on Software
Agents. IEEE Engineering in Medicine & Biology Society (EMBC),
2014.
[6] B. Balachandran and M. Enkhsaikhan. Developing multi-agent ecommerce applications with JADE, in: Knowledge-Based Intelligent
Information and Engineering Systems, Springer, pp. 941-949, 2007.
[7] S. Russell and P. Norvig. Articial Intelligence: A Modern Approach.
Pearson Education (2), 2003.

60