Journal of Transport & Health xxx (xxxx) xxx–xxx

Contents lists available at ScienceDirect

Journal of Transport & Health

journal homepage: www.elsevier.com/locate/jth

Happy-anger emotions classifications from electrocardiogram

signal for automobile driving safety and awareness
⁎
Khairun Nisa Minhad , Sawal Hamid Md Ali, Mamun Bin Ibne Reaz
Department of Electrical, Electronic and Systems Engineering, Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia, 43600
Bangi, Selangor, Malaysia

AR TI CLE I NF O AB S T R A CT

Keywords: Developing a system to monitor the physical and psychological states of a driver and alert the
Emotion recognition driver is essential for accident prevention. Inspired by the advances in wireless communication
Feature reductions systems and automatic emotional expression analysis using biological signals, an experimental
Stimuli process protocol and computational model have been developed to study the patterns of emotions. The
Wireless sensors
goal is to determine the most efficient display stimuli to evoke emotions and classify emotions of
individuals using electrocardiogram (ECG) signals. A total of 69 subjects (36 males, 33 females)
participated in the experiment and completed the survey. Physiological changes in ECG during
the stimulus process were recorded using a wireless device. Recorded signals underwent a fil-
tering process and feature extraction to determine meaningful features, define the model based
on data assumption, and finally select algorithms used in the classification stage. Two extracted
ECG features, namely root mean square successive difference and heart rate variability, were
found to be significant for emotions evoked using the display stimuli. Support vector machine
classification results successfully classify the happy-anger emotions with 83.33% accuracy using
an audio-visual stimulus. The accuracy for happy recovery is 90.91%, and an excellent accuracy
was also acquired for anger recovery. Findings of this work show that ECG can be used as an
alternative to automatic self-reflective test procedures or additional source with which to vali-
date the emotional state of a driver while in an automobile.

1. Introduction

Inappropriate and aggressive driving behavior is a contributing factor for accidents due to the strong emotional state of drivers
which impact the level of focus and attention and task performance. Research shows that driving with extreme emotions, particularly
anger, can be as dangerous as driving under the influence of alcohol or fatigue (Roidl et al., 2013). Human behaviors, particularly
frustration, anger, and hostility, have been repeatedly linked to risky and aggressive driving (Berdoulat et al., 2013; Schwebel et al.,
2006). In Malaysian traffic, the Malaysian Institute of Road Safety Research identified and reported human errors, such as risky
driving, reckless speeding, fatigue, and driving under the influence, which contributed to 68% of road accidents (Ahmad Noor Syukri
et al., 2012). Identifying human emotions evoked while driving a vehicle can help in efficiently understanding human behavior to
avoid damages and loss due to accidents. Thus, an immediate response system in vehicles can be prepared to reduce high emotional
states and lessen the accident risk of a driver.
Investigating smart, quick, and unique physiological responses which reflect human emotions, specifically those of automotive
and commercial vehicle drivers, remains a challenging task. The high cost of implementation, complexity of the integrated system,

⁎
Corresponding author.
E-mail addresses: khairunnisa.minhad@siswa.ukm.edu.my (K.N. Minhad), sawal@ukm.edu.my (S.H.M. Ali), mamun.reaz@gmail.com (M.B.I. Reaz).

https://doi.org/10.1016/j.jth.2017.11.001

2214-1405/ © 2017 Elsevier Ltd. All rights reserved.

Please cite this article as: Minhad, K.N., Journal of Transport & Health (2017), http://dx.doi.org/10.1016/j.jth.2017.11.001
K.N. Minhad et al. Journal of Transport & Health xxx (xxxx) xxx–xxx

and lack of road user safety awareness have been identified as the root causes. This work explores the theory and model of emotions,
as well as the development of stimulus database and protocols to evoke human emotions. We adhered to the ethical guidelines for
conducting experiments. The relationship between physiological signals and human emotion during driving and the specific signature
of biological signals are studied. Physiological signals that can capture the changes in emotions during the stimulus process are
deliberated. The type and specifications in the selection of the sensor and electrodes are crucial in minimizing noise and loss of raw
signal data. Pre- and post- signal processing is limited not only to signal filtering but also to meaningful features extracted from the
signal. Then, suitable classifiers are used to categorize the extracted features into their respective emotion groups. The local in-
stitutional ethical review board approved this study protocol, and informed consent is obtained from the study participants.

2. Methods

2.1. Participants

Participants must be of Malaysian origin and aged between 18 and 60 years old. A total of 69 participants (age mean 25.07; SD
4.61) volunteered in this study. A total of 36 males (age mean 24.19; SD 5.19) and 33 females (age mean 26.03; SD 3.75) partici-
pated. Participants were excluded from the study if they met any of these criteria: refusal to provide informed consent, corrected
normal vision, hearing difficulties, and cardiovascular issues. The participants were further informed that an emolument will be
granted for their participation. Hypothesis testing using two population means was used to determine the sample population size.

2.2. Stimuli set

Mental state or feeling spontaneously occurs, which reflected physiological changes in the human body. Stimulus is something
that causes a physiological or psychological response. A stimulus set is a tool setup with specific contents to encourage the subject to
provide respective responses and evoke emotions accordingly (emotion elicitation). Various research reported four main techniques
in emotion elicitation: (1) real world car driving which equipped with sensors (Wang et al., 2013a, 2013b; Rebolledo-Mendez et al.,
2014; Vicente et al., 2015) where the dynamic driver's information includes the human (driver), vehicle and driving environment.;
(2) in-lab experiment using driving stimulator (Eyben et al., 2010; Katsis and Rigas, 2014; Wulf et al., 2014; Wang et al., 2015); (3)
activities designed to evoke the stimuli such as flash cards and games (Ring and Kaernbach, 2015; Seoane et al., 2014); and (4) in-lab
experimental setting using selected display stimuli to evoke the emotions.
In this work, the human emotion elicitation process was conducted in-lab using a display stimulus method. Various reported
works consider image stimulus as a globally accepted method to elicit emotion (Henderson et al., 2014; Hosseini and Khalilzadeh,
2010; Huang et al., 2015). The use of dynamic stimuli (video clips) has also demonstrated specific emotions and captured effective
features well (Maffei et al., 2014). Video clips have been reported to induce stronger positive and negative effects than the music clips
(Lazar and Pearlman-Avnion, 2014). In other studies, music (Chin et al., 2014; Kim and André, 2008) and real singers were used to
evoke specific emotions.
We designed an in-lab experimental display protocol which consists of three types of stimuli database for emotion elicitation:
display of selected images, video clips, and audio-video clips. The subjects watched the images and clips displayed on the screen
according to the designed protocol. Electrocardiogram (ECG) data with other physiological signals were simultaneously recorded
during the emotion elicitation process. The goal is to determine the most efficient display stimuli to evoke the respective studied
emotions. To our knowledge, this is the first protocol designed for learning specific features extracted from changes in biological
signals which evoke emotions using images, videos, and audio-visual display interactions in one session.
The selected stimuli method was based on two factors theory by Schachter and Singer (Picard et al., 2001). The stimuli database
was modeled according to multimodal bio-signal sensor data handling (Canento et al., 2011). Two of the most contradicting emotions
defined in the circumplex model of Russell (Posner et al., 2005) were investigated: happy from high positive valence-arousal and
anger from high negative valence-arousal. However, negative emotions are critical for survival and can enhance attention-arousal
mechanism (Chan and Singhal, 2013).
The emotion baseline was determined by neutral state stimulated at the beginning or during the recovery period at the end of the
process stimuli. Neutral stimuli contain neutral images and soothing music selected from a public domain, which comprises scenery
pictures (ocean, blue sky, mountain, view, lake, garden, historic buildings, fruit farms) without humans or animals (Yuen et al.,
2009). The recovery period at the end of the experiment does not demonstrate any display of images or contain video clips or sounds.
Instead, the participants were instructed to relax with eyes closed. Emotion elicitation started with happy followed by anger emotion

Fig. 1. Proposed stimuli protocols.

2
K.N. Minhad et al. Journal of Transport & Health xxx (xxxx) xxx–xxx

blocks (Murugappan, 2014). Fig. 1 summarizes the display database arrangement and duration of each segment. The stimulus
presentation protocol was developed using a free FlashIntegro video editor.
Adaptation of emotions to the environment for most scenarios is subject to the perspective of an individual due to cultural and
geographical factors (Sneddon et al., 2011), gender (McKeown et al., 2015), age (Sullivan et al., 2007), life style and history, level of
education and professional status. For example, in a fully developed country, the behavior of drivers in a big city is likely law
obedient compared to that of a non-developed or developing country. The level of aggressiveness in driving is also different. Hence,
we proposed a new protocol and display contents to efficiently evoke the emotions to be studied. The selected images and video clips
were based on real-life scenarios and incidents, such as clips obtained from a closed-circuit television camera but not scenes from a
movie. The selected images and clips were not limited to driving and traffic incidents only but more human body related contents for
efficient emotion elicitation (Uhrig et al., 2016).
A survey was conducted in a pilot study prior to the full-scale experiments to validate the selected display items from the public
domain which are closer to the culture, native language, and geography of participants compared with the International Affective
Picture System (IAPS) (Bradley and Lang, 2007) database. A total of 60 respondents participated and provided feedback on the
display stimuli that they believe effectively evoked their emotions. From this survey, we confirmed that the proposed stimuli
achieved high efficacy at 72.9%, 77.17%, 80.34%, 76.87%, and 67.36% of the studied emotions, which include happiness, sadness,
anger, disgust, and fear, respectively. Conversely, the efficacy of the IAPS database used in the stimuli process respectively obtained
62.8%, 57.03%, 59.8%, 45.07%, and 65.65% efficacy for the same set of investigated emotions.
The timing of each segment has been carefully defined to ensure that the displays have sufficient time to detect the changes in the
ECG signals characteristics during the emotion elicitations. The duration parameter refers to the minimum time span of the recorded
signal to complete the action detection process. Studies haves shown that heart rate will be increased upon with anxiety and surprise.
Also, the heart rate will also be at resting state after 60 s. Furthermore, the mean of P, R, and T wave peak points or amplitude (in
millivolt) are is found higher when these emotions are evoked (Hardani et al., 2014). The cooling period between the stimuli episodes
was 25 s, which sufficiently neutralizes the evoked emotions based on cognitive and electroencephalography research (Bos, 2006).
Eight digital images without sound for happiness and anger stimuli were displayed for 40 s or 5 s each (Agrafioti et al., 2012).
Psychologists suggested that videos should be 1 to 10 min long to evoke a particular emotion (Soleymani et al., 2012). Other studies
also showed that participants recognized multiple emotions in video clips that last for more than 2 min (Handayani et al., 2015). This
work retained the original duration of the selected video and audio-visual contents of the studied emotions to avoid disappointment
over an exciting or impressive series of stimuli events viewed by the participants (Konar et al., 2014).

2.3. Study participants

The participants rested for 10 min after being equipped with the sensors for wireless signal calibration. Participants were briefed
on the overall objective of the project and were instructed to focus their attention and follow the experiment guidelines during
stimulus presentation. They were informed not to move any part of their body and stay relaxed for artifact prevention (Fatourechi
et al., 2007), especially during the signal data acquisition stage. Hence, signal artifacts can be efficiently separated from meaningful
frequencies in the signal preprocessing stage. Electronic devices, such as mobile phones, wireless connections, mobile modems,
tablets and laptops, watches, and metal accessories, were removed to reduce noises of the transmitted raw data. Subjects were
allowed eye movement while looking at the visual stimuli. A cooling down period was scheduled after each stimulus and the subjects
were allowed to relax to return to their neutral state. The entire session lasted for approximately 40 min, including briefing, sensor
preparation, electrode placement, and signal data acquisitions.

2.4. Electrocardiogram (ECG) – a devoted biological signal

Driving a car is a complex cognitive process. All incoming information from the external (road, weather, billboard, signage) and
internal (dashboard, passengers, temperature) environment must be processed by the human brain, including other information, such
as vision, hearing, and balance (Wang et al., 2015). Any level of distraction or loss of attention can disrupt the attention of a driver,
which might lead to car crashes. The cognitive process is also frequently affected by human emotions. Emotion can be modeled as a
discrete limited set of basic emotions (anger, disgust, fear, happiness, sadness, and surprise), while other emotions are grouped as a
subset of these basic emotions. Each emotion is activated in unique and specific neural pathways. Hence, these physiological changes
in the human body due to emotions can be measured and investigated.
The state of the human emotion can be obtained from speech (Busso et al., 2013), body gesture and motion (Knapp et al., 2009)
and facial expressions (Gruebler and Suzuki, 2014). Captured emotions are exposed to ambiguity due to individual perception of the
same events used to evoke the emotions (stimuli process). Unexpressed and masked emotions (sad yet smiling) are invisible to the
natural eye and cannot be tracked by face, voice, or body gestures. However, physiological signals originate from the activity of the
autonomous nervous system (ANS) in the human brain and cannot be triggered by any conscious or intentional control (Hosseini and
Khalilzadeh, 2010). Negative emotions and stress levels of drivers detected in the ANS can provide warnings using continuous
detection for safe driving.
Physiological changes can provide meaningful information in identifying emotional states. Fast identification and pure emotions
of drivers can be obtained by measuring physiological signals, such as specific electrical properties of ECG signals, including
heartbeat (Agrafioti et al., 2012; Canento et al., 2011), heart rate variability (HRV) (D. Kim et al., 2011a, 2011b; Kim et al., 2004),
skin conductance and response (Bach, 2014; Lin, 2011), and skin and body temperature (C. J. Kim et al., 2011a, 2011b; Kim et al.,

3
K.N. Minhad et al. Journal of Transport & Health xxx (xxxx) xxx–xxx

2004). Previous works reported that physiological parameters, such as heart rate, HRV, respiration, and body temperature, may likely
be measured in a nonintrusive manner, while measurement of other parameters may be more intrusive. This study presents the
changes in physiological signals obtained from ECG signals as a reaction to the proposed stimulus display which uses an emotion
induction method.

2.5. Signal data processing

ECG signal plot is frequently used by medical doctors to investigate cardiac diseases or disorders by referring to its generated
output waves. Frequencies of ECG range between 0.05–150 Hz for diagnoses and 0.5–40 Hz for monitoring. Raw ECG signals contain
various noises during signal acquisition. The sources of noise may come from the equipment, lights, radios, and other biological
signals. In this study, a bandpass signal filtering technique using a 4th order Butterworth-type bandpass filter with a cutoff frequency
of 100 Hz was used to eliminate muscle artifacts (which is low-pass filtered to remove high-frequency signals) and 0.5 Hz to discard
low-frequency components mainly due to motion artifacts and respiration variation (which is high-pass filtered to remove low-
frequency signals). The Butterworth filter is an infinite impulse response type, and the implementation requires a small number of
operations (Jeon et al., 2014). A notch filter was used to filter 50 Hz external electrical line noises. Electrical power interference has
been filtered using a band stop filter with cutoff frequencies of 45 Hz and 55 Hz. At this point, ECG characteristics, such as P, Q, R, S,
and T waves can be clearly identified. These meaningful ECG features were coded and extracted using the signal processing tool based
on the following characteristics.(Agrafioti et al., 2012; Shoeb and Clifford, 2006):

• R wave amplitude of maximum (peak) point can up to 1.6 mV but higher than 0.3 mV
• The location of each peak point of R wave
• The distance of R wave to the next point of R wave (RR intervals)
• QRS intervals occurred every 0.08 – 0.12 seconds or approximately 10 – 50 Hz of frequency
• Signal segmentations according to the process stimuli episodes
• Calculate the extracted features.
Feature extractions of ECG, notably heart rate, heart rate variability, and blood pressure, have considerable effects on the car-
diovascular system by increasing diastolic blood pressure, cardiac output, and heart rate (Sinha et al., 1992). Fear, action, and
happiness are reported to increase systolic pressure while diastolic pressure remained unchanged. Heart rate was observed to be very
sensitive to anger as opposed to joy and sadness (Agrafioti et al., 2012). These features can be collected in every occurrence of an
event (complete cycle), defined window size (in s), or sliding window size with certain stepping or overlapping values. Segmentation
is trivial, but a good technique will maintain all signal information of specific events in discrete time intervals. In this study, a fixed
occurrence of an event size segmentation technique was used to detect the signal peak in a holistic pattern to preserve the signal
without the overlap windowing technique (Uy et al., 2013; J. Wang et al., 2013a, 2013b).
All data processing was performed off-line using a commercial software package (MATLAB R2015). The filtered ECG signals were
segmented according to their respective event stimuli. Hence, the process involved short-term data with a length of less than 15 min.
The time domain method is one of the prominent techniques used to deal with short-term ECG windowing. This method is especially
used to evaluate variations in heart rate or HRV, which can be obtained from the value of peak (QRS wave) and interval of adjacent
peaks (Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology, 1996). A
total of 10 features was calculated: the value of R wave peak mean (R mean), R peak standard deviation (R std), mean distance of two
R peak points (RR mean), RR standard deviation (RR std), R beat counts, heart rate or beats per minute (BPM) with an acceptable
range between 66 and 100 (Sameni et al., 2007), heart rate variability mean (HRV mean), root mean square successive difference or
RMSSD of detrended fluctuation analysis (DFA) technique (Oweis and Al-tabbaa, 2014), power (Murugappan et al., 2013) and the
permutation entropy (Ravelo-garcía et al., 2015) were calculated.
Statistical analysis is one of the most important techniques in selecting significant features. Tens or hundreds of features are
expected to be extracted from a finite time of signal. However, a large number of features will determine the overall computational
time. In this work, statistical analysis information using a high-performance language for technical computing, such as Matlab and
statistical tool SPSS, was used to perform the feature selection. Then, the selected extracted features were trained using the Matlab
computational analysis tool in the classification stage. The classifier was used to categorize feature information obtained from a
signal, and the signal was categorized into different classes of emotion. The efficiency of a classifier is measured by its accuracy rate.
Accuracy = (1−error rate ) × 100%

In this work, the classification methods, which include two classes of support vector machine (SVM) (Saini and Saini, 2014),
Naïve Bayes (Wagner et al., 2014), k-nearest neighbor(k-NN) (Murugappan et al., 2013) and Gaussian classifier (Dong and Zhou,
2014; Ghasemzadeh et al., 2013) have been selected due to their ability to successfully classify the physiological changes of an
individual after the evoked emotions. In terms of linear classification, SVM is known as a parametric algorithm. The nonlinear data
usually comprise nonlinear SVM data sets (random or unpredictable data situations) and has a clever kernel option to efficiently
separate two different classes. The k-NN algorithm is generally known as a nonparametric algorithm, but this algorithm can still
detect linear or non-linear distributed data. In this work parameter, k in k-NN was selected as an odd number. The initial value was
obtained by calculating k = sqrt (n)/2 , where n is the number of samples in the training set. Several cross-validations were performed
to determine which k value works best for the data. A large k value running over labeled examples of big data leads to the

4
K.N. Minhad et al. Journal of Transport & Health xxx (xxxx) xxx–xxx

consumption of more computational time to find the k-NN.

2.6. Signal data recording

We focus on ECG because it is reported to be one of the biological signals that produce the best classification of mental status
(Seoane et al., 2014). Heart and respiration rates can be obtained from ECG, and both rates are highly mediated by the ANS. Hiding
emotions with regard to cardiac reactions is difficult. In addition, ECG is reported to exhibit an emotion specific-pattern after ANS
ends in each of the four chambers of the heart (Agrafioti et al., 2012).
Pre-signal processing is to determine the sampling rate (Hz), bit size, and resolution of a recorded signal (Ghasemzadeh et al.,
2013). ECG signal information is obtained between 0.5 Hz and 100 Hz. In this work, a radio frequency (RF) type of device was used
for recording, and multiple channels were utilized for simultaneous recording of several biological signals. Hence, a high sampling
rate of 960 Hz with a 12-bit resolution was used to ensure a sufficient bandwidth for data transmission to avoid distorted signals and
information loss (Braithwaite et al., 2013; Selvaraj et al., 2013). A PC was used for recording the measured ECG data. During the
recording, a signal output voltage was sampled at a discrete point of time of the original analog signal. This process means that each
sampled data point is converted into a binary value (in bits). A large bit size leads to more precise discrete value representation and
better signal resolution. The process was repeated for the next sample frequency, and a computer processing tool will reconnect these
points to form a continuous graph.
Furthermore, an electric current exceeding 100 mA may pose a considerable risk of inducing ventricular fibrillation in the heart.
For safety reasons, the subject must be elevated from the ground. For this work, the left leg at ankle position was selected as ground. A
conductive adhesive hydrogel metal electrode, such as Kendall Meditrace 500, was used to record signals from the skin surface. The
ionic current flow of bioelectric potentials generated in the body is called ionic potentials. Electrodes convert ionic potentials into
meaningful electronic potentials (Kimura, 2013). Then, the electrode potential is the voltage developed at the electrode–electrolyte
interface. Efficient measurement of these ionic potentials requires conversion into electronic potentials before measurement using a
selected wireless device (BioRadio, 2011). The electrodes were firmly placed on the skin for a stable current flow. A sufficient amount
of time was provided for the conductivity gel to penetrate the hairy skin (Huang et al., 2015). In this work, standard recording
disposable conductive hydrogel snap electrodes with solid connectivity gel made of Ag/AgCl were used because they displayed
minimal bias potentials. Electrodes also generate electrical noises due to the resistance of the dead skin layer, conductance, and
impedance. Careful skin preparation prior to signal acquisition provides a stable output over time and performs DC recording, low
noise, and a small amount of drift.
Three channels in the device sensor were used to record the ECG signals, and each channel comprised differential input con-
nections. Four ECG wire leads and three jumper connections were utilized to form Einthoven's triangle, which is a standard bipolar
limb lead ECG configuration. The participants were seated approximately 60 in. away from a 50″ flat screen TV used to display the
proposed stimuli. ECG raw signals were then converted to Excel format and transferred onto a computer signal processing tool for
signal processing. Each session was video-recorded using a Sony DSLR camera for verification and safety purposes.

2.7. Survey questions

A survey was conducted at the end of the data acquisition session. Findings from the survey were used to validate the emotions of
an individual and rate the efficacy of the selected video clips and images in the proposed stimuli database. The questionnaire was
created in Google Form and was filled up by the respondent after the stimuli session using a 10-point Likert scale (Schwebel et al.,
2006), where “0” is least effective and “10” is the most effective. The demographics of the respondents were obtained by preparing
questions, such as gender, age, blood pressure reading, and general health condition. The list of questions is important to measure the
correlation of the self-reported results with their physiological reaction to the elicited emotion captured in the stimuli session. Their
subjective attitudes can be identified by using a selected statistical or engineering machine-learning tool.

3. Results

In the experiment, a total of seven subjects were excluded due to high movement artifacts, and two were eliminated due to
truncated signals during wireless signal data acquisition. At the signal processing stage, the recorded ECG signals obtained from the
remaining 30 females and 30 males were processed. The hypotheses of the study are as follows: (1) images, video clips, and audio-
visuals can evoke human emotions when the subject is provided ample time, and (2) values of extracted features value measured via
individual's electrocardiography (ECG) are not the same for the different detected emotions.

3.1. Experimental results

Classifiers are commonly used in signal processing event modeling. SVM, naïve Bayes, k-NN, and Gaussian classifiers were se-
lected to classify the performance of the extracted ECG features. These classification methods obtain mathematical relationships
between a set of descriptive and qualitative variables. For comparison with the preferred SVM classifier, the other selected methods
were utilized to justify the performance of SVM in this investigation. The aim is to confirm the significance of the proposed biological
signals and determine whether the method is sufficient for the recognition of human emotions. Table 1 summarizes these classifiers.
Figs. 2 and 3 show the Matlab covariance matrix based on the calculated coefficients of the ECG extracted features, which were

5
K.N. Minhad et al. Journal of Transport & Health xxx (xxxx) xxx–xxx

Table 1
Accuracy comparison of classifiers.

Stimuli Emotions Number of class Accuracy (%)

SVM Naïve Bayes k-NN Gaussian

Image Happy 2 69.23 53.83 61.83 70.00

Video Anger
Video audio
Image Happy 2 58.33 48.10 52.50 51.67
Anger
Video Happy 2 75.00 49.17 53.93 50.00
Anger
Video audio Happy 2 83.33 54.64 61.90 60.06
Anger
Image Happy Neutral 2 81.82 54.40 64.88 49.29
Happy 2 100.00 50.00 87.86 49.29
Recovery
Video Happy Neutral 2 66.67 50.00 54.05 51.43
Happy 2 90.91 62.98 81.43 73.81
Recovery
Video audio Happy 2 83.33 58.57 64.76 68.1
Neutral
Happy 2 90.91 50.00 83.21 76.07
Recovery
Image Anger 2 75.00 54.52 66.31 56.07
Neutral
Anger Recovery 2 100.00 52.14 86.67 50.59
Video Anger Neutral 2 66.67 50.00 70.83 53.93
Anger Recovery 2 100.00 71.55 90.48 78.93
Video audio Anger Neutral 2 72.73 48.69 67.86 56.43
Anger Recovery 2 100.00 51.43 82.98 71.55

Fig. 2. Covariance matrix of ECG features before selection.

recorded during an audio-visual display of the process stimuli. Earlier, exploratory data analysis was performed to determine whether
the features are normally distributed to choose between discriminant analysis and parametric testing for statistical analysis. These
figures also showed that the correlation coefficient successfully returned a matrix where the columns represent random variables and
rows represent observations. Fig. 2 shows that more than one feature is highly correlated with one another while Fig. 3 shows only
one feature will be selected for the classifications stage.
Figs. 4–6 show the mean value of the selected features observed in different evoked emotions using the audio-visual display
stimulus obtained from all participants. ECG features were placed on the y-axis and beats within the ECG segment were on the x-axis
to display a visual representation of RR intervals, heart rate, and HRV over the stimuli time. The mean value of RR was found to be
inversely proportional to HRV because the RR intervals decreased as beat count R increased over a period of time during the anger
stimulus. Fig. 7 summarizes different emotional expressions produced during changes in autonomic activity, such as the increased
count of R peaks and HRV when emotions were evoked using the anger stimulus. The R–R intervals of the recovery stage shown in the

6
K.N. Minhad et al. Journal of Transport & Health xxx (xxxx) xxx–xxx

Fig. 3. Covariance matrix of ECG features after selection.

Fig. 4. Mean value of R peak.

last subplot were found to be stable and within a narrow range, demonstrating less variability compared with the elicitation of
happiness and anger. The anger HRV pattern is characterized by its jagged and irregular waveform.

3.2. Statistical results

Five demographic (gender, age, race, body mass index, and height) and health (systolic/diastolic blood pressure and body
temperature) variables were collected prior to the experiment and were used as independent variables. The dependent variables were
the mean values of R, RR, BPM, RMSSD, HRV, power, and permutation entropy. Emotion elicitation involved anger, happiness,
neutral, and recovery, which was evoked using images, video clips, and audio-video clips.

3.2.1. Emotion features reduction/selection

In the statistical analysis, angry and happy responses to the emotional stimuli recorded using ECG signals were investigated to
determine whether the mean of the measurement variables (extracted features discussed in Section 2.6) remains the same. The mean
of the extracted features was smoothened using a log transform technique (log 10) to make normally distributed data less skewed
(McDonald, 2014). The result of the normality test only shows that the mean value of beat, RR intervals, BPM, RMSSD, and HRV
features fulfilled the Kolmogorov–Smirnov principles. The value of Sig. must be > 0.05 using Lillie for significance correction.
We conducted the statistical analysis using two-way analysis of variance (ANOVA). A parametric statistics using repeated mea-
sures was used to observe the detected emotions (such as image stimuli used to evoke happiness and anger) conducted on the same
individual more than once. Repeated measures comparing the nominal significance level (p < 0.05) were corrected by applying

7
K.N. Minhad et al. Journal of Transport & Health xxx (xxxx) xxx–xxx

Fig. 5. Mean value of RR intervals.

Fig. 6. Mean value of HRV.

Bonferroni's corrections.
Tables 2–4 summarize the significant differences of selected features obtained from repeated measures and pairwise comparisons.
The measurement variable is the stimuli used to evoke emotion. The subject and studied emotion features are the two nominal
variables. The effect of RMSSD and HRV features as indexes of emotion detection is significant in each stimuli (p < 0.05); whereas,
the effect of beat, RR, and BPM of emotion is insignificant (p > 0.05). Both features were analyzed in repeated measures.

3.2.2. Effect of gender on emotions

Two independent t-tests were used to determine whether the mean for the selected normally distributed features extracted from
the stimuli is similar to that of the 30 male and 30 female subjects. The equal variance was assumed for the test because the standard
deviations for the two groups are similar. Any feature in which the independent t-test was insignificant (p > 0.05) was rejected. The
results indicate that a statistically significant difference exists between the mean of the ECG signal feature extraction scores for males
and females of the t and Sig (2-tailed) values summarized in Table 5. The back-transformation of the value obtained using 10value , for
example the actual male ECG bit per minute value on the anger image stimuli was originally obtained at 101.87 = 74.13 BPM. Females
have a statistically significant higher mean score on BPM and HRV features of all stimuli used to evoke happiness, anger and neutral
state. Additionally, males have higher mean on the RR intervals.

8
K.N. Minhad et al. Journal of Transport & Health xxx (xxxx) xxx–xxx

Fig. 7. Plot of filtered ECG and meaningful extracted features during audio visual stimulus.

Table 2
Image stimuli in repeated measures and pairwise comparisons.

Emotion stimuli Features Factor 1 (I) Factor 2 (J) Sig.b

Digital image Beat counts Neutral Anger 0.007

Happy 0.020
Happy Anger 1.000
RR Neutral Anger 0.036
Happy 0.042
Recovery Anger 1.000
Happy 1.000
Happy Anger 1.000
BPM Neutral Anger 0.023
Happy 0.067
Recovery Anger 0.453
Happy 1.000
Happy Anger 1.000
RMSSD Neutral Anger 0.928
Happy 0.256
Recovery Anger 0.000
Happy 0.000
Happy Anger 1.000
HRV Neutral Anger 0.011
Happy 0.008
Recovery Anger 1.000
Happy 1.000
Happy Anger 1.000

3.3. Post-experiment survey analysis

A self-assessment online survey was conducted after each session. In the survey, the questions were designed to collect the degree
of feelings that occurred when the subjects watched the display stimuli. An image snapshot from each event was used to help the
participants recall the respective stimuli and select one answer listed under the questions. The participants were instructed to provide
a weighted scale of 0 for least feeling to 10 for most feeling for each observed stimuli. Answers are neither right nor wrong; however,
each unmatched answer of the used stimuli will be classified as true-false, and the provided score of the selected feelings will be
accumulated. All 69 subjects involved in these experiments voluntarily participated in the survey.
In this survey, the participants found that the anger stimuli was completely effective to evoke the anger emotion. If the perfect

9
K.N. Minhad et al. Journal of Transport & Health xxx (xxxx) xxx–xxx

Table 3
Video stimuli in repeated measures and pairwise comparisons.

Emotion stimuli Features Factor 1 (I) Factor 2 (J) Sig.b

Video stimuli Beat counts Neutral Anger 0.000

Happy 1.000
Happy Anger 0.000
RR Neutral Anger 1.000
Happy 1.000
Recovery Anger 0.350
Happy 1.000
Happy Anger 1.000
BPM Neutral Anger 1.000
Happy 1.000
Recovery Anger 0.628
Happy 1.000
Happy Anger 1.000
RMSSD Neutral Anger 0.022
Happy 1.000
Recovery Anger 0.000
Happy 0.000
Happy Anger 1.000
HRV Neutral Anger 1.000
Happy 1.000
Recovery Anger 0.002
Happy 0.145
Happy Anger 1.000

Table 4
Audio-video stimuli in repeated measures and pairwise comparisons.

Emotion stimuli Features Factor 1 (I) Factor 2 (J) Sig.b

Video audio stimuli Beat counts Neutral Anger 1.000

Happy 0.100
Anger Happy 0.636
RR Neutral Anger 1.000
Happy 0.284
Recovery Anger 1.000
Happy 1.000
Anger Happy 1.000
BPM Neutral Anger 1.000
Happy 0.352
Recovery Anger 1.000
Happy 1.000
Anger Happy 1.000
RMSSD Neutral Anger 1.000
Happy 0.025
Recovery Anger 0.000
Happy 0.000
Anger Happy 0.042
HRV Neutral Anger 1.000
Happy 0.005
Recovery Anger 1.000
Happy 0.289
Anger Happy 0.157

score is given at 690 (provided a weighted score of 10 by each of the 69 participants), then the scores for anger image, video, and
audio-video clips are 572, 571, and 642, respectively; meanwhile, the audio-video clips show the highest score among the other
stimuli methods. The scores for happy image, video, and audio-video clips used in the display stimuli are 478, 473, and 515,
respectively, where audio-video clips remain to obtain the highest score. From the analysis, we discovered that the happy stimuli was
rated as neutral by a small group of participants. Fig. 8 shows the efficiency rating of the proposed stimuli database of the selected
images, videos, and audio-video clips rated by the participants.

4. Discussion

Driving a car is a cognitive process and it frequently affected by the emotions of a driver. An effective and immediate response
system integrated into the vehicle is a major step for safe and pleasant driving. We aim to first develop a technology, such as

10
K.N. Minhad et al. Journal of Transport & Health xxx (xxxx) xxx–xxx

Table 5
Gender differences in the simulated emotions.

Emotions Type of stimuli Extracted features from ECG signal Transformed mean t Sig. (2-tailed)

Male Female

Anger Image R −0.162 −0.162 −0.130 0.989

RR 2.889 2.873 1.648 0.105
BPM 1.872 1.889 −1.643 0.106
RMSSD 1.609 1.619 −0.112 0.911
HRV 1.891 1.908 −1.713 0.092
Video R −0.163 −0.161 −0.039 0.969
RR 2.895 2.884 1.135 0.261
BPM 1.867 1.877 −1.240 0.220
RMSSD 1.549 1.599 −1.040 0.303
HRV 1.884 1.896 −1.144 0.258
Video Audio R −0.162 −0.160 −0.038 0.969
RR 2.895 2.873 2.508 0.015
BPM 1.864 1.890 −2.543 0.014
RMSSD 1.613 1.659 −0.613 0.542
HRV 1.885 1.910 −2.360 0.022

Happy Image R −0.161 −0.162 0.023 0.982

RR 2.891 2.869 2.253 0.028
BPM 1.868 1.891 −2.336 0.023
RMSSD 1.618 1.569 0.692 0.492
HRV 1.889 1.919 −2.037 0.047
Video R −0.162 −0.160 −0.047 0.963
RR 2.897 2.877 2.015 0.049
BPM 1.863 1.883 −1.975 0.050
RMSSD 1.612 1.613 −0.007 0.995
HRV 1.883 1.903 −1.875 0.066
Video Audio R −0.160 −0.159 −0.017 0.986
RR 2.878 2.876 1.926 0.200
BPM 1.882 1.883 −1.288 0.203
RMSSD 1.651 1.799 −1.447 0.153
HRV 1.904 1.907 −1.497 0.140

Neutral Image R −0.162 −0.162 0.001 0.999

Soothing music RR 2.897 2.877 2.063 0.044
BPM 1.862 1.883 −2.065 0.044
RMSSD 1.712 1.639 1.008 0.318
HRV 1.883 1.903 −1.920 0.060

Recovery Closed eyes R −0.160 −0.159 −0.018 0.986

RR 2.889 2.878 1.275 0.208
BPM 1.871 1.882 −1.281 0.205
RMSSD 1.977 1.986 −0.198 0.844
HRV 1.893 1.905 −1.283 0.205

Fig. 8. Efficiency (%) of proposed stimuli obtained from the survey.

intelligence facilities focusing on emotional state detection to aid drivers. The system will be used to eliminate driving risk factors,
such as those in rescue teams (ambulance and fire engines) and commercial vehicles (airplanes, trucks, and buses).

4.1. Useful ECG features in emotion classifications

A stimulus protocol was designed to overcome prolonged process stimuli reported in previous studies (Berna et al., 2014; Selvaraj

11
K.N. Minhad et al. Journal of Transport & Health xxx (xxxx) xxx–xxx

et al., 2013). The proposed time setting and arrangement of each episode were sufficiently justified in the signal processing stage. The
selected filter type and feature extractions and reduction approach have successfully optimized the classification performance. The
observed R wave peak was given for the highest point of a complete cardiac cycle, and the location of this peak in a studied window
(epoch) was given for the intervals of R peaks. From the two extracted features, meaningful information, such as total beat counts,
total BPM (heart rate), HRV, and RMSSD, can be calculated. The power and permutation entropy values were also calculated for the
examined window. Moreover, the statistical values of these features, such as mean, variance, and standard deviation, were used in the
experiment. However, certain features were significantly different with regard to emotion recognition. Hence, a feature reduction
method is vital in reducing the number of features.
In this work, beat counts and RR intervals, as well as BPM, RMSSD, and HRV values, were identified as normally distributed. The
results are significant to accept the null hypothesis: “same features tested on different emotions are strongly significant (p < 0.05).”
These selected features produced a higher accuracy of selected emotion classification compared to the other feature types.
Classifiers performance derived from statistical learning theory and accuracy rate are two of the performance measurements. All
extracted and calculated features can be lumped together into the classifier. However, the most significant features which contributed
to high accuracy were clearly observed using repeated measures of statistical analysis. For example, Table 1 shows that happiness-
anger evoked using the audio-video stimuli obtained 83.33% of SVM classification accuracy. Table 5 summarizes its significant
features and shows that RMSSD and HRV played an important role in the higher accuracy of audio-video clips compared to the rest of
the calculated features.

4.2. Effective stimuli to evoke emotions using ECG

A stimuli database developed using sound, music, audio-video clips, videos, and images to evoke individual affective state of
emotions was reported in previous research. In this study, the use of a dynamic stimulus (video clips) demonstrated specific emotions,
captured effective features, and induced stronger positive and negative affects compared to the image display. Factors, such as native
language, culture, and environment, played an important role in the selection of images and clips with which to evoke the human
emotions demonstrated in this work. The ultimate preference is to employ real images and clips in the emotion elicitation process.
The statistical and experimental results using the proposed design framework have shown that the studied emotions have been
successfully classified, and significant differences of the tested variables were observed. Moreover, based on the classification results,
the 25 s standby (cooling) period between two stimuli was sufficient to bring the ECG signal back to the emotion baseline. This
interval is 35 s shorter than that in a previous research (Berna et al., 2014).

4.3. Classifier performance

Support Vector Machines (SVM) defined an optimum decision boundary (hyper-plane) to separate of two classes using radial basis
functions (RBF) kernel functions The SVM obtained the support vectors that give the best separating hyper-plane and closest point for
both tested classes. The SVM classifier was most suitable for the data distribution in this work and shows the highest accuracy in
recognizing two different sets of data and successfully separates them into classes. We found that the emotion baseline used in the
proposed recovery session frequently obtained a higher accuracy rate compared to neutral stimuli. Moreover, anger and happy
emotions were affectively evoked by using audio-visual stimuli compared to images or video (without sounds) clips. The SVM results
also show that combining all selected and significant ECG features of the three stimuli (image, video, and audio-video clips) did not
increase the accuracy of the system. In SVM classification, the program optimized the penalty of coefficients (constraints) and kernel
function with which the RBF value of this work was set from −5 to 5. Apart from the cross-validation process, the kernel function and
constraints had been optimized in the SVM model training (loop) by substituting the values between −5 and 5 into the kernel and
constraint of RBF. The kernel with high classification accuracy was selected. The number of k-fold has been tested for 10- and 20-
folds. The classification results show that k-fold 10 produced the optimum accuracy.
On the other hand, a non-linear k-NN computed the distance of each sample from all the other samples. Then, the samples were
sorted according to distance. In this study, the classification results show lower classification accuracy compared to SVM due to the
robustness of the RBF kernel setting, which is a more suitable data distribution in this work.
The naïve Bayes classifier assumes that the presence of a particular feature in a class is unrelated to the presence of any other
feature. The naïve Bayes model is particularly useful for very large data sets. Along with its simplicity, the model is known to
outperform even highly sophisticated classification methods. However, this model was found to work less effectively on the data set
((2emotions (3stimulus ) + 2control emotions ) × 6selected features × 60subjects ) in this study. The Gaussian classifier calculates the means
and variances of each class using maximum likelihood estimation.
The multivariate Gaussian distribution is used to calculate the class conditional probability density functions and posterior
probabilities calculated using Bayes' rule. This work shows that the accuracy of Gaussian's classifications is similar to that of naïve
Bayes classification performance. We observed the differences in classification accuracy between two and three classes. The accuracy
of the three classes is lower than the two classes because only a single continuous kernel function was generated for the classifier to
bypass the total number of classes. For example, the kernel function that separates neutral, happy, and anger definitely has a
drawback in providing favors between emotions, such as neutral-happy or neutral-anger. However, classifications of two classes only
provided favors to the classes defined. The two classes frequently obtain high classification accuracy.
The classification accuracy of happy and angry was found to be lower compared with happy-neutral and anger-neutral accuracy.
Happy and anger are emotions expressed with high intensity, wherein classification based on prosody features is extremely difficult

12
K.N. Minhad et al. Journal of Transport & Health xxx (xxxx) xxx–xxx

due to a high level in the arousal (activation) dimension. However, these emotions are situated opposite to each other in the valence
axis (pleasure). Therefore, quality features will effectively contribute to enhance the classification results (Cowie and Cornelius,
2003).

4.4. Women are more emotional than men

Previous research reported that in a simulation, males appear to distinguish cues of simulation yet judge the simulated signals to
be genuine compared to females who judge based on perception to obtain higher levels of simulation (McKeown et al., 2015). In this
wok, significant differences in gender also exist, which indicate that females have higher ECG mean values of BPM and HRV when
presented with anger audio-video clips, happy images and video stimuli compared to males. An effect of gender was evident in the
arousal scales, suggesting that women reported higher levels of emotional arousal than men when viewing video-clips. No significant
difference in selected features between female and male during neutral and recovery stimuli was observed in this study. This point
proves that the stimuli used in both sessions have successfully kept the female and male participants relaxed and stress-free. These
significant ECG features also indicated that they can act as high potential bio-markers in investigating the effects of gender by
efficiently using ECG biological signals. Two ECG features, namely RMSSD and R in this investigation, were found insignificant in the
gender effect of emotions.

4.5. Survey analysis

Survey analysis conducted after the stimuli and signal data acquisition sessions. From the respondent feedbacks, we can deduced
that negative emotions obviously easier to evoke compare to happy emotions. In this study we found the participants tend to have
mixed emotions of neutral or relax state with happy emotions. A total of 34 subjects thought they were feeling happy when natural
images and soothing music were played. However, from emotions classifications using ECG signal processing and statistical analysis,
we confirmed that physiological changes in cardiac cycle originated from the activity of ANS had successfully recognized neutral and
happy emotions as two different entities. Moreover, anger perceptions for most males and females to the anger stimuli are the same
compare to the happy emotions. On the other hands, level of happy emotion for male frequently more difficult to obtain compare to
female (McKeown et al., 2015).

5. Conclusions

Advancement in wireless connectivity has contributed to technological advancements in healthcare and improved safety in daily
life. This study presents the design framework, protocols, and signal processing required to develop an affordable automotive driver
emotion detection and recognition system. The main focus is to reduce the cost by simplifying the computation involved and in-
vestigate the correlation of multi-signal responses to changes in emotions. The low-complexity and high-efficacy experiment vali-
dated the designed framework. A new stimuli protocol which considers the local culture, native language, and gender sensitivities is
proposed. In this study, we conducted a statistical analysis to evaluate the features extracted from ECG signals. In the future, we will
explore artificial neural networks, including deep learning, which is a sophisticated classification method of this era, to enhance the
developed model and classification algorithms.

Funding

This work was financially supported by Ministry of Science Technology and Innovation, Malaysia (Grant No. 01-01-02-SF1061).
Funders were not involved in the conduct of the research.

Conflict of interest

The authors declare that they have no conflict of interest.

References

Agrafioti, F., Hatzinakos, D., Member, S., Anderson, A.K., 2012. ECG pattern analysis for emotion detection. IEEE Trans. Affect. Comput. 3 (1), 102–115.
Ahmad Noor Syukri, Z.A., Siti Atiqah, M.F., Fauziana, L., Abdul Rahmat, A.M., 2012. MIROS Crash Investigation and Reconstruction Annual Statistical Report 2007-
2011. Malaysian Institute of Road Safety Research, Kuala Lumpur.
Bach, D.R., 2014. A head-to-head comparison of SCRalyze and Ledalab, two model-based methods for skin conductance analysis. Biol. Psychol. 103, 6–11. http://dx.
doi.org/10.1016/j.biopsycho.2014.08.006.
Berdoulat, E., Vavassori, D., Sastre, M.T.M., 2013. Driving anger, emotional and instrumental aggressiveness, and impulsiveness in the prediction of aggressive and
transgressive driving. Accid. Anal. Prev. 50, 758–767. http://dx.doi.org/10.1016/j.aap.2012.06.029.
Berna, G., Ott, L., Nandrino, J.-L., 2014. Effects of emotion regulation difficulties on the tonic and phasic cardiac autonomic response. PLoS One 9, e102971. http://dx.
doi.org/10.1371/journal.pone.0102971.
BioRadio, 2011. ® 150 User’s Guide., Cleveland Medical Devices Incorporation.
Bos, D.O., 2006. EEG-Based Emotion Recognition: The Influence of Visual and Auditory Stimuli, pp. 1–17.
Bradley, M.M., Lang, P.J., 2007. The International Affective Picture System (IAPS) in the study of emotion and attention. Handb. Emot. Elicitation Assess. 29. pp.
29–46.
Braithwaite, J., Watson, D., Robert, J., Mickey, R., 2013. A Guide for analysing electrodermal activity (EDA) & skin conductance responses (SCRs) for psychological

13
K.N. Minhad et al. Journal of Transport & Health xxx (xxxx) xxx–xxx

experiments. Psychophysiol. Res. 49, 1–42.

Busso, C., Mariooryad, S., Metallinou, A., Narayanan, S., 2013. Iterative feature normalization scheme for automatic emotion detection from speech. IEEE Trans.
Affect. Comput. 4 (4), 386–397.
Canento, F., Fred, A., Silva, H., Gamboa, H., Lourenço, A., 2011. Multimodal biosignal sensor data handling for emotion recognition. IEEE Sens. 647–650.
Chan, M., Singhal, A., 2013. The emotional side of cognitive distraction: implications for road safety. Accid. Anal. Prev. 50, 147–154.
Chin, Y.-H., Lin, C.-H., Siahaan, E., Wang, J.-C., 2014. Music emotion detection using hierarchical sparse kernel machines. Sci. World J. 2014, 1–7. http://dx.doi.org/
10.1155/2014/270378.
Cowie, R., Cornelius, R.R., 2003. Describing the emotional states that are expressed in speech. Speech Commun. 40 (1), 5–32.
Dong, W., Zhou, M., 2014. Gaussian classifier-based evolutionary strategy for multimodal optimization. IEEE Trans. Neural Netw. Learn. Syst. 25 (6), 1200–1216.
Eyben, F., Wöllmer, M., Poitschke, T., Schuller, B., Blaschke, C., Färber, B., Nguyen-Thien, N., 2010. Emotion on the road - necessity, acceptance, and feasibility of
affective computing in the car. Hidawi Adv. Human.-Comput. Interact. 2010, 1–17. http://dx.doi.org/10.1155/2010/263593.
Fatourechi, M., Bashashati, A., Ward, R.K., Birch, G.E., 2007. EMG and EOG artifacts in brain computer interface systems: a survey. Clin. Neurophysiol. 118, 480–494.
Ghasemzadeh, H., Ostadabbas, S., Guenterberg, E., Pantelopoulos, A., 2013. Wireless medical-embedded systems: a review of signal-processing techniques for clas-
sification. IEEE Sens. J. 13, 423–437. http://dx.doi.org/10.1109/JSEN.2012.2222572.
Gruebler, A., Suzuki, K., 2014. Design of a wearable device for reading positive expressions from facial emg signals. IEEE Trans. Affect. Comput. 5, 227–237.
Handayani, D., Wahab, A., Yaacob, H., 2015. Recognition of emotions in video clips: the self- assessment manikin validation. TELKOMNIKA 13, 1343–1351. http://dx.
doi.org/10.12928/TELKOMNIKA.v13i4.2735.
Hardani, D.N., Wahyunggoro, O., Nugroho, H.A., Faisal, N., 2014. Analysis of emotional condition based on electrocardiogram signals. In: Proceedings of the IEEE
Electrical Engineering and Computer Science, pp. 152–157.
Henderson, R.R., Bradley, M.M., Lang, P.J., 2014. Modulation of the initial light reflex during affective picture viewing. Psychophysiology 51 (9), 815–818. http://dx.
doi.org/10.1111/psyp.12236.
Hosseini, S.A., Khalilzadeh, M.A., 2010. Emotional stress recognition system using EEG and psychophysiological signals: Using new labelling process of EEG signals in
emotional stress state. In: Proceeding of the IEEE Biomedical Engineering and Computer Science Confrence.
Huang, J., Xu, D., Peterson, B.S., Hu, J., Cao, L., Wei, N., Zhang, Y., Xu, W., Xu, Y., Hu, S., 2015. Affective reactions differ between Chinese and American healthy
young adults: a cross-cultural study using the International Affective Picture System. Biomed. Cent. Psychiatry 15 (60), 1–7. https://doi.org/10.1186/s12888-015-
0442-9.
Jeon, T., Kim, B., Jeon, M., Lee, B., 2014. Implementation of a portable device for real-time ECG signal analysis. Biomed. Eng. Online 13, 1–13.
Katsis, C.D., Rigas, G., 2014. Emotion recognition in car industry. In: Konar, A., Chakraborty, A. (Eds.), Emotion Recognition: A Pattern Analysis Approach. John Wiley
& Sons, New Jersey, pp. 515–544.
Kim, C.J., Whang, M., Lee, E.C., 2011a. Identification of arousal and relaxation by using SVM-based fusion of PPG Features. In: Proceeding World Academy of Science,
Engineering and Technology Conference 5, pp. 1704–1708.
Kim, D., Kim, J., Lee, E., Whang, M., Cho, Y., 2011b. Interactive emotional content communications system using portable wireless biofeedback device. IEEE Trans.
Consum. Electron. 57 (4), 1929–1936. http://dx.doi.org/10.1109/TCE.2011.6131173.
Kim, J., André, E., 2008. Emotion recognition based on physiological changes in music listening. IEEE Trans. Pattern Anal. Mach. Intell. 30 (12), 2067–2083. http://dx.
doi.org/10.1109/TPAMI.2008.26.
Kim, K.H., Bang, S.W., Kim, S.R., 2004. Emotion recognition system using short-term monitoring of physiological signals. Med. Biol. Eng. Comput. 42 (3), 419–427.
http://dx.doi.org/10.1007/BF02344719.
Kimura, J., 2013. Electrodiagnosis in Diseases of Nerve and Muscle: Principles and Practice, Third ed. Oxford University Press.
Knapp, R.B., Jaimovich, J., Coghlan, N., 2009. Measurement of motion and emotion during musical performance. In: Proceeding of the IEEE Affective Computing and
Intelligent Interaction Conference, pp. 1–5.
Konar, A., Halder, A., Chakraborty, A., 2014. Introduction to emotion recognition. In: Konar, A., Chakraborty, A. (Eds.), Emotion Recognition: A Pattern Analysis
Approach. John Wiley & Sons, New Jersey, pp. 1–45.
Lazar, J.N., Pearlman-Avnion, S., 2014. Effect of affect induction method on emotional valence and arousal. Psychology 5, 595–601. http://dx.doi.org/10.4236/psych.
2014.57070.
Lin, Y., 2011. A natural contact sensor paradigm for nonintrusive and real-time sensing of biosignals in human-machine interactions. IEEE Sens. J. 11 (3), 522–529.
http://dx.doi.org/10.1109/JSEN.2010.2041773.
Maffei, C., Roder, E., Cortesan, C., Passera, F., Rossi, M., Segrini, M., Visintini, R., Fossati, A., 2014. Kinematic elicitation of basic emotions: a validation study in an
Italian sample. Psychology 5, 1065–1078. http://dx.doi.org/10.4236/psych.2014.59119.
McDonald, J.H., 2014. Handbook of Biological Statistics, Third ed. Sparky House Publishing, Baltimore, Maryland.
McKeown, G., Sneddon, I., Genuine, W.C., 2015. Gender differences in the perceptions ofand simulated laughter and amused facial expressions. Emot. Rev. 7 (1),
30–38.
Murugappan, M., 2014. Frequency band localization on multiple physiological signals for human emotion classification using DWT. In: Konar, A., Chakraborty, A.
(Eds.), Emotion Recognition: A Pattern Analysis Approach. John Wiley & Sons, New Jersey, pp. 295–313.
Murugappan, M., Murugappan, S., Zheng, B.S., 2013. Frequency band analysis of electrocardiogram ( ECG) signals for human emotional state classification using
discrete wavelet transform ( DWT). J. Phys. Ther. Sci. 25 (7), 753–759.
Oweis, R.J., Al-tabbaa, B.O., 2014. QRS detection and heart rate variability analysis: a survey. Biomed. Sci. Eng. 2 (1), 13–34. http://dx.doi.org/10.12691/bse-2-1-3.
Picard, R.W., Vyzas, E., Healey, J., 2001. Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23
(10), 1175–1191. http://dx.doi.org/10.1109/34.954607.
Posner, J., Russell, J.A., Peterson, B.S., 2005. The circumplex model of affect: an integrative approach to affective neuroscience, cognitive development, and psy-
chopathology. Dev. Psychopathol. 17 (3), 715–734.
Ravelo-garcía, A.G., Navarro-mesa, J.L., Casanova-blancas, U., Martin-gonzalez, S., Quintana-morales, P., Guerra-moreno, I., Canino-rodríguez, J.M., Hernández-
pérez, E., 2015. Application of the permutation entropy over the heart rate variability for the improvement of electrocardiogram-based sleep breathing pause
detection. Entropy 17, 914–927. http://dx.doi.org/10.3390/e170.
Rebolledo-Mendez, G., Reyes, A., Paszkowicz, S., Domingo, M.C., Skrypchuk, L., 2014. Developing a body sensor network to detect emotions during driving. IEEE
Trans. Intell. Transp. Syst. 15 (4), 1850–1854. http://dx.doi.org/10.1109/TITS.2014.2335151.
Ring, P., Kaernbach, C., 2015. Sensitivity towards fear of electric shock in passive threat situations. PLoS One 10, e0120989. http://dx.doi.org/10.1371/journal.pone.
0120989.
Roidl, E., Frehse, B., Oehl, M., Höger, R., 2013. The emotional spectrum in traffic situations: results of two online-studies. Transp. Res. Part F: Psychol. Behav. 18,
168–188. http://dx.doi.org/10.1016/j.trf.2013.08.002.
Saini, V., Saini, R., 2014. Driver drowsiness detection system and techniques: a review. Int. J. Comput. Sci. Inf. Technol. 5 (3), 4245–4249.
Sameni, R., Shamsollahi, M.B., Jutten, C., Clifford, G.D., 2007. A nonlinear bayesian filtering framework for ECG denoising. IEEE Trans. Biomed. Eng. 54 (12),
2172–2185. http://dx.doi.org/10.1109/TBME.2007.897817.
Schwebel, D.C., Severson, J., Ball, K.K., Rizzo, M., 2006. Individual difference factors in risky driving: the roles of anger/hostility, conscientiousness, and sensation-
seeking. Accid. Anal. Prev. 38 (4), 801–810. http://dx.doi.org/10.1016/j.aap.2006.02.004.
Selvaraj, J., Murugappan, M., Wan, K., Yaacob, S., 2013. Classification of emotional states from electrocardiogram signals: a non-linear approach based on hurst.
Biomed. Eng. Online 12 (1), 44. http://dx.doi.org/10.1186/1475-925X-12-44.
Seoane, F., Mohino-herranz, I., Ferreira, J., Alvarez, L., Llerena, C., Gil-pita, R., 2014. Wearable biomedical measurement systems for assessment of mental stress of
combatants in real time. Sensors 14 (4), 7120–7141. http://dx.doi.org/10.3390/s140407120.
Shoeb, A., Clifford, G., 2006. Wavelets; multiscale activity in physiological signals short-time fourier transform. In: Proceeding Biomedical Signal and Image

14
K.N. Minhad et al. Journal of Transport & Health xxx (xxxx) xxx–xxx

Processing, pp. 1–29.

Sinha, R., Lovallo, W.R., Parsons, O., 1992. Cardiovascular differentiation of emotions. Psychosom. Med. 54 (4), 422–435. http://dx.doi.org/10.1097/00006842-
199207000-00005.
Sneddon, I., McKeown, G., McRorie, M., Vukicevic, T., 2011. Cross-cultural patterns in dynamic ratings of positive and negative natural emotional behaviour. PLoS
ONE 6 (2), e14679. https://doi.org/10.1371/journal.pone.0014679.
Soleymani, M., Lichtenauer, J., Pun, T., Pantic, M., 2012. Multimodal emotion recognition in response to videos. IEEE Trans. Affect. Comput. 3 (1), 211–223.
Sullivan, S., Ruffman, T., Hutton, S.B., 2007. Age differences in emotion recognition skills and the visual scanning of emotion faces. J. Gerontol. Ser. B Psychol. Sci.
Soc. Sci. 62 (1), 53–60. https://doi.org/10.1093/geronb/62.1.P53.
Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology, 1996. Heart rate variability: standards of
measurement, physiological interpretation, and clinical use. Eur. Heart J. 17, 354–381.
Uhrig, M.K., Trautmann, N., Baumgärtner, U., Treede, R., 2016. Emotion elicitation: a comparison of pictures and films. Front. Psychol. 7, 1–12. http://dx.doi.org/10.
3389/fpsyg.2016.00180.
Uy, G., Ngo, C.F., Trogo, R., Legaspi, R., Suarez, M., 2013. Correlation of stress inducers and physiological signals with continuous stress annotations. Theory Pract.
Comput. 178–183.
Vicente, F., Huang, Z., Xiong, X., Torre, F., De, Zhang, W., Levi, D., 2015. Driver gaze tracking and eyes off the road detection system. IEEE Trans. Intell. Transp.
Systems16 4, 2014–2027.
Wagner, J., Lingenfelser, F., Andr, E., 2014. Building a robust system for multimodal emotion recognition. In: Konar, A., Chakraborty, A. (Eds.), Emotion Recognition:
A Pattern Analysis Approach. John Wiley & Sons, New Jersey, pp. 379–410.
Wang, J., She, M., Nahavandi, S., Kouzani, A., 2013a. Human identification from ECG signals via sparse representation of local segments. IEEE Signal Process. Lett. 20
(10), 937–940. http://dx.doi.org/10.1109/LSP.2013.2267593.
Wang, X., Zhang, J., Ban, X.J., Tan, D., 2013b. Dynamic feature extraction method of driver's propensity under complicated vehicle group. Hindawi Adv. Mech. Eng. 5,
287653.
Wang, Y.-K., Jung, T.-P., Lin, C.-T., 2015. EEG-based attention tracking during distracted driving. IEEE Trans. Neural Syst. Rehabil. Eng. 23 (6), 1085–1094. http://dx.
doi.org/10.1109/TNSRE.2015.2415520.
Wulf, F., Rimini-döring, M., Arnon, M., Gauterin, F., 2014. Recommendations supporting situation awareness in partially automated driver assistance systems. IEEE
Trans. Intell. Transp. Syst. 1–7.
Yuen, C.T., San, W.S., Seong, T.C., Rizon, M., 2009. Classification of human emotions from EEG signals using statistical features and neural network. Int. J. Integr. Eng.
1 (3), 71–79.

15