Vous êtes sur la page 1sur 10

Can FOUR Score replace GCS for assessing neurological

status of critically ill patients - An Indian Study

Anita Mercy S, Ramesh Thakur, Sandhya Yaddanapudi, Hemant Bhagat

Abstract Since its development in 1974, the Glasgow Coma Scale(GCS) has been the gold standard
for assessing the patients Level Of Consciousness(LOC) and acute changes in neurological status. Despite its
widespread use, the GCS has many limitations that are well documented. The Full Outline of
UnResponsiveness(FOUR) score is a new coma scale that was recently validated as a proposed alternative to
GCS. The main purpose of the study was to assess the reliability of the FOUR score in critically ill patients of
main ICU of PGIMER, Chandgiarh. In this study, scoring of GCS and FOUR score were performed by anesthetists
and nurses on 21 patients. In a total of 100 pair-wise ratings, the inter-rater agreement for both the FOUR score
(k = 0.65) and the GCS (k = 0.66) was good among nurse anesthetist pair. The internal consistency for both
the FOUR score ( = 0.97) and the GCS scale ( = 0.94) was excellent. A good correlation was found between
the FOUR score and the GCS ( = 0.94, p = 0.001). The predictive validity of the FOUR score [Area under the
receiver operating curve(AUROC)= 0.8] is slightly higher than the GCS (AUROC = 0.76) in this population.
But the results could not be concluded with the availability of very limited number of patients. Majority of the
raters strongly agreed that the FOUR score can be used as an alternative to the GCS scale, though there is no
significant difference in the raters opinion about both the scales. The study suggests that the new FOUR score
has precise clinical acumen in detecting subtle changes in neurological status as the GCS scale. Inclusion of
assessment of brainstem function along with the assessment of cortex function adds value to the new tool.

Keywords Introduction
Glasgow Coma Scale (GCS), Full Outline of
Consciousness is a state of general
UnResponsiveness score (FOUR),
Unconsciousness awareness of oneself and the environment
and includes the ability to orient towards new
Correspondance at stimuli.1 Despite advances in technology, a
thorough clinical assessment is still the key
Anita Mercy to identify subtle changes in a patients
MSc. Nursing 2nd yrs student (Oncology Nursing) neurological status and is fundamental to the
National Institute of Nursing Education (NINE), management of neuroscience patients. To
PGIMER, Chandigarh provide quality patient care, the bedside nurse
must therefore be able to accurately and

Nursing and Midwifery Research Journal, Vol-9, No.2, April 2013 63


consistently assess and communicate these response(E), motor response(M), brainstem
changes.2 reflexes(B), and respiration(R). In each of
these categories, a score of 0 indicates
Scales have been constructed to
nonfunctioning status, and a score of 4
improve communication among health care
represents normal functioning. The number
personnel.3 The most commonly used scale
of components and the maximal grade in each
is the GCS scale.3 GCS scale misses key
of the categories is four (E4, M4, B4, R4).3 In
essential elements of a comprehensive
contrast to the GCS, the FOUR score eye
neurological examination for comatose
response category assesses eye tracking in
patients.4 Failure to assess the verbal score in
addition to eye opening, which allows it to
intubated patients and inability to test brain
differentiate vegetative state from minimally
stem reflexes are shortcomings of GCS.5, 3, 6
conscious state (MCS) patients.2, 7, 12 The FOUR
The ability to assess the GCS motor score is
scale also more accurately and expeditiously
often impacted by the administration of
diagnose the locked in syndrome by
sedatives or neuromuscular blocking agents
specifically assessing voluntary eye
and the presence of confounders such as
movements. 7, 12 The motor assessment
spinal cord injury. The GCS is skewed toward
includes response to pain, ability to follow
motor assessment, with a maximum of 6
simple commands, and the presence of
points. This affects the ability to assign an
generalized myoclonus status epilepticus, a
accurate GCS to patients who are receiving
poor prognostic sign in comatose survivors
medications or who have injuries that interfere
after cardiac resuscitation.2, 3, 13 Brainstem reflex
with motor assessment. 2 Subsequent
category was created to assess the function
investigations of the GCS had revealed
of the mesencephalon, pons, and medulla,
disagreement among the raters, especially
which allows diagnosis of uncal herniation.2, 7
between experienced and inexperienced
Lower brainstem function is evaluated using
users.5, 7, 8, 9 Prior efforts to modify or replace
the respiration category to identify irregular
this scale have been unsuccessful because
breathing patterns, including Cheyne-Stokes
no scale could improve on its simplicity and
respirations.2
practical usefulness.
The FOUR score has been well received
To address the many limitations to the
in and outside the United States and has been
GCS, Wijdicks et al, at the Mayo Clinic
implemented at the Mayo Clinic Saint Marys
designed the FOUR score as a proposed
Hospital. The nursing staff too have
alternative in 2005. 7, 10, 11 The FOUR score has
enthusiastically embraced the new coma scale
been developed to assess the depth of coma
in the United States. Studies on its usefulness
in a more detailed manner than the GCS
outside the boundaries of the Neurological-
scale.11 The FOUR score assigns a value of 0
to 4 to each of four functional categories: eye Neurosurgical ICU are under way.11

Nursing and Midwifery Research Journal, Vol-9, No.2, April 2013 64


Wijdicks et al (2005) and other and the raters (nurses and anesthetists) and
investigators, have conducted validation of the accessible population was critically ill
FOUR score in different clinical settings like patients admitted in main ICU of PGIMER,
medical ICU, neurology ICU and emergency Chandigarh and the raters (nurses and
department of Mayo Clinic involving different anesthetists) of the same unit. Patients were
group of raters including experienced and selected by purposive sampling i.e.) all
inexperienced nurses and neuro physicians. critically ill patients of four different categories
Validation of FOUR score has also been done (alert, drowsy, stuporous and comatose)
in different group of patients including stroke admitted in main ICU, patients with age >12
patients,15 pediatric population,2 critically ill yrs during 16th July till 31st august 2011 and
patients11, 14 and neuroscience patients6, 11, 16 in 21 patients were included in the study. All
different settings by various groups of raters patients who were on neuromuscular
in other places outside the Mayo Clinic. No blocking agents were excluded from the study.
studies related to FOUR score have been done Raters were selected by purposive sampling
in India. It is very important to have a simple i.e.) all nurses and anesthetists who were on
and reliable clinical scale system to determine duty were included in the study and 32 raters
the level of consciousness in our clinical were included in the study. All raters who were
settings with which both the nurses and not willing to participate were excluded from
the study. The sample size was determined
doctors are comfortable.
before the study and a total of 100 pair-wise
In this preview, the present study was ratings of FOUR score and GCS were included
planned to assess the reliability and feasibility in the study. The tools used for data collection
of the FOUR score in critically ill patients were demographic proforma of the patients
admitted in main Intensive Care Unit. and the raters, the FOUR score, the GCS scale
Materials and Methods and the raters opinionnaire.
GCS scale & four score were
This study was conducted at main ICU
standerised tools and other tools were
of Nehru hospital, PGIMER, Chandigarh, a
prepared by thorough literature review and
premier institution of medical education and
experts opinion and validated by different
research in India which has been functioning
experts of the medical and nursing field.
as a tertiary care hospital. The main ICU is a
Ethical approval for the study was obtained
12 bedded multi-specialty ICU under the
from the Ethics Committee of the institution.
Department of Anesthesia and Intensive Care,
A written permission was obtained from the
which caters to medical, neurological,
HOD, Department of Anesthesia and main ICU,
surgical, gynecological, and other intensive
PGIMER. Sister-in-charge of main ICU was
and critical care conditions. The target
informed about the study. The anonymity and
population of the study was critically ill patients
confidentiality of the participants (raters) in

Nursing and Midwifery Research Journal, Vol-9, No.2, April 2013 65


relation to findings was protected while Among 100 observations, 29 were done by
repor ting the study. The period of data nurse nurse pair, 41 by nurse anesthetist
collection was from 16th July to 31st August pair and 30 anesthetist anesthetist pair. In
2011. The techniques of data collection total, 31 raters were enrolled in the study.
followed were observation and records. Among 21 patients, 9 (42.9%) belong
Scoring on each patient was performed at the to the middle adulthood (41 65 Yrs) group
first possible admittance by the researcher for and the mean age of the patients was 44.4
predictive validity analysis. Raters were 1.9 years (range 17- 78 years). Among the
oriented to the FOUR score by a teaching 21 patients, 14 (66.7%) were males and 7
session using PowerPoint presentation on (33.3%) were females, 11 (52.4%) were
FOUR score along with demonstration on intubated using orotracheal tube and 10
actual patients. Return demonstration was (47.6%) were intubated using tracheostomy
also taken from the raters. Posters on FOUR and 8 (38.1%) died in ICU and 13 (61.9%)
score was put up on walls of each cubical of were alive and got transferred from ICU.
main ICU. Written instructions about the FOUR Among 21 patients, 6(28.6%) were admitted
score were given to each evaluator at the time with head injury.
of rating. Raters pair were nurse-nurse, Among 32 raters, 12(37.5%) were
anesthetistnurse and anesthetist holding Diploma in Nursing (GNM), 13(40.6%)
anesthetist. Paired observations were made were B. Sc Nursing (Nsg) holders, 6(18.8%)
by the raters on each patient and each patient had professional qualification as
was observed by different pair of raters at M. B. B. S, currently undergoing M.D and
different timings. Within the same hour, each 1(3.1%) had M. D as qualification. The mean
evaluator in the pair recorded a FOUR score professional experience of the raters was 4.6
and a GCS score for the patient. At the end of 5.5 years (range 2 months - 23 years). The
data collection, raters were given opinionnaire mean experience of the raters in ICU was 2.7
on GCS and FOUR score and the responses 2 years (range 2 months - 6 years & 8
were taken by 5-point Likert scale. The data months).
was analyzed using both descriptive and
inferential statistics. Calculation was carried Internal consistency of the FOUR score and
out manually using a calculator and with the the GCS
help of Microsoft excels (2007) and Statistical Internal consistency of the FOUR score
Package for Social Science (SPSS) and the GCS was analyzed using Cronbachs
programme version 16. and spearmans correlation coefficient.
value of 0.5 or less is considered
Results unacceptable, values between 0.5 and 0. 6
During the study period of 6 weeks, are considered poor, values between 0. 6 and
100 observations were done on 21 pateints. 0.7 are considered questionable, values
between 0.7 and 0.8 are considered
Nursing and Midwifery Research Journal, Vol-9, No.2, April 2013 66
acceptable, values between 0. 8 and 0.9 are are shown in table. 1. Cronbachs for the
considered good and values above 0.9 are FOUR score (0.95) and the GCS (0. 94)
considered excellent internal consistency.6,18 indicate excellent internal consistency for both
values for both the FOUR score and the GCS the scores.
Table. 1: Internal consistency of the FOUR score and the GCS N = 100

Variable Cronbach s alpha () p value


FOUR score 0.97 <0.001**
GCS 0.94 <0.001**
**
Internal consistency is significant at the 0.01 level
Spearman correlation coefficient was used
to analyze the correlation of the FOUR score 12.50

and the GCS for 200 observations. A highly


positive correlation of = 0.94 between the 10.00
FOURscore

FOUR score and the GCS score (Figure. 1) 7.50

Inter-rater agreement of the FOUR score and 5.00

the GCS
Inter-rater agreement of the FOUR 2.50
R Sq Linear = 0.871

score and the GCS was analyzed using 0.00

Cohens kappa. A kappa value of 0.4 or less 2.00 4.00 6.00


GCSscore
8.00 10.00 12.00

is considered poor, values between 0.4 and


0.6 are considered fair to moderate, values Figure 1. Correlation between the GCS
between 0.6 and 0.8 are considered good and score and the FOUR score
values above 0.8 are considered excellent anesthetist in 68. 3% of the observations of
agreement.5, 2, 6 the total FOUR score (k = 0.65) and in 71%
Kappa statistics of the FOUR score is of the observations of the total GCS score (k
shown in table-2 and kappa statistics of the = 0.66) which shows that the degree of
GCS is shown in table-3. In 29 pair-wise agreement between Nurse Anesthetist pair
ratings, the two nurses agreed exactly in is higher than that of other pairs (Nurse
51.7% of the observations of the total FOUR Nurse pair and Anesthetist Anesthetist pair),
score [k = 0.47(fair to moderate observer for FOUR score, k = 0.65(good observer
agreement)] and in 68.9% of the observations agreement) and for GCS, k = 0.66 (good
of the total GCS score [(k = 0.65(good observer agreement), in 30 pair wise ratings
observer agreement)], in 41 pair wise the two anesthetists agreed exactly in 53. 3%
ratings the nurse agreed exactly with the of the observations of the total FOUR score

Nursing and Midwifery Research Journal, Vol-9, No.2, April 2013 67


(k = 0. 48) and in 53.3% of the observations FOUR score (k = 0. 55) and in 65% of the
of the total GCS score [k = 0.46(fair to observations of the total GCS score (k = 0.6)
moderate agreement)] Overall among 100 pair which shows that there is fair to moderate
wise ratings the two group of raters agreed agreement among all pairs.
exactly in 59% of the observations of the total

Table. 2: Kappa values for the inter-rater agreement of the FOUR score N = 100
Kappa value (k)
Rater Pair Total Eye Motor Brainste Respiration
n Score response response reflexes Score
Score Score Score
Nurse Nurse 29 0.47 0.58 0.74 0.74 0.90
Nurse Doctor 41 0.65 0.70 0.64 0.86 0.78
Doctor- Doctor 30 0.48 0.69 0.59 0.78 0.32 `
Overall 100 0.55 0.67 0.66 0.80 0.69

Table. 3: Kappa values for the inter-rater agreement of the GCS score N = 100
Kappa value (k)
Rater Pair n Total Eye response Motor response ROC Curve

Score Score 1.0


Score
Nurse - Nurse 29 0.65 0.79 0.8
0.75
Nurse - Doctor 41 0.66 0.81 0.69
Sensitivity

30 0.46 0.77 0.61


0.6
Doctor- Doctor
Overall 100 0.6 0.8 0.4
0.69
Predictive value of the FOUR score and the GCS 0.2

ICU mortality status prediction using the FOUR score was analyzed using ROC curve
0.0
0.0 0.2 0.4 0.6 0.8 1.0
1 - Specificity

Diagonal segments are produced by ties.

------------ GCS score

Reference Line

FOUR score

Figure 2. Predictive value of the GCS and the FOUR score for ICU Mortality

Nursing and Midwifery Research Journal, Vol-9, No.2, April 2013 68


Figure 2. shows the predictive value of opinion about the practical feasibility of the
the GCS (AUROC is 0.76, 95% confidence GCS is shown in table. 4. Among 31 raters,
interval 0.55 0.96, p = 0.55) and the 18(58.1%) agreed to the statement that GCS
predictive value of the FOUR score (AUROC is is reliable, 20(64.5%) agreed to the statement
0.8, 95% CI: 0.6 1, p = 0.25). Out of 8 that GCS is easy to use, 19(29%) agreed to
patients who died in ICU, 1 (12.5%) got the the statement that GCS is reproducible,
minimum GCS score of 3 and 1 (12.5%) got 16(51.6%) agreed to the statement that GCS
the minimum FOUR score of 2. gives more clinical information and 16(51.6%)
Practical feasibility of the FOUR score agreed to the statement that GCS takes less
The descriptive statistics of the raters time to perform.

Table. 4: Raters opinion on practical feasibility of the GCS N = 31


Statements regarding Strongly Disagree No opinion Agree Strongly
GCS Disagree agree
n % n % n % N % n %
GCS is reliable 1 03.2 7 22.6 18 58.1 5 16.1
GCS is easy to use 1 03.2 20 64.5 10 32.3
GCS is reproducible 2 06.5 16 51.6 19 29.0 4 12.9
GCS gives more clinical 1 3.2 5 16.1 8 25.8 16 51.6 1 03.2
information
GCS takes less time to 3 9.7 1 03.2 2 06.5 16 51.6 9 29.0
perform

The descriptive statistics of the raters score can be used as an alternative to GCS.
opinion about the practical feasibility of the Very few among both the raters group
FOUR score is shown in table. 5. Among 31 were disagreeing that FOUR score can be used
raters, 20(64.5%) agreed to the statement that as an alternative to GCS. Most of the raters
FOUR score is reliable, 20(64.5%) agreed to expressed that they find difficulty in eliciting
the statement that FOUR score is easy to use, withdrawal response to pain and often face
13(41.9%) agreed to the statement that FOUR confusion between withdrawal response to
score is reproducible, 20(64.5%)agreed to the pain and flexion response to pain in GCS.
statement that FOUR score gives more clinical Some of them verbalized difficulty in
information, 16(51.6%) agreed to the assessing the motor component of GCS in
statement that FOUR score takes less time to quadriplegic patients. Raters felt that the
perform & 13(41.9%) agreed & 10(32.3%) addition of brain stem reflexes in FOUR score
strongly agreed to the statement that FOUR would give them more clinical information.

Nursing and Midwifery Research Journal, Vol-9, No.2, April 2013 69


Table. 5: Raters opinion on practical feasibility of the FOUR score N = 31
Statements regarding Strongly Disagree No opinion Agree Strongly
FOUR score Disagree agree
n % n % n % n % n %
FOUR score is reliable 2 6.5 20 64.5
FOUR score is easy to use 1 3.2 4 12.9 20 64.5 6 19.4
FOUR score is reproducible 3.2 12 38.7 13 41.9 5 16.1
FOUR score gives more 3 3 20 64.5 8 25.8
clinical information
FOUR takes less time to 3 9.7 4 12.9 5 16.1 16 51.6 3 9.7
perform
FOUR score can be used 1 3.2 1 3.2 6 19.4 13 41.9 10 32.3
as an alternative to GCS
Discussion Studies have been conducted among
Neurological disturbances pose a different group of patients (stroke patients,15
greater challenge in the critically ill patients and pediatric population,2 critically ill patients11,14
patients with neurological disorders. The GCS and neuroscience patients6, 11, 16). This is the
has been the gold standard for assessing the first study to evaluate this newly validated
LOC in patients with significant brain injury FOUR score in India for its application in
since it was developed in 1974.2 The GCS is critically ill patients. Wijdicks et al (2005) and
widely used and accepted but gives relatively other investigators, have conducted validation
limited information about brainstem function, of FOUR score in different clinical settings like
eye opening and tracking, and respiratory medical ICU, neurology ICU and emergency
patterns.13 Since its introduction in 2005, FOUR department of Mayo Clinic involving different
score has been refined in clinical use, and its group of raters including experienced and
usefulness has been confirmed by hundreds inexperienced nurses and neuro
of neurosurgical patients and dozens of physicians.2,3,5,6 This study involved nurses
doctors.7 FOUR score maintains simplicity and and anesthetists of diverse experience and
provides far better information, particularly for different qualification.
intubated patients. The FOUR score is a good Study results demonstrated that the
predictor of the prognosis of critically ill predictive value of the FOUR score for ICU
patients and has important advantages over mortality is slightly on the higher side than
the GCS in the ICU setting.14 The FOUR score the GCS as Cohen et al (2009) found a higher
has been developed to assess the depth of predictive value for the FOUR score than the
coma in a more detailed manner than the GCS. 2 The study also demonstrated an
GCS.11 excellent internal consistency for both GCS and

Nursing and Midwifery Research Journal, Vol-9, No.2, April 2013 70


FOUR score and good correlation between of use and global acceptance raises the
GCS and FOUR score. High degree of internal potential of GCS scoring in the critical care
consistency for both GCS and FOUR score and neurological assessment, provided one keeps
good correlation between GCS and FOUR in mind its limitations.
score were also elicited in 2 studies The raters agreed that Four score is
conducted by Wijdicks et al (2005)3 and Iyer reliable, easy to use, reproducible gives more
VN et al (2009).14 clinical information and takes less time to
The study conducted by Wolf et al perform and can be used as alternate to GCS.
The study findings recommend that the
(2007) found that the inter-rater agreement
Four score can replace the GCS scale since
among experienced and inexperienced
the inter-rater agreement between nurse-nurse
neuroscience ICU nurses was good to
pair and anesthetist-anesthetist pair was fair
excellent with the FOUR score and the GCS6
to moderate and the inter-rater agreement
but in this present study only fair to moderate
between nurse-anesthetist pair was good in
agreement was found for the FOUR score and
total FOUR score. It is also recommended that
GCS among all 100pairs of raters and at the
the health care team need to be trained and
same time, good inter-rater agreement existed
made proficient in using the FOUR score before
between Nurse Anesthetist pair for the FOUR
its implementation in the clinical settings and
score. In other pairs (Nurse-nurse pair and
a similar study can be replicated with large
anesthetist-anesthetist pair) the inter-rater
sample size involving more number of patients
agreement was fair to moderate. of different population for a longer period of
Majority of the raters of this present time to confirm the usability and reliability of
study agreed and 10 of them strongly agreed the FOUR score.
that the FOUR score can be used as an
alternative to the GCS. References:
The FOUR score and the GCS were 1. Hickey JV. The clinical practice of
almost equally able to predict mortality in this neurological and neurosurgical nursing. 4th ed.
population. The excellent internal consistency Philadelphia: Lippincott; 1997.134-7
and good level of inter-rater agreement among
2. Cohen J. Interrater reliability and
Nurse Anesthetist pair suggests that the new
predictive validity of the FOUR score coma scale
scale is consistent and reliable and that nurses
in a pediatric population. Journal of neuroscience
with differing levels of experience and
nursing 2009 Oct; 41(5): 261-7
expertise are more likely to correctly assess
the patient and assign the same score using 3. Wijdicks EF, Bamlet WR, Maramattom BV,
the FOUR score. The FOUR score has the Manno EM, McClelland IR. Validation of a new
potential to become an important measure in coma scale: The FOUR score. Annals of neurology
prospective clinical studies. However, the ease 2005; 58: 585- 93

Nursing and Midwifery Research Journal, Vol-9, No.2, April 2013 71


11. Wijdicks EFM, Bamlet WR, Maramattom
4. Murthy TVSP. A new score to validate
BV, Manno EM, McClelland RL. Does the JFK
coma in emergency department FOUR score.
Revised Coma Recovery Complement the FOUR
Indian journal of neurotrauma. 2009; 6(1):59-62
Score? Annals of neurology 2006 Dec; 60(6): 745
5. Fischer M, Regg S, Czaplinski A, 12. Schnakers C, Giacino J, Kalmar K, Piret
Strohmeier M, Lehmann A, Tschan F, et al. Inter- S, Lopez E, Boly M, et al. Does the Four Score
rater reliability of the Full Outline of Correctly Diagnose the Vegetative and Minimally
UnResponsiveness score and the Glasgow Coma Conscious States? Annals of neurology 2006 Dec;
Scale in critically ill patients: a prospective 60(6): 746
observational study. Critical care 2010;14(2): R64
13. Wijdicks EFM, Kokmen E, OBrien PC.
6. Wolf CA, Wijdicks EFM, Bamlet WR, Measurement of impaired consciousness in the
McClelland LR. Further validation of the FOUR neurological intensive care unit: a new test.
score coma scale by intensive care nurses. Mayo Journal of neurology neurosurgery psychiatry.
clinic proceedings. 2007 Apr; 82(4):435-8 1998; 64:117 - 19
7. Ankavipat P. Endorsement of the FOUR 14. Iyer VN, Mandrekar NJ, Danielson RD,
score for consciousness assessment in Zubkov AY, Elmer JL, Wijdicks EFM. Validity of
neurosurgical patients. Neurolgia Medico- the FOUR score coma scale in the medical
Chirurgica 2009 Dec; 49(12): 565-71 intensive care. Mayo clinic proceedings 2009
8. Balestreri M et al. Predictive value of August; 84(8): 694-701
Glasgow coma scale after brain trauma: change 15. Idrovo L, Fuentes B, Medina J, Gabaldn
in trend over the past ten years. Journal of L, Ruiz-Ares G, Abenza MJ, et al. Validation of the
Neurology Neurosurgery Psychiatry (2004); 75: FOUR score (Spanish Version) in acute stroke:
161, 162 An interobserver variability Study. European
9. Rowley G, Fielding K. Reliability and Neurology 2010; 63: 364-69
accuracy of the Glasgow Coma Scale with 16. Ledoux D, Bruno M, Jonlet S, Choi P,
experienced and inexperienced users. Lancet. Schnakers C, Damas F, et al. Full Outline of
1991; 337:535 38 Unresponsiveness compared with Glasgow
10. SoRelle Ruth. FOUR score challenges coma scale assessment and outcome prediction
Glasgow Coma Scale. Emergency Medicine in coma. Critical Care. 2009;13(1): 107
News [online] August 2006; 28(8): 50-55.
Available from: URL: http://journals.lww.com/
em-news/Fulltext/2006/08000/
FOUR_Score_Challenges_
Glasgow_Coma_Scale.2.aspx [accessed
January 2012]

Nursing and Midwifery Research Journal, Vol-9, No.2, April 2013 72