2010; 32: 947–952

BEME RAPID REVIEW

Measuring the educational environment in

health professions studies: A systematic review
DIANTHA SOEMANTRI1, CRISTIAN HERRERA2,3 & ARNOLDO RIQUELME2,4
1
Department of Medical Education, Faculty of Medicine, Universitas Indonesia, Indonesia, 2School of Medicine, Pontificia
Universidad Católica de Chile, Chile, 3Evidence Based Medicine Unit, Pontificia Universidad Católica de Chile, Chile,
4
Department of Gastroenterology, Pontificia Universidad Católica de Chile, Chile

Abstract
Background: One of the determinants of the medical student’s behaviour is the medical school learning environment.
Aim: The aim of this research was to identify the instruments used to measure the educational environment in health professions
education and to assess their validity and reliability.
Med Teach Downloaded from informahealthcare.com by University of Melbourne on 11/26/10

Methods: We performed an electronic search in the medical literature analysis and retrieval system online (MEDLINE) and Timelit
(Topics in medical education) databases through to October 2008. The non-electronic search (hand searching) was conducted
through reviewing the references of the retrieved studies and identifying the relevant ones. Two independent authors read, rated
and selected studies for the review according to the pre-specified criteria. The inter-rater agreement was measured with Kappa
coefficient.
Results: Seventy-nine studies were included with the Kappa coefficient of 0.79, which demonstrated a reliable process, and 31
instruments were extracted. The Dundee Ready Education Environment Measure, Postgraduate Hospital Educational Environment
Measure, Clinical Learning Environment and Supervision and Dental Student Learning Environment Survey are likely to be the
most suitable instruments for undergraduate medicine, postgraduate medicine, nursing and dental education, respectively.
For personal use only.

Conclusions: As a valid and reliable instrument is available for each educational setting, a study to assess the educational
environment should become a part of an institution’s good educational practice. Further studies employing a wider range of
databases with more elaborated search strategies will increase the comprehensiveness of the systematic review.

Introduction Practice points

One determinant of a medical student’s behaviour is the
medical school learning environment (Genn 2001a). The
environment related to the student’s behaviour is as perceived
by the student. Genn and Harden (1986) suggested that
although the concept is rather intangible, the effects of an
educational environment are substantial, real and influential.
The climate makes a notable contribution to the prediction of
student achievement, satisfaction and success (Genn 2001b).
The relationship between the educational environment as
perceived or experienced by the students and their achieve-
ment, satisfaction and success in medical school has brought
into awareness the need to study or evaluate the medical
school’s environment. The World Federation for Medical
Education (1998) has considered the learning environment as
one of the targets for the evaluation of medical education
programmes. Based on all of the above, a study of the
environment should become an inseparable part of the study
of a medical school’s curriculum.
Genn (2001a) provides a definition for educational envi-
ronment as the manifestation of a curriculum. Rothman and
Ayoade (1970) also consider the learning environment as ‘a
manifestation of the effects on students of the various parts of
the curriculum’. Maudsley (2001) states that ‘a learning
. Measuring the educational environment should become
a part of educational institutions’ good practices.
. Each educational context or setting demands different
aspects of environment to be measured.
. Various instruments to measure educational environ-
ment in different health professions education settings
are available.
. Some instruments have proven their psychometric
values.
environment exists wherever and whenever students
gather . . . ’ and it includes various factors, which contribute
to an effective education and becomes the background in
which the curriculum exists.
Pace and Stern (1958) suggest that an educational environ-
ment is characterized by the pressures, stresses, practices,
policies, rewards and values within the classroom or school of
which they refer to as ‘environmental press’. Furthermore, one
of the 10 questions to be asked when planning a course or
curriculum is ‘what educational environment or climate should
be fostered?’ (Harden 1986). The course or curriculum planner

Correspondence: Diantha Soemantri Department of Medical Education, Faculty of Medicine, Universitas Indonesia, Jakarta Pusat 10430, Indonesia.
Tel/Fax: 62 21 3901814; email: dianthasoemantri@yahoo.com; diantha.soemantri@ui.ac.id
ISSN 0142–159X print/ISSN 1466–187X online/10/120947–6 ß 2010 Informa UK Ltd. 947
DOI: 10.3109/01421591003686229
 D. Soemantri et al.
should evaluate those areas that are fostered and encouraged
in the institution.
The term environment and climate are often used inter-
changeably in educational literature. Genn (2001b) defines the
term climate as the educational environment of an institution
as perceived by students. Roff and McAleer (2001) use the term
educational climate as the synonym of educational environ-
ment. In the study conducted by Rothman & Ayoade (1970),
the learning environment is defined as what the students
perceive and thus no differentiation is made between the term
environment and climate. Therefore, in this article the educa-
tional climate will be regarded as similar to educational
environment.
However, it is the students’ perceptions of the environment,
rather than the environment itself, that determines the behav-
iour (Hutchins 1961; Rothman & Ayoade 1970; Genn 2001a;
Konings et al. 2005). Students’ perceptions of the classroom
Med Teach Downloaded from informahealthcare.com by University of Melbourne on 11/26/10
environment have been used to predict their outcomes (Fraser
et al. 1986). Students, as the ‘inhabitants’ of the (Genn 2001a),
are the ones who experience the environment and it is their
perceptions of the environment, which then becomes one of
the determinants of their behaviour.
A quantitative measurement of the educational environ-
ment requires the use of an inventory or instrument. The
selection of such an instrument should be based on the
qualities of the measurement process using a particular
For personal use only.
instrument or the suitability of that instrument to measure
educational environment. The qualities, or often referred to as
psychometric features, usually come under two main head-
ings, which are validity and reliability. A valid and reliable
learning environment tool allows a meaningful measure of the
learning environment of an institution, and thus appropriate
measures to improve the environment can be taken. The
differences among various educational settings may require a
different educational environment inventory, which suits the
specific situation of a particular institution.
In any measurement process, it is necessary to ascertain
whether the instrument or inventory measures what it
supposes to measure; and validity deals with this specific
question. Content validity refers to the extent that the
instrument or inventory measures the intended subject
matter content (Gronlund 1976a). Criterion-related validity
refers to the extent to which the measurement results can be
used to predict a future outcome and also the extent to which
they correlate with current results ( performance) obtained by
other valid measures (Gronlund 1976a). Construct validity may
be defined as the extent to which the test can be used to
measure certain psychological constructs and the results
interpreted in terms of those constructs (Gronlund 1976a).
Reliability refers to the reproducibility of the measurement
or assessment results. A result may be reliable over different
periods of time, over different raters or over different samples
of questions (Gronlund 1976b). An unreliable or inconsistent
measurement result cannot possibly permit valid interpretation
of the result. A high correlation between the scores of
individual items, in other words a high internal consistency,
would indicate that the scores measure a single construct. The
degree of reliability adequate for a particular measurement
depends on: the purpose of the measurement, the use of the
948
results, the importance of the decision that will be made and
ultimately the consequences resulting from the measurement.
Reliability coefficient of 0.60 is considered acceptable for
questionnaires (Nunnally 1978).
Objectives
The objectives of this systematic review were to respond to the
following two research questions:
(1) Which instruments or inventories are used to measure
the educational environment in health professions
education?
(2) How is the suitability of each instrument in measuring
the educational environment in health professions
education?
Methods
A systematic review approach is applicable in medical
education research. The purpose is similar to that of the
systematic review in medicine, which is to reflect the best
evidence available in medical education (BEME Collaboration
2006). Steps in conducting a systematic review can be
summarized as follows: formulation of a specific review
question, comprehensive search and inclusion of studies,
quality assessment of included studies and data extraction,
synthesis of study results, interpretation of results and report
writing (Pai et al. 2004). These steps should result in a
summary of evidence related to a specific research question.
Search strategy
We performed an electronic search in the MEDLINE and
Timelit (Topics in Medical Education) databases through to
October 2008. The non-electronic search (hand searching) was
conducted through reviewing the references of the retrieved
studies and identifying the relevant ones. The MEDLINE search
strategy combined the available and appropriate headings
medical subject (MeSH) which are (exp) educational mea-
surement/, (exp) education/, (exp) learning/ with the free
keywords ‘education* environment’, ‘climate’, ‘learning envi-
ronment’ and ‘education* climate’. In the Timelit database in
which the search using subject headings is not possible, a
search using the relevant keywords (‘educational climate’,
‘educational environment’, ‘education environment’ and
‘learning environment’) was conducted.
Inclusion and exclusion criteria
We included studies that were in the area of health professions
education (undergraduate or postgraduate), which measures
the educational or learning environment with a quantitative
method (employs an instrument or inventory). A study was
excluded if it was a review article, a study in non-health
professions education, did not measure the overall educational
or learning environment of the institution and employed a
qualitative research method.

Data extraction
One author read all the titles retrieved by the searches and
eliminated the studies that were clearly of another subject.
From the resultant list, two independent authors read the
abstracts in the records to identify publications that appeared
to be eligible for this review. From the chosen abstracts, two
independent authors read the full texts, rated and selected
studies for the review according to the pre-specified (inclusion
and exclusion) criteria. Disagreements were resolved by
discussion. The inter-rater agreement was measured with
Kappa coefficient.
Data synthesis
To respond to the first question of this review, the search
strategy allowed us to identify the inventories or instruments
Med Teach Downloaded from informahealthcare.com by University of Melbourne on 11/26/10
that were used to measure the educational environment in
health professions education.
To answer the second research question, the data regarding
the validity and reliability of the educational environment
measurement results using each particular inventory was
extracted, analyzed and synthesized. Some instruments were
used in several studies, therefore the information was gathered
from all the included studies utilizing those particular instru-
ments. Based on the available information, the most suitable
instrument or inventory to measure educational environment
For personal use only.
in different fields of health professions education was identi-
fied. The information concerning the educational environment
measurement instrument was extracted from the included
studies: the health professions education setting in which the
instrument was developed and/or used; the general informa-
tion about the instrument, including its subscales; the validity
of the results (content validity, criterion-related validity,
construct validity) and the reliability of the results.
Results
Following the search strategy, the electronic databases
(MEDLINE 1966 to October 2008 and Timelit) yielded 951
titles. Duplication of titles from the two databases were
identified and removed. As a result of the search strategy and
subsequent screening, 178 studies (1958–2008) were consid-
ered to be relevant for further review. Full papers of the studies
were obtained. Two reviewers worked independently to
review the studies and make the decision on each of them.
The reviewers disagreed on 19 studies. Therefore, the inter-
rater agreement was substantial, with 90% agreement and the
Kappa coefficient of 0.79. Kappa coefficient takes into account
the occurrence of chance agreement.
Thirty one studies were excluded because they were not
primary studies of educational environment. Those studies
were literature reviews, editorials or commentaries/letters
around the topic of educational environment. Another exclu-
sion criterion was the primary educational environment study,
which was conducted in non-health professions education
settings, such as secondary education or other subject disci-
plines in higher education level, 14 studies were excluded
following this criterion.
BEME Rapid Review
Eighteen studies employed qualitative research methods,
such as interviews or observation, to assess the educational
environment or certain aspects of it. Thus, those studies were
excluded. Another 36 studies were excluded because they
only measured a certain aspect of the educational environ-
ment, such as students’ stress, coping, support services,
student abuse and harassment, perceptions of teaching and
course experience. The reviewers considered those studies to
have too narrow coverage of educational environment, and
therefore could not be regarded as measuring the full
educational environment.
Following the flowchart, 79 studies from 1961 to 2008 were
included (Figure 1). These were primary studies, which
measured the educational environment in health professions
education using specific instruments. The area of health
professions education included undergraduate and postgrad-
uate medicine, nursing and dental education.
Thirty one instruments were identified from the 79 included
studies. Some instruments were developed specifically to be
used in particular settings, while others were adapted from
another instrument. Ten instruments were used in undergrad-
uate medical education, seven instruments for postgraduate
medical education, eight and four instruments were used in
nursing and dental education, respectively. One instrument
was used for both undergraduate and postgraduate medical
education and one instrument was used in all areas (under-
graduate and postgraduate medicine, nursing, dental, chiro-
practic education). Table 1 summarizes the identified
instruments along with the settings in which the instruments
were used.
Data regarding the use of the identified instruments in each
article were extracted and synthesized. They include the
psychometric properties presented in the articles as the results
of the use of the instruments in the educational environment
Studies on educational or learning
environment (178)
Review studies (31)
Studies in non health
professions education (14)
Original undergraduate and
postgraduate studies (133)
Studies which measure only Studies with qualitative
a particular aspect of the
environment (36)
methods (18)
Studies with quantitative method (79)
(Kappa coefficient=0.79)
Instruments which measure educational environment in health
professions education (31)
Figure 1. Disposition of 178 articles on educational/learning
environment.
Note: Solid and dash lines indicate included and excluded
studies, respectively.
949
 D. Soemantri et al.

Table 1. Undergraduate and postgraduate health profession educational environment measurement instruments identified in
the systematic review.

No. Instrument/Inventory Setting

1 Medical school learning environment Survey – modified (MSLES) Medicine
2 Learning environment questionnaire (LEQ) Medicine
3 Questionnaire from Parry et al. Medicine
4 Veteran affairs (VA) learners’ perceptions survey Medicine ( postgraduate, PG)
5 Learning environment assessment (LEA) Medicine (PG)
6 Medical school environment questionnaire (MSEQ) Medicine
7 Questionnaire from Robins et al. Medicine
8 Medical school environment inventory (MSEI) Medicine
9 Questionnaire from Rotem, Godwin and Du Medicine (PG)
10 Course valuing inventory (CVI) Medicine
11 DREEM Medicine (undergraduate and PG), nursing,
dental and chiropractic
12 Operating room educational environment measure (OREEM) Medicine (PG)
13 Instrument from Pololi and Price Medicine
14 Surgical theatre educational environment measure (STEEM) Medicine (undergraduate and PG)
15 Anaesthetic theatre educational environment measure (ATEEM) Medicine (PG)
16 Practice-based educational environment measure Medicine (PG)
Med Teach Downloaded from informahealthcare.com by University of Melbourne on 11/26/10

17 Postgraduate hospital educational environment measure (PHEEM) Medicine (PG)

18 Questionnaire from Patel and Dauphinee Medicine
19 Learning Climate Measure from Wangsaturaka Medicine
20 Questionnaire from Orton – Modified Nursing
21 Questionnaire from Hart and Rotem Nursing
22 Clinical learning environment scale (CLE) Nursing
23 Clinical learning environment inventory (CLEI) Nursing
24 College and university classroom environment inventory (CUCEI) Nursing
25 Clinical learning environment and supervision (CLES) Nursing
26 Clinical post-conference learning environment survey (CPCLES) Nursing
27 Postgraduate research experience questionnaire (PREQ) Nursing
28 Questionnaire from Gerzina, McLean and Fairley Dentistry
29 Clinical education instructional quality (ClinEd IQ) Dentistry
For personal use only.

30 Learning environment survey (LES) Dentistry

31 Dental student learning environment survey (DSLES) Dentistry

measurement processes. The synthesis of data were presented
in the form of table for each area of the health profession study
(Tables 2–5, available at www.medicaltacher.org). The row
lists the educational environment measurement instrument
whereas the column lists the psychometric properties of each
instrument, which consist of content validity/content evidence,
criterion validity/relation other variables, construct validity and
reliability/internal structure.
Discussion
Specific databases which contain publications in medical
education are of limited availability. It becomes more chal-
lenging to conduct a comprehensive search because of this
issue. Relevant papers are more likely to be missed and large
numbers of false hits occur. Moreover, finding the appropriate
subject headings may become a problem because the database
(MEDLINE) is not specific to medical education and thus, the
availability of subject headings relevant to medical education is
limited. It was then necessary to combine the search with
relevant keywords (‘education environment’, ‘education cli-
mate’, ‘learning environment’ and ‘climate’) to increase the
specificity. Notwithstanding, our search strategy was likely to
mislay relevant studies due to the lack of specific search tools
for the medical education area. In that order, the use of only
two databases and a low-sensibility search strategy become
limitations of this study. This is caused by the limited resources
on the researcher’s side. For a systematic and comprehensive
search to be successfully conducted, adequate resources
950
(human resources, educational material, time and cost) are
essential. In order to improve the comprehensiveness of the
search, the hand searching of the references cited in the
retrieved articles was conducted. The process yielded a
substantial number of additional articles, which were not
retrieved through the database search.
The decision to include or exclude each study was based on
the criteria visualized in the flowchart (Figure 1). Although the
flowchart was clear and the reviewers understood the criteria
beforehand, some disagreements occurred inadvertently
during the process. The disagreements mostly emerged when
making the decisions on studies that only measured a certain
aspect of the educational environment; they resulted from
differences in interpretation about those particular studies.
A high percentage of agreement (90%) between the two
independent reviewers and the Kappa coefficient of 0.79 show
a fairly reliable result of the inclusion/exclusion (review)
process. The systematic search yielded a substantial number of
instruments for measuring educational environment in health
professions education. The measurement of educational envi-
ronment has received important attention from educationalists,
highlighted by the availability of various instruments to be
utilized in different educational settings.
Analysis of the validity and reliability of educational
environment measurement instruments
Medicine (undergraduate). There were 12 educational envi-
ronment measurement instruments identified in this study.

The summary of the psychometric qualities of the instrument-
sare presented in Table 2, available at www.medicalteacher.
org. Several instruments (questionnaire from Parry et al.
(2002), Pololi & Price (2000), Patel and Dauphinee (1985),
Robins et al. (1997), MSEI (Hutchins 1961) and MSEQ
(Wakeford 1981, 1984)) had considerable weaknesses in
their validity and reliability. The Learning Climate Measure
was an instrument developed specifically to measure the
educational environment of Thai medical education
(Wangsaturaka 2005). Therefore, it was a nation-specific
instrument and its use was limited to that specific context.
The STEEM was originally developed for postgraduate surgical
learning setting. When applied to the undergraduate setting, it
has demonstrated some weaknesses in terms of its validity
(Nagraj et al. 2006).
The LEQ (Rothman 1970; Rothman & Ayoade 1970; Levy
et al. 1973; Kaufman & Mann 1996; Schwartz & Loten 2003,
Med Teach Downloaded from informahealthcare.com by University of Melbourne on 11/26/10
2004a, b) MSLES (Feletti & Clarke 1981a, b; Clarke et al. 1984;
Lancaster et al. 1997) and DREEM (Roff et al. 1997, 2001;
Al-Qahtani 1999; Al-Zidgali 1999; Baozhi 2003; Bassaw et al.
2003; Vieira et al. 2003; Zaini 2003; Al-Hazimi et al. 2004a, b;
Mayya & Roff 2004; Till 2004, 2005; Jiffry 2005; Varma et al.
2005; Dunne et al. 2006) demonstrated the robustness in terms
of their psychometric qualities. Their content validity was
established and they also had high internal consistency. The
construct validity of the MSLES and DREEM were indicated by
For personal use only.
their ability to significantly differentiate the students’ percep-
tions of the learning environment between medical schools
with the traditional curriculum and ones with the more
innovative curriculum (Feletti & Clarke 1981b; Al-Qahtani
1999; Zaini 2003; Al-Hazimi et al. 2004a). The DREEM showed
an additional strength because it could be applied in medical
schools in different countries, cultures and contexts and still
demonstrated consistent reliability. In relation to criterion
validity, there was a relationship between the DREEM scores,
which reflected the students’ perceptions of the learning
environment and academic achievement (Baozhi 2003; Mayya
& Roff 2004).
According to the data presented in the related articles, the
DREEM is likely to be the most suitable instrument to be
applied in undergraduate medical education settings. The
validity had been established, and the instrument demon-
strated highly reliable results consistently throughout its
administration in different contexts. The results also indicated
the ability of the DREEM to differentiate between the learning
environment of the traditional and one of the more innovative
medical schools.
Medicine (postgraduate). Most of the identified educational
environment measurement instruments in postgraduate med-
ical education were developed for use in specific postgraduate
specialty areas. The psychometric qualities of the identified
instruments are summarized in Table 3, available at www.
medicalteacher.org. Postgraduate medical education pro-
grammes consisted of many different specialties and levels of
training. Each speciality had its own uniqueness in terms of the
educational environment. This situation often required a
specific instrument which could address the certain aspects
of the educational environment in the particular setting.
BEME Rapid Review
Four instruments: the STEEM (Cassar 2004), the OREEM
(Kanashiro et al. 2006), the ATEEM (Holt & Roff 2004) and the
practice-based educational environment measure (Mulrooney
2005) were developed for use in the surgical/operating theatre
learning setting, anaesthetic theatre learning setting and
practice-based component in general practice (GP) training,
respectively. The VA Learners’ Perceptions Survey was
designed for residents from different specialties within the
United States VA clinical training setting (Keitz et al. 2003). In
the process of deciding the most suitable instrument for use in
postgraduate medical education, it was appropriate to exclude
those instruments. The decision was not based on the quality
of those instruments, since most of them demonstrated good
validity and reliability, but rather because of the specific
content of the instrument, which limited their use in a more
general postgraduate education setting.
The content validity of the LEA was not established,
although it showed fairly high internal consistency (Roth
et al. 2006). The questionnaire from Rotem et al. (1995) and
the PHEEM established content validity through clear descrip-
tion of the development process of the instruments (Roff et al.
2005). The strengths of the DREEM were the high internal
consistency and concurrent validity. However, the content and
construct validities were not established since the factor
analysis demonstrated that some modifications were necessary
for the instrument to be applied in postgraduate setting
(Bassaw et al. 2003; de Oliveira Filho & Schonhorst 2005;
de Oliveira Filho et al. 2005a, b).
The PHEEM consisted of 40 items divided into three
subscales ( perceptions of role autonomy, perceptions of
teaching and perceptions of social support). The reliability of
the PHEEM was better compared to that of the Rotem, Godwin
and Du questionnaire. It has also been shown that by using
PHEEM, reliable results can be obtained with feasible sample
size. In addition, the PHEEM had been administered to several
different sample groups, such as senior house officers and
specialist registrars from different specialties, and demon-
strated almost similar reliability coefficients in those groups
(Jayashree 2004; Roff et al. 2005; Aspegren et al. 2007; Boor
et al. 2007; Clapham et al. 2007). Some researches have
conducted factor analysis on PHEEM and they demonstrated
different results regarding the factors, which the instrument
measures (Aspegren et al. 2007; Boor et al. 2007; Clapham
et al. 2007). The construct validity of the Rotem, Godwin and
Du questionnaire was not clearly established. Therefore, the
PHEEM is likely to be the most suitable instrument for use in
postgraduate medical education because of its content validity,
high reliability and also its ability to be used in different
postgraduate settings.
Nursing. Most of the identified instruments were used to
assess the clinical learning environment in nursing education
setting. A summary of the psychometric qualities of the
instruments’ is presented in Table 4, available at www.
medicalteacher.org. All instruments, except CUCEI (Fisher &
Parkinson 1998), provided moderate to high reliability coef-
ficients. Several instruments, such as the CPCLES (Letizia &
Jennrich 1998), the questionnaire from Hart and Rotem (1995),
and the DREEM (Pimparyon et al. 2000; Al-Sketty 2003;
951
 D. Soemantri et al.
O’Brien et al. 2008) demonstrated some weaknesses in regard
to the construct validity of the instruments. Almost all
instruments have demonstrated a high degree of content
validity. The PREQ covered several aspects of an educational
environment (supervision, intellectual climate, clarity, infra-
structure, skills development and thesis examination process).
However, according to the validity analysis, there are some
subscales or statements in the instrument that were too specific
for a particular nursing education setting (Drennan 2008),
which then limit its use at different nursing education settings.
The content and construct validities of the CLES have been
established. It was administered to the groups of nursing
students from different countries (Finland and United
Kingdom) and it has demonstrated marginal to high reliability
coefficient (internal consistency) in those studies (Saarikoski &
Leino-Kilpi 1999, 2002; Saarikoski et al. 2002). The CLEI had
also been administered in several contexts of nursing educa-
Med Teach Downloaded from informahealthcare.com by University of Melbourne on 11/26/10
tion. However, the internal consistency was fairly poor
compared to that of other instruments (Chan 2001a, b, 2002,
2003; Ip & Chan 2005; Henderson et al. 2006; Midgley 2006;
Chan & Ip 2007). The reliability coefficients of the CLE Scale
(Dunn & Burnett 1995; Dunn & Hansford 1997) were lower
than those of the CLES, although both the instruments’
construct validity has been confirmed. Therefore, based on
the available information and analysis, the CLES is likely to be
the most suitable instrument to assess nursing students’
For personal use only.
perceptions of their clinical learning environment.
Dentistry. There were five instruments for dental educational
environment measurement identified in this study. The validity
and reliability of those instruments varied and are summarized
in Table 5, available at www.medicalteacher.org.
There was only one instrument that was originally devel-
oped for dental educational environment (Gerzina et al. 2005).
The others were modifications or identical version of instru-
ments, which were originally designed to be used in medical
school. The content validity of those instruments was ques-
tionable since there were no processes of validating them
before they were applied in dental education settings,
although the original versions may have proven to be valid.
In addition to that, two instruments (LES and DSLES) were
developed from the MSLES. It was an instrument first devel-
oped in 1970s and it might not encompass current trends or
changes in educational practice nowadays. The DSLES was
more identical to the MSLES than the LES, because the LES was
a shorter version of the MSLES whereas the changes in the
DSLES were only minor.
The LES (Stewart et al. 2006) demonstrated the highest
overall reliability coefficient (Cronbach’s alpha of 0.97). The
reliability of the ClinEd IQ (Henzi et al. 2006), the DREEM
(Zamzuri et al. 2004) and the questionnaire from Gerzina et al.
(2005) were not tested. The DSLES also demonstrated good
reliability (Cronbach’s alpha of 0.91), which was comparable
to those of the MSLES (Henzi et al. 2005).
Based on the information provided in the studies and the
analysis conducted, the DSLES is likely to be the most suitable
instrument for measuring educational environment in dental
education setting. It demonstrated good reliability and since
952
the DSLES was more similar to MSLES than the LES, it was
more likely to have better content validity.
Conclusion
The systematic search yielded 178 studies, which were
considered to be eligible for further review. The pre-specified
criteria were applied to those studies and finally 79 studies,
which were the original/primary studies measuring the overall
educational/learning environment in health professions edu-
cation using specific instruments, were included. A substantial
number of educational environment measurement instruments
(31 instruments) were extracted from the included studies. The
use of a wider range of databases with more elaborated search
strategies will increase the comprehensiveness of the system-
atic review.
The DREEM, PHEEM, CLES and DSLES are likely to be the
most suitable instruments for undergraduate medicine, post-
graduate medicine, nursing and dental education, respectively.
The content validity was established through elaborate
description of the development process of the instruments.
Some instruments have added values with the establishment of
their construct validity. These educational environment mea-
surement instruments also demonstrated consistency through-
out their applications in different contexts/settings.
Further analysis will be useful to explore the ability of each
instrument as a predictor variable for a particular educational
achievement. Furthermore, a study on students’ perceptions of
the educational environment should become a part of good
educational practice of an institution, as a suitable, valid and
reliable instrument for each educational context is available.
Acknowledgements
The authors thank Sue Roff and Sean McAleer for their
comments and support in completing the project and finalizing
the manuscript. This work is partially supported by grants of
the National Commission for Scientific and Technological
Research (CONICYT), FONDECYT no. 11004336 to A.R.
Declaration of interest: The authors report no conflicts of
interest. The authors alone are responsible for the content and
writing of the article.
Notes on contributors
DIANTHA SOEMANTRI, MD, MmedEd, is a lecturer in medical education in
the Department of Medical Education, Faculty of Medicine, Universitas
Indonesia.
CRISTIAN HERRERA is member of the Evidence Based Medicine Unit and
the Health Policy and Systems Research Unit from the School of Medicine,
Pontificia Universidad Católica de Chile.
ARNOLDO RIQUELME, MD, MmedEd, is an undergraduate and postgrad-
uate clinical tutor in Internal Medicine and consultant in the Department of
Gastroenterology, Pontificia Universidad Catolica de Chile School of
Medicine, Chile.
References
The references for this article can be viewed at www.medicalteacher.org.