Qual Quant

DOI 10.1007/s11135-012-9706-3

Developing measures for performance excellence:

is the Baldrige criteria sufficient for performance
excellence in higher education?
Muhammad Asif Abdul Raouf Cory Searcy

Received: 12 January 2012 / Accepted: 10 April 2012

Springer Science+Business Media B.V. 2012

Abstract The implementation of the education criteria for performance excellence

(ECPE)the education version of the Baldrige criteriais driven by the growing need
for performance excellence in higher education. Ironically the criteria meant for performance excellence have no theoretical underpinning and are based on normative performance
improvement requirements. The lack of content and internal validity is reflected in some
critical weaknesses in the criteria. Based on a critical review, this paper highlights potential
improvements in the areas of aligning processes with the organisational mission, operations
focus, customer focus, workforce focus, and performance measurement. To help enhance
the ECPE, the paper develops performance measures for systematic implementation of the
criteria. This is discussed in terms of defining desired outcomes, developing key measures,
identifying specific indicators to track performance, and choosing methods to assess performance. The discussions provide a basis for future revisions to the ECPE.
Keywords Baldrige criteria Education criteria for performance excellence (ECPE)
Performance excellence Higher education Performance measurement

M. Asif (B)
Prince Sultan University, KSA, Riyadh, Saudi Arabia
e-mail: Muhammad.assif@gmail.com
A. Raouf
University of Management and Technology, Lahore, Pakistan
e-mail: abdulraouf@umt.edu.pk
C. Searcy
Ryerson University, Toronto, ON, Canada
e-mail: cory.searcy@ryerson.ca

123

M. Asif et al.

1 Introduction
The Baldrige criteria for performance excellence (BCPE) consists of three versions: business, healthcare, and education. The education criteria for performance excellence (ECPE)
are used by educational institutes worldwide to impact bottom-line improvements in performance including operational, financial, and market outcomes. The criteria require an
organisation to assess its improvement efforts, diagnose its overall performance management systems and identify strengths and opportunities for improvement (NIST 2010). The
Baldrige criteria is considered a comprehensive means for achieving performance excellence
in education. Since the criteria share some common elements with TQM (Alonso-Almeida
and Fuentes-Fras 2012; Seetharaman et al. 2006), it is also considered a means for TQM
implementation (Bou-Llusar et al. 2009; Black and Porter 1996; Dean and Bowen 1994;
Flynn and Saladin 2006).
This paper challenges the basic structure of the ECPE and identifies a number of areas for
improvement in the ECPE. The main focus of the ECPE is on describing processes that are
necessary for achieving excellence. In doing so, the ECPE provides organisations flexibility
in terms of achieving the desired targets. However, the ECPE does not guide the systematic
implementation and performance measurement of the criteria. This paper has two main contributions. First, it identifies a number of areas for improvement that should be addressed
in future revisions of the ECPE. Second, the paper develops performance measures for key
aspects of the ECPE. The purpose is to elaborate the process of development of such measures
so that measures could be developed for the rest of the processes in the ECPE.
The rest of the paper is structured as follows. The next section provides an overview of
the structure and contents of the ECPE. This is followed by a critical review of the criteria
and areas for improvements are identified. The paper then discusses the need to develop
performance measures. This is further explained through examples of academic processes.
The paper ends with conclusions.

2 ECPE
2.1 The basic structure of the ECPE
The ECPE claims to interpret generic principles of performance excellence in the particular
context of education (ECPE 2011; Houston 2008). The criteria consist of 11 core values
which are embedded in seven categories. Six of these categories are labelled as systematic
processes, which include leadership, strategic planning, operations focus, workforce
focus, customer focus, and measurement, analysis and knowledge management. A focus
on these systematic processes yields performance results, including leadership and governance outcomes, budgeting, financial and marketing outcomes, workforce-focused outcomes, customer-focused outcomes, and student learning and process outcomes (Fig. 1).
Unlike many of the standardised management systemssuch as ISO 9001 and ISO 14001
which are prescriptive in naturethe ECPE is descriptive in nature. This means that the ECPE
does not pose compulsory requirements for organisations but, rather, provides flexibility in
achieving the desired ends. This approach is advantageous because it allows organisations
flexibility in achieving desired targets and organisations can design their processes according
to their own unique context. However, this approach also leaves practitioners with limited
specific guidance in performance improvement, particularly with respect to questions such
as how to achieve targets and what techniques and methods to be employed.

123

Developing measures for performance excellence

Core values
Visionary leadership
Focus on the future
Managing for
innovation
Agility
Organizational and
Embedded in
personal learning
Valuing workforce
members and partners
Student-centered
excellence
Societal responsibility
Management by fact
Focus on results and
creating value
Systems perspective

Systematic Processes
1.Leadership
2.Strategic planning
Yielding
3.Customer focus
4.Measurement, analysis,
and knowledge
management
5.Workforce focus
6.Operations focus

7. Performance Results
Student learning and
process outcomes
Customer-Focused
outcomes
Workforce-Focused
outcomes
Leadership and
governance outcomes
Budgeting, financial and
market outcomes

Fig. 1 The basic structure of the ECPEadapted from the ECPE (2011)

2.2 A snapshot of the literature on the ECPE

The underlying theoretical framework of the Baldrige Criteria is of critical importance, since
it conveys a message about the route to competitiveness (Flynn and Saladin 2006). However,
the ECPE has no theoretical underpinning and is based on normative practices and process
improvement ideas of quality experts and practitioners. While the research on the validation
of BCPEthe business version of the criteriahas expanded in the last few years; this has
not been the case with the ECPE. As Badri et al. (2006, p. 1119) noted, there are only a few
studies that fully address Baldrige in the area of education.
The existing literature can be classified into two categories. The first type of literature
explores the impact of the Baldrige criteria on the performance of educational institutes.
Some representative examples in this type include Belohlav et al. (2004); Evans (1997), and
Weinstein et al. (1998). Belohlav et al. (2004), for instance, found that implementation of
the Baldrige criteria led to a higher level of student engagement in the learning process,
as evidenced by more abundant and higher-quality feedback to the instructors. The overall
message of the first type of research is that implementation of the Baldrige criteria leads to
performance improvements. The second type of research focuses on the contents/categories
and validity of the criteria. However, while there is ample research on the validity of the business version of the criteria (i.e., BCPE), little is known about the validity of the ECPE. The
few exceptions include Badri et al. (2006) and Winn and Cameron (1998). Badri et al. (2006)
found that all categories of the criteria are significantly linked with organizational outcomes
in terms of organizational performance results, and student, stakeholder and market focus.
While that study focused on determining the relationship between the criteria constructs and
performance, Winn and Cameron (1998) examined the validity of the proposed relationships
among the Baldrige dimensions using data from higher education. The findings suggested
that modifications to the criteria were required and that questions remain as to whether the
ECPE constitutes a valid model. This concern is also raised by a number of authors. For
instance, Badri et al. (2006, p. 1119) noted that while developing a curriculum based upon
Baldrige principles has received noteworthy attention, what is not readily evident within the
literature is the actual application of the MBNQA concepts as part of the educational deliv-

123

M. Asif et al.
Table 1 Key areas needing improvement in the ECPE
Key areas needing
improvement in
the criteria

Explanation

Alignment of the
processes with
the mission

The criteria provide an overarching emphasis on the alignment of organisational processes with the mission of the HEI. The underlying assumption
is that organisations are always able to develop a well-formulated mission.
Inability to develop an appropriate missiondue to incomplete understanding of the environment, stakeholders and academic excellence
could result in inefficient processes.
The supply chain in higher education is fundamentally different from the
traditional supply chain in manufacturing. The conventional concepts of
supply chain management discussed in the ECPE do not apply readily in
higher education. Further supplier selection, supplier performance evaluation, and dealing with poorly performing suppliers, as discussed in
the ECPE, reflect a traditional manufacturing approach to supplier management.
The customersupplier relationship in higher education is complex and fundamentally different from other sectors of the economy. Relying on the
input of only one type of stakeholdersuch as studentscould risk the
interests of other stakeholders and the overall integrity of the system.
Management needs to balance the requirements of various stakeholders
to design best-in-class academic programs. The ECPE does not address
this aspect.
This category is too generic and does not discuss workforce management
from an academic perspective.
The ECPE has two main weaknesses regarding performance measurement:
first, performance measures currently mentioned in the criteriasuch as
productivity, cycle time, and efficiencyare rather vague and their
application in an academic setting needs to be discussed further. Second,
the ECPE does not discuss systematic performance measurement. The criteria needs to have a solid approach towards performance measurement.

Operations focus

Customer focus

Workforce focus
Performance measurement

ery process. On a similar note Arif and Smiley (2004) noted that while the Baldrige Award
in education has captured the attention of decision makers, there has been little empirical
research examining the usefulness of the award criteria to guide the actions of organizations
that seek to improve performance.
The gap in existing research is clear. The research needs to advance beyond the relationships between the ECPEs elements and performance to question the existence and
(in)adequacy of the existing elements. As the well-known axiom says you cannot solve a
problem at the same level of consciousness at which it was created. Future research should
establish the validity of the criteria. As a first step of the needed research, this paper provides
a critical analysis of the ECPE. The next section discusses some of the weaknesses of the
criteria.

3 Weaknesses of the ECPE

The weaknesses of the ECPE are summarised in Table 1. A detailed discussion of the weaknesses highlighted is provided in the sections below.

123

Developing measures for performance excellence

3.1 Alignment of organisational processes with the mission

As a fundamental starting point, the ECPE requires educational institutes to develop a mission
and then align organisational processes with that mission. The criteria requires:
What are your stated purpose, vision, values, and mission? What are your organizations core competencies and their relationship to your mission?...What are the key
elements that engage them [workforce] in accomplishing your mission and vision?
(ECPE 2011, p. 4)
How do you select, collect, align, and integrate data and information for tracking daily
operations and overall organizational performance, including progress relative to strategic objectives and action plans? (ECPE 2011, p. 16)
The criteria allow educational institutes considerable flexibility in their approach to performance excellence. Performance excellence, however, is not always about alignment of
organisational processes with the mission. If an organisation fails to develop the right mission, the alignment of the processes with the mission would be of little use. Performance
excellence is also about using the existing knowledge of process management and total quality
management to achieve excellence. For instance, the mission of the institute may require the
organisation to improve satisfaction of stakeholders. The mission, however, does not guide
the institute on how to achieve customer satisfaction. Simply designing academic processes
to align with the mission without using the existing knowledge of quality management may
not be very fruitful. An example is of program design. The mission of the organisation may
require developing an effective syllabus, curriculum, or a program. The existing knowledge
of performance management shows that quality improvement methods, such as quality function deployment (QFD), can be used to systemize program design and to facilitate inclusion
of the voice of the customer. However, the criteria do not draw on the existing knowledge
base. This is despite the fact that applying tools such as QFD in an education context finds
wide support in the literature (see, for example, Aytac and Deniz 2005, Lo Franco and La
Rosa 2012, Martins and Aspinwall 2001, and Sirvanci 2004).
To summarise, future revisions of the criteria should focus on two key issues. First, higher
education institutes (HEIs) should carefully scan the broader external and internal environment and understand their stakeholders and meanings and means of academic excellence
before formulating the mission. This is because only a properly formulated mission and
subsequent alignment of organisational processes with the mission is likely to give rise to performance excellence. In other words, the ECPE should go beyond mere alignment. Second,
the ECPE should place an explicit focus on using the advanced knowledge of performance
management during program design and the development of other processes. Program design
aligned with the mission is not sufficient on its own for developing best-in-class programs
and, thus, performance excellence.
3.2 Operations focus
This category in the ECPE consists of two clauses: (1) Work systems: how do you design,
manage, and improve your work systems? and (2) Work processes: how do you design,
manage, and improve your key work processes? The criteria requires:
Supply-Chain Management: How do you manage your supply chain? How do you
ensure that suppliers you select are qualified and positioned to enhance your per-

123

M. Asif et al.
Table 2 Customersupplier relationship in higher education-based on Asif and Raouf (2012) and Raouf
(2004)
Customer

Supplier

Services by supplier

Student

Faculty

Curriculum design, teaching,

professional and intellectual
development
Support services, material and
equipment, provision of required
resources
Learning in group and team work

Administrators

Students
Faculty members

Faculty members
Students

Industry and
parents of
students

Higher Education
institutes

Higher Education Institutes

Funders
Community
Professional and accreditation bodies

Learning from peers and senior

faculty members
Feedback from students about
teaching methods, course
evaluation, teacher evaluation, and
the overall program design
Knowledge, wisdom, know-how,
know-what, know-why,
professional and intellectual
development
Financial resources
License to operate, provision of
required resources
Regular program assessment,
provision of professional
requirements and guidelines

formance and student and stakeholder satisfaction? How do you evaluate supplier
performance? How do you deal with poorly performing suppliers? (ECPE 2011,
p. 22)
This category is too generic and does not discuss operations from an academic perspective.
The supply chain concept mentioned in the ECPE needs to be explained in terms of curriculum development, program design, program contents, delivery methods, student assessment,
and faculty management. The ECPE, however, portrays a picture of traditional supply chain
by requiring how do you ensure that your suppliers are qualified?; how do you evaluate
their performance?; and how do you deal with poor performance of suppliers?. This type
of customersupplier relationship works well in a business setting where two parties are
dependent on each other for financial gains. However, in academics, conventional suppliers
such as the government, private funders, and the community work independently and many
times voluntarily provide resources to the educational institutes. The customer-institute may
have little or no influence over the mentioned suppliers. This clearly highlights that the conventional supply chain terms of business parlance need to be used carefully in academics due
to the unique nature of its processes. Such conventional supply chain concepts can be used,
at their best, in non-academic functions of the educational institutes such as administration
and support functions. However, such concepts are difficult to apply in academics which is
the core area of any educational institute.
Further, the traditional concepts of customer, and supplier are also different in education. This is shown in Table 2, which is based on Asif and Raouf (2012), Raouf (2004),
and Tribus (1995). The Table makes it clear that different stakeholders may assume multiple roles at a given time. Students, faculty and HEIs assume the role of both cus-

123

Developing measures for performance excellence

tomer and supplier. As an example, consider the student in a role of supplier of knowledge for other students and faculty. The ECPE requirements ensuring that suppliers are
qualified, evaluating their performance, and dealing with poor performance are a misfit
in an academic setting. This category, thus, needs to be revised in future versions of the
criteria.
3.3 Customer focus
This category of the ECPE examines how an organisation engages with students and other
stakeholders for long term market success. An essential outcome of customer focus is the
incorporation of the voice of the customer and other stakeholders into program design. This
helps bring legitimacy. The stakeholders in an academic program could include students, faculty, HEIs, government, regulatory agencies, professional and accreditation bodies, industry
or future employers, funders, community, etc.
Some stakeholders may have requirements not aligned with the requirements of other
stakeholders. One example is the dual role of students as customers of higher education
and grade seekers (Meirovich and Romar 2006; Asif and Raouf 2012). Further students may
be unaware of the knowledge and skills required by their future employers. The problem of
conflicting stakeholder-requirements is discussed in the literature by Asif and Raouf (2012),
Chen (2010), Cruickshank (2003), Meirovich and Romar (2006), and Sirvanci (2004). Such
a scenario highlights the need for wider stakeholder consultation and balancing the stakeholder requirements for optimal results. Omitting the latter step could risk the incorporation
of only powerful entities and ignoring less powerful stakeholders. This, in turn, could distort
the whole purpose of stakeholder consultation. The ECPE, however, is mute on this critical
aspect. The ECPE requires:
Listening to Current Students and Stakeholders: How do you listen to students and
stakeholders to obtain actionable information? How do your listening methods vary
for different student groups, stakeholder groups, or market segments? How do you
use social media and Web-based technologies to listen to students and stakeholders,
as appropriate? How do your listening methods vary across the stages of students and
stakeholders relationships with you?
Given that defining the customer in higher education has been problematic in the past
(Sirvanci 2004; Meirovich and Romar 2006) and HEIs have struggled to balance stakeholder-requirements (Houston 2007), this is a critical issue. Some authors have discussed
this issue from the perspective of the systems approach (Houston 2007; Galbraith 1999).
In any case, this critical aspect needs to be discussed in the future revisions of the
criteria.
3.4 Workforce focus
This category in the ECPE consists of two clauses: (1) Workforce environment: how do you
build an effective and supportive workforce environment and (2) Workforce engagement: how
do you engage your workforce to achieve organisational and personal success? Academic
processes are the core processes in a HEI (cf. administrative processes meant to support
the core processes). The workforce for academic processes mainly consists of faculty and
researchers. The faculty members, in an academic setting, are usually required to do multiple
tasks including research, teaching, student counselling, administrative tasks, services to pro-

123

M. Asif et al.

fession, and services to community, etc. To achieve high performance, the workload of the
faculty needs to be balanced for various activities. This would also require commensurate
development of supporting structures such as setting criteria for workload allocation, assessment, and rewards systems. However, the workforce category of the ECPE is too generic
and the mentioned issues are not addressed in the criteria.
3.5 Performance measures
The results category of the ECPE requires determining operational process effectiveness in
addition to other measurements. The operational process effectiveness could be determined
in terms of program design, admissions, faculty management, learning and development,
etc. The ECPE, however, does not explicitly address the mentioned issues. Rather, it makes
vague linkages to productivity, cycle time, and other measures of process effectiveness,
efficiency, and innovation. The criteria states:
Operational effectiveness: what are your current level and trends in key measures of
indicator of the operational performance of your key work systems and processes,
including productivity, cycle time, and other appropriate measures of process effectiveness, efficiency, and innovation?(ECPE 2011, p. 23)
The concepts of productivity, cycle time, and efficiency are more common in manufacturing which is characterised by tangible inputs and outputs. While it is easy to measure
productivity and efficiency and their various types (Krajewski et al. 1999; Slack et al. 2010)
in the manufacturing sector, these concepts do not readily apply in education because input
and output are often intangible and it is difficult to make measurements and comparisons. The
future revisions of the criteria need to eliminate vague measures and offer in-depth insights
into performance measurement.
On a similar note, the measurement, analysis, and knowledge management category of
the ECPE, requires organisations to use performance measures for various organisational
processes. The use of performance measures is in line with one of the core values of the
ECPE, i.e., fact-based decision making to align organisational direction and resources with
key processes. The ECPE requires:
Performance Measures: [] What are your key organizational performance measures,
including key short-term and longer-term budgetary and financial measures? How frequently do you track these measures? How do you use these data and information to
support organizational decision making and innovation? (ECPE 2011, p. 16)
The general approach of the ECPE is that it requires academic institutes to use performance measures. However, it does not explain how to develop such performance measures.
In other words it discusses what to do but not how to do it. This leaves practitioners with
difficulty in implementation. The question that remains unanswered is how to systematically
develop performance measures for various academic processes such as admission, faculty
recruitment and development, teaching, and learning and development.
To summarise, currently there are two issues with performance measurement in the
ECPE. First is the existence of vague performance measurement metrics including productivity, cycle time, and efficiency; and second is the lack of clear guidance on development
of performance measures. Without the latter, performance measurement, improvement, and
excellence will remain elusive.

123

Developing measures for performance excellence

Fig. 2 Systematic approach to
development of performance
measures

Desired outcomes
to be measured
in terms of
Key measures
through development
of specific
Indicators
using
Methods

4 Developing performance measures for academic processes

As noted earlier, the ECPE is based on normative performance management practices, and
in doing so, it requires the use of performance measures for various processes. Performance
excellence, however, requires a more systematic approach characterised by upfront planning
of clear outcomes, developing measures and indicators for such outcomes, and methods to
track performance (Veleva and Ellenbecker 2001; Chen 2012). This systematic approach is
shown in Fig. 2. The Figure shows that systematic performance measurement requires development of key measures. The key measures need to be assessed through the development
of specific indicators and appropriate methods (Ball and Wilkinson 1994). While the key
measures are generic and represent a higher level of abstraction; indicators are more specific,
measurable and represent the operational level. For example, a HEI interested in entrepreneurial leadership would measure the performance of its initiatives in terms of new business
ventures by its graduates. New business ventures is, thus, the key measure for entrepreneurial leadership. The key measure must be measured through development of more specific
indicators (Veleva and Ellenbecker 2001), which in this case could be number of graduates
starting new business. Finally, this measurement needs to be carried out using appropriate
methods. This focuses on assembling the required supporting information through data collection. The data collection could be carried out using different methods including surveys
of graduates or developing a database of graduates initiating new businesses. To summarise,
while entrepreneurial leadership is the intended outcome; new business ventures is a possible key measure; number of graduates starting new businesses is a possible indicator; and
the use of surveys or database to get relevant data is a potential method. The development of
key measures, indicators, and methods makes performance measurement more systematic,
helps in making policy decisions, and better allocation of resources (Kells 1992). The ECPE,
however, does not address such issues.
While the use of indicators to measure performance is common (Kells 1992); the use of a
structured approach to develop performance indicators aligned with the mission of the institute requires further attention by HEIs (Ball and Wilkinson 1994). Processes in an academic
setting would undoubtedly vary from one setting to another. However, there are several general processes that are common to any academic setting. These processes include: developing
a mission and vision for the institute, program design, admissions, faculty recruitment and
development, learning and development, support services, and financial aspects. These pro-

123

M. Asif et al.
Table 3 The key processes in an academic setting
Area

Description

Reason for inclusion

Mission and
vision
development

Setting the direction and defining the

focus of the HEI.

Program design

A structured approach to define

content, delivery methods, and
assessment to achieve desired
learning outcomes.

Admissions

Admissions provide input to HEIs in

the form of students for the
developed programs.

Faculty
recruitment and
development

How the faculty is recruited,

developed, and managed in the
overall academic processes to
achieve performance excellence.

Learning and development

How the HEI ensures learning and

development of students and
faculty, and develops it for
organisational excellence.

Support services

Such services include security,

medical, maintenance, housing,
sports, and logistics.

Financial aspects

Financial aspects concern the overall

management of the financial
portfolio of an organisation.

A strategic essential that defines

desired outcomes and determines
allocation of resources, direction of
efforts, and execution of processes.
HEIs could lose their focus and
direction without setting a vision
and mission. Mission and vision
development is also a requirement
of the ECPE.
Academic programs are the main
product of HEIs. A focus on this
key area is required for
performance excellence and
satisfaction of a broad range of
stakeholders.
Admissions is a critical process that
affects the rest of the processes and
determines the performance
outcomes of the HEI. Like any
performance improvement
program, HEIs need to control this
critical process.
The faculty is an important type of
stakeholder who have a critical role
in any academic program including
program design, delivery,
assessment, research, and other
functions of the HEI. Effective
management of the faculty entails
better performance.
Being the seats of learning, HEIs
bear the responsibility to ensure
learning and development of
students and faculty. Organisations
can capitalise on this feature to
achieve performance excellence.
Since the support services are meant
to support the main processes, they
need to be designed and managed
to support academic excellence.
Financial viability is an important
contributor to performance
excellence. Financial outcomes is
also a construct of the performance
results category of the ECPE.

cesses are central to any academic setting and also represent the key processes at strategic,
tactical, and operational levels. A description of the processes and a summary of the rationale
for their inclusion is provided in Table 3.
Performance excellence hinges on performance measurement and subsequent improvement. As the well-known axiom says, what gets measured, gets managed. However, tracking
the overall performance of the HEI requires a structured approach (Ligarski 2009) characterised by the development of suitable measures, indicators, and methods (Ball and Wilkinson

123

Developing measures for performance excellence

1994). A structured approach to performance measurement, thus, lies at the core of performance excellence. The ECPE does not provide any guidance on this critical area of performance excellence. Addressing this area in future versions of the criteria could make performance excellence more systematic. Through a number of representative examples, Table 4
illustrates the development of performance measures for the above mentioned academic processes. The performance measures are discussed in terms of desired outcomes, key measures,
indicators, and methods.
One example of a structured approach to performance measurement discussed in Table 4 is
development of the vision and mission of the HEI. A HEI may have a mission which emphasizes achievement of world-class teaching, entrepreneurial leadership, innovative research
that impacts social and technological change, active role in the development of society, or a
desire to develop a stimulating campus, or a multi-cultural university. A structured approach
to the realisation of these outcomes consists of developing key measures for each outcome.
The key measures for world-class teaching could include quality of teaching, student success, program completion, and students knowledge, skills and values. However, these key
measures are not sufficient on their own and organisations need to develop more specific
indicators to ensure the realisation of the mentioned outcomes. The indicators in this case
could include number of students completing the program, dropout rate, median score of
students, % of students with a particular GPA, satisfaction rate of students, graduates, and
alumni for academic and support activities, course rating, and graduates employment rate.
The measurement of these indicators could be carried out through data collection, surveys,
and formal and informal feedback from the relevant stakeholders (Ball and Wilkinson 1994).
Reliability and validity must be key considerations throughout the process. In particular,
care must be taken to ensure that the indicators measure what they are intended to measure.
Further examples are provided in Table 4.

The areas missing

in the ECPE
Key measures
Performance
indicators
Assessment

Hybrid approach to performance

assessment through combined use
of audits, self-assessment and
benchmarking

Self-assessment

Audit

Benchmarking

Carried out by
process owner

Carried out by
independent
party

Comparison
against industry
best-practices

Fig. 3 The modes of assessment approaches to track performance along key measures and indicators

123

123

Mission and
vision
development

Process

Multicultural
university

Stimulating campus

Innovative research
that impacts social,
economic and
technological change

Entrepreneurial
leadership

World-class teaching

Ou tco mes

Nationalities of students,
past graduates, and faculty

Events involving local

community

Student activities

Athletic events

Satisfaction rate of students for design, layout, and visual

appeal of the campus
Campus design-for-purpose
Number of social activities per year

Students satisfaction for

campus layout and outlook

Number of nationalities of current and past students

Number of nationalities of faculty

Number of events involving local community

Number of athletic events per year

Overall sports profile of the institute
Number of student activities per year
% of student members of various clubs per year

Number of journal papers addressing local needs

Number of research projects
Number of technology projects
Number of research projects addressing local needs

Number of patents
Number of spin-offs
Number of patents addressing local needs

Patents

Ideas and models of social,

economic, and technological
relevance

Number of graduates starting new businesses

New business ventures

Students median score

Graduates employment rate

Students and graduates

knowledge, skills, and
values

Data collection for nationalities of students and faculty

Data collection for each activity

Students and other stakeholders feedback through
formal and informal conversations
Reports from newspapers, magazines, and other
independent sources.

Data collection about each indicator such as data on

number of spin-offs, patents, journal papers, research
projects.
Expert evaluation of patents, research projects, and
journal papers about their relevance to the societal
needs

Survey / database of graduates initiating new

businesses

Students GPA data collection

Survey of graduates employment rate
Survey of graduates employed at high positions

Course feedback from students

Survey involving students, alumni, and employers

Students GPA data collection

Median score of students

% of students with a particular GPA

Student success rate

Satisfaction rate of students, graduates, and alumni for
academic and support services
Course rating median evaluation of the course by students

Data about number of passed and dropped students

Number of students completing the program

Dropout rate

Program completion rate

Quality of teaching

Potential methods

Indicators

Key measures

Table 4 Performance measures for various academic processes

M. Asif et al.

O ut come s

Best-class academic
program

Ample enrolment of
high quality students

Competent faculty

Professional
development; and
State-of-the-art
scholarly activities

Program design

Admission

Faculty
management

Learning and
development

Academic processes

Process

Table 4 continued

Exchange programs

Scholarly activities
including conferences,
seminars, and research
projects

% of facultyusing sabbatical leaves

Number of exchange programs available and availed by students and
researchers

Data collection about number of faculty

using sabbaticals
Data about number of students availing
exchange programs

Data collection regarding faculty

participation in scholarly activities and
number of scholarly events

Data about research project, publications,

and presence on the editorial boards of
journals

Data collection about average course

evaluation and teachers evaluation

Teaching performance

Research profile

Data of students enrolment

Number of students enrolled

Percentage of faculty with excellent teaching rating

Number of enrolments

Number of publications
Number of secured academic grants
% of faculty winning academic grants
Number of research projects secured / completed
Percentage of faculty on editorial board / reviewers panels of journals
% of faculty attending conferences and seminars
Number of conferences, seminars, workshops, grants, exchange
programs per year
Number of on-going / completed research projects

Qualification test results

Prep-year results

Data collection of number of approved

programs
Benchmarking against best performers
Feedback from stakeholders about relevant
indicators
Reports from newspapers, magazines, and
other independent sources.
Data collection about students median score
Formal and informal feedback from students,
alumni, employer, and other stakeholders
Benchmarking against best performers
Entrance exams
High school results

Data collection about relevant indicators

Potential methods

Prep-year program completion rate

Median high school / college grades / percentage

Students median score

Graduates employment rate
Time to first employment after graduation

Approval of program from competent authorities such as

professional bodies (yes/no)
Total number of approved programs offered by the institute
Stakeholders satisfaction for existing programs
Employer satisfaction for academic programs and graduates

Number of programs, courses, and course contents comparable with

the best-class programs in the market

Indicators

Median qualifying tests score (such as GMAT, GRE, TOEFL, and

IELTS)

Quality of incoming
students

Professional
development and
development of critical
thinking and analytical skills

Meeting demands of the

industry, professional
bodies, and other
stakeholders

Comparability with the bestclass programs offered in

market

Key measures

Developing measures for performance excellence

123

123

Financial

Financial viability
and
Profitability

Support
services

Income generated from research projects

Income generated from consultancies
Income generated from spin-offs / patents
Amount generated from various sponsorships

Sponsorship / endowments
from various sources

Total budget or % of budget allocated to research, teaching, and

support services
Total teaching and research costs

Satisfaction rate of service users

Data collection about each of the mentioned

indicators

Survey about satisfaction of users from the

provided services
Formal and informal feedback from users

Data collection about number of service

users and service availability

Number of people availing the services

Number of hours of service availability

Potential methods

Indicators

Revenues

Expenditures (Research,
teaching, and support
services)

Satisfaction of students from

services

Best-class support
services including
security, medical,
maintenance,
housing, sports, and
logistics

Financial portfolio

Key measures

O u tc o m e s

Process

Table 4 continued

M. Asif et al.

Developing measures for performance excellence

One precaution that needs to be exercised in the use of indicators is to supplement them
with peer review (Ball and Wilkinson 1994). While the performance indicators provide
objective evaluation, peer reviews, although subject to bias, could offer deeper insights to
performance improvement. The measures and indicators noted in the Table provide a compass for performance measurement. HEIs could use and benchmark these key measures and
indicators to develop a comprehensive list of performance measures for their own unique
contexts. While a structured approach to the realisation of vision and mission is critical for
performance excellence (Ball and Wilkinson 1994); the ECPE is mute on this critical aspect.
Once such measures and indicators are developed and used by the institute then the next
step is performance assessment. This could be carried out using self-assessment, audit, and
benchmarking, or a combination of these approaches. Self-assessment is a dominant mode of
assessment in performance excellence. The main advantage of self-assessment is that it promotes greater involvement of the process owner as compared to an audit. Audits are a popular
means of regular assessment. Their main advantage is that they are usually carried out against
certain criteriasuch as clauses of a standardand are carried out by trained personnel.
Owing to these two characteristics audits provide a more systematic means of performance
assessment. For a detailed comparison of self-assessment and audits, see, Karapetrovic and
Willborn (2001) and Ni and Karapetrovic (2003). HEIs could also use a hybrid approach
whereby the two approaches are used together to overcome the limitations and benefit from
the advantages of each. For example, audits could be used for the follow up of a previous
self-assessmentfor instance, to see the extent to which points noted in self-assessments are
addressed. Benchmarking is the comparison of organisational processes against best practices in the market. Its purpose is to calibrate performance against best performers and also
to glean new insights. This is shown in Fig. 3.
Essentially organisations could use the mentioned modes in a synergistic manner. This
could enhance the effectiveness of performance evaluation and could provide more comprehensive performance assessment. However, the ECPE is mute on the synergistic use of
the mentioned assessment methods. Future revisions of the criteria should address this point
(Fig. 3).

5 Conclusions
The ECPE has evolved as a powerful means for performance excellence in education. However, the ECPE do not have theoretical underpinnings. This paper identifies a number of
areas for improvements in the ECPE which need to be addressed in future revisions of the
model. The most important is that, as noted throughout this paper, the descriptive approach
employed by the criteria is too generic and does not provide specific guidance for performance excellence. For example, the ECPE requires effective management of the supply
chain. The conventional concepts of supply chain, however, are different in academia than
in manufacturing and, thus, pose difficulty in understanding and application in an academic
context.
The ECPE requires organisations to develop a mission and then align organisational processes with the mission. The underlying assumption is that the mission is appropriate and
alignment would serve the required purpose. However, if an educational institute fails to
develop the right missionin terms of understanding and addressing stakeholder needs and
developing best-class programthen alignment of academic processes with the mission may
lead to inefficient processes. The ECPE does not address this point. The descriptive nature
of the ECPE requires the organisations to consider a number of processes for performance

123

M. Asif et al.

excellence. However, it does not provide guidance on how to do that. The use of the existing
knowledge of performance excellence is not a requirement. As a result organisations may
end up designing inefficient and ineffective processes even though they may be aligned with
the mission. The weaknesses of the ECPE are also discussed in terms of workforce management, supply chain management, customer focus, and performance measurement.
The key point is that the criteria are too generic and do not discuss the requirements from an
academic perspective. This in turn could pose difficulties in their implementation.
This paper highlights the need for systematic development of performance measures to
facilitate performance excellence implementation. The systematic development of performance measures consists of defining upfront the desired outcomes, determining key measures, and developing indicators and methods for measurement. The paper develops performance measures for a number of academic processes including developing mission and
vision, program design, admission, faculty recruitment and management, learning and development, support processes, and financial portfolio. The implementation approach discussed
in this paper is metrics-oriented and its starting point is defining clear outcomes, performance
measures, and measurable indicators. Further, the assessment of performance measures is
discussed in terms of synergistic use of self-assessment, audits, and benchmarking. While
self-assessment remains a regular means of assessment; audits can be used to track progress
of non-conforming processes identified in self-assessment. Benchmarking helps calibrate
organisational performance against the best practices in industry. Synergistic use of the three
means of performance measurement is currently missing in the ECPE and needs to be considered in future revisions of the model.

References
Alonso-Almeida, M.M., Fuentes-Fras, V.G.: International quality awards and excellence quality models
around the world. A multidimensional analysis. Qual. Quant. 46(2), 599626
Arif, M., Smiley, F.M.: Baldrige theory into practice: a working model. Int. J. Educ. Manage. 18(5),
324328 (2004)
Asif, M., Raouf, A.: Setting the course for quality assurance in higher education. Qual. Quant. (2012). doi:10.
1007/s11135-011-9639-2
Aytac, A., Deniz, V.: Quality function deployment in education: a curriculum review. Qual. Quant. 39(4),
507514 (2005)
Badri, M.A., Selim, H., Alshare, K., Grandon, E.E., Younis, H., Abdulla, M.: The Baldrige education
criteria for performance excellence framework: empirical test and validation. Int. J. Qual. Reliab.
Manage. 23(9), 11181157 (2006)
Ball, R., Wilkinson, R.: The use and abuse of performance indicators in UK higher education. High.
Educ. 27(4), 417427 (1994)
Belohlav, J.A., Cook, L.S., Heiser, D.R.: Using the Malcolm Baldrige national quality award in teaching: one
criteria, several perspectives. Decis. Sci. J. Innov. Educ. 2(2), 153176 (2004)
Black, S.A., Porter, L.J.: Identification of the critical factors of TQM. Decis. Sci. 27(1), 121 (1996)
Bou-Llusar, J.C., Escrig-Tena, A.B., Roca-Puig, V., Beltran-Martn, I.: An empirical assessment of the EFQM
excellence model: evaluation as a TQM framework relative to the MBNQA Model. J. Oper. Manage. 27(01), 122 (2009)
Chen, S.H.: A performance matrix for strategies to improve satisfaction among faculty members in higher
education. Qual. Quant. 45(1), 7589 (2010)
Chen S.H.: The establishment of a quality management system for the higher education industry. Qual. Quant.
46(4), 12791296 (2012)
Cruickshank, M.: Total quality management in the higher education sector: a literature review from an international and Australian perspective. Total Qual. Manage. Busi. Excell. 14(10), 11591167 (2003)
Dean, J.W., Bowen, D.E.: Management theory and total quality: improving research and practice through
theory development. Acad. Manage. Rev. 19(3), 392418 (1994)

123

Developing measures for performance excellence

ECPE (Education Criteria for Performance Excellence) 20112012. National Institute of Standards and Technology (NIST), Gaithersburg (2011)
Evans, J.R.: Critical linkages in the Baldrige award criteria: research models and educational challenges. Qual.
Manage. J. 5(1), 1330 (1997)
Flynn, B., Saladin, B.: Relevance of Baldrige constructs in an international context: a study of national culture. J. Oper. Manage. 24(5), 583603 (2006)
Galbraith, P.L.: Systems thinking: a missing component in higher education planning?. High. Educ.
Policy 12(2), 141157 (1999)
Houston, D.: TQM and higher education: a critical systems perspective on fitness for purpose. Qual. High.
Educ. 13(1), 317 (2007)
Houston, D.: Rethinking quality and improvement in higher education. Qual. Assur. Educ. 16(1), 6179 (2008)
Karapetrovic, S., Willborn, W.: Audit and self-assessment in quality management: comparison and compatibility. Manage. Audit. J. 16(06), 366377 (2001)
Kells, H.R.: Performance indicators for higher education: a critical review with policy recommendations.
Education and Employment Division, Population and Human Resources Department, The World Bank,
(1992)
Krajewski, L.J., Ritzman, L.P., Malhotra, M.K.: Operations management. Pearson Prentice Hall, Upper Salle
River (1999)
Ligarski, M.J.: Appraisal of lecturers for the assessment of the quality of post-graduate studies. Qual.
Quant. 43(1), 137144 (2009)
Lo Franco, E., La Rosa, S.: Quality as an autonomous body of knowledge. An in-depth survey on Italian higher
education. Qual. Quant. 46(3), 751776 (2012)
Martins, A., Aspinwall, E.M.: Quality function deployment: an empirical study in the UK. Total Qual.
Manage. 12(05), 575588 (2001)
Meirovich, G., Romar, E.J.: The difficulty in implementing TQM in higher education instuctionthe dualtiy
of instuctor/student roles. Qual. Assur. Educ. 14(4), 324337 (2006)
Ni, Z., Karapetrovic, S.: Perennial self-audit: model and applications. Manage. Audit. J. 18(5), 363373 (2003)
NIST: Baldrige National Quality Program. http://www.nist.gov/baldrige/ (2010). Accessed 05 May 2010
Raouf, A.: Quality in higher education. Proc. Pak. Acad. Sci. 41(2), 165174 (2004)
Seetharaman, A., Sreenivasan, J., Boon, L.P.: Critical success factors of total quality management. Qual.
Quant. 40(5), 675695 (2006)
Sirvanci, M.B.: Critical issues for TQM implementation in higher education. TQM Mag. 16(6), 382386 (2004)
Slack, N., Chambers, S., Johnston, R.: Operations management with myomlab, 6th ed. Financial Times,
Prentice Hall (2010)
Tribus, M.: Total quality in schools of business and engineering. In: Roberts, H.V. (ed.) Academic initiatives
in total quality for higher education., ASQ quality press, Milwaukee (1995)
Veleva, V., Ellenbecker, M.: Indicators of sustainable production: framework and methodology. J. Clean.
Prod. 9(6), 519549 (2001)
Weinstein, L., Petrick, J., Saunders, P.: What higher education should be teaching about qualitybut is not.
Qual. Prog. 31(4), 9195 (1998)
Winn, B.A., Cameron, K.S.: Organizational quality: an examination of the Malcolm Baldrige National quality
framework. Res. High. Educ. 39(5), 491512 (1998)

123