THESIS RESEARCH REPORT NOTE

New approaches in project

performance evaluation
techniques
Douglas C. Bower
Ryerson University, Toronto, Canada and
Government of Ontario, Toronto, Canada, and
Andrew D. Finegan
Entrepreneurship, Commercialisation and Innovation Centre,
University of Adelaide, Adelaide, Australia
Abstract
Purpose The purpose of this paper is to describe a Doctor of Project Management research study
including summary of the literature review, the application of the combination of case study, survey and
theory building research methodologies, key research ndings and potential areas for future research.
Design/methodology/approach The research investigates the reasons for the limited adoption of
earned value management (EVM) as a project performance evaluation technique. It proposes new
extensions to this technique that will be benecial to project management practitioners. The multifaceted
research approach incorporates the following elements: a review of previous and current literature on
EVM; a survey of project management practitioners on their practices and attitudes towards EVM;
analysis of the known challenges of the EVMtechnique; development of techniques to address andresolve
the EVMchallenges; consolidationof those techniques intoa single frameworkandimplementationmodel;
and validation of that framework and model through multiple methods.
Findings The research conrms that EVM can be greatly enhanced and simplied though three key
initiatives: include the cost assurance (i.e. risk transfer) provided by procurement contracts; measure
project achievement and progress on the completion of each phase, rather than monthly; and combine the
above into a simplied, single model.
Originality/value This paper provides practitioners with an insight into how EVM can be enhanced
and applied inproject management organisations. Inparticular, the integrated PAVAtechnique should be
particularly useful to projects using the rolling wave approach, as its recognition of phases provides a
framework for short- and long-term planning.
Keywords Project management, Project evaluation
Paper type Research paper
Summary of the research thesis
Many organisations have adopted the management of projects, programs and their
portfolios as their preferred approach for a wide range of initiatives related to newproduct
development, strategy implementation, and business transformation (Winter et al., 2006).
The current issue and full text archive of this journal is available at
www.emeraldinsight.com/1753-8378.htm
Thesis URL link: http://adt.lib.rmit.edu.au/adt/uploads/approved/adt-VIT20080130.135140/public/
02Whole.pdf
Project
performance
evaluation
435
Received January 2009
Accepted March 2009
International Journal of Managing
Projects in Business
Vol. 2 No. 3, 2009
pp. 435-444
qEmerald Group Publishing Limited
1753-8378
DOI 10.1108/17538370910971072
Despite that progress within organisations, the conceptual base for project management
theory has attracted criticism for its lack of relevance to practice (Morris, 1994, 2000) and
also for its failure to contribute signicantly to improved performance of projects across
various industries. A particular target for this criticism is the earned value management
(EVM) technique. This is somewhat of a paradox. EVM is widely recognised as a core
project management technique with obvious benets; however, its utilisation is not
widespread beyond a few specic industries, notably the defence and aerospace
organisations in the USA.
Supporting this observation, Fleming and Koppelman (2004, p. 1) recount their
conversationwiththe leadeditor of the HarvardBusiness Review(HBR), towhomtheyhad
provided information on EVM in response to his inquiry about the topic. Apparently, he
wanted assurances that EVMwas for real. After a fewweeks to reviewthat material, the
HBR editor called back and asked this key question: If EVM is so good [. . .] why isnt it
used on all projects? Fleming and Koppelman (2004) then cited three underlining reasons
why, in their opinion, EVM has not been universally accepted on most projects. Their
rst reason is that EVM advocates often speak in a foreign tongue, and refer to the
obtuse acronyms (e.g. BCWS) that were originallyemployed. This has some validity, but it
is not clear that the revisedversions (e.g. plannedvalue PV) will make a major difference.
Their second reasonis because initiallythe DoD(Department of Defence) denedEVMto
acquire major systems. In other words, EVM is meant for major defence projects not
regular ones. There is a great deal of validity to this observation. Conventional EVM as
codied in US standards may be overkill for most projects. Third, they content that
sometimes management [. . .] doesnt really want to know the nal cost! This statement
is borne out by the fact that EVM in the USA has been promoted mainly by nancial
controllers, such as those at the US Federal Ofce of Management and Budget (Evans,
2005), rather than by practicing project managers or directors.
In recent literature, some authors (Williams, 2005) have suggested that many of the
problems inherent in project management practice (cost overruns, delays, etc.) reside in
the prescriptive, functionalist, and quantitative tradition inherited by project
management from the narrow perspective of operations management. For example,
the contributors to Making Projects Critical (Cicmil and Hodgson, 2006a) further
expanded research on project management, by considering it within a wider
organizational and societal context. The practical experience of project managers
conrms this perspective; it is not possible to successfully complete complex and
dynamic projects through simply adhering to normative methodologies. Managers
must also view their projects within a broad environment that includes the goals of the
organisation and the needs of society. That broader perspective, however, does not
relieve researchers of the opportunity or obligation to further improve current project
management methodologies such as EVM. This thesis suggests that some researchers
can develop and demonstrate more effective project management techniques in the
area of project performance evaluation while others may advance knowledge and
practices in less-prescriptive areas. Project management is a relatively immature area
of management, one that can benet from re-examination of existing models as well as
the development of new and varied ones.
Other authors (Cicmil and Hodgson, 2006b) have asserted that the decision to
study and investigate a management topic in a particular fashion involves a
philosophical choice by the researcher on what is important. This is based on the
IJMPB
2,3
436
premise that what one decides to study has methodological consequences (Holstein
and Gubrium, 1995, p. 72). Cicmil (2006) further contends that the chosen theoretical
approaches will determine the way in which the subject phenomenon will be
represented.
EVM has become the predominant theoretical tradition for the evaluation of project
performance and progress over the past several decades. That key role is underlined
not only by its prominence in most PM textbooks and bodies of knowledge, but also its
codication within the US Government Requirements and Standards (NDIA, 1998). It
provides a narrow perspective on the project status, as it evaluates performance and
progress using a metrics approach in which just three measures are derived, all based
on cost or other resource calculations. That narrow focus is a weakness; obviously the
EVM indicators and forecasts are only as good as the quality of the information on
which they are based. Further, EVM only considers that internal performance of the
project that is, how well the project is performing in relation to its cost, time, and
scope objectives. EVM does not address the performance of the results of the project,
nor the impact of those results on society, the environment, etc. That said, the narrow
focus if EVM may also be a strength, as it allows the underlying concepts of EVM to be
readily understood, and limits the performance evaluation process to quantiable
components.
The question posed by the HBR editor If EVM is so good [. . .] why isnt it used
on all projects? has been adopted as the central problem for this thesis, and has led
to the formulation of the following research questions:
RQ1. What is the basis and foundation of the earned value (EV) methodology?
RQ2. To what extent is EVM utilised and accepted by project management
professionals?
RQ3. What are the strengths of EVM in the management of projects and
programmes?
RQ4. Does EVM have serious challenges, issues or aws that may be slowing its
adoption?
RQ5. What new concepts or approaches could address those challenges, and
further enhance or alter EVM?
RQ6. Could those enhancements to EVM be combined to form a valid new
methodology?
RQ7. Would project managers accept a new EV methodology?
Yin (2003, p. 21) identies these ve components of research design as being
particularly important:
(1) a studys questions;
(2) its propositions, if any;
(3) its unit of analysis;
(4) the logic linking the data to the propositions; and
(5) the criteria for interpreting the ndings.
Project
performance
evaluation
437
The units of analysis for this thesis are represented by the measures and indices within
conventional EVM methodology as described in the previous section. Their validity and
effectiveness have been analysed and conrmed by numerous studies identied in the
thesis. Through the development of hypothesis and models, this thesis adopted
the established EVMunits of analysis, and developed complementary units. In this thesis
the new phase-assured value analysis (PAVA) methodology introduced the related
measurement units of assured value and expected cost. Inthe phase-earned value analysis
(PEVA) approach, the units of phase planned value, phase earned value, and phase
expected cost are introduced. These PEVA values and calculations are illustrated in
Tables I and II.
In this thesis, the data or information is represented by the new algebraic
expressions that have created to express the value relationships inherent in two new
models: AVA and PEVA. Further data are contained within the combined model
(PAVA) that provides the benets of both new techniques.
The logic linking that data to the propositions is contained in the analysis and
discussion that applies four types of validation to verify these new models: mathematical
verication, practitioner evaluation, compliance with standards, and a retrospective case
analysis.
Context of the thesis
This research study was the nal and examinable submission of work for the Doctor of
Project Management (DPM) degree at RMIT University in Melbourne, Australia. The
1 Phase planned value (PV
p
) is the total budget for a specic phase
2 Phase EV (EV
p
) is equal to the phase PV, once that phase is completed
3 Phase AC (AC
p
) is the total of all internal and external costs attributed to that phase
4 Phase cost variance (CV
p
) is the phase EV less the phase AC: CV
p
EV
p
2AC
p
5 Phase cost performance index (CPI
p
) is phase EV divided by phase AC: CPI
p
EV
p
=AC
p
6 Cumulative cost variance (CCV) is cumulative EV less cumulative AC, for all completed phases:
CCV CEV 2CAC
7 Cumulative CPI (CCPI) is the cumulative EV divided by cumulative AC, for all completed phases:
CCPI CEV=CAC
Source: Bower (2007, pp. 140-1)
Table I.
PEVA cost measures and
indicators
1 Project start date (PSD) is the planned and actual date for the initiation of the project
2 Planned phase end date (PPED) is the calendar date on which the phase is planned to nish
3 Actual phase end date (APED) is the point when the phase deliverables are accepted as 100 percent
complete
4 Phase schedule variance (SV
p
) is the difference between planned and actual completion dates for
that phase
5 Phase schedule performance index (SPI
p
) is the current planned duration (in days), divided by the
current actual duration, both counted from the project start to the end of the last completed phase
6 Forecast phase end date (FPED) is the calendar date on which an incomplete or unstarted phase is
now forecast to nish, based on the planned duration multiplied by the SPI
P
for the last completed
phase
Source: Bower (2007, p. 141)
Table II.
PEVA time measures and
indicators
IJMPB
2,3
438
doctoral programis structuredto provide a distance-learningapproach(usinginformation
communication technologies such as blackboard and e-mail) together with telephone and
voice over the internet protocol facilities for communication. This doctorate is targeted at
practising project managers who can take part in the program from anywhere in the
world. DPM candidates enter the programwith a masters degree and a minimumof eight
years of experience in PM environments.
The candidate was attracted to the DPM program as it provided a exible but
rigorous academic environment in which he could develop ideas, concepts and models
that would address some of the questions related to project management such as the
paradox of EVM being high in value, but low in usage. The program design facilitates
not only the discipline that is necessary for performing doctoral level research, but also
the collaboration that is possible within the community of practice that is formed by
two dozen project management practitioners, some with decades of varied experience.
The structure of the DPM program is shown in Figure 1.
In this case, the PM procurement and ethics course, together with its reective
learning course, initiated an interest in examining the relationship between EVM and
procurement. An initial paper on the variation of EVM when it is applied in projects
with different forms of contract and delivery scenarios focussed thinking in this area.
In undertaking this work in research preparation 1, the candidate became curious to
know why EVM did not take into account the added certainty or assurance that was
provide by procurement techniques, particularly in the forecasting of the expected total
project cost. Simply put, if a project manager had just signed one or more rm
xed-price contracts for work to occur in the remained of the project, why did not EVM
formulae include that information? That research resulted in the development of the
assured value analysis (AVA) model, and the presentation of the resulting paper at two
project management conferences at the end of 2004 (Bower, 2004a, b).
Figure 1.
DPM program outline
Reflective
learning
(12 credit points)
PM knowledge
management
(12 credit points)
Research
preparation 1
(24 credit points)
Research thesis 1 (Draft) (48cp)
Research thesis 2 (Submission) 36cp
Reflective
learning
(12 credit points)
Research
preparation 2
(24 credit points)
PM
leadership
(12 credit points)
Reflective
learning
(12 credit points)
Research
preparation 3
(24 credit points)
PM Ethics &
procurement
(12 credit points)
Reflective
learning
(12 credit points)
Research
preparation 4
(24 credit points)
PM
practice 2
(12 credit points)
Research thesis 3
(Clean-up) 12cp
D
P
M

d
i
r
e
c
t
o
r

Y
1
S
u
p
e
r
v
i
s
o
r

f
o
r


y
e
a
r
s

2

a
n
d

3
48 credit points = part time year
96 credit points = full time year
Project
performance
evaluation
439
Similarly, while proceeding with research preparation 2, the candidate returned to his
interest in project phases to question why the EVM methodology disregarded phases
in any of its concepts and calculations. That line of inquiry led to the development of
the PEVA concept, which was presented at several major conferences in 2005 and 2006
(Bower, 2005, 2006a, b).
This progression of research and inquiry is shown in Figure 2, which represents the
progression that a DPM candidate may experience in the program. The candidate
began in the lower left quadrant (Q1) studying the as is situation regarding project
procurement, phases and performance evaluation. During the program he moved to Q2
in proposing new concepts and techniques that could or should be utilised. From
there, he continued to Q4 in developing new approaches to evaluating the progress and
performance of a project, specically the new models for AVA and PEVA. This
enabled several reective learning papers to be written including Bower and Walker
(2007) that explored the concepts of rollout projects that linked to this research.
Discussion and conclusions
Project management is a relatively new area of business management studies, one that
is still struggling to establish its legitimacy and importance within academic
institutions and within the business community. That struggle has been reinforced by
the efforts to establish standards by professional associations such as Project
Management Institute (PMI) and the International Project Management Association.
It is further supported by the increasing level of project management research that has
been carried out by practitioners and educators.
Project management standards and research are mutually dependent and interrelated.
Research results can be widely recognised and endorsed by professional standards,
and those standards may be legitimized by the results of rigorous research. In a new
area such as project management, this need may be more obvious that in an
established area of science. However, research also has the potential to contradict
Figure 2.
DPM program
contribution
N
e
w

c
o
n
c
e
p
t
/
T
o
o
l
/
P
e
r
f
o
r
m
a
n
c
e

m
e
a
s
u
r
e
m
e
n
t

a
g
e
n
t
s
New application/Geographical area
Low High
High
Auditing as is
Situation analysis
Mapping/plotting
Reflection
Modifying, studying
new context
influences and
refining, action
learning
Developing new
systems, new
approaches, new
tools
Q1
Q2
Proposing could
or should be
Modifying + trial &
error, action
learning
Q3
Q4
Knowledge
management
PM leadership
PM
procurement
+ Ethics
PM practice 2
DPM program
contribution
IJMPB
2,3
440
standards, and rigid enforcement of standards can wrongly imply that further research
is unnecessary.
EVM has been developed and promoted within the project management community
as the singular methodology for project performance measurement, particularly for
cost control. In the USA, a major organisation has invested signicant effort in creating
a national standard for EVM (NDIA, 1998), and the dominant PMI (2005) has created
an EVM Practice Standard. Both of these documents tend to reinforce the notion that
there is only one acceptable way to measure project performance, and that way is
EVM. The only signicant exception is the recent recognition given to earned schedule
as an interesting variant for improving on project progress measurement.
In this thesis, PAVA is developed as an alternative to conventional EVM techniques,
and in doing so have challenged the idea that EVM is the only suitable way to assess
project cost performance. This research has opened the door not onlyto PAVA, but also to
the possibility that other improvements may be made by other researchers in this area.
The PAVA model takes note of the earned schedule concept and recent research on that
initiative; however, it does not incorporate the earned schedule approach into PAVA as
this study found it to be procedurally dubious and unnecessarily complex. Rather, by
simplymeasuring the progress of each project phase with the PAVAtechnique, managers
can readily identify the schedule variance expressed in days, and also arrive at a reliable
schedule performance indicator.
As PAVA is an intuitively simple and effective technique, it may also be adopted
due to its value as a communication tool. To this end, PAVA facilitates phase end
reviews, at which the status of the project may be assessed. Therefore, PAVA
assessment can be a key component of the reporting that is delivered to key
stakeholders, such as project executives and client groups.
The results produced by PAVA lend themselves to graphic portrayal not only as
trend lines to accompany the PMB S-curve, but also as horizontal bar charts that
permit the viewer to quickly visualise the project status. When applied to a portfolio of
projects, these PAVA diagrams encourage side-by-side comparison of a range of
current projects. This serves to not only inform key stakeholders, but also to support
decision-making within a program portfolio.
Over time, the simplicity and features of PAVA should lead to increased
understanding, acceptance and implementation of it in a range of industries and project
types. It is suggested that adoption of PAVA is likely to occur in two major arenas:
(1) PAVA may be utilised as a compliant EVM system by organisations that must
adhere to the performance measurement standards that the USA, the UK and
possibly other government agencies require. The research demonstrates that
PAVA is fully compliant with NDIAs (1998) 32 criteria for an EVMS. Several
implementation routes can be surmised. Some organisations may chose to
implement a form of PAVA for all of its performance measurement needs, as it
produces results inconformitywith accepted standards for EVM. Other rms may
decide to use PAVA on a trial basis to quickly implement performance
measurement, with the option of eventually easing into the full use of conventional
EVM. Of course, such rms may well decide that PAVA addresses all of their
needs. Finally, some rms may already be using conventional EVM, but realise
that theycanachieve all of their performance measurement objectives with PAVA,
and devise a process for converting to it.
Project
performance
evaluation
441
(2) PAVA may also be adopted by organisations that do not need to comply with
specic project management standards. The recognition of the real benets to
be realised from improved control of cost and time will drive this adoption.
Where there is apprehension about the formidable administrative burden of
conventional EVM, PAVA might be more readily accepted if it is promoted
within such organisations as a completely new approach, and not simply as a
variant on standard EVM techniques.
There are a few situations in which PAVA may not be appropriate. A project that cannot
be readily divided into project phases would obviously be a poor candidate for PAVA;
however, it is difcult to imagine that any signicant project would not have several
phases. Similarly, it would be awkward to implement PAVA on a project with many
phases that are scheduled to end at the same time. Such a situation would denitely be
highly unusual, and might be addressed through re-planning of those phases so that they
are divided into sequential sub-phases. PAVA would not be appropriate on a project for
which no cost information is available. For example, in some organisations projects that
are undertaken by internal staff do not have formally recognised budgets. Certainly, this
would make PAVA implementation impossible, but the same would be true for EVM
implementation or any other form of cost control.
Since PAVA includes the value and cost of procurement contracts, it would appear
more valuable in organisations with projects that include a signicant degree of
procured services and deliverables. However, no complications result when PAVA is
used on a project with no procurement that aspect of the model is simply not utilised.
In the interest of simplicity, it would be advantageous to use PAVA consistently for all
projects, so that procurement agreements would be captured in any projects that
include them.
As already noted, EVM has not been widely adopted on architecture, engineering
and construction projects. Owing to the maturity of the construction industry, it has
evolved alternate mechanisms for cost and schedule control. Those mechanisms may
actually approximate the PAVA concept, particularly if they include the AC of vendor
contracts, and their budgeted value, in the cost to complete calculation.
The recognition of phases in PAVA facilitates its application to projects that utilise
the rolling wave approach to project planning. While one phase is underway, detailed
planning can be performed on the next phase, and future phases may remain planned
at a high level of both budget and schedule. By the same token, PAVA combined with
rolling wave is highly appropriate for projects that are not fully dened at the outset.
Budgets allocations and target milestones can be identied for the long-term phases,
then revisited as the project proceeds and its true nature becomes more clearly
understood by the organisation, project team and client.
Finally, this research into conventional EVM and the development of new and
improved techniques for project performance measurement was undertaken to conformto
the intentions of the DPM program. The thesis describes the creation, description and
validation of PAVA, and makes a unique and valuable contribution to project
management. Therefore, it is most gratifying that at the RMIT Universitys research
awards in November 2008, Bower (2007) was awarded the University Research Prize in
recognition of the achievement of excellence in this thesis. This is a tting tribute to the
researchers inspiration and effort, and to the DPM program.
IJMPB
2,3
442
Future research
As the name implies, PAVA addresses three aspects of project management: project
phases, assurance (reduced uncertainty) and EV. Of these three topics, project phasing
is an area that deserves more research attention. Phases are unique to projects or
similar endeavours, and may be used to frame all aspects of project planning and
implementation. This is potentially a fertile eld for further research.
Uncertainty is an underlying component of project management, one that is
addressed through many established techniques: scheduling, budgeting, quality
control, etc. In this thesis, procurement is selected as an element of PAVA, not simply
because procurement practices address uncertainty (through risk transfer), but also
because they can be expressed as values (i.e. budgets and costs), and therefore can
readily assist in predicting a revised project nal cost (EAC). However, there are other
risk management techniques that should be incorporated into project performance
assessment. Some exploratory work has been undertaken (Hillson, 2004) on the
integration of risk management predictive techniques (e.g. Monte-Carlo analysis) into
conventional EVM. These techniques have not been incorporated into PAVA at this
point, as the intention was to address the shortcomings of EVM one of which was its
apparent administrative burden. Further research may be warranted in this area, to
determine if risk management techniques should be used to predict a range of possible
outcomes for the nal cost and completion date of the project phases, or of the project
as a whole.
References
Bower, D.C. (2004a), Assured value analysis: earned value extended, paper presented at the
PMI Global Congress North America 2004, Anaheim, CA, October.
Bower, D.C. (2004b), Assured value analysis: EVM meets procurement, paper presented at the
16th Annual International Integrated Program Management Conference, Tysons Corner,
VA, November 14-17.
Bower, D.C. (2005), Phase earned value analysis, paper presented at the PMI Global Congress
North America, Toronto, September.
Bower, D.C. (2006a), Phase earned value analysis: a proposal for simplifying yet enhancing
EVM, paper presented at the PMI Research Conference 2006, Montreal, July 16-19.
Bower, D.C. (2006b), Phase earned value analysis: a proposal to simplify yet enhance EVM,
paper presented at the Business Performance and Project Management Summit, London,
April 27.
Bower, D.C. (2007), New directions in project performance and progress evaluation,
unpublished research, RMIT University, Melbourne.
Bower, D.C. and Walker, D.H.T. (2007), Planning knowledge for phased rollout projects, Project
Management Journal, Vol. 38 No. 3, pp. 45-60.
Cicmil, S. (2006), Understanding project management practice through interpretive and critical
research perspectives, Project Management Journal, June, pp. 27-37.
Cicmil, S. and Hodgson, D. (2006a), Making Projects Critical, Palgrave, New York, NY.
Cicmil, S. and Hodgson, D. (2006b), Making projects critical: an introduction, in Cicmil, S. and
Hodgson, D. (Eds), Making Projects Critical, Palgrave, New York, NY, pp. 1-28.
Project
performance
evaluation
443
Evans, K.S. (2005), Improving Information Technology (IT) Project Planning and Execution,
Ofce of Management and Budget, Executive Ofce of the President, Washington, DC,
Memorandum.
Fleming, Q.W. and Koppelman, J.M. (2004), If EVM is so good . . . Why isnt it used on all
projects?, The Measurable News, Spring.
Hillson, D. (2004), Earned value management and risk management: a practical synergy, paper
presented at the PMI Global Congress North America, Anaheim, CA, October.
Holstein, J.A. and Gubrium, J.F. (1995), The Active Interview, Sage, Thousand Oaks, CA.
Morris, P.W.G. (1994), The Management of Projects: A New Model, Thomas Telford, London.
Morris, P.W.G. (2000), Researching the unanswered questions of project management, paper
presented at the PMI Research Conference, London.
NDIA (1998), ANSI/EIA-748 Guide: Earned Value Management System (Standard), American
National Standard Institute/Electronic Industry Association, Washington, DC.
PMI (2005), Practice Standard for Earned Value Management, Project Management Institute,
Newtown Square, PA.
Williams, T. (2005), Assessing and moving on from the dominant project management
discourse in the light of project overruns, IEEE Transactions on Engineering
Management, Vol. 52 No. 4, pp. 497-508.
Winter, M., Smith, C., Morris, P.W.G. and Cicmil, S. (2006), Directions for future research in
project management: the main ndings of a UK government-funded research network,
International Journal of Project Management, Vol. 24, pp. 638-49.
Yin, R.K. (2003), Case Study Research: Design and Methods, Sage, Thousand Oaks, CA.
Corresponding author
Douglas C. Bower can be contacted at: dbower@ryerson.ca
IJMPB
2,3
444
To purchase reprints of this article please e-mail: reprints@emeraldinsight.com
Or visit our web site for further details: www.emeraldinsight.com/reprints
Reproducedwith permissionof thecopyright owner. Further reproductionprohibited without permission.