PROJECT EVALUATION

JUSTICE G. DJOKOTO
Ph. D. (cand.), M.Phil, P.G.D.E., B.Sc.
Pan African Institute for Leadership and Governance Studies
 Overview
• What is it?
– A systematic and objective assessment of an
ongoing or completed project
• Design
• Implementation
• Results
– Involves gathering, analysing, interpreting and
reporting information
– Should be based on credible data
 Purpose
• Learning and improvement
• Accountability
• provide useful feed back to stakeholders;
– entrepreneurs,
– sponsors,
– donors,
– client-groups,
– administrators,
– staff,
– and other relevant constituencies
 Types of Evaluation
• There are several types of evaluation.
• The classification is based on:
– purpose of the evaluation,
– methodology,
– timing,
– who is involved in the evaluation
– position of the evaluators.
• Based on purpose (Trochim, 2002)
– formative
– summative
 Types of Evaluation
• Based on Timing
– Ex-ante evaluation
– Ex-Post evaluation
• Based on position of evaluator
– External evaluation
– Internal evaluation or self-assessment
 Types of Evaluation
• Ex–ante evaluation
– Conducted before the implementation of a
project as part of the planning
– Also referred to as appraisal or quality at entry
• Ex-post evaluation
– Conducted after the project is completed
– Used to assess sustainability of project
effects, impacts
– Identifies factors of success to inform other
projects
 Types of Evaluation
• External evaluation
– Initiated and controlled by the donor as part of
contractual agreement
– Conducted by independent people – who are
not involved in implementation
– Often guided by project staff
 Types of Evaluation
• Internal or self assessment
– Internally guided reflective processes
– Initiated and controlled by the group for its
own learning and improvement.
– Sometimes done by consultants who are
outsiders to the project
– Need to clarify ownership of information
before the review starts
 Types of Evaluation
• By methodology employed
– Quantitative
– Qualitative
 Steps in Managing a Project
Evaluation
1. Establishing the need for an evaluation
2. Initial Planning and Resourcing
3. Developing Terms of Reference
4. Engaging the Evaluator or Evaluation Team
5. Approving the Workplan
6. Implementing and Monitoring the Evaluation
7. Assessing the Results of the Evaluation
8. Developing a Plan for Follow-up
Step 1: Establishing the need for
an Evaluation
• AGRIS project manager(s) need to clarify
the purpose of evaluations. E.g.
- Donor requirement
– Accountability
– Innovation
– Learning and change
– Responding to changed circumstance
 Step 2: Assessing the ability and
readiness to evaluate
• Evaluations take up significant time and resources
• Need to ensure that the costs are appropriate for the
anticipated benefits. Some considerations:
– Importance of the evaluation to AGRIS or the donor-does it
need to take place?
– Ability and readiness of AGRIS and partners to engage in
the evaluation-Is there a reason that the evaluation should
be postponed? Cancelled?
– Size of the evaluation. Setting the focus and scope for the
evaluation
– Resourcing the evaluation: Money? Technical expertise?
• Defining scope and size
– Clarify
• if external or internal
• Level of effort and resources required / available
• Stakeholder groups to be involved and how.
• Full stakeholder desirable, but could be limited to
the following:
– Deciding whether or not to evaluate.
– Defining the type of evaluation, its scope, and criteria.
– Defining the evaluation questions, what are the key
issues to explore in the evaluation?
– Defining evaluation workplan.
– Evaluation activities must be scheduled and fit into the
stakeholders' agendas.
– Deciding which recommendations to adopt and which to
reject.
– Disseminating and gathering feedback on the results.
 Providing resources for the
Evaluation
• Evaluations require substantial investments of financial
and human resources.
– Funding source would have been indicated in the
project document
 Developing Terms of Reference
(TORs)
• TOR are the key guide for an evaluation.
• They should
– clarify reasons for the evaluation
– highlight issues that have become apparent
– indicate the general depth and scope required
– spell out any imperatives for the evaluators
– provide details about methodology, scheduling, cost
and the qualifications of the members on the evaluating
teams
 Developing Terms of Reference
(TORs)
• The project manager is responsible for
ensuring clear and focused TORs
• This is as far as the Manager is
responsible for development of the TOR
Contents of Terms of Reference
1. Context for the evaluation
2. Rationale or purpose for the evaluation
3. Evaluation issues and questions
4. Evaluation stakeholders
5. Methodology
6. Qualifications of evaluators
7. Schedule
8. Outputs and Deliverables
9. Cost
10.Action Plan
11.Appendices - Evaluation Matrix, Evaluation Policy,
LFA
 Engaging the Evaluator or
Evaluation Team
• Evaluators can be selected by you, imposed
by donors or jointly agreed to.
• Which ever it is some guide is useful here:
– The appropriate level of technical expertise or
evaluation expertise
– The previous experience or profile of the
evaluator
– Suggested profile of a good evaluation team
– Using peers as evaluators
– Roles and responsibilities
 Reviewing and Approving the
Workplan
• The evaluation work plan is developed by the
evaluator and the evaluation team
• It should:
– provide roadmap for conducting the evaluation
(interprets TORs)
– include proposed methodology and means of analysis
• A poor work plan leads to poor evaluation
• Important that the leadership of the project review
and approve the evaluation work plan
 Reviewing and Approving the
Workplan
• Suggested outline of a Workplan
– Introduction - purpose and stakeholders
– Evaluation Questions (framework)
– Methodology (sources, methods)
– Schedule (Gantt chart)
– Resource Allocation and Budget
– Evaluation Team
– Outline of Evaluation Report
 Implementing and Monitoring the
Evaluation Work
• Managers required to facilitate evaluators work by:
– Supporting field data collection
– Making documents available
– Responding to regular evaluation reports and feedback
– Distributing draft reports for comments to appropriate
partners
– Participating in donor and evaluator meetings when
requested
– Reviewing drafts of findings and reports and providing
feedback
 Different Audiences may have
Different Needs
• Internal staff might need a verbal report and a
memo with key points
• Donors and external stakeholders might need
a full report
• Ministries might need an abstract
• Public at large might need an abstract of
findings only
• Know your audience and match your reporting
approach
 Effective Communication of
Evaluation Results
• Captures the data in its conclusions
• Speaks in language of users
• Detached, non-possessive stance
• Objective - “truth” to power, but
• Is pragmatic - goes only as far as the key
stakeholders will accept
Assessing the quality of an evaluation
report and process
• Meeting needs – commissioning managers,
stakeholders
• Relevant scope
• Suitable methods
• Reliable data
• Sound analysis
• Credible findings
• Impartial conclusions
• Clear reporting
Doing It Yourself
 What to Evaluate
• Outcomes
• Processes
 Steps In Evaluation
• Planning
• Selecting object (setting objectives)
• Methodology
– Deciding on standards
– Choice of measures
– Data collection
– Data analysis
• Implementing evaluation
• Reporting
Thanks for Participation