Strategies, Role and

Impact of Project
Evaluation
Brussels, 22 January 2010
Radu Szekely
radu.szekely@vnf.fi
 Project Evaluation
CONTENTS

1. Stages of Evaluation
2. Types of Evaluation
3. Evaluation Processes and Strategies
4. The Role of the External Evaluator

Brussels, 22 January 2010 Radu Szekely 2

 Stages of Evaluation
Thinking back to evaluation plans you
have proposed to use in your projects,
when is evaluation planned?

Honestly…

Brussels, 22 January 2010 Radu Szekely 3

 Types of Evaluation (1)
1.Internal Evaluation – performed by staff of
partnership members, preferably directly
involved in the project work

1.External Evaluation – performed by an expert

not attached to any of the partnership
members, who has no other task in the
project
NOTE: The evaluation performed by experts of the EACEA is not external
evaluation.

Brussels, 22 January 2010 Radu Szekely 4

 Types of Evaluation (2)
1. Planning Evaluation (leading to an
Inception Report and a QA Plan)
2. Formative Evaluation (leading to
Progress Reports and Quality Assurance
Recommendations)
3. Summative Evaluation (leading to Impact
Reports and Quality Assessment)
NOTE: Each one of the above can be conducted as internal or external evaluation,
or preferably as a combination of the two. All three types are necessary.

Brussels, 22 January 2010 Radu Szekely 5

 Types of Evaluation - Group task
Discus the evaluation plans you have
proposed to use in your projects focusing
on:
- Types of evaluation proposed
- Criteria for evaluation
- Techniques and strategies for evaluation
- Expected impact of evaluation

Brussels, 22 January 2010 Radu Szekely 6

 Planning Evaluation (1)
Planning Evaluation
-will assess the understanding of the
project’s goals, objectives, strategies, and
timelines
-through questionnaires and one-to-one
interviews

Brussels, 22 January 2010 Radu Szekely 7

 Planning Evaluation (2)
Impact of Planning Evaluation
-starts during the proposal stage, or at early stage of the
project
- it is very intense and short-term
-helps clarify expectations and motivation
-focuses attention on the areas of highest potential
impact
-possibly brings about a re-distribution of tasks
-draws attention to possible problems in the cooperation
between partners

Brussels, 22 January 2010 Radu Szekely 8

 Planning Evaluation (3)
• Why was the project developed? Are the aims and objectives
still the same?
• What is the need it is attempting to address? How have these
needs changed over time and what imapct they have on re-
planning project activities?
• Who are the stakeholders? Who are the final target-group to
be served?
• Do we know the original situation? What are the expected
outcomes? What will be our indicators of change?
• How much does it cost? What is the budget for the program?
What human, material, and institutional resources are
needed?
• What are the measurable outcomes to achieve?
• What is the expected impact in the short run? The longer run?

Brussels, 22 January 2010 Radu Szekely 9

 Planning Evaluation (4)
Critical factors that are likely be identified at planning
evaluation stage
• lack of clear general goals that all the partners share
• vague task identification for the Project groups and
individuals
• loose commitment to the project implementation of
some partners, leading possibly to drop-outs, reduced
contribution or ’pushing own agenda’
• insufficient usage of the versatile expertise of the
partners
• too little time for the core task in relation to
administration and finances, supporting activities, etc

Brussels, 22 January 2010 Radu Szekely 10

 Formative Evaluation (1)
-will assess ongoing project activities
-provides clear and transparent benchmarks for
measuring progress
-begins at network start-up and continues throughout the
life of the project
-aims to provide information to improve the project and to
steer it back on track
-points out unexpected developments before they have
an impact
-points out delays, even small ones

Brussels, 22 January 2010 Radu Szekely 11

 Formative Evaluation (2)
Implementation Evaluation
- to assesses whether the project
development is being conducted and
delivered as planned
Implementation questions:
¾ Were the appropriate participants selected and involved in the
planned activities?
¾ Do the activities and strategies match those described in the
plan? If not, are the changes in activities justified?
¾ Were activities conducted in line with the proposed aims and
objectives?
¾ Was a management plan developed and followed?

Brussels, 22 January 2010 Radu Szekely 12

 Formative Evaluation (3)
Progress Evaluation
- to assess progress in meeting the project’s goals
- collecting information to learn whether or not the
benchmarks of progress were attained and to point
out unexpected developments
- quantitative and qualitative information is collected
to determine the impact of the activities and
strategies used, and wether some of the project
activities need to be changed or deleted to improve
the work

Brussels, 22 January 2010 Radu Szekely 13

 Formative Evaluation (4)
Progress Evaluation Questions

• Are the partners moving towards the anticipated

goals of the network?
• What hinders the advance towards those goals?
• Which of the activities and strategies are aiding
the partners to move toward the goals?

Brussels, 22 January 2010 Radu Szekely 14

 Summative Evaluation
- to assess the project’s success
- takes place after ultimate modifications and
changes have been made, after the project
is stabilized and after the impact of the
project (or individual project activities) has
had a chance to be realised
-it should be conducted at project level, at
partner organisations level, for different
Workpackages, and for individual activities
and products
Brussels, 22 January 2010 Radu Szekely 15
 EXAMPLE: Event Evaluation
Planning evaluation:
Send a questionnaire to all the participants before the
event to assess their expectations

Formative evaluation:
Enable the participants to reflect on the progress of
the event (e.g. training, meeting, seminar) while it is
taking place (e.g. through reflective diaries, individual
interviews, post-it notes stuck on a wall, etc.)

Summative evaluation:
Gather final evaluative information at the end of the
event, discuss it, use to to improve the next event

Brussels, 22 January 2010 Radu Szekely 16

 External Evaluator
Role, rationale and requirements for the
external evaluator:
• to provide objectivity to the evaluation
process (“second pair of eyes”)
• to bring additional experience and a wide
range of evaluation methodologies
• to be (preferably) a recognised expert,
familiar with both the subject area and the
management and evaluation of European
projects
• to be acceptable to the project team

Brussels, 22 January 2010 Radu Szekely 17

 Summary
• Begin early in the project
• Involve others (external evaluator, but also
target group representatives)
• Use all types of evaluation
• Use evaluation mechanisms that predict
the development of the project
• Record all data and data analysis and use
them in project implementation
Brussels, 22 January 2010 Radu Szekely 18