Foundations in Monitoring

and Evaluation
Niamh Barry
Niamh.Barry85@gmail.com
February 2010
 Session outline
1. What is M&E?
2. Why do M&E?
3. What happens without M&E?
4. Tools and Approaches to M&E
5. Logical frameworks
6. Performance indicators
7. Practical work
8. Data collection and MOV’s
9. Reporting and reviewing
10.Documentation
 What is M & E?
• Monitoring – routine & regular collection ,
analysis & use of information to track
progress towards goals
• Evaluation : assessment of the extent to which
a project is achieving or has achieved its
stated goals
 Or
• Monitoring : Improvement and development of the
community.
• Evaluation : A process design to show the
relationship


 What's the difference?

– Timing
– Analysis

 What do we M & E?
 Inputs
 Process
 Progress towards goals
 Impact of actions against stated goals
 What M&E is can not do
• M&E ‘s fundamental tenet is to inform judgments about project
performance

•
• Measure things that are immeasurable! i.e. if all your activities do
not have a goal then how can progress be measured???


• The key is in program design if your program design is weak and

inconsistent then you will not have a quality M&E framework


• First question should look at your program ‘is this the best way to
achieve our goals’ then you can answer the question of how to
best M&E the program


• M&E should always be built in – it must work with the program

 Why do M&E?

• M&E of development activities provides organizations

with a better means for learning from past experience,
improving service delivery, planning and allocating
resources, and demonstrating results as part of
accountability to donors & partners.


• Within the development community there is a strong

focus on results— this helps explain the growing
interest in M&E.
•
Why do M&E, when we should focus on
implementation?


 “If your not keeping score, your only

practicing”


•
 What happens without M&E?
• If you do not monitor progress, you do not know if you can or are
succeeding
• If you do not monitor progress, you can never recognize program
failures OR success
• If you do not evaluate Impact, you do not know if you have succeed or
failed
• You prevent learning and sharing
• If you never M&E you may repeat programs that have no impact
and are wasting organizational and target participants time,
resources and donor funds.

•
•
 Tools and Approaches to M&E

• Various tools and methods exist to begin to help us M&E, the

choice depends what is being being M&E (ed)
•
 Progress/impact indicators
 The logical framework approach
 Formal surveys – Baselines
 Needs assessments
 Participatory methods (FGDs)
 Cost-benefit and cost-effectiveness analysis
 Impact evaluation
•
• Some of these tools & approaches are complementary; some
are substitutes. Some have broad applicability, while others
are quite narrow in their uses.
•
QUESTIONS?
 What do we use?

PERFORMANCE INDICATORS
AND LOGICAL FRAMEWORKS


 A combination of logical framework and

performance indicators is the most commonly
used in long term programming and
planning
LOGICAL FRAMEWORKS
Causal Pathway (same but with out indicators)
 Causal Pathways
• This is the initial step in creating the logical framework
• It is a program planning tool that ensure that program activities
will cause the desired impact
• This is a very important step – all inputs must be enough to produce
activities and these activities must eventually produce the goal
• It’s logic is based on cause and effect or the ‘theory of change’ but
you must be careful to think of challenges during this exercise i.e.
you can not assume condom distribution will lead to condom use.
 Simple Logical Framework
Goal Indicators MoVs Assumptions

Objectives

Outcome

Output

Activities Inputs
 Terminology
• Input-resources needed in the project
•


• Activities- process of using inputs to achieve results




• Output –the results of the completed project activities O

u
 t
 si
d
• Outcome/objectives-takes place when the project targets uses e
p
 outputs as anticipated r
 o
j
 e
• Impact/goal-long term c
t
 change/effectiveness of the project c
o
 n

t
r
• Many other terms are used-but effectively mean the same thing …goals-
o
objectives-results-effects-purpose…the point is to know whenl to
use them in the process
•
M&E Components The Soup Analogy HIV Program

Inputs Chicken, vegetables, Broth, Spices Condoms, Testing Kits, Staff time,
Pot, Stove Transport, Funding

Activities Chopping vegetables, cooking Training

chicken, boiling and stirring Delivery of service

Outputs Soup has been made VCT available

Outcomes Appeasement of hunger
Satisfaction of eating soup
Goal Improved nutritional status of
person eating soup
Condoms available
Knowledge on transmission
Trained staff
Increase in condom use
Increase in uptake of VCT
Changes in sexual behavior
Decrease in STI trends
Decrease in HIV transmission,
morbidity and mortality
Change in social norms
Economic impact
Important Notes on Log Frames
• You must know what you want to set out to
achieve before doing the log frame, is your
programme:
• Causal: strategies aimed at directly changing a
situation
• Persuasive: strategies aimed at influencing a
situation or thinking about a situation
• Supportive: strategies aimed at influencing the
environment with which a focal problem is
situated
• If you do not know exactly what you want to do
then you can’t realistically expect it to be
M&E(ed)
Important Notes on Log Frames
Remember that change happens of a period
of time (Theory of Change)
That is, how a particular project is

anticipated to contribute to social change

through time.
Often and incorrectly the vertical logic is

used to systematically disaggregate the

problem – without consideration of time
 Right and Wrong Approaches
Goal :a wall
Objectives :the bricks that make up the wall

Outputs :the sand and cement that make up the bricks

Activities :the molecules that makes up the grains of sand

and cement


Goal :significant and lasting changes anticipated in the lives

of the ultimate beneficiaries (e.g. households with improved
livelihoods)
Objectives : Change to action – peer educators using

knowledge and increased capacity

Outcomes :changes in knowledge /attitude /practice influenced
among intermediary stakeholders (e.g. Peer educators with
better capacity)
Outputs :deliverables of the project implementing team aimed
to initiate the change process (e.g. training of peer
educators)
Activities : Trainings, follow ups
QUESTIONS?
PERFORMANCE
INDICATORS
 Performance Indicators
• Performance indicators are measures of inputs, processes, outputs,
outcomes, and impacts for development projects, programs, or
strategies.

• Only beneficial when supported with sound data collection—

perhaps involving formal surveys—analysis and reporting,

• Indicators enable us to track progress, demonstrate results, and take

corrective action to improve service delivery.

• Participation of key stakeholders in defining indicators is important

because they are then more likely to understand and use
indicators for management decision-making.
 Developing indicators
• Working with indicators is not difficult
• Indicators are no more than criteria that will help us
monitor and evaluate our project
• To develop them, we have to ask ourselves:
 What aspect will tell us whether or not something
went well?
 Whether we achieved our objectives or not?
 Whether we followed our plans?

Example of Performance Indicators
• A program has the goal to increase community
knowledge of HIV prevention and access to HIV
testing treatment and care through targeted health
talks
• What could the performance indicators of this
programme be?
• Number of community members reached
• Improvement in community members knowledge on HIV
• Improvement in number of people testing
• Improvement in the number of people receiving treatment
• Increase in condom use
• Reduction in HIV transmission


•
 Important Note on Indicators
• When we talk about any of the below type of indicators:
 Increase
 Decrease
 Improvement
 Change
 Etc
• We can only measure these if we have a BASELINE to
compare it with
• A baseline is information on whatever we wish to impact
prior to our program intervention.
 Important note on Indicators
 Keep them SMART


S – Specific
M - Measurable

A – Achievable

R – Realistic

T – Timely


Resources to Help on Indicator
Development
QUESTIONS?
 Devising your own Logical frame work

PRACTICAL WORK
 Case study
• Country XX has one of the highest burdens of TB
worldwide. XXX has quite a number of challenges that
correspond to this high burden of TB:


v Limited clinical infrastructure

v Poorly trained clinic staff
v Low health seeking behavior in the community
v Stigma regarding TB
v Low knowledge of TB
v Low adherence/Lost to follow up for those on
medications
• An TB NGO establishes itself in country XX and wants to
decrease the incidence of TB. Of course to do this it
most address all of the challenges.
• The NGO’s overall Goal is the decrease the incidence of
TB mortality and morbidity in country XX
•
• Design a logical framework with activities, Outputs,
Objectives and the stated goal.
• Start with your goal and then work with activities up!
• Draft a number of indicators for each of these steps.
Key Questions to Consider
• How will you track or measure the success of
this project?


• What key indicators will allow to measure

results of the work?


• What are some sources and/or means of data

collection for these indicators?


• What challenges do you anticipate? What

assumptions do you have?
•
DATA COLLECTION AND
MOV’S
 Means of Verification (MOV)
• For every indicator you create you must be able to verify it.
• That is you must be able to collect data to prove that it is true.


Example
Indicator: Decrease in the number of TB patients lost to follow up

MOV: National TB registers

Indicator: Increase in TB/HIV knowledge among clinic staff

MOV: TB/HIV Knowledge questionnaire

•
• Essentially these are the data collection tools you will use to monitor
progress and evaluate impact .

 Data collection Tools
• These are integral to any M&E Framework with them you can not
monitor progress or evaluate impact.
• You must ensure all data collection tools capture the indicators you have
established
• When designing data collection tools – Less is more
• Do not ask for unnecessary information
• Do ensure the tool is user friendly
• Do pilot test the data collection tool (if possible) before you finalise it
• Do design the tools as you are also designing the indicators – this will
tell you if your indicator is SMART
 Important Notes
• Does your data collection tool also need to collect baseline information?

• Does each indicator require a new data collection tool? Are there MOVs
such as National Registers, KAP surveys already available with
information on your target population.

• Remember: data collection tools are anything that collects information on

your indicators
 Attendance lists
 Training evaluations
 National registers
 Questionnaires
 Monthly reports
QUESTIONS?
REPORTING AND
REVIEWING
 Reporting
• One of the key reasons for implementing an M&E
framework is because of reporting requirements

• Most funding agencies require this annually

• Additionally the organization should have its own

reporting structure, not only to managers but to the
beneficiaries and all stakeholders
 Reporting ctd
• Monthly reports on progress are recommended to
ensure that the programme is progressing without
challenges and to review progress made against
targets
• Monthly reports are more desirable as they will detect
any challenges early
• Additionally data collected should be entered into a
program data base
•
 Reviewing
• All programs need an internal review and replanning
process at least yearly.
• Recommended that this take place as a prelude and
guide to implementing new work plans
• Reviewing should focus on program’s overall
achievements, challenges and emerging issues, based
on information obtained through the M&E processes
described in the preceding slides.
• Through this process, work plans are reviewed
• The process should include Project partners and
beneficiaries
DOCUMENTATION
 Documentation
• Information provided by a strong M&E framework is invaluable

• Organizations should always documenting: good practices, success stories

and case studies.

• Case studies will facilitate the sharing of ideas for action.

• They will provide information on what people are doing and what's working,
what communities have learned from their experience and how it made a
difference. They will provide inspiration and show what's possible.

• Disseminating the M&E findings including the documentation should form

a critical part of the overall M&E activities
Concluding Messages
• Keep it simple
• Build it in
• Meaningful as well as measurable
• Develop evaluation capacity
• Learn and do it yourself!
•
Thank you!