Managing for Results

Programme Cycle Management

Trine Eriksen and Inger K. Stoll

Financial Mechanism Office
November 2012

1
 RESULTS BASED MANAGEMENT
What is Results Based
Management (RBM)
RBM is a management strategy
by which all actors,
contributing directly or
indirectly to achieving a set of
results, ensure that their
processes, products and
services contribute to the
achievement of desired results
(outputs, outcomes and
impact)
What is a result and a
risk?
In the context of the Grants:
A result is the output, outcome
or impact of a development
intervention
A risk is an event that may
occur and impede the
objective
2
 KEY CONCEPTS

Results achievement

Beneficiary oriented

Managing for results and reduced risks

3
RESULTS BASED MANAGEMENT
IN PRACTICE

1. Clearly identifying program beneficiaries and designing

programs to meet their needs.
2. Defining realistic expected results, based on appropriate
analysis.
3. Identifying and managing risks.
4. Monitoring progress towards results with the use of
appropriate indicators.
5. Increasing knowledge by learning lessons and integrating
them into decisions, and
6. Reporting on results achieved and the resources involved.

4
Part I
Quality at entry -
Basis for good management

5
PROGRAMMING CYCLE

The Programming Cycle and Continuous Feedback

Planning

Programming
Reporting

Budgeting
Evaluation

Monitoring Implementation

6
The Programme and the Context

National/ Impact National/

sector area -------------------------------- sector area
Outcome
Programme -------------------------------- Programme
area area
Inputs Outputs

Activities

Present situation Future situation

Targets
Baseline
 RESULTS CHAIN

Impact Widespread improvement in society/sector Long term

Outcomes Intermediate effects on target groups and systems Medium term

Outputs Products and services delivered Short term

Activities Tasks transforming inputs to outputs Ongoing

Inputs Financial, human & material resources Ongoing

8
Identify main programme
elements of EEA/N Grants
Objective (impact) 32 predefined objectives
Widespread improvements
in societ or a sector

Outcome(s) 160 predefined outcomes In DoRIS

Intermediate effects on data base
target groups and systems
Outputs Self-defined (at least two In DoRIS
Products and services outputs per outcome) data base
delivered
Activities Self-defined Not in DoRIS
Tasks transforming inputs
to outputs

Inputs Budget + Budget in DoRIS

Financial, human and
material resources 9
WHAT IS AN OUTCOME?

• Outcomes can occur on multiple levels:

• Individual level;
• Family or household;
• Community or population group;
• Organization;
• System (involving multiple institutions or agencies); or
• State.

10
DIFFERENT TYPES
OF OUTCOMES
• The nature of the changes can vary as well. Outcomes can involve
different types of changes:

• Changes in perceptions, attitudes, feelings, beliefs, or values;

• Knowledge;
• Skills or level of functioning;
• Behaviours or practices;
• Conditions;
• Status (such as socio-economic status, and status can reflect formal or
informal status); or
• Policies.

11
Defining outcomes and
outputs
• Express an outcome and an output
as a positive condition.
• A measurable, desired result: Wait list for a
cancer operation has been reduced from 16
moths to 10 months at hospital x in y province
• Avoid «double»objectives as these blur what
will be measured.
in order to through
12
Cross-Cutting issues as
outputs and targets
• Good governance
• Gender equality
• Sustainable development: Environmental, Economic
and Social
Cross-cutting issues are either reflected in the outputs
or the indicators (targets), or assumptions.
If the programme substantially addresses cross-cutting
issues, it is advisable to define outputs and targets that
reflect the cross-cutting issues. This strengthens follow-
up.
13
Programme logic
Overall which will contribute to the two
Contribute to
objectives overall objectives

which will contribute to the fulfilment of the

Contribute to Objective objective
if the outputs are produced, then the
Expected
Likely to outcomes expected outcome is likely to be
happen achieved

if the activities take place, then

Should be Outputs the outputs will be produced
able to
guarantee
Activities
that this
if the inputs are available, then
will happen
Inputs the activities will take place
14
ASSUMPTIONS

• The intervention logic is an aim-means chain and a

cause-effect chain.
• It is assumed that “certain events” need to occur at
one level in order to achieve the desired effect at the
next level.
• It is not certain that these events happen; they
represent uncertainties.

15
 Impact/Objective
There are
Assumptions
Outcome
between each
level:
Outputs (deliverables)
Identify
Assess
Activities Monitor
Manage and
Mitigate
Inputs

16
 Identify important
assumptions
• Assumptions are worded as positive conditions (see objectives)
• Assumptions are linked to the different levels of the
Programme Matrix. Each level must contain the necessary and
sufficient conditions for the next level above.
• Start from bottom of matrix and work upwards
• Are weighted according to importance and probability

Example: Scholarship recipients return to assigned jobs

17
Check the significance
of the assumptions
1. Eliminate the obvious factors which are:
• Not important for the outcome
• Very likely to occur

2. Assess the probability of occurrence

for the remaining factors:

a) Quite likely but b) Not likely to

not certain: occur (killing factor!)
Include the factor Redesign the
into the PM and programme
make sure to: c) If this is not
• Monitor it possible:
• Report
Reject the
changes
programme
• If possible
proposal
influence it
RISKS

• If important assumptions do no occur, they might

represent a risk.
• A risk is any event that could occur and adversely
impact the achievement of results at all levels
• Risks are an expression of uncertainty
• Assumptions are expressed as a positive condition (a
necessary event), whilst risks are expressed as
negative conditions (un-wanted events)

19
Identify main programme
elements
Objective (impact) Assumptions and risks
Widespread improvements
in societ or a sector

Outcome(s) Assumptions and risks

Intermediate effects on
target groups and systems
Outputs Assumptions and risks
Products and services
Delivered
Activities Assumptions and risks
Tasks transforming inputs
to outputs

Inputs Assumptions and risks

Financial, human and
material resources 20
Cross-Cutting issues as
assumptions
• Good governance
• Gender equality
• Sustainable development: Environmental, Economic
and Social
Cross-cutting issues are either reflected in the outputs
or the indicators (targets), or assumptions. If not taken
care of they can represent a risk to the sustainability of
the programme results. Therefore cross-cutting issues
should be monitored in the same way as assumptions
and risks.
21
The basic elements of the
Programme Matrix

Impact (Objective)

Outcome

Assumptions describe the

necessary conditions that
must exist for the cause and
effect relationships between
the levels to occur.
Outputs (deliverables) If an assumption does not
occur, it might represent a
Activities risk

Inputs
22
 The Matrix is the point of reference
throughout the life of the programme

National/ National/
sector area sector area
Programme implementation
Programme Programme
area area

Programme

Present situation Future situation

Baseline
PART II
Risk Management

24
PART III
Programmes and
projects

25
Programmes and
projects

Pre-defined projects

Strategic
approach
Programme
proposal

Projects Pre-defined projects

26
 Programme design

OBJECTIVE (INDICATORS) (ASSUMPTION

S)
Programme

PROGRAMME OUTCOME
OUTCOME INDICATORS ASSUMPTION
S
OUTPUTS INDICATORS ASSUMPTION PROGRAMME OUTPUTS
S

OBJECTIVE (INDICATOR (ASSUMPTIO OBJECTIVE (INDICATOR (ASSUMPTI OBJECTIVE (INDICATOR (ASSUMPTI

S) NS) S) ONS) S) ONS)
OUTCOME INDICATORS ASSUMPTION OUTCOME INDICATORS ASSUMPTIO
Projects

OUTCOME INDICATORS ASSUMPTIO

S NS NS
OUTPUTS INDICATORS ASSUMPTION OUTPUTS INDICATORS ASSUMPTIO OUTPUTS INDICATORS ASSUMPTIO
S NS NS
ACTIVITIES INPUTS ASSUMPTION ACTIVITIES INPUTS ASSUMPTIO ACTIVITIES INPUTS ASSUMPTIO
S NS NS

27
 Example – Renewable
Energy
Increased share of
Objective (pre-defined) renewable energy in
energy use

Expected outcome(s) Increased renewable Increased awareness of and

energy production education in renewable
(pre-defined) energy solutions

More energy X policies at local 100 MW capacity X awareness X training courses in

Outputs efficient RES in and regional level RES constructed raising RES provided to
place to stimulate RES and in operation programmes at officials at local and
developed local level regional level
carried through

•Modernised RES •Training in RES •Windmills •Awareness raising •Training courses

Types of infrastructure planning
•Solar systems
campaigns at local for officials at
competence level regional level
projects •R&D on RES
•Hydropower
•Plans/policy •Train the trainers •Training courses
•Feasibility of RES
development •Bioenergy for officials at
mapped out in •
regional level
relation to local • •
•
conditions •
• •
• •
•

28
 EXAMPLE Gender-based
violence
Gender-based
Objective (pre-defined) violence prevented
and tackles

Expected outcome(s) Gender-based

violence reduced
Domestic violence reduced

(pre-defined)
Services for A national Awareness around Services for Procedures for work
Outputs women at risk of
violence designed
coordination centre
for service provision
gender based
violence increased
victims of
domestic violence
in crisis center
standardised
established improved

• Support to • New • Media campaign • Training for • Support to

Types of existing centers coordination
• Education counselors develop new
• center standards and
projects Support to start
new centers • Support for
activities • Support to start methodologies
• Curriculum new counseling
• Support to coordination centers • Training
development
provide services activities
• Manual
• Training of
for women at • Training developed
risk such as police and
counseling, • Methodologies medical
phone-lines and guidelines personnel at
development local and
• Training regional levels
• Studies

29
Call for proposals

Be clear about which outcomes and outputs you

want the projects to contribute to.
Which projects are necessary and sufficient?

Project template in DoRIS:

http://www.eeagrants.org/id/3192

30
Logical relationship

31
PART IV
Basis for results
measurement

32
BASIS FOR RESULTS
MEASUREMENT

Result based management

33
"The starting point for a results-oriented approach is ex
ante setting of clear and measurable targets and
outcome indicators. Indicators must be clearly
interpretable, statistically validated, truly responsive
and directly linked to policy intervention, and promptly
collected and publicised."

From the conclusions of the 5th Cohesion Report, adopted by the

European Commission in November 2010
QUALITY AT ENTRY!
30
 INDICATOR

• A means to measure
achievement, or to help assess a
development

35
HOW SHOULD INDICATORS
BE FORMULATED?
• Once indicators have been identified, they should be
formulated to reflect measurable change. Indicators
frequently utilize the following metrics to depict measurable
amounts:
• Number; LESS IS MORE
• Ratio;
• Percentage; or
• Rate;
• Time
Indicators shall whenever relevant, be broken down per gender
In DoRIS, always specify unit of measurement!
36
EXAMPLES OF INDICATORS
Example 1: Outcome: Improved health status of young children in region/district X

Indicator: Percentage of children under 6 years who

are immunized against x y z diseases in district A

Example 2: Outcome: Increased participation of worker’s

organisations in policy development

Indicator: Number of cases in which the submitted

comments of worker’s organisations on draft legislation
are reflected in final legislation

37
INDICATORS
To describe how results will be measured, indicators
must be specific, observable and come from reliable
data.

• Does the indicator provide the most direct evidence

of the condition or result to be measured?
• Is the indicator important, understood, and accepted
by important stakeholders?
• Is the indicator defined in the same way over time?

38
• Are data for the indicator collected in the same way
over time?
• Will data be available for the indicator?
• Will data be collected frequently enough?
• Are data currently being collected?
• If data are not currently collected, can cost effective
instruments for data collection be developed
quickly?

39
Indicators related to
Bilateral Relations
• All programmes should contribute to
strengthened Bilateral Relations
• Use the Bilateral guideline.
Four types of bilateral results expected:
Extent of cooperation
Shared results
Wider effects
Knowledge and mutual understanding
40
Indicators related to
«Horizontal issues»
• A human-rights based approach and concerns for
horizontal issues like fighting discriminiation, racism,
hate speech etc. should guide defining the outputs
and the indicators.

Example: X% of members of Parliament are minorities.

Example: Incidences of hate speech has decreased in Y

school

41
Programme matrix -
indicators
 Sources of Verification
• When defining the indicators, consider the data sources.
Where can you easily get data that are reliable and timely?

• Specify your sources of information

• Assess where you can get easy access to data

Example
Ministry of Health’s statistics on cancer (health status of
population) and various statistics on cancer detection and
treatment services
Programme matrix –
Sources of Verification
WHAT ARE BASELINES?

• Baselines provide context for the setting of

targets and capture the situation before a
development intervention begins, or at the
beginning of a time period that will be
monitored and assessed.
• Baselines describe the conditions prior to
programming efforts. The present situation!
• Essential to make credible and meaningful
assessments on progress towards outcomes.
45
Programme matrix –
Baseline data
WHAT ARE TARGETS?

• Targets set the level for the magnitude of change

expected by the end of a predetermined time period.
They reflect what organizations commit themselves to
achieve by the end of some duration of time.

• Targets provide tangible and meaningful points of

discussion with beneficiaries, stakeholders, and
partners, and allow us to add further specificity to the
outcomes from the results logic.

47
TARGETS

• When you have defined the indicators and know the baseline
situation, you set the targets.
• What do you want the situation for the beneficiaries to be in 3 or 5
years?
• Examples:
• 1. 60% of W 40+ called in for cancer test in Southern Region by year
2016.
We want to increase the % of W 40+ called in from 30% in 2010 to
60%
in 2016.
• 2. Time lag between detection of cancer to operation is not more
than 3
months in 2016. 48
BASELINES AND TARGETS
SUMMARY
• A baseline is the value of the indicator before the
implementation of the project starts
• Targets orients stakeholders to the tasks to be
accomplished
• Targets help establish clear expectations
• Targets serve as the guideposts for monitoring
whether progress is being made
• Targets promote transparency

49
Programme matrix -
Targets

43
Experience: The EU

In 2010 the EU evaluated the 4,131 impact indicators under the ERDF
programmes. Among the programmes;
• 94% had final achievements;
• 58% had targets;
• 6% had baselines;
• 55% had targets and achievements;
• 5% had baselines, targets and achievement; and finally
• 0.5% had baselines but no targets and no achievements.

How can we assess impacts if we have no idea of the starting point?

Conclusion: Much of the data collected are not meaningful and do not
represent the impact of the Structural Funds.

51
Part IV Quality at implementation
• Monitoring
• Reporting on results and risks
• Evaluation

52
PROGRAMMING CYCLE

The Programming Cycle and Continuous Feedback

Planning

Programming
Reporting

Budgeting
Evaluation

Monitoring Implementation

53
PERFORMANCE
MONITORING
• Define processes and systems for performance
measurement;
• Decide the intervals at which measurement will occur;
• Assign roles and responsibilities;
• Set standard requirements for project promoters
• Build in adequate time and resources for analysis of
performance measurement data so that progress is reviewed
and assessed.
• Use web site for communication with project promoters and
other stakeholders

54
Performance monitoring

• Status regarding the output and outcome

indicators is measured and reported
towards the baseline situation (“where
we were”), and also towards the future,
desired situation (“the targets”).
 The Matrix is the point of reference
throughout the life of the programme

National/ National/
sector area sector area
Programme implementation
Programme Programme
area area

Programme

Present situation Future situation

Baseline
Programme performance
framework
Reporting requirements in annual
report from Programme Operators

Programme outcome
• Report and analyse status of the selected indicators for the expected programme outcome(s)
• Analyse how the projects and the programme outputs have contributed to the expected
programme outcomes.

Programme outputs
• Report and analyse status of the selected indicators
• Summerize and analyse how projects have contributed to the planned programme outputs

Outcome and outputs of bilateral relations

• Report and assess achievemenet of outcome and outputs of bilateral relations

Assess risks and risk mitigation, cross cutting issues, sustainability, and outreach to target groups

58
EVALUATIONS

• Evaluations have three key functions:

(1) Learning: As an input to provide decision-makers
with knowledge and evidence about performance
and good practices;
(2) Accountability: To donors, funders, political
authorities, stakeholders and the general public, and
(3) Communication: Results for the beneficiaries

THANK YOU!
59