Social

impact measurement in the

philanthropic sector

The ultimate guide: from success to significance

Preface

We are happy to share a new guide with you: Social Impact measurement in the
philanthropic sector the ultimate guide.

The importance of social impact measurement in the philanthropic sector (philanthropists,
family offices, (corporate) foundations and Venture Philanthropy Organisations / VPOs) has
slowly but steadily increased over the last decades. In these years funding grantees has
transformed from the traditional giving without looking back to a more impact-based
approach. This is caused in part by the growing call of both institutional as private donors for
transparency in spending and results. Concurrently, philanthropic organisations receive an
increasing amount of request for their funding (due to budget cuttings in all European
national governments), and are looking for ways to make better choices where to allocate
their resources most effectively in a way that drives improvements and maximises social
impact.

This guide is based on several years of experience measuring social impact of the donations
of philanthropy organisations, various discussions, talks and meetings within the sector and
drawing from existing literature and research.

While we hope this document helps all readers gain a better understanding of social impact
measurement, it is important to note here that there are other frameworks than the ones
mentioned in this guide and that there is not one right or best way to measure social impact.

With this guide we do however hope to provide philanthropists, (corporate) foundations,
VPOs, (potential) grantees and other interested parties with a concise and accessible
overview of social impact measurement for the philanthropic sector, in which we see
measuring social impact as a vital element for making better decisions and thus maximising
the change in the lives of beneficiaries.

This guide includes general information on what social impact measurement is and why it is
important to measure it including explanations of often-used terms and concepts -,
followed with information about the current status of social impact measurement in the
philanthropic sector. The guide also highlights several often-used impact frameworks
including their similarities and differences, and concludes with advice on how to choose the
framework that is right for you.

Thank you for your interest in our work!

The ultimate guide: from succes to significance 2

Table of contents

Defining social impact ............................................................................................................... 4

Defining social impact measurement ....................................................................................... 4
Scope of your analysis ........................................................................................................................ 4
Problem analysis ................................................................................................................................. 5
Stakeholders ....................................................................................................................................... 5
Theory of Change ................................................................................................................................ 6
Continuous forecasting, measuring, verifying and monitoring .......................................................... 6
Reporting ............................................................................................................................................ 7
EVPA framework ................................................................................................................................. 7

Why measure? The strategy behind the measurement ........................................................... 7
Selecting grantees ............................................................................................................................... 7
Understanding impact ........................................................................................................................ 7
Maximising impact .............................................................................................................................. 8

Current status of social impact measurement in the philanthropy sector ............................... 9

Which framework and models do they use? ............................................................................ 9

Are they really that different? ................................................................................................ 11

Choosing a framework ............................................................................................................ 12
Proportionality .................................................................................................................................. 12

Social impact measurement in practice with Sinzer: templates, benchmarking, aggregation 13
Templates ......................................................................................................................................... 13
Benchmark grantees ......................................................................................................................... 14
Aggregate the impact of multiple projects ....................................................................................... 14
Case study ......................................................................................................................................... 15

Time to start measuring! ........................................................................................................ 15

The ultimate guide: from succes to significance 3

Defining social impact

Social or societal impact, as it is sometimes termed today, is generally understood as the
proportion of the changes that take place for a group of stakeholders as a result of certain
actions or interventions. The proportion aspect takes into account changes that would have
happened anyway, regardless of the actions or interventions, as well as any proportion of
the changes caused by other organisations or people. Moreover, social impact can be both
positive as negative, intended or unintended.

Defining social impact measurement

Social impact measurement is an iterative process of forecasting, reviewing and evaluating
the impact activities have on beneficiaries and other stakeholders. There are multiple
methods and tools on how to measure social impact; the general consensus is that any social
impact measuring process contains the following elements:
Scope of your analysis
It is recommended to start all efforts of measuring social impact by determining the scope of
what you will be measuring. Deciding what to in- and exclude of your measurement before
you start will help structure the process.

The following question can help you determine the scope of your social impact
measurement:

Level of assessment: Will you be assessing social impact on the project-, programme-
or organisational level?
Forecast or evaluation: Will you be using social impact measurement as a tool to
forecast expected impact of certain projects or programmes that have yet to be
undertaken, or will you use it to evaluate projects or programmes in the past or
currently running? Combining the two is also an option start with a forecast and
evaluate whether your forecasted social impact is realised once the programme runs.
It is recommended however to determine what your starting point is before setting
up your measurements.
Audience: Whom are you measuring for? Who is the audience of your social impact
measurements, i.e. who will be viewing results and reading the reports? Will results
be shown to your own board, your employees, aspiring grantees or potential donors
or other funders (to show what the social impact of their investments is)?
Purpose: What is the purpose of your social impact measurement? Social impact
measurement can serve a number of purposes. Determining your goal beforehand
can help you make choices in how rigorous your assessments will need to be. A few
commonly named purposes of social impact measurement:
o Select: determine which projects to fund or not (forecast);
o Understand: know whether you are actually creating change or not, and how
much, understand where you create the most value and for whom.

The ultimate guide: from succes to significance 4

o Benchmark and improve: evaluate these projects and use the gained
(management) information to help further improve them: maximise impact
o Prove: ground and prove claims, be accountable: show that you are doing
what you are saying you are doing and achieving
o Communicate: create and increase awareness and support for your mission
o Attract funds: attract new donors, investors or funders

Problem analysis
After determining scope, the second step is to conduct a problem analysis. As with your
social impact measurement, a problem analysis can be drafted on the organisational,
programme or project level.

Answering the following questions can draw up a problem analysis:

What is the social issue you are (or your grantee is) trying to solve?
Why is this urgent? And for who?
Whats the scale of the social issue?
Whats your (or your grantees) solution?
Why does this solve the issue? Is there a need for the solution?
What are the objectives?

Stakeholders
Stakeholders are both individuals and organizations who experience change created by your
activities. In other words: these are the people and organisations that you have an impact
on. Although social purpose organisations (SPOs), like charities, social enterprises and
NGOs, often have multiple stakeholders, the main stakeholder for philanthropic
organizations are generally the grantees beneficiaries: the targets groups of the various
projects and programmes.

It is important to conduct a thorough stakeholder analysis in your social impact
measurement. One way to do this is to start with listing all stakeholders that experience
some sort of change because of your activities, and then narrowing this list down to the
stakeholders that need to be included in your analysis. A few things to keep in mind when
considering which stakeholders to include:

Think of what changes for stakeholders. Remember that these changes can be
positive or negative, intended or unintended.
Include those stakeholders who experience material change: only those changes that
are relevant and significant in relation to your activities.

The ultimate guide: from succes to significance 5

Theory of Change
The Theory of Change describes how the envisioned changes for beneficiaries (and other
stakeholders) will take place. This is sometimes described as If-conditions-then-change.

Although not necessarily bound to this format, the Theory of Change is frequently visualized
by means of the Impact Value Chain, consisting of a set of building blocks, which are
described below:

Inputs: the resources, whether capital or human, invested in the activities of the SPO
Activities: the concrete actions undertaken by the SPO in order to achieve their social
objectives
Outputs: the tangible products and services from the activity
Outcomes: the changes (both long and short term) in the social situation of the
target population resulting from the activity
Impact: the long-term outcomes adjusted for what would have happened anyway
and the actions of others and for unintended consequences (which could be positive
or negative).

The below figure demonstrates this with an example:

Continuous forecasting, measuring, verifying and monitoring
The fifth element of social impact measurement is the actual forecasting, monitoring and
evaluation of the results: to measure the outputs, outcomes and impact of your project,
programme or organisation by using a set of indicators or metrics that tell you whether the
expected or intended change has actually happened, and by how much.

In this stage we would advise you to use generally accepted performance metrics. A useful
catalogue is IRIS, developed by the Global Impact Investing Network (GIIN), where you can
browse through and pick a set of metrics (mostly output based) that make sense for your
portfolio. Another one is called the Global Value Exchange, which is as well an open-source
database with more than 5.000 entries of stakeholders, outcomes, indicators and proxies to
value outcomes.

The ultimate guide: from succes to significance 6

Reporting
The last element of social impact measurement is reporting about your social impact.
Reporting can help share knowledge and good practices, spread awareness and create
support and contribute to transparency and accountability.







EVPA framework

The European Venture Philanthropy

Association (EVPA) has developed
together with an Expert Group of
members, universities and consultants
(our own managing director Marlon
van Dijk was also involved!) a 5-step
approach which defines the same
elements as described above
(although put in slightly different
terms, e.g. setting objectives is
captured in the problem analysis). We
highly recommend you to read this
practical guide too.

Why measure? The strategy behind the measurement

Selecting grantees
Most foundations receive a lot of requests for grants. If youre mission is to create social
impact, then it makes sense to grant funding to those projects that have the highest impact
potential.

Utilising impact measurement as a forecasting tool can provide valuable insight in the
expected impact per grantee and thus help you select projects. It allows you to prioritise
where to invest you resources for greatest impact.
Understanding impact
Measuring outputs alone, like the number of participants or workshops, or the total amount
of volunteer work, only scratches the surface of the total impact of a project or program.
This does not provide enough information to determine whether a project is effective or not.
How can you be sure the projects you fund are really making a difference for the people they
are aiming to help, and achieving their social objectives? Are they only creating positive
social impact for all stakeholders, or are they perhaps also creating negative social impact
for others? Can you confidently claim that the social impact created can be attributed to the
grantee, or did other organisations or people contribute to it as well?

The ultimate guide: from succes to significance 7

Understanding impact an example

A programme you fund aims to cure participants from their alcohol addiction/dependency. Of the
100 participants in the first year, 80 were cured and 20 were not.

Outputs:
-
100 participants in the program
-
80 cured
-
20 not cured

When judging the created social impact of the program based on these outputs alone, you would
conclude that the program was very effective and advise to scale it up in its current form.

Outcome: how did success or failure affect the participants?
-
80 cured were happy, gained confidence, reconnected with family, found jobs
-
20 were not cured ended up severely depressed with their situation and their failure. This
group valued their participating in the programme as very negative.

Impact
Asking the participants about their situation, the programme initiator found that the 80 people
who succeeded all had family support to fall back on, whereas the 20 who failed had not. Part of
the success is attributed to the support of participants family.

Conclusion
After this discovery, the program initiators decided to screen participants before entering the
program and to allow only participants with family support to enter the program.
Applicants without family support were referred onto another program that offered a better-suited
service (a partnership between the two organisations was soon to be developed).
The program was scaled up in its new form (including screening participants for family support) and
now had a near-100% success rate, resulting in maximal social impact.

Maximising impact
Philanthropic organisations with a social mission are united by a common goal: creating
social change. In order to reach that common goal (social change) and further amplify it, it is
crucial to know which approaches work (best) and why. Evaluating the social impact of past
and current projects, can help you understand which actions have most social impact in
which situations, and can thus help you make better decisions in the future and improve the
performance of the grantee along the way.

If for example the measurements show that the success rate of a certain intervention is
near zero, than this could be an indication that funding such interventions is no longer
sensible unless perhaps the intervention is adapted or put in another context.

Only by measuring social impact consequently and consistently can you drive improvements
to maximize its social impact you and your grantees create, make better decisions, bring
initiatives to scale and be accountable to stakeholders.

The ultimate guide: from succes to significance 8

Current status of social impact measurement in the

philanthropy sector

Although the concepts and importance of social impact measurement have been buzzing all
over the philanthropic world for a while now, there is still much debate about how social
impact should be measured (presumably due to the complexity of the concept). Many
foundations have developed their own frameworks on how to measure social impact.
Nevertheless, the 2013/2014 European Venture Philanthropy Association (EVPA) survey
reports an important trend of increased focus on social impact measurement. Over 96% of
respondents to the 2013/2014 survey report to be measuring social impact, 87% of the
respondents focus their objectives on measuring outcome while 85% still measure outputs
(such as the number of people reached) only. There is an increase though in measuring
impact (which requires an assessment of attribution): 70% replied on this survey using
impact measurement. However, social impact measurement still occurs less frequently
(mostly once per year) than financial performance management (mostly on a quarterly
basis).
Additionally, there is an increase of respondents that use social impact measurement to
unlock new funding, as the below chart from the survey report shows:

Despite the assumption that some of these respondents state they are measuring impact
when really they are measuring output, these results do show that European VPOs are
increasingly concerned with measuring social impact.

Which framework and models do they use?

Presumably due to the complexity of the concept, there is much debate about how social
impact should be measured. This has resulted in some 100+ tools, methods, frameworks,
databases, ratings systems, metrics and taxonomies.1

1

Check out: http://trasi.foundationcenter.org for more information

The ultimate guide: from succes to significance 9

This guide lists several of the most-used frameworks: Social Impact Assesment (SIA), Social
Return on Investment (SROI), Participatory Impact Assessment (PIA), Public Value Scorecard
(PVSc), Bottom of the Pyramid (BoP) Impact Assessment Framework, Logframe or Logical
Framework Approach (LFA).2

More
information
www.redf.org

Framework
Social Return
on
Investment
(SROI)

Description
SROI is a framework based on social generally accepted accounting
principles (SGAAP) that can be used to help manage and understand the
social, economic and environmental outcomes created by your activity or
organisation. SROI will help you understand, manage and communicate
the social value that your work creates in a clear and consistent way with
customers, beneficiaries and funders. It can help you manage risks,
identify opportunities to improve your impact and raise finance.

Social
Impact
Assessment
(SIA)

Includes adaptive management of impacts, projects, and policies (as well

as prediction, mitigation, and monitoring) and therefore plays a role in
project or policy planning. Can be applied to a broad range of
interventions, and not just within a regulatory framework. Understood to
be an umbrella or overarching framework that embodies all human
impacts.

http://www.st
.nmfs.noaa.go
v/tm/spo/spo
16.pdf

Public Value
Scorecard
(PVSc)

Based on the concept of the Balanced Scorecard, with three crucial

differences:
1. In the PVSc, the ultimate value created by the organization is measured
in nonfinancial terms
2. The PVSc focuses not just on paying customers, or clients/beneficiaries
who benefit from the organisations operations, but also on third-party
funders
3. The PVSc focuses on the capabilities for achieving large social results
outside the boundary of the organisation itself
Attempts to answer the question: What difference are we making? by
means of a participatory approach to measuring impact on livelihoods
Offers a useful tool for discovering what change has occurred and why.
Describes an eight-stage approach, including examples of tools which may
be adapted to different contexts; does not provide a detailed step-by-
step formula or set of tools to carry out project impact assessments.

http://exinfmv
s.securesites.n
et/workshop_
files/public_se
ctor_scorecar
d.pdf

Participatory
Impact
Assessment
(PIA)

http://fic.tufts
.edu

Bottom of
the Pyramid
(BoP) Impact
Assessment
Framework

Aims to understand who at the base of the pyramid is affected by BoP

http://wdi.um
ventures and how. Contributes to deeper insight into how the link
ich.edu
between profits and poverty alleviation. Based on the different well-being
constructs as developed by 1998 Nobel Prize winner Amartya Sen

Logframe or
Logical
Framework
Approach
(LFA)

The Logical Framework Approach (LFA) is a systematic, visual approach to

designing, executing and assessing projects that takes into account the
relationships between available resources, planned activities, and desired
changes or results. A logframe matrix serves to tranform this broader LFA
into action, and forms the basis of an actionable work plan to guide
implementation through the project/programme lifecycle.

http://bettere
valuation.org/
evaluation-
options/logfra
me

Maas, K. (2014), Classifying Social Impact Measurement Frameworks, The Conference Board Initiative on
Corporate Philantropy.

The ultimate guide: from succes to significance 10

In the below table the listed frameworks are compared with the EVPA 5 step-approach:
+ means little emphasis, ++ sufficient and +++ excellent.

Framework Step 1:
Step 2:
Step 3:
Step 4:
Step 5:
Setting
Analysing
Measuring
Verifying
Monitoring
objectives
stakeholders results
and valuing & reporting
impact
SROI
+++
+++
+++
+++
+
SIA

+++

+++

++

PVS

++

+++

PIA

+++

++

++

++

BoP

++

++

+++

++

Logframe

+++

+++

+++

Are they really that different?

Despite the differences between these frameworks a bridging factor is that most of these
are in some way or other based on the Theory of Change (as the above figure 1
demonstrates).

As described in an earlier chapter of this guide, the Theory of Change is the description of
the impact value chain of the funded project, programme or organisation. The Theory of
Change captures the necessary resources (inputs), the activities that take place, the tangible
products and services of these activities (output), the short and long term changes of these
activities (outcomes) and the extent of these long term outcomes that can be attributed to
the project, programme or organisation (impact).

Drawing up a projects Theory of Change will result in an impact map: the structure of your
analysis. When indicators are linked to the various building blocks (outputs, outcomes and
impact) in the impact map, the impact map can be used as a tool for on-going impact
measurement, monitoring and reporting.

The ultimate guide: from succes to significance 11

HOW%
How(do(you(measure?(((
How(do(you(report(
about(it?(
Op8onal:(How(much(is(
it(worth?(

Stakeholders(
Problem(
analysis(

Input(

Social
impact
measurement

Ac8vity(

Output(

Indicators(

Outcome(

Valua8on(

Impact(

Theory of Change

WHAT%
What(is(the(issue?(
How(urgent(is(it?(
What(is(the(scale?(
What(is(the(solu8on?(
What(are(the(objec8ves?(

Measuring & monitoring

WHO%
Who(are(involved?(
Who(contributes?(
Who(does(what?(

HOW%MUCH?%
What(do(you(deliver?(
What(are(you(changing?(
How(much(can(be(
aBributed?(
What(happend(anyway?(

Choosing a framework

When choosing a framework the most important element to take into account is the scope
of your analysis. The level, audience and purpose of your analysis will determine how the
social impact needs to be structured and visualised as well as how rigourous your analysis
needs to be. After determining the scope you can then choose the framework best suitable
for you.

Proportionality

An important concept related to the scope of your analysis, is proportionality:
Extensive randomized controlled trials can be useful sometimes even necessary before
scaling an intervention, but they are expensive and cant be used to manage an organisation.
An ongoing stream of high-quality information is desired, but the resources to acquire are
lacking. Superficial data will not be useful, but overdoing it will jam operations and confuse
those involved. What you need therefore is an approach to social impact measurement that
is simple enough to do, but rigorous enough to hold ground. An approach that is
proportional to the scope of the project, programme or organisation that is being measured.

The ultimate guide: from succes to significance 12

Social impact measurement in practice with Sinzer:

templates, benchmarking, aggregation

Sinzers mission is to make social impact measurement better accessible and cost-efficient
through software, in order to enable philanthropic organizations and their grantees to
manage their social impact consistently and maximize their value.

In order to achieve this mission the Sinzer software platform for social impact measurement
was designed: easy to use and flexible to adapt to match the social impact measurement
needs of a wide variety of organisations.

The Sinzer platform supports several existing frameworks, such as SROI, SCBA and the
recently new launched Strategic Impact Framework (read more about it in this guide: 8
steps to becoming an impact investor with Sinzer), but also allows for the easy
development of custom impact frameworks.

Social impact measurement can be operationalised on the platform through the following
functionalities:

Management tool
Results dashboard with dynamic charts and graphs
Build in survey module to collect data from beneficiaries and other
stakeholders for time-efficient impact monitoring
User management: invite grantees or co-workers as viewers and editors
Reporting module to export raw data in an Excel and Word file
Database integration: Global Value Exchange integration (including 200 IRIS metrics!)
White Labelling: create a personalised look & feel

The ultimate guide: from succes to significance 13

Templates
Defining a separate Theory of Change for all grantees can take a lot of
time and hinder comparability since most grantees will use project-
specific indicators.

To overcome these challenges the Sinzer platform allows the creation
of templates. A template can be understood as a sector or domain
specific Theory of Change containing predefined stakeholders, inputs,
activities, outputs and outcomes for a certain domain (e.g. health care,
education, agriculture, etc.). A template can include a standardized
survey for data collection from beneficiaries and/or other stakeholders
to conduct continuous impact measurement.

A template can be used to measure the social impact of multiple
projects within the choosen domain. Since a big part of the Theory of
Change is standardized in the template, 70% of the data on the
projectlevel will already be filled out, which saves a lot fo time in the
analysis process.
Benchmark grantees
Impact information can be used to better understand grantees impact relative to their
peers. Projects analysed with the same template can be benchmarked in Sinzer to gain a
better understanding of which initiatives work best and why, as well as help improve
grantees impact by knowledge sharing.
Aggregate the impact of multiple projects
Moreover, using templates allows data to be aggregated. This enables you to gain insight
into the impact of an entire sector or domain, and even into the impact of your foundation
or VPO as a whole.

The ultimate guide: from succes to significance 14

Case study
At Sinzer we advice our clients in creating sector specific templates, including collecting the
relevant data by developing appropriate survey-questions. Interested in reading a case study
about how we developed templates for a foundation that funds 1.300 projects per year?
Click here to go to the case study of Dutch VSB Foundation.

Time to start measuring!

Are you ready to start measuring impact of your grantees and projects? We would like to
finish this guide with a few recommendations that may help structure your thinking and get
you started with measuring social impact.

1.
2.
3.
4.

Firmly embed social impact measurement in your strategy

Allocate budget for social impact measurement
Create an open culture for the sharing of knowledge and best practices
Start, pilot, learn, improve: measuring social impact is an iterative process

Need advise or help with measuring social impact? Curious about the Sinzer platform and all
of its functionalities? Contact us via http://www.sinzer.org or info@sinzer.org.

The ultimate guide: from succes to significance 15