Policy Brief January 2011

Evaluating for Effective

Development Assistance
Problem Recommendations & Actions
Understanding what InterAction welcomes the prominence given to evaluation and evidence-based decision-mak-
works and why is crucial ing in the Presidential Policy Directive on Global Development, the Quadrennial Diplomacy and
to the success of U.S. Development Review and the draft Global Partnerships Act of 2010. The U.S. Government
development assistance should prioritize development assistance evaluation so it can strengthen our work with develop-
and to achieving the ing country governments and civil society to design and undertake cost-effective interventions
most cost-effective that build their capacity to address their development needs and thus contribute to lasting
use of limited aid development change. The U.S. Government must promote a range of evaluation approaches
resources. A number that suit different contexts, interventions and evaluation questions, as well as provide guidelines
of challenges inhibit for evidentiary rigor that address a range of situational circumstances and anticipated changes.
the U.S. Government’s • Support USAID’s efforts to reinvigorate its evaluation function and strengthen interna-
ability to achieve and tional development evaluation practices.
demonstrate results, • Prioritize evaluation within development assistance by:
including: (1) project »» Setting aside a minimum portion of program budgets for evaluation purposes;
timeframes not aligned »» Requiring USAID missions to reserve a portion of their funding to conduct evaluations
with the time required across portfolios of related projects to promote learning in areas of strategic interest;
to meet development »» Focusing on outcome-level changes; and
outcomes; (2) reporting »» Changing incentives to reward those who learn from their interventions, rather than
requirements that those who report positive results.
focus more on how • Make evaluation responsive to the host governments, civil society organizations and
money is spent than people with whom we work by mandating their inclusion as partners in evaluation design,
on lives changed; implementation, dissemination and use.
(3) placing donors’ • Ensure evaluation methods and notions of rigorous evidence are well-suited to the con-
demands over local texts and questions raised by the evaluation.
priorities for long-term • Where security concerns allow, present evaluation findings transparently and publicly
changes; (4) reluctance in-country and in the United States.
to acknowledge
development’s Results
complexity, take risks
Greater commitment to evaluating development programs, and to evaluating
and learn from failures;
them in the appropriate ways, reap positive results including:
and (5) insufficient
• The U.S. Government, through USAID, will emerge as a leader in development pro-
funding for evaluation.
gram evaluation and be enabled to undertake a more effective, efficient and respon-
sive U.S. foreign assistance program.
• Development assistance implementers will be able to reach a larger number of peo-
www.InterAction.org ple and contribute to better outcomes with the limited resources available. In more
concrete terms, this means having an aid program more responsive to the govern-
1400 16th Street, NW
Suite 210 ments, communities and individuals with whom we partner, and more capable of
Washington, DC 20036 producing lasting improvements in people’s lives.
202-667-8227
Background
Within USAID, steps are already being taken to rebuild the
agency’s evaluation capacity. In June 2010, for example,
USAID established the Office of Learning, Evaluation and
Research within the new Bureau of Policy, Planning and
Learning. The new office promises to elevate the promi-
nence of results measurement and the use of evidence in
USAID. This same office is working on a new Evaluation
Policy (due for release in January 2011) that will articu-
late expectations regarding evaluation practices, with
an emphasis on higher methodological standards, trans-
parency and utility performance and impact evaluations.
USAID is also developing new evaluation training programs
for program managers and evaluation specialists. In addi-
tion, through an interagency process, it is actively engaged
in developing the monitoring and evaluation frameworks
for several presidential initiatives (Global Health, Feed the
Future and Climate Change) in a way that is consistent with
a renewed emphasis on impact measurement.
Evaluation at USAID has an uneven history, marked by
a sharp decline in capacity over the past decade.1 One
particularly troubling trend has been the growing emphasis
on performance monitoring at the expense of evaluation,
resulting in a decline in the number of evaluations con-
ducted and too much attention paid to collecting data on
large numbers of low-level indicators. Much of the remain-
ing evaluation work is done by implementing partners,
preventing USAID from gaining the experience and advan-
tages that stem from undertaking such work. In 2006, the
Center for Development Information and Evaluation, which
had been charged the previous year with leading an initia-
tive to revitalize evaluation within USAID, was abolished.
These and other factors have resulted in several prob-
lems that must be addressed for development assistance
to be successful. These include:
• Focus on outputs rather than outcomes: Evalua-
tion at USAID has suffered due to the overriding focus
on tracking how money is spent, rather than on what
results aid has achieved. Too often, the focus has been
on measures such as the number of schools built or
teachers trained, rather than on the ultimate objective
of such activities—improvements in children’s learn-
ing. Furthermore, these measures are often decided in
Washington, with insufficient input from those actually
overseeing, implementing or affected by programs in
the field.
• Aversion to risk: Just as no one would expect every
business start-up to become profitable, it is unrea-
sonable to expect every development program to
1 Andrew Natsios, “The Clash of the Counter-bureaucracy and Develop-
ment,” Center for Global Development, July 2010.
be successful. Fear that negative evaluation results
will undercut support for aid (or funding for a specific
organization) means evaluation findings are not widely
shared, limiting opportunities to correct and learn from
failures.
• Demand for quick results: Development takes time.
Yet organizations are often asked to report on impact
over time periods too short for that impact to be
achieved, much less evaluated. While some meaning-
ful impacts, such as increases in agricultural yields or
patients treated, can be measured within a relatively
short time period, others, such as increases in income
or improvements in health status or learning, may take
years to materialize.
• Inadequate funding for evaluation: Monitoring and
evaluation-related budget items are among the first to
be cut when budgets need to be trimmed. This is a sign
that the agency does not prioritize evaluation.
There are encouraging signs that some of these problems
will be addressed and that the decline of evaluation within
USAID is starting to be reversed. The prominence given to
evaluation and evidence-based decision-making by senior
Administration officials is particularly welcome. Setting the
tone for all agencies involved in development assistance,
the Presidential Policy Directive on Global Development
calls for the rigorous evaluation of the impact of policies
and programs, and for the results of such evaluations to
drive policy and budget processes. In addition, the direc-
tive calls for more “substantial investment of resources in
monitoring and evaluation.”2
Previewing the contents of the Quadrennial Diplomacy
and Defense Review, Secretary of State Hillary Clinton set
out a vision of USAID as an agency in which decisions are
“based on hard evidence to ensure that investments deliver
results” and which measures success on the basis of
improvements in people’s lives rather than on “the number
of programs run.”3 She also reinforced statements made by
USAID Administrator Rajiv Shah about being transparent
about successes and failures and learning from them.
2 “Fact Sheet: U.S. Global Development Policy,” September 22, 2010.
http://www.whitehouse.gov/the-press-office/2010/09/22/fact-sheet-
us-global-development-policy (accessed October 27, 2010)
3 Hillary Rodham Clinton, “Leading Through Civilian Power: Redefining
American Diplomacy and Development,” Foreign Affairs, November/
December 2010. http://www.foreignaffairs.com/articles/66799/hillary-
rodham-clinton/leading-through-civilian-power