File 5 - Module e - Monitoring & Evaluation For Ffa 20 July 2011

Food Assistance for Assets (FFA) Manual
MODULE E
MODULE E: MONITORING AND EVALUATION OF FFA
This section is about monitoring and evaluation and what it takes to monitor and track the performance of FFA. An interventions success can only be known through the monitoring of ones work and documenting lessons learnt. This is often at the basis of building the case for replicable and scalable resilience building efforts. Such experience may be also shared with wider audiences through visual best-practice documentation with the greatest success achieved when your work is replicated in other locations.
ODXP PREVENTION & RECOVERY WORLD FOOD PROGRAMME
FFA Manual Module E: Monitoring and Evaluation for FFA
FFA Manual Module E (2011): version 1. This module was published and made electronically available in July 2011. Where relevant, this module supersedes previous guidance on FFA interventions. Please inform ODXPs Prevention and Recovery team if you identify outdated information that causes confusion with the information presented here. Any updates to Module E will be outlined below (and include page numbers) to allow FFA practitioners with an older version to identify where changes have occurred: No changes as yet.
Contents
E.1 E.2 1. E.3 1. OVERVIEW ................................................................................................. 6 WHAT IS MONITORING AND EVALUATION ................................................. 7 Purpose of M&E ................................................................................................. 7 MONITORING FFA AND KNOWLEDGE MANAGEMENT ................................. 8 Designing an M&E Strategy for Food for Asset Programmes ........................... 8 M&E systems and the National Government ........................................................ 8 When to develop your M&E strategy .................................................................... 8 Who is involved in the development and management of the M&E strategy ....... 8 2. Building Your Logical Framework ................................................................... 10 2.1. Defining the Objectives of the Food for Asset Programme ...................... 10 Identifying Intended Outcomes............................................................................ 11 Indicators.............................................................................................................. 12 Setting Targets ..................................................................................................... 15 Assumptions and Risks ........................................................................................ 15 3. Building the M&E Plan .................................................................................... 16 3.1. The M&E Plan as an organizational tool................................................... 16 Indicators, baselines and targets .......................................................................... 16 Data sources ......................................................................................................... 16 Data collection tools ............................................................................................ 19 Responsibility for Data Collection....................................................................... 20 Frequency of Data Collection .............................................................................. 20 4. Quality Assurance Mechanisms ........................................................................ 20 4.1. 5. 6. 7. 8. Data quality assurance ............................................................................... 20 Quality of services ............................................................................................... 21 M&E Plan Matrix ............................................................................................. 22 Use of Information ............................................................................................... 22 Evaluation and Reviews .................................................................................... 24 Types of Evaluations in WFP .............................................................................. 24 M&E Capacity Building and Systems Strengthening ....................................... 27 M&E Coordination and Budget ........................................................................ 28 M&E Coordination .............................................................................................. 28 Budget .................................................................................................................. 28 9. M&E Resources ................................................................................................ 28
E.4 1.
BEST PRACTICES .........................................................................................29 Best Practice Templates .................................................................................... 30 1.1. Infotechs or Best Practices ........................................................................ 30 What to remember when gathering Best Practices: 5W & 1H ............................ 31 Field Form ............................................................................................................ 33 A sample Best Practice gathering experience ...................................................... 33 Main advantages and risks of Best Practices ....................................................... 35
2. 3. 4. E.5
Case studies and research papers ...................................................................... 36 Evaluations ........................................................................................................ 38 Others (pamphlets, leaflets, flyers, posters, photo-series, films) ................... 38 GLOSSARY.................................................................................................45
AN INTRODUCTION TO THE FFA MANUAL:

Overall WFP uses approximately 12 to 15 percent of its yearly resources for assets restoration, rehabilitation or creation under emergency, recovery and enabling development operations. Most countries receiving food assistance increasingly promote policies and strategies requiring various forms of conditional transfers (productive safety nets, special operations to improve access to food, disaster risk reduction, and resilience building). It is therefore important for WFP staff (and its partners) to meet these challenges and emerging demands. The purpose of this manual is to strengthen WFP staff understanding of the contexts that require FFA, their selection and programmatic coherence to WFP global and local commitments, as well as main design aspects. The manual is divided into four modules: Module A provides the overall rationale and framework for FFA within the WFP toolbox of assistance Module B provides the analytical lens in which to determine if FFA is appropriate within specific contexts Module C helps define the specific FFA projects to be undertaken within these specific contexts, depending on various factors Module D provides the practical elements of implementing FFA Module E provides the key elements that informs M&E for FFA Caveats . A limitation of this FFA manual is that it cannot be fully comprehensive the nature of FFA can be so diverse that it would be impossible to capture all possible approaches and interventions. Therefore, this guidance focuses largely on the response options and assets that are commonly related to WFP operations. . A second limitation relates to the range of response options and FFA interventions related to pastoral and urban settings. These are simply insufficient as documented experience regarding FFA from these areas has been limited. However, there has been increased attention in several CO to both pastoral and urban livelihoods in recent years that will bring further lessons and best practices. Furthermore, the current FFA guidance is largely built upon documented evidence from a few countries where FFA have demonstrated significant impact and have been documented both in terms of the processes that lead to positive results to technical standards and work norms. It became clear to the authors that there are several other countries with important experience (past or recent) that could not be taken into consideration or only marginally in the drafting of these guidelines because of insufficient information. . Another limitation is the level of insufficient research information regarding FFA under different programme contexts and the often anecdotal assumptions that tend to underplay the role and impacts of FFA (positive and negative). . A final limitation is the lack of guidance on Food for Training (FFT) which is largely absent in these guidelines as cutting across all programme design components (school, feeding, HIVAIDS, nutrition, etc). In relation to FFA, these guidelines include FFT only in relation to the range of assets that would impact on disaster risk reduction and resilience building.
MODULE E: Monitoring and Evaluation of FFA

E.1 OVERVIEW
The purpose of this document is to provide basic guidance for the development and management of Food for Asset M&E systems, including the use of best practices. The intended audience is WFP Country Office staff involved in the design, implementation, monitoring and evaluation of WFP-assisted Food for Asset activities. For more in-depth information on the design and implementation of M&E systems in general, visit the Indicator Compendium on WFPgo or contact the M&E unit in ODXP at HQ. The monitoring and evaluating of programs at local and corporate level is a dynamic and evolving process. As contexts and priorities change, the response too will change. This requires M&E systems to be flexible and adaptive to the changing context. Country offices are strongly encouraged to regularly visit the Programme Guidance Manual on WFPgo for the most current guidance related to M&E.
E.2
WHAT IS MONITORING AND EVALUATION
1. Purpose of M&E
Project monitoring and evaluation serves two main purposes: To provide regular information on project performance in turn promoting efficient and effective implementation and operation of projects (accountability) To provide lessons for the planning and design of future projects
Fundamentally, it is a tool to be used to support the basic managerial functions of planning, directing, and decision making with the ultimate goal of continuous programme improvement. Monitoring Monitoring is the day to day management task of collecting and reviewing information that reveals how an operation is proceeding and what aspects of it, if any, need adjustment. The purpose of programme monitoring is to refine and optimize the approach to delivering the programme to the beneficiaries. It assesses how consistently a programme is operated relative to its design. Routine programmatic monitoring includes context, process, output, and outcome monitoring. Evaluation Evaluation should happen at key milestones during a program lifecycle to inform relevance, efficiency, effectiveness, sustainability and impact measuring the success of the project relative to its stated objectives. As such, it focuses on programme outcomes and the impact of the intervention on beneficiaries. Evaluations provide recommendations which can be used to inform the design of a subsequent project (if there is one). In addition, evaluations contribute to the debate related to strategic issues and promote accountability. How are they linked? Monitoring and evaluation are closely linked and mutually supportive and complementary. Evaluation needs and feeds on a good monitoring system with sound performance indicators and reliable tracking. Whereas monitoring is a day-to-day management tool, evaluation is a strategic tool that involves comparing data to draw conclusions not only for the intervention, but for programming in general (M&E Knowledgebase).
Key messages: M&E systems are the primary tool for measuring programmatic performance and impact Data derived from monitoring systems and evaluations inform future program design Monitoring and evaluation are closely linked, mutually supportive and complementary
E.3
MONITORING FFA AND KNOWLEDGE MANAGEMENT
1. Designing an M&E Strategy for Food for Asset Programmes

A comprehensive M&E system requires a clearly outlined strategy for ensuring that the monitoring and evaluation of data collected is of high quality, timely and informative. The strategy should outline in detail what data will be collected, how, when and by whom it will be collected, the process for data analysis, persons responsible including government and implementing partners, budgetary requirements and other such relevant details. M&E systems and the National Government To the degree feasible, the government, through the appropriate institution (often the Ministry of Agriculture), should be responsible for the management and implementation of FFA M&E activities. In any case, government counterparts should to be involved in the design of the M&E system and participate in data collection, processing and analysis. It is critical to build government capacity in this regard to ensure the sustainability of the programme.
When to develop your M&E strategy The M&E strategy is designed in conjunction with the general project design. This happens after a needs assessment and context and seasonal livelihood analysis has been undertaken, and the objectives for WFP food assistance have been defined. In reality, M&E systems are rarely developed from scratch. More likely Country Offices will already have a current M&E system in place which can be modified accordingly to measure identified objectives and outcomes for the programme. The design (or re-design) of a new project provides an ideal opportunity for the review and assessment of the current M&E systems. This process is described in more detail in the M&E Capacity Building and Systems Strengthening section.
Who is involved in the development and management of the M&E strategy The development and implementation of the M&E strategy should be participatory, lead by the government with support from WFP staff (usually the Head of Programme, with support from programme and VAM officers) and inclusive of key stakeholders such as community representatives and cooperating partners. A participatory approach to designing M&E systems has multiple benefits. It fosters ownership which is linked to better data monitoring uptake, and improved responses to identified issues. As a general rule, M&E systems should be as simple and user-friendly as possible, focusing on the essential types of data but avoid overloading - any M&E system can serve its purpose
only if it is actually used and functioning! The degree of complexity or sophistication will depend on the M&E capacity of each country; however, as a general rule, it is preferable to collect only a few indicators in a regular and reliable manner rather than setting up an ambitious list of indicators which turn out to be impossible to collect. Key messages: A clearly outlined strategy is essential to a comprehensive M&E system Effective and sustainable M&E systems are participatory and have strong Gvt involvement M&E systems should be as simple and user-friendly as possible
2. Building Your Logical Framework

Building your M&E strategy begins with the development of a logical framework (logframe). The logframe is the link between project design and the design of the M&E system. The process of developing this tool can help clarify and outline the objectives of your food for asset project and the final product serves as an outline of your strategy and thought process. The logframe feeds into your M&E plan matrix, which outlines in greater detail how data will be collected and used. For a detailed description on building a comprehensive logical framework, click here http://docustore.wfp.org/stellent/groups/public/documents/other/wfp007751.pdf An abridged version of the logframe is submitted with your project document, and should reflect what outcome and output indicators you will be measuring and reporting on. All indicators reported in the logframe should be reported on at some point in the life of the project. The project document logical framework outlines: Clearly defined and realistic outcomes and outputs A minimum set of indicators for each outcome and output that are feasible to collect and analyse, including targets Assumptions and risks that may influence success and failure For Country Programmes and Development projects, logical frameworks should be aligned with the UNDAF and its Results Matrix. It is thus critical for Country Offices to engage in the interagency policy dialogue and UNDAF relevant working groups from the beginning, so that WFP inputs can be fully integrated in the UNDAF or any UN common programming process. 2.1. Defining the Objectives of the Food for Asset Programme 2.1.1. Formulating Objectives Formulating objectives entails identifying precisely the results WFP is trying to achieve within the FFA programme. The basis for all program design and corresponding M&E strategy should flow from these clearly defined objectives, which are appropriate, rationale, and correspond to the national context and policy frameworks. When objectives are clearly outlined they are easier to evaluate. Formulating objectives entails identifying precisely the results you are trying to achieve and is based on a contextual analysis of the needs in a specific country, paying particular attention to government priorities and strategies. The objectives of your food for asset activities should be stated in terms of support to national and regional policies and frameworks, such as the Comprehensive Africa Agriculture Development Programme (CAADP). They should also be linked to WFPs Strategic Objectives, in most cases Strategic Objective 2 (Prevent Acute Hunger and Invest in Disaster
10
Preparedness and Mitigation Measures) and Strategic Objective 3 (Restore and Rebuild Lives and Livelihoods in Post Conflict, Post Disaster or Transition Situations). The type of operation (i.e. EMOP, PRRO, DEV/CP) will influence the objective of WFPs FFA intervention. For example, an emergency operation may have improved food consumption as the primary objective with FFA activities used as a conditional transfer in line with WFPs Strategic Objective 1 to Save lives and protect livelihoods in emergencies. In comparison, a PRRO may focus on the longer term objective of enhanced resiliency to shocks and reestablished livelihoods through asset creation (SO2 and SO3 respectively). For more information on the link between the programme categories (type of operation) and defining objectives, click here http://one.wfp.org/eb/docs/2010/wfp220540~2.pdf The objective of your food for asset activity depends on the context and the specific needs and priorities in a country or region, as well as the capacity and technical skills of your cooperating partners. Different results or outcomes can be expected when asset creation is used in different contexts, with different objectives in mind. Complementary activities and enabling factors are critical the scale and complexity of a food for asset intervention depends on the capacity and expertise of cooperating partners and the ability to procure non-food items (NFIs) such as tools and equipment. The lack of these essential items could mean that low-tech asset creation (e.g. desilting of a water pond or compost making) may be more appropriate with the primary objective of the food for asset activity being conditional transfer and increased food consumption, and the secondary objective being the asset created.
Identifying Intended Outcomes Outcomes are the mid-term results or changes that the food for asset activity is supposed to bring about, usually within the course of the project. Outcomes must contribute to the achievement of the interventions objective and each intended outcome must address the causes of a specific problem identified in the problem analysis1. Properly defined outcomes are a key feature of good design. There must be a logical link between the problem, the outputs of the intervention, and the intended outcome. Outcomes should be specific, measurable, achievable, relevant, and time bound (SMART). Outcomes that have all five of these criteria can be more easily evaluated at the project conclusion. Food for Asset Outcomes and the Strategic Results Framework The Strategic Results Framework is a core component of WFPs strategic, management and accountability frameworks. It is also the basis for WFPs performance measurement system, which enables WFP to translate its mandate and strategy into tangible outcomes. The framework helps WFP to demonstrate to what degree it has achieved its Strategic Objectives
1
A sound problem analysis helps ensure that WFP interventions address the underlying causes of food insecurity, and so make a permanent improvement in the lives and livelihoods of poor people Programme Guidance Manual
11
and the Millennium Development Goals (MDGs). In addition, the Strategic Results Framework provides the basis for aligning country-level monitoring and reporting with the five Strategic Objectives: it enables WFP to track outcomes and outputs at the project level, which can then be aggregated to show achievements at the corporate level (WFP Outcome Measurement Strategy, 2009). Outcomes are to be aligned with WFPs Strategic Results Framework (SRF) and its corporate outcomes. Country offices will select intended outcomes and outputs from the corporate outcomes and outputs outlined in the Strategic Results Framework under the relevant Strategic Objective. The corporate outcomes are intentionally generic and should be used as the basis for the development of project outcomes which will meet the SMART criteria. Indicators Indicators play the critical role of informing management as to whether an operation is being implemented as planned and achieving the desired results as articulated in its logical framework. As such, indicators are also referred to as performance indicators. A baseline or benchmark value is determined at the beginning of your project and the indicator is used to measure change against this baseline value. The measurements of an indicator may be disaggregated by any number of factors. For WFP, gender and age group are critical factors for disaggregation. Disaggregation of indicators can also be operation-specific and should be defined in terms of the subgroups that are to be compared with one another. Finally, it is important to note that selecting indicators should be as consistent as possible with those used by the government or partners. There are five different types of indicators outcome, output, context, input, and process: 2.1.2. Outcome Indicators Selecting Outcome Indicators from the Strategic Results Framework All outcomes listed in an M&E plan need to be aligned with at least one indicator, the first choice being a corporate indicator. Corporate indicators are those indicators found in the WFP Strategic Results Framework and are aligned with the corporate outcomes and outputs to be achieved under specific Strategic Objectives (SOs). The primary corporate indicators for food for asset activities are the Household Asset Score and Community Asset Score. In cases where the primary objective of the food for asset activity is improved food consumption, the Household Food Consumption Score may be a more appropriate indicator. Household and Community Asset Scores The asset score measures the increase in functioning assets that enable a community or a household to be more resilient, or less negatively impacted, by shocks.
12
A checklist of appropriate household or community level assets is created by the country office in consultation with community members. The type of assets listed will depend on the objective of the activity SO2 activities will focus on resiliency-based assets whereas SO3 activities will focus on infrastructure and takes into account livelihood contexts. Prior to the projects start or in the early stages of the project, a community assessment is done to determine how many of the assets on the list are available and functioning. For each asset that is available and functioning, the asset is assigned a score of 1. For those assets on the list that are not available or functioning, a score of 0 is assigned to each. The scores are then tallied and the final number is used as a baseline. At the completion of the project or at pre-determined intervals (i.e. annually), the same communities or households are reassessed and the same process is undertaken as was used for the baseline. Functioning assets in the community are counted and a score is tabulated. Note that all the assets in the community are counted, not just the ones attributed to WFP. The intent is to measure the overall resilience or enhanced capacity of the community, which can only be done if all assets are included in the count. The asset score is calculated by subtracting the baseline score from the follow up survey score. This score is calculated for all of the communities surveyed and is either reported as the percentage of communities with an increased asset score or as an average asset score across communities. To find more information on the Household Asset Score and the Community Asset Score, including methodology, see Annex E-1 or visit the Indicator Compendium on WFPgo http://wiki.wfp.org/M_and_E_Kit/index.php/Indicator_Compendium Corporate indicators may be complemented with project specific indicators, which can provide a more accurate and in depth performance measurement and a better understanding of the range of factors influencing the outcome. They allow for flexibility at operational level to provide data being collected for donor or partner reporting requirements or to align with national frameworks indicators (as framed in a Disaster Risk Reduction Strategy or the UNDAF, for instance). Project-specific indicators, unlike corporate indicators, are optional. The WFP Indicator Compendium provides a comprehensive selection of project-specific indicators. Where possible, country offices are encouraged not to create new indicators but use the ones already available, which have been compiled from various active projects within WFP. Projects are encouraged to measure as many additional indicators as will provide a comprehensive picture of the project dynamics. It is important to note that adding 2-3 optional indicators can greatly increase measurement capabilities but can also dramatically increase the cost and burden on both the beneficiaries who provide the data and the staff processing the data collected.
13
The following table provides an overview of outcome and outcome indicators (corporate and project specific) relevant to food for asset activities as described in the SRF and the Indicator Compendium. It also provides the aligned strategic objective and corresponding operational context. Note that this is a general guideline which will not always apply to every context or situation a country office encounters.
Outcome Indicators Outcome Corporate Indicator(s) Project Specific Indicators
Strategic Objective
Operational Context
SO1 Save Lives and Protect Livelihoods SO2 - Prevent Acute Hunger and Invest in Disaster Preparedness and Mitigation Measures
Emergency
Outcome 1.3 Improved food consumption over assistance period for targeted emergencyaffected households
No. of HHs reporting increased Household income from asset created Food Consumption No of HHs with increased Score (HFCS) employment opportunities Household Asset Score (HAS) Community Asset Score (CAS) No of HHs/communities with access to clean and safe water % of assets created through FFA/FFW managed and maintained on a regular basis by communities/households % of beneficiaries with increased access to social services (schools, clinics, markets, etc.) due to road construction/rehabilitation Community Asset Score (CAS) % of HHs who have recovered their pre-crisis asset level % of beneficiaries who have reestablished their livelihood post crisis
Cross-cutting
Outcome 2.3 Hazard risk reduced at community level in targeted communities
Protracted Relief Outcome 3.2 Targeted SO3 Restore & Recovery communities have and Rebuild increased access to Lives and Development or assets in fragile Livelihoods CP transition situations
2.1.3. Output Indicators Outputs are the actual deliverables (products, goods and services), which the operation is expected to produce and can be held accountable for. An example of a FFA output indicator is Hectares (ha) of gully land reclaimed as a result of check dams and gully rehabilitation structures. Outputs should lead to one or more of the intended outcomes.
14
2.1.4. Context Indicators Context indicators inform on the context of the operation. These indicators may not be specific to food for asset activities but to the country context in general. Context indicators can be linked to the risks and assumptions outlined in your logframe. Examples of context indicators include rainfall patterns do not change significantly or suitably qualified government staff available and willing to be trained and to remain in government disaster management and/or response departments and/or ministries. 2.1.5. Input Indicators Input indicators inform on the resources allocated to food for activities, such as human resources. An example of an input indicator is the number of field monitors employed. 2.1.6. Process Indicators Process indicators inform on the quality of the intervention as it relates to timeliness and efficiency. An example of a process indicator is the determination if food and/or cash are distributed to FFA participants as scheduled. Setting Targets For each indicator a realistic target should be identified. A target is the desired level of performance to be accomplished within a specific period of time; this can be a specific indicator value or a trend (WFP Indicator Compendium). The Strategic Results Framework provides project targets for all corporate indicators but this is to be used as a guideline only. Targets should be set based on the context in which the food for asset intervention operates, and reflect a change that is realistic based on the context. Assumptions and Risks Assumptions are factors that are taken as a given in the design of an intervention, and that are expected not to change, or to change in foreseeable ways. For example, in the case of Bangladesh, an assumption for their enhanced resiliency activity was that a Supportive government policy for comprehensive risk reduction strategy continues. Good planning requires the recognition of major assumptions, and enough flexibility to permit design changes in response to changes in conditions. Risks are about uncertainty that can result in positive or negative outcomes. An example of a risk is continued donor funding for the food for asset intervention. Key messages The logframe is the link between project design and the design of the M&E system The objective of the FFA intervention is often dependent on the context and country priorities, as well as the capacity and skills of your cooperating partners. Outcome indicators should be selected based on the primary objective of the FFA intervention in many cases it will be the HAS and the CAS
15
3. Building the M&E Plan

3.1. The M&E Plan as an organizational tool Effective implementation of an M&E strategy requires a clearly outlined plan to ensure thorough and timely data collection and analysis. The main components of an M&E plan include: Indicators, baselines and targets Data sources, including routine data collection and surveys Quality assurance mechanisms data quality and program quality M&E coordination Evaluation reviews M&E capacity building and system strengthening M&E implementation plan and budget Each of these components will be discussed in greater detail throughout these guidelines. Indicators, baselines and targets The first component of the M&E Plan is Indicators, Baselines and Targets. As discussed in the previous section on building a logical framework, there are five types of indicators to be used for monitoring your food for asset activity. The choice of indicator and corresponding target is dependent on country context, programme objectives and other consideration. The WFP Indicator Compendium can help with the selection of appropriate indicators and can provide guidance on targets for corporate indicators. 3.1.1. Baselines Baseline data is the data collected prior to intervention and is used to measure what, if any, change has occurred at outcome level as a result of the FFA intervention. What baseline data you collect depends on the outcome indicator youve selected to measure your FFA intervention. For the Community Asset Score or the Household Asset Score, a baseline will be conducted to determine the current status of assets in the community based on the number of functioning and available assets. For FFA activities that will look to improve food consumption of participants and families, a baseline will be conducted to determine the Household Food Consumption Score. WFP frequently collects this data during the Comprehensive Food Security and Vulnerability Analyses (CFVSAs), the Emergency Food Security Assessments (EFSAs) and through Food Security Monitoring Systems (FSMS). The Food Consumption module can be easily included in Post-Distribution Monitoring Surveys. Data sources Quality data sources are essential to well functioning M&E systems. Aspects of reliability, accuracy, timeliness, completeness and integrity should all be considered when determining the quality of a data source.
16
Data may be obtained from primary or secondary sources. Primary data are obtained through direct contact with respondents, such as beneficiaries and community members, and entail face-to-face information sharing between the surveyor and the representatives of the population under survey. Secondary data is information that has already been collected by others i.e. routine data collected by cooperating partners or data collected by governmental departments. Making use of solid secondary data (with accurate collection methodology and verification of validity) is very resource effective. Data collection can use qualitative or quantitative methods. Qualitative methods include interviews, focus group and stakeholder meetings, field visit and direct observation as well as spot-check visits. Quantitative methods include surveys, such as those used for collecting data on the household and community asset scores and questionnaires for post distribution monitoring, where counts are provided, such as in output data. Data Collection Data collection is the backbone of monitoring. It provides the information that allows WFP staff to assess planned vs. actual achievements, progress and changes of the operation. As such, it is crucial that data is collected in a systematic and strategic way that garners accurate, timely and reliable information. This is done by articulating the necessary steps required to collect the data, known in the M&E plan as the means of verification. 3.1.2. Routine Monitoring Beneficiary Contact Monitoring (BCM) & Post Distribution Monitoring (PDM) There are two types of regular monitoring activities in WFP: Beneficiary Contact Monitoring (BCM) and Post Distribution Monitoring (PDM), which serve the dual purpose of monitoring the operation against the planned implementation, and providing an up-to-date assessment. Post Distribution Monitoring monitors process and outputs indicators, such as planned vs. actual numbers of targeted beneficiaries, amount of metric tonnes distributed, appropriate ration size, quality of commodities, and quantitative output data such as number of bridges constructed. Beneficiary Contact Monitoring is the monitoring of beneficiary perceptions of the food assistance provided. It focuses on beneficiary access to, use of and satisfaction with outputs by seeking feedback directly from the beneficiary. BCM and PDM can be conducted simultaneously during field visits.
17
3.1.3. Survey-based Monitoring Household Asset Score (HAS) and Community Asset Score (CAS) Surveys To measure outcomes related to FFA activities, specifically the community asset score and the household asset score, a survey should be undertaken if routine monitoring processes are not sufficient to collect the outcome data required. The HAS and CAS survey data collection tool will collect information on assets in the community, their level of functionality and the percentage of community members using or benefiting from the asset (in the case of the CAS). A baseline survey is conducted prior to project implementation and a follow up survey at project completion or on an annual basis. For more information on the HAS and CAS see the section on indicators and data collection tools. Food Security Monitoring Systems Food Security Monitoring Systems (FSMS) monitor changes in peoples food security situations. They aim to alert the humanitarian community when a situation deteriorates and to ensure that assistance can be provided in a timely and appropriate way. For example, an FSMS collects information on individual households' food consumption, their income and the strategies they develop to cope with difficult situations. It also monitors market prices and rainfall patterns. For more information on FSMS contact the VAM unit (ODXF) at HQ or reference the FSMS handbook (Source: WFP.org) The Community and Household Surveillance (CHS) model The Community and Household Surveillance (CHS) system provides information on the food security and nutrition situation at critical times of the year. A specific feature of the system is that it allows the measurement of short and longer-term effects (outcomes) of food assistance interventions by capturing comparable data from beneficiaries and nonbeneficiaries. The system has been tested over many years in seven countries in the SADC region. The system fits best for interventions where targeted households are receiving household rations (in the form of food, cash or vouchers) over several months and the main objective is to enhance the household food security status (e.g. unconditional or conditional transfers). It enables food security and nutrition trend analysis of beneficiaries and non beneficiaries over time and allows for impact assessment of different programme categories and transfer modalities. Besides outcome monitoring, the system also covers post-distribution monitoring information including household perceptions on targeting, access to assistance, satisfaction with the quality, and use of food.
18
Conducting Field Visits For quality assurance purposes, field visits should occur regularly and on a random basis. Ideally, all food for asset work sites should be visited at least once, however for very large operations a representative sample can be used. Irregularities observed during field visits should result in corrective action by WFP and the Government, or implementing partner.
Sampling makes it possible to analyse the impact of a WFP operation. Sampling is used when a subset of the population (or other unit) under study is selected from the larger group (the entire population under study). By studying the findings from that sample it is hoped that valid conclusions can be drawn about the larger population from which the sample was taken. Sampling is commonly employed in order to avoid the expense and time associated with total enumeration of the population, as is done during a census. Source: M&E Knowledgebase
Data collection tools Once a quality data source has been established, tools are required in which to collect the data. FFA data collection tools generally take the form of a questionnaire or a checklist and need to be carefully designed to collect only necessary information to ensure efficient use of M&E resources. In other words, each question asked in a questionnaire or checklist should be useful and necessary for the analysis. The indicators selected in the M&E Plan are the basis on which the data collection tools should be designed. See Annex E-1 for examples of CAS/HAS and PDM data collection tools.
19
Responsibility for Data Collection Data collection is generally conducted by WFP field monitors and in some cases programme officers, in collaboration with government and partners. When the government is responsible for the implementation of WFP FFA activities, roles and responsibilities for M&E are outlined in the Letter of Agreement. WFP role is to support monitoring through capacity development (training) and technical assistance and through ongoing supervision and oversight. If the FFA activity is operated by a cooperating partner, the responsibilities for data collection, analysis and reporting are laid out in the Field-level Agreement (FLA), a contract that spells out what data needs to be collected, how often and formats for reporting the required information back to WFP.
Frequency of Data Collection The frequency of data collection is decided by the country office prior to implementation of the activity and is dependent on the indicator and the scale and capacity of the operation. PDM data can be collected monthly or quarterly and in many cases, a monthly report is submitted by FFA coordinators to track progress made on FFA projects. The nature of outcome measurement requires that outcome data is collected less frequently than process indicators and is dependent on the duration and type of implemented activities that are necessary to create the assets. At minimum, data for the HAS and the CAS should include two rounds of data collection per project cycle one at baseline and a second to collect follow up data. Outcome data can be collected annually for reporting purposes but this more feasible for the CAS than the HAS which is much more time consuming and costly.
4. Quality Assurance Mechanisms

Quality assurance mechanisms are an essential part of any M&E system as way to assess both the quality of the data being reported and the quality of services delivered. Routine quality assurance checks on the data being collected and the quality of services provided allow programme staff and management to be responsive to issues that arise to ensure proper corrective action is taken.
4.1. Data quality assurance Data quality at all stages (data collection, transfer, compilation, analysis and storage) is a key consideration and requirement for quality M&E systems. This is in particular relevant for
20
routine program monitoring data, for which data may be collected by many different partners with many different capacities and at many different levels. Data quality should be considered along the following five dimensions:
DATA QUALITY
Reliability: Accuracy: Timeliness: Completeness: Integrity: Integrity is when data generated by the information system at all analisis levels are protected from deliberate manipulation or bias for political or personal resons. How well does This dimension Completeness The data the data inside or refers, among generated are means that the derieved from a others, to how based on information database or current up-to-date protocols and system is registry reflect data is at the time capturing all that it procedures that the reality is is of release. In this do not change should. This supposed to respect there acording to the however also measure. might be a gap user, when and without capturing between the how often they to mutch. reference peiod of are used. The data the data and the is reliable because date on which the they are collected data becomes consistently. available.
Source: WFP ODXP M&E Unit Data quality assurance requires adequate human resources and technical capacity (training, equipment) to implement. Examples of data quality assurance mechanisms are random supervision visits at FFA sites and systematic checks on data entry. Feedback mechanisms should be put in place to address identified data quality issues It is recommended in any case to triangulate (compare) WFP monitoring and survey data with other data available to ensure it is consistent. This work can help to identify data collection issues in WFP or issues with secondary data sources.
Quality of services Assessing the quality of services is linked with the monitoring of process indicators that will further inform whether for example the food was received on time, or the right amount of food was distributed. It also includes the beneficiary perception of services and beneficiary satisfaction. The monitoring of the quality of services delivered may be done in a statistical manner, ensuring representativeness at beneficiary level, or in a purposive manner, ensuring services are assessed in areas where issues are most likely to occur. A combination of both types of assessment would be most effective, however with increased cost.
21
Similar to data quality assurance, when considering quality of services assurance it is important to ensure that adequate resources are allocated to it and that protocols include feed-back mechanisms to stakeholders on identified quality issues and actions to be taken to address issues.
5. M&E Plan Matrix

An M&E plan matrix can be useful in organizing essential information by establishing clear roles and responsibilities of WFP and partners; planning for routine collection of monitoring data based on identified indicators, and ensuring that M&E reports are disseminated to all stakeholders in a timely manner. The M&E Plan matrix begins with identifying indicators, as discussed For each indicator, the matrix identifies: the means of verification (i.e. data source, collection method and tool, responsibility and frequency): and the use of information (i.e. which reports the data will be presented in).
Use of Information The Use of Information column outlines how the information collected will be used. This is important for ensuring that each piece of data collected is done so for a reason, with a practical plan for the use of the information. The submission of timely and accurate reports is essential for management follow-up at the regional bureau and/or headquarters, particularly when prompt corrective action needs to be taken. Regular, authoritative reports help maintain donor confidence and support resource mobilisation efforts: they form the basis of WFP's accountability to its donors and the international community. Country Offices annually report on each operation in the form of a Standard Project Report (SPR). Other reports can include executive briefs and country office situation reports, and external reporting requirements for donors and governments.
22
Example M&E Plan Matrix

M&E Plan Matrix Operation Name: Operation Number: Start Date: [enter name] [enter name] (dd.mm.yyyy) Means of Verification Indicator Name Data source Collection Method/Tool Responsibility for Data Collection Frequency of Data Collection Use of Information Reports
Context Input Process Output Outcome
23
6. Evaluation and Reviews

Evaluation is the systematic and objective assessment of an on-going or completed project, programme or policy, its design, implementation and results. The aim is to determine the relevance and fulfilment of objectives, efficiency, effectiveness, impact and sustainability. The monitoring data collected throughout the life of the operation, is used to inform the evaluation. Types of Evaluations in WFP There are several different types of evaluations used within WFP, each with benefits and drawbacks. 6.1.1. Conducting Self Evaluations of FFA projects The most common evaluation in WFP is the internal evaluation conducted by the Country Office. This is referred to as a self-evaluation or a project review. A self evaluation is less costly than an external evaluation and can be done quickly, using information garnered throughout the course of the project. The drawback of self-evaluation is they do not meet WFPs evaluation requirements of independence because the commissioner of the evaluation is at the same time the manager of the operation under evaluation. As such, self evaluations are often perceived as lacking objectivity. Nonetheless, these exercises are important for internal learning and accountability and for providing lessons which can be shared across countries and regions. Self evaluations are undertaken by WFP staff in country offices without involvement of consultants other than for the facilitation of the exercise, if necessary. They report on the implementation of the annual work plan, providing a comparison of planned versus actual results. Given that they are self-evaluations, they do not have to meet the requirements for independence spelled out in this policy. Source: WFP Evaluation Policy 2008 Evaluations should provide learning about key aspects of an operation: Timing, Budget and activities selected Targeting and outputs Planned and achieved outcomes Stakeholders involvement in the process How to maintain benefits A country office will generally conduct a self evaluation or project review at the end of the project. If the project is being extended or if activities are being continued in another operation (i.e. if an EMOP has transitioned to a PRRO), an evaluation should be conducted midway through the life of the project so that recommendations and lessons learned can feed into project design. Generally an evaluation will be planned for the entire project, of which the asset creation activity might be one element. There are several aspects to consider when planning a self evaluation which will ultimately impact and influence the design of the evaluation
24
1) Determine the rationale for evaluating the project - that is why is the evaluation being undertaken and why now. Common reasons for conducting a self evaluation include the upcoming start of a new project, donor pressure and good practice. 2) Determine what is to be gained from the evaluation that is what the objectives of the evaluation are. Evaluations generally have two objectives: accountability and learning. Self evaluations are primarily geared towards learning as the accountability aspect is compromised by the lack of independence. The rationale and the objective of the evaluation will determine how the self evaluation will be conducted. 3) Determine who will benefit from the evaluation? That is, who is your target audience for the evaluation? Commonly this is the country office, donors, cooperating partners, and/or beneficiaries. 4) Determine what the evaluation will include? That is, what is the scope of the evaluation? Will it evaluate all activities or just food for assets? Will it be confined to one geographical area or all? Will it evaluate the duration of the operation or just a portion of it? Perhaps it will evaluate food for asset activities over the course of several operations. Additional Considerations Self evaluations require a considerable human and financial investment. It is important to be realistic about the amount of time and resources that can be committed to the evaluation, as this can greatly impact the design of the evaluation. Finally, it is important to remember that evaluations need data. This is where your monitoring system comes into play. A strong logical framework and corresponding monitoring system are key components of an evaluation, and will provide the information needed to assess your project. Guidelines for conducting self-evaluations are being developed by OEDE and should be available by September 2011 on their website: http://go.wfp.org/web/evaluation/home 6.1.2. De-centralized Evaluations Decentralized evaluations are evaluations which are conducted by consultants hired by the country office. While often more rigorous than self evaluation, these evaluations still do not meet the requirement of independence as it is the country office that manages and oversees the consultants conducting the evaluation. However, a well-planned and executed decentralized evaluation can be extremely valuable for learning purposes. An example of one is provided below: Review and Appraisal of WFP FoodForAssets (FFA) Programme in Kenya, 20042009 In 2009, the Kenya country office conducted a project appraisal of food for asset activities that had been implemented over the course of five years. The evaluation involved stakeholder consultation, field visits and desk review. The primary objective of the review was to evaluate the relevance, effectiveness, efficiency and - to the extent possible - impact and sustainability of WFP-supported FFA activities. The review was undertaken for learning purposes, to inform design of the next PRRO and was targeted for use by WFP Country office team, Government and NGO partners.
25
Recommendations were made in the following areas: Geographic/asset targeting; project cycle and project design; technical support and capacity building; food distribution; project sustainability; and overall project management. Recommendations and lessons learned were shared with stakeholders and incorporated in the design and implementation of the subsequent PRRO. To access FFA Kenya Evaluation Report
6.1.1. OEDE-led evaluations The most rigorous evaluation is an external evaluation conducted by the Office of Evaluation at WFP. Unlike self evaluations, an OEDE-led evaluation meets the requirements of independence, meaning that it is free from influences that would bias the conduct, findings, conclusions or recommendations of the evaluation2. Types of OEDE-led evaluations include: Country Portfolio Evaluations include the entirety of WFP's work in one country. The evaluations provide answers to these questions: How strategically we have been placed and aligned within the country context? What drives our strategy in a country? What results have we achieved? CPEs normally cover a five-year period, but are tailored to the situation in the country. The lessons from CPEs help Country Offices prepare their country strategies and their new operations. Operations Evaluations focus on one operation at a time. They evaluate whether an operation is relevant, effective, and efficient, and seek to provide reasons for the successes and shortcomings of the project. These findings help Country Offices design new operations. Operations Evaluations can be managed by the Office of Evaluation (independent operations evaluation) or by the Regional Bureaux and Country Offices (decentralized operations evaluation). Strategic Evaluations: take a global perspective and focus on strategic issues for WFP as a whole. Policy Evaluations: examine how effective WFP policies have been in achieving their objectives. Impact Evaluations: provide an in-depth analysis of the impact that WFP work has on beneficiaries. Impact Evaluations focus on one programme activity at a time, in one country, to allow the evaluation team enough time and resources to collect data through a mix of methods.
WFP Evaluation Policy 2008
26
7. M&E Capacity Building and Systems Strengthening

M&E systems should be continuously reviewed to assess ways in which they can be strengthened both internally (through improved infrastructure and equipment as well as human resources capacity building) and externally (M&E capacity building of the government and partners). System strengthening can take place at any point in the project lifecycle but the design or redesign of a project provides an ideal opportunity for reviewing the current system in place. Capacity building and systems strengthening activities include: Capacity needs assessment Development of a capacity building plan Capacity building workshops and meeting Multi-stakeholder M&E system assessment
27
8. M&E Coordination and Budget

For an M&E system to be effective, it needs to be properly coordinated and adequately funded. M&E Coordination The management structure and roles within WFP and with partners - the institutional set up will vary from country to country. Some COs have specific M&E units while in other COs, generally smaller, the responsibility of M&E will fall to the programme staff and/or the VAM unit. Coordination would begin at the project planning stages and continue through every stage of the program development process, throughout program implementation and evaluation.
Budget A budgeted M&E work-plan is a time-bound, activity-based tool which describes and budgets for all M&E activities and clarifies agreed roles and responsibilities of everyone involved in the program. The time-period covered by the work-plan is the length of the project and provides an overview of: 1. Budget requirement for each of the different components of the M&E plan 2. The source of funding The planned M&E items are to be allocated to the appropriate source. DSC, ODOC and LTSH should be budgeted at design stage for the whole duration of the operation. The total M&E budget should be reported as a percentage of total programme/project budgets.
9. M&E Resources
WFP Indicator Compendium http://wiki.wfp.org/M_and_E_Kit/index.php/Indicator_Compendium
28
E.4
BEST PRACTICES
Field staff across WFP do not often document what works regarding FFA. This is not about overall inputs, outputs and outcomes measured through the project cycle and specific evaluations - it is about describing why specific techniques or approaches worked in a given area, the steps used to their planning, design and implementation, how management of assets was achieved and why, visual descriptions, cost effectiveness and impacts on people. At the 2001 workshop on Supporting Livelihoods in Situations of Chronic Political Instability (SCPI) hosted by the Overseas Development Institute (ODI) and attended by some 40 specialists from a dozen organizations, the attendees (including WFP staff) referred to the lack of Best Practices (in general livelihoods operations) with the following responses (The arrows point to how this PGM intends to deal with these challenges, which are prevalent throughout WFPs operations):
Good, innovative practice at field level may simply not be documented due to lack of time or resources and therefore headquarter is unaware and lessons do not get learnt. The Prevention and Recovery unit is suggesting incorporating the BPs into the COs advocacy, and M&E strategy so as to use standard practices like information gathering, field visits, site supervision, etc as part of the BP gathering process. Approaches adopted at the theoretical level may not have been put into practical how to do guidelines and therefore remain inaccessible to the field. The units BPs, Infotechs and How-to sections in this PGM should make approaches adaptable to the field. Approaches adopted at headquarter by policy and management people may have practical implications for implementation (particularly in SCPI) which mean they cannot be put into practice in the field. This PGM compiled all its components from WFPs experiences in the field, in order to maintain a bottom up approach to the policy implications of the implementation of FFA programs.
These three, often cited challenges at WFP, should begin to be resolved with the Best Practice approach to be rolled out with this guidance. There are a number of good practices and achievements that WFP has contributed to realize through FFW in countries such as in China and Vietnam during the 80s and 90s that have never been documented properly. This in spite of often spectacular government led
29
successes to which WFP contribution through FFW has been very significant. Similarly, some achievements such as the flood protection measures in Bangladesh and the road construction works in Nepal can be traced in Country Office reports and in some of the brochures produced. However, they are often referenced in these documents amidst other interventions and rarely presented as a clear detailed description of the type of assets created, from planning to implementation and management. Films and photographic materials are even rarer than documented reports. As a result, a repository of best practices for FFA is to be created and filled with examples of what works in different contexts. This can take the form of simple templates (see below example) as well as more robust case studies and evaluations specific to FFA. The following are specific tools suggested for CO and field staff to produce or promote through partners such as NGOs and Government:
1. Best Practice Templates

These templates serve as a tool to present examples of excellent interventions that other Country Offices can replicate. Mainstreaming this information will contribute to foster the replication of Best Practices throughout WFPs Food for Assets operations. This tool builds a lot from previous ODK and CO work on Infotechs and advocacy materials. Monitors and Programme Officers can take advantage of being equipped with the Best Practices format when supervising a site, if that site meets the highest standards of quality and particularly if participants on that site did something exceptional in the process or is showing exceptional results. Country Offices are encouraged to modify the BP tool to fit their strengths. For example: some countries will see the benefit of featuring seasonal programming, while others with very long dry seasons and very short rainy seasons may only need to mention it. 1.1. Infotechs or Best Practices While Best Practices are a tool to show exceptional FFA work that could be replicated, Infotechs provide technical guidance for the design of FFA techniques. They compliment and link to each other, providing a comprehensive how to approach for other Country Offices to consider.
30
What to remember when gathering Best Practices: 5W & 1H
HOW WHO WHERE WHEN WHAT & WHY
When gathering Best Practices, a monitor/ Programme Officer/ Technical Expert should focus on the five Ws and one H. The above circle can be considered as an approach to information gathering, where WFP staff start compiling information on what happened and why, and finish with the backbone of the Best Practice, the how. Remember:

Who was involved? What happened (what's the story)? Where did it take place? When did it take place? (specifics are important here: When did FFA phase out for example) Why did it happen? How did it happen? (This is where the case is built for the best practice)
The steps used to plan them Why specific techniques or approaches worked in a given area
The following information will not necessarily be included in the Best Practice but it is necessary to know the answers to them to be able to build your case:
Describe Design and implementation
The big picture: Why is this flagship/best practice intervention? o What evidence-based positive changes show that this Best Practice is worth replicating?
How management of assets was achieved and why
31
Background: Geographical context of area where best practice is taking place o Seasonal calendar o Shocks o What is the regional food security situation? The Story: The intervention timeline o When did the intervention start and why o Population (demographics-Female-male split, children) Before and After: o Soil situation o Food security o Beneficiary population for the intervention o How many meals was the household consuming per day when the project started? How many when the project ended? (after distributions stopped)
Cost effectiveness Show Impacts on people Visual descriptions
The How: o Cross-cutting issues: o How was Participatory planning implemented? o Gender focus? o Local partnerships? o Technical Capacity (was there low technical capacity or sufficient technical capacity?) Technical How: o Please provide the essential technical steps to completing this intervention (i.e. measuring the size of the trench) o If you have a particular way of implementing a step (i.e.: when monitoring activity make sure to have meetings on x day because other days people less likely to come.) o What kind of expertise is required (external/ internal)? o What, if any, are the necessary Non-Food Items? Replicability: o What are the key points to keep in mind when replicating this case (cannot apply one-size-fits-all approach to other countries so need to specify commonalities (type of soils, composition of households, etc) that would allow for replication. Cost-effectiveness: o Consider including cost-effectiveness information if available. If data is not available, a qualitative assessment can be conducted by asking community
32
members how much they lost during a recent shock and comparing that to the investment in the community. Images: o Before and after landscape shots: Consider organizing site photos in a folder with images that show the beginning of the intervention and the ongoing efforts. Organizing these images will not only allow monitors to complete BPs but also other staff will be able to sort through for advocacy, M&E and funding purposes. o First steps/Middle steps/Final Steps (Focus on images where labour is portrayed (action shots, instead of images of the final product) o Stand at the bottom of large structures to show height. o Make sure there is always someone in the frame. A person provides context and scale to a process. Some areas are so vast that it becomes difficult to understand how big a particular structure is, thus misrepresenting the amount of work that a structure of that size requires. o Always take a portrait image of a beneficiary if their answer may be included in the BP. You can decide if you will feature the beneficiary later.
Field Form When Country Offices embark on BP gathering, it is important to have a tool to compile the amount of information that will be presented at the site. This can be a survey form or it can be a list of questions attached to a field notebook. What is important is that the person gathering the information, has a list of questions to fill out so that when beneficiaries provide broad answers, specific questions can help narrow the information down to what is necessary. See Annex E-2 for an example. A sample Best Practice gathering experience 1.1.1. At the office: Country Office staff consults with the head of monitors on which FFA sites, if any, are performing exceptionally well with unique approaches. Field monitors are consulted and the selected sites are mapped out for follow up. Programme officers, in consultation with field monitors, prioritize sites according to topography, type of soils, climate, food security situation and other factors that will provide a varied set of BPs. The field monitors are then advised to prepare to gather BPs on the selected sites, while at the same time being encouraged to share information and experiences with other field monitors in the country. Field monitors contact partner organizations and government counterparts to inform them about the BP process. Partners are encouraged to develop BPs with WFP field staff. Monitors gather all the available information on the sites before carrying out the onsite process.
33
Tip: M&E systems will usually have a good amount of data on food security, assets built, capacity building, etc.
Contact the local representative and counterpart tech support staff, to tell them what WFP needs to take from this visit (follow the 5W & 1H approach). This allows the supporting team to plan a visit that logically answers the BP questions. Tip: To tell the story of the BP at a gully rehabilitation site, the team may start at the upper catchment of the watershed in order to better understand the overall process carried out by the planning team. The team will be able to observe the initial interventions carried out by the community and then proceed to the bottom catchment structures, leading down to the gully at the specific site. The key is to keep a chronological order to understand the overall strategy.
1.1.2. At the site Monitors explain to community members what Best Practices are and why they are important for the community.
Tip: During the initial (technical) stage of BP gathering, its not necessary to have a meeting with the community. A representative from the local planning team and a local counterpart supervisor should be able to provide all the technical information. Monitors and counterpart support staff examine the site, focusing on the before and after picture. Questions should always refer to how things are better or worse and why. Example: When developing a BP on Shallow Wells, the focus should be on the situation of the community/ household before and after the intervention.
Questions to consider: When was the well built? How many people were involved in building it? For how long did the participants receive food assistance for the project? Where they receiving food relief before? Have they increased the number of meals they eat per day? How deep do they have to dig to find water? What about before? How long did it take for water to dry up before? What about now? How many households are benefitting from the well and the wells around it? What about the other techniques supporting the well?
34
In order to keep your BP focused while also tied to strategic WFP goals, try to visualize the intervention as part of a larger operation. A small technique is always part of something bigger. Remember to get data for both the people benefitting from the specific technique and data for the overall site intervention. Frame your questions based on this. If the gully has been rehabilitated and people are reaping the benefits of this, is there a watershed intervention in place? If there is, is this part of a regional strategy? Is the regional strategy also a national strategy? You will be able to summarize this information in a digestible format for other countries to understand the impact of your FFA project. After understanding the overall strategy of the site, the team can focus on the technique. Remember 5W & 1H. These answers will be reinforced later on with information from the community members. See 15.3 to come up with questions that are relevant to the community and the work that WFP is doing. If data is not available from M&E indicators and or other sources, it is always possible to acquire the information directly from the community. Consider triangulating information in the community to acquire a solid understanding of the information. It is important to make sure that people understand that this is a qualitative study and should not be taken as definitive research. Once the information has been collected, the monitor will edit it to fit the BP format. The BP is then sent, as a preliminary draft for the CO to edit. The CO will then confirm the information and add any data as required. The BP is sent to HQs Prevention & Recovery unit for feedback and for HQ to understand what type of BP methodology works for the particular CO. An editing process, presented in Annex E-2, will begin. COs are likely to face less challenges during the development of the upcoming BPs, after having gone through this interaction with HQ. BP's are then presented in a 2-page format. See Annex E-2
Main advantages and risks of Best Practices Main advantages: (1) The possibility to document the main features and technical aspects of many specific FFA their performance and impact, the approach and steps undertaken to complete the work, and the partners involved.
35
(2) The possibility to provide quick hands on experience to other staff within and outside the country. The latter can be supported by programme staff dedicated to network those practices with regional bureaus. (3) To be used for advocacy purposes and documentation.
Risks: (1) Some of the approaches or techniques documented may not withstand the test of time (e.g. an asset described that is destroyed a few months later) therefore to be documented when it becomes clear that specific benefits and correct management can be observed. (2) The template may not be sufficient to capture in sufficient detail a given project and approach. In this case either more than one template can be developed e.g. series 1-2-3-4, etc - or a more detail case study prepared (see below).
2. Case studies and research papers

Case studies are more elaborated efforts that include detail description of activities and of their impact. There are few FFA projects that undertake case studies and quite often those case studies tend to describe the effects and impact of a given project on people and their livelihoods. While this is important, specific technical aspects that relate to the key interventions that positively impacted on people are described only in general terms. More attention is required to describe the following aspects: Participatory approach used, Targeting mechanisms, Role of women in decision making and benefits, Costs efficiency and effectiveness (measure relevance of the transfer modality such as cash, vouchers or food), Management and maintenance of assets created, Impact on reducing hardships, Seasonal livelihood analysis, Labour market analysis, Technical design of FFA interventions, Sequence of FFA activities and the visual description of these interventions through photos or drawings, Work norms, Self-help contributions (as a proportion of the total) and solidarity efforts, Role of Government institutions, Integration requirements (e.g. between FFW/FFA and other IGAs, etc).
36
Another important aspect to document is how these interventions fit within national policies and strategies. This includes to which extent they have been able to influence approaches or specific efforts undertaken through joint UN programmes, major food security coalitions and safety nets. Considering the role FFA can play in resilience building and the whole adaptation agenda, of considerable interest will be research and case studies undertaken in close collaboration with the policy unit on DRR and Climate Change and with research institutes and universities at country and international level. The following examples albeit not comprehensive provide some indication of the type of case studies that CO may want to undertake to describe specific FFA interventions and their impact. Note that following case studies would have been more powerful with a cost effectiveness analysis as well as more detail description of specific activities and techniques used for implementation: i) A WFP Country Office commissioned case study on the type of FFW
interventions around refugee camps in Northern Kenya Kakuma, and their impact on local production and food security.
ii) A research paper on Can food-for-work encourage agricultural production by Sosina Bezu and Stein Holden (2006) http://www.sciencedirect.com/science?_ob=MImg&_imagekey=B6VCB-4T1SFS2-11&_cdi=5950&_user=1922573&_pii=S030691920800050X&_origin=browse&_zone= rslt_list_item&_coverDate=12%2F31%2F2008&_sk=999669993&wchp=dGLbVzWzSkzS&md5=e4441583ded67b539369fb5bf0543453&ie=/sdarticle.pdf iii) An IFPRI-case study on the role of participatory planning in FFW projects The Empowering Communities through Food-based Programmes In Crisis-Prone Situations: Ethiopia Case Study (Report Prepared for the World Food Programme Under the IFPRI-WFP Linking Research and Action Initiative) by Marc J. Cohen and Mariagrazia Rocchigiani et al, September 2008 http://www.wfp.org/sites/default/files/WFP_Discussion_PaperEmpowering_communities-Ethiopia_0.pdf
37
3. Evaluations
Several CO undertake periodic evaluations of their operations that often include specific or extended evaluations of the FFA (FFW/CFW) component of CP or PRROs. There are few in depth evaluations of FFW specific projects from a programme and technical design perspective. However, a number of key lessons can be taken from these evaluations to inform the choice of FFA and the type of techniques to apply in relation to objectives such as resilience building and risk reduction. Additional aspects that evaluations can provide is in depth cost benefit analysis and the way effectiveness and efficiency are determined.
Report on the cost-benefit analysis and impact evaluation of soil and water conservation and forestry measures (MERET) WFP, 2005
1. 2. Country Programme and PRRO past Evaluations, and current Country Portfolio Evaluations offer a number of lessons learned for FFA which influence programme design: countries such as Nepal, Ethiopia, Bangladesh, Sierra Leone, Madagascar, Kenya, and Haiti have been through important evaluations that provided relevant info regarding performance of FFA and recommendations for improvements. These documents are often discussed and shared with a number of stakeholders (donors, governments, etc) to identify what works, adjust objectives and strategies, improve technical aspects and enhance the dissemination of best practices. Evaluations also help to identify areas to focus research work and specific studies. For example an impact evaluation undertaken in 2002 for the MERET project led to a specific case study on a much broader cost benefit analysis of the same.
4. Others (pamphlets, leaflets, flyers, posters, photo-series, films)

There are a number of materials that can be produced both as quick snapshots for advocacy purposes as well as to effectively summarize a strategy, an approach or an activity. To the extent possible they should be effective visuals with limited text. The following are few examples linked to both an overall FFA strategy and specific land rehabilitation interventions linked to resilience building.
38
4.1
Visual on the MERET Strategy (FFA Participatory Land Rehabilitation Project) and implementation phases
39
4.2
Transformational Role of MERET using an Environmental Credit and Savings approach
40
4.3 Posters showing FFA interventions and impacts (Before and after effect)
41
4.4
Short brochures on FFA activities and Impacts (Kenya)
WFP KENYA and Climate Change Adaptation Activities (2009 CO Brief) Major achievements: WFP Kenya and its partners have been assisting thousands of communities to develop and implement various climate change adaptation projects in ASAL since 2004. These projects helped target beneficiaries to recover from hunger related problems through supporting implementation of appropriate technologies such as rainwater harvesting and management structures, gully reclamation measures, improved crop and pasture production and tree planting activities. WFP has supported capacity building initiatives targeting staffs of its cooperating partners (NGOs and Government line ministries) and its field monitors working at different levels. The purpose of this capacity building is twofold: 1) improve technical knowhow of staff and community members in selection, design and implementation of rainwater harvesting and management (RHM) technologies and, 2) enhance a participatory and integrated RHM project planning approaches and skills. In this regard, in 2009 alone, 88 staffs from 9 districts (Isiolo, Baringo, Tana River, Tharaka, Taita Taveta, Turkana, Kilifi, Moyale and Mandera) were trained both theoretically and practically. This training has already impacted on participatory project planning and implementation of good quality climate change adaptation activities. Between 2004 and 2009, food for asset (FFA) activities supported the creation of water structures holding 3.3 million m3 of water, treated 13,506 ha of lands using soil and water conservation measures, and supported food production in agricultural projects on 8,134 ha in 17 major districts (details in Table below). Major assets created by WFP FFA: 20042009. District
Kitui Mwingi Taita / Taveta Kinango Kilifi Laikipia Narok Malindi Machakos Isiolo Mandera-EMOP Mandera-DPF Moyale Garissa Samburu Tana River Turkana (DPF+Refugees) Total
Water (m3)
20,418 367,522 74,866 728,238 680,596 74,866 74,866 279,044 142,925 149,731 81,672 0 95,283 13,612 68,060 40,836 428,775 3,321,308
Agriculture (ha)
Soil and water conservation (ha)

3,047 6,838 1,422 68 102 0 0 0 1,286 34 203 0 508 0 0 0 0 13,506
0 0 1,188 111 1,114 111 149 149 111 334 483 186 520 37 0 223 3,599 8,316
42
Plate 1: Water pan (left), rock catchment (middle) and sand dam (right). Reclamation and the productive use of marginalized lands was achieved through the construction of water harvesting structure such as trapezoidal bunds, semi-circular bunds, zai pits, etc. The reclaimed land has helped to produce drought tolerant crops such as sorghum, millet, green grams, watermelons, and animal fodder. The outcome of this activities delighted target beneficiaries and motivated them to replicate similar activities (see Plate2 for the structures and Plate3 for the crops).
Plate 2: Trapezoidal bund Garba Tulla district (left) and semi-circular bund Baringo district (right).
Plate 3: Trapezoidal bund supporting the production of drought tolerant crops (Tana River district).
43
Construction of irrigation schemes and increasing irrigation efficiency (Plate 4)
Plate 4: Irrigation scheme boosting crop yield (Taita Taveta district).

Construction of farm terraces to reduce soil erosion problems and increase crop productivity (Plate 5)
Plate 5: Fanya juu terrace in Kitui district (left) and Mwingi district (right)
Note:
The above summary of FFA activities is simple and clear it could perhaps benefit from additional information on sustainability and effects on reduction of food gaps and other environmental impacts however, the combination of text-table-photos is effective and offers a quick yet useful account of what has been achieved in Kenya using FFA. The overall title may be however questioned by some stakeholders it would be then helpful to present the note as a contribution to adaptation and not as a primary objective as the title and first paragraphs in text appears to suggest. The adaptation benefits came as a result of what has been achieved through FFA whilst their primary objective was to offset consumption needs of able bodied hungry people following periods of droughts.
44
E.5
GLOSSARY
All definitions, unless otherwise specified, are from WFP sources Evaluation The systematic and objective assessment of an on-going or completed project, programme or policy, its design, implementation and results. The aim is to determine the relevance and fulfilment of objectives, efficiency, effectiveness, impact and sustainability (OECD-DAC, 2010) Indicator A variable that measures one aspect of a program or project (an input, process, output, outcome) that is directly related to the programs objectives (OECD-DAC, 2010) Input The financial, human, and material resources used for the development intervention Monitoring - A continuing function that uses systematic collection of data on specified indicators to provide management and the main stakeholders of an ongoing intervention with indications of the extent of progress and achievements of objectives and progress in the use of allocated funds (OECD-DAC, 2010) M&E Logical Framework Management tool used to improve the design of interventions, most often at the project level. It involves identifying strategic elements (inputs, outputs, outcomes, impact) and their causal relationships, indicators, and the assumptions or risks that may influence success and failure. It thus facilitates planning, execution and evaluation of a development intervention M&E Plan A document developed at the beginning of a project that describes the M&E strategy over a project cycle (e.g. 3-5 years) and the different components of the M&E system. M&E Strategy - A strategic approach to ensure that adequate resources and capacities are allocated to the M&E system to support the monitoring and evaluation of an operations implementation and achievement of results M&E System Assessment a review of the strengths and weaknesses of the M&E system with the objective to plan, mobilize resources for and implement strengthening measures. Objective The intended physical, financial, institutional, social, environmental, or other results to which a project or programme is expected to contribute Outcome The intended or achieved short term and medium term effects of an interventions outputs. Outcomes represent the changes in conditions that occur between the completion of outputs and achievement of impact Output The results of the program activities regarding quality and quantity of goods and services delivered and actual number of participants/beneficiaries that is expressed as a count, percentage or ratio. Participatory M&E - approach which involves local people, development agencies, and policy makers deciding together how progress should be measured, and results should be acted upon (Guijt et al, 1998)
45

File 5 - Module e - Monitoring & Evaluation For Ffa 20 July 2011

Transféré par

Informations du document

Description originale:

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

File 5 - Module e - Monitoring & Evaluation For Ffa 20 July 2011

Transféré par

Droits d'auteur :

Formats disponibles

Food Assistance for Assets (FFA) Manual

ODXP PREVENTION & RECOVERY WORLD FOOD PROGRAMME

FFA Manual Module E: Monitoring and Evaluation for FFA

FFA Manual Module E: Monitoring and Evaluation for FFA

FFA Manual Module E: Monitoring and Evaluation for FFA

FFA Manual Module E: Monitoring and Evaluation for FFA

AN INTRODUCTION TO THE FFA MANUAL:

FFA Manual Module E: Monitoring and Evaluation for FFA

MODULE E: Monitoring and Evaluation of FFA

FFA Manual Module E: Monitoring and Evaluation for FFA

WHAT IS MONITORING AND EVALUATION

FFA Manual Module E: Monitoring and Evaluation for FFA

MONITORING FFA AND KNOWLEDGE MANAGEMENT

1. Designing an M&E Strategy for Food for Asset Programmes

FFA Manual Module E: Monitoring and Evaluation for FFA

FFA Manual Module E: Monitoring and Evaluation for FFA

2. Building Your Logical Framework

FFA Manual Module E: Monitoring and Evaluation for FFA

FFA Manual Module E: Monitoring and Evaluation for FFA

FFA Manual Module E: Monitoring and Evaluation for FFA

FFA Manual Module E: Monitoring and Evaluation for FFA

Outcome 2.3 Hazard risk reduced at community level in targeted communities

FFA Manual Module E: Monitoring and Evaluation for FFA

FFA Manual Module E: Monitoring and Evaluation for FFA

3. Building the M&E Plan

FFA Manual Module E: Monitoring and Evaluation for FFA

FFA Manual Module E: Monitoring and Evaluation for FFA

FFA Manual Module E: Monitoring and Evaluation for FFA

FFA Manual Module E: Monitoring and Evaluation for FFA

4. Quality Assurance Mechanisms

FFA Manual Module E: Monitoring and Evaluation for FFA

FFA Manual Module E: Monitoring and Evaluation for FFA

5. M&E Plan Matrix

FFA Manual Module E: Monitoring and Evaluation for FFA

Example M&E Plan Matrix

Context Input Process Output Outcome

FFA Manual Module E: Monitoring and Evaluation for FFA

6. Evaluation and Reviews

FFA Manual Module E: Monitoring and Evaluation for FFA

FFA Manual Module E: Monitoring and Evaluation for FFA

WFP Evaluation Policy 2008

FFA Manual Module E: Monitoring and Evaluation for FFA

7. M&E Capacity Building and Systems Strengthening

FFA Manual Module E: Monitoring and Evaluation for FFA

8. M&E Coordination and Budget

FFA Manual Module E: Monitoring and Evaluation for FFA

FFA Manual Module E: Monitoring and Evaluation for FFA

1. Best Practice Templates

FFA Manual Module E: Monitoring and Evaluation for FFA

What to remember when gathering Best Practices: 5W & 1H

HOW WHO WHERE WHEN WHAT & WHY

Describe Design and implementation

How management of assets was achieved and why

FFA Manual Module E: Monitoring and Evaluation for FFA

Cost effectiveness Show Impacts on people Visual descriptions

FFA Manual Module E: Monitoring and Evaluation for FFA

FFA Manual Module E: Monitoring and Evaluation for FFA

FFA Manual Module E: Monitoring and Evaluation for FFA

FFA Manual Module E: Monitoring and Evaluation for FFA

2. Case studies and research papers

FFA Manual Module E: Monitoring and Evaluation for FFA

FFA Manual Module E: Monitoring and Evaluation for FFA