Tools for Sustainable Livelihoods: Livelihoods Monitoring and Evaluation Kath Pasteur IDS st February 2001 21 DRAFT FOR

COMMENTS Table of contents TABLE OF CONTENTS......................................................................................1 1. INTRODUCTION.............................................................................................1 2. WHAT ARE THE CHARACTERISTICS OF LIVELIHOODS MONITORING AND EVALUATION?..........................................................................................2 2.1 TAKING A BROAD SL FOCUS..................................................................................3 2.2 THE SL PRINCIPLES.............................................................................................4 2.2.1 People centred and participatory.............................................................4 2.2.2 Holistic and cross-sectoral,.....................................................................5 2.2.3 Linking micro to macro............................................................................5 2.2.4 Dynamic and Sustainable.........................................................................6 2.2.5 Support a process, learning approach...................................................6 2.3 THE SL FRAMEWORK...........................................................................................6 3. HOW DO I IMPLEMENT A LIVELIHOODS APPROACH TO M&E?....9 3.1 HOW DOES LIVELIHOODS M&E FIT IN THE PROJECT CYCLE?.......................................9 3.2 HOW DOES LIVELIHOODS M&E RELATE TO THE LOGICAL FRAMEWORK?....................11 3.3 WHAT INSTITUTIONAL ARRANGEMENTS SUPPORT LIVELIHOODS M&E?.......................13 3.4 WHAT METHODS ARE APPROPRIATE FOR LIVELIHOODS M&E?..................................14 3.6 HOW SHOULD I DESIGN LIVELIHOODS INDICATORS?.................................................17 3.7 HOW CAN LIVELIHOODS M&E DATA BE ANALYSED?..............................................20 3.8 WHAT ARE THE COSTS AND BENEFITS OF LIVELIHOODS M&E?...............................21 3.9 SUMMARY.........................................................................................................22 4. LINKS TO FURTHER INFORMATION ONLINE....................................23 1. Introduction Are DFID projects, programmes or country strategies having a sustainable and long term impact on improving the livelihoods of poor people? Whether or not projects have been designed using a livelihoods approach, the Sustainable Livelihoods framework and principles offer a useful tool and guide for measuring and interpreting the outcomes and impacts of development activities. Livelihoods M&E can also help to improve our understanding of the Sustainable Livelihoods approach as an effective development process
1

for achieving poverty reduction. A Livelihoods approach takes a holistic view of tackling poverty, and puts poor people and their priorities at the centre of development. The Sustainable Livelihoods Framework and principles are tools to guide our understanding and practice when planning and implementing development activities. Livelihoods Monitoring and Evaluation (Livelihoods M&E) draws on the best practice of conventional M&E and Participatory M&E, but the focus, principles and framework of the Sustainable Livelihoods (SL) approach add further value and insights. Livelihoods M&E is still in a process of evolution and experimentation. This tool draws on some early experience and conceptual thinking and outlines the added value that a Livelihoods approach brings to M&E. It is written primarily for DFID staff and partners, but it is hoped will also be of relevance to a much wider audience. Three key features of the Sustainable Livelihoods approach help to provide further insight, and improve on the focus, priorities and methods of conventional styles of M&E: 1) Its broad focus highlights not just project impacts, but wider livelihoods goals; 2) The livelihoods principles form the basis for innovative methodologies; 3) The livelihoods framework helps with the design of better oriented and more relevant indicators. Hence, livelihoods M&E outputs will differ from those of conventional M&E in a number of ways. They provide a broader, more dynamic view of project impacts throughout the project lifetime, rather than a one-off, ex poste assessment. They try to link the context with the outcomes and impacts helping to explain why and how. Instead of policing, Livelihoods M&E builds a culture of learning and reflecting, and aims to provide relevant information for planning and policy-making. Section 2 of this tool outlines the conceptual features that characterise a Livelihoods approach to M&E, whilst Section 3 details the practical implications of this new thinking in terms of planning and implementation of Livelihoods M&E. 2. What are the Characteristics of Livelihoods Monitoring and Evaluation?

This section looks at the ways in which the focus, principles and framework of the Sustainable Livelihoods approach are relevant to Monitoring and Evaluation, and can be translated into an improved approach to M&E. 2.1 Taking a broad SL focus A Sustainable Livelihoods approach to development demands a more holistic understanding of poverty, and of the linkages between different livelihoods components. The goals of livelihoods projects tend to have a broader scope, beyond a narrow focus on the direct outcomes of project or programme activities within a particular sector. Hence, Livelihoods M&E should aim to assess the bigger picture of livelihoods beyond the project environment, and the links between different aspects of the livelihoods context. Three forms of M&E can be identified: process, outcome and impact (see table below). Livelihoods M&E addresses all three, but places greater emphasis on the third. It furthermore tries to test the assumption that project processes and outcomes will result in the achievement of intended impacts or livelihoods goals. Table 1. Process, outcome and impact M&E. Process: monitoring the Did the project process achieve the SL principles? Was the SL

implementation of activities and how intended outcomes? Did it incorporate effectively this is done, i.e. the project the strategy, work styles, management framework an effective tool for planning arrangements, etc. appropriate activities? Outcome: measuring the effect of the Do the outcomes correspond to the needs identified in the SL analysis? Are they activities that have been undertaken, cross-cutting (including technical, social, and their progress towards achieving human, financial)? the project purpose. Impact: ascertaining the longer term Has the project resulted in peoples access to a greater and more balanced and more widespread, intended and range of livelihood assets? Has it unintended, consequences of an reduced vulnerability and improved livelihood outcomes? Has there been a intervention, and monitoring progress beneficial policy impact? towards broader livelihoods goals. Livelihoods M&E can be used to monitor and evaluate the impact of an individual
3

project, a sector wide programme, or a country strategy. Ideally nested systems of M&E at these different levels can provide opportunities for making additional linkages and building a broad and comprehensive picture of how livelihoods are being impacted for poverty reduction. 2.2 The SL Principles The principles of the SL approach demand a shift in focus from outputs to people, and an exploration of poor peoples own priorities. The SL approach poses the challenge of how to incorporate these principles into an M&E methodology. The implications of the principles outlined in the table below are further elaborated in this section. Table 2. The implications of the SL principles for Livelihoods M&E Principle People-centred participatory Holistic and Implication and Poor people are the central focus
Involvement of a wider range of beneficiaries cross- Study changes across a range of livelihoods aspects

sectoral Linking micro to macro Dynamic and sustainable

Study impacts at different levels both local and

policy - and the links between them Ensure that the dynamic and fluctuating nature of livelihoods is reflected

Support

Long term viability of interventions and sustainability process, Not just for project learning but for policy input wider

learning approach

understanding of resource allocation processes (reword)

Capacity building such that learning continues beyond the donor involvement

2.2.1 People centred and participatory The Sustainable Livelihoods approach focuses on development activities that are based on the priorities of poor people. For such an approach to be effective poor people themselves must play a key role in identifying and addressing those priorities. Thus, Livelihoods M&E should be:

People-centred i.e. focus on evaluating whether the livelihoods of the poor are being addressed Participatory i.e. ensure that poor people are key stakeholders in the M&E process.

Rather than concentrating on resources, or income-based project outputs, a peoplecentred approach centres on peoples lives and their own understandings of poverty and wellbeing. It recognises also that poor people are not homogeneous, and will not all benefit equally from project interventions. More participatory styles of M&E embrace a range of stakeholder perceptions, they encourage a joint analysis of lessons, and they improve learning within partner organisations, and amongst the project beneficiaries themselves. Participatory approaches should exhibit the following characteristics:

The M&E process is designed and managed with partners and beneficiaries; Indicators are identified and negotiated with partners and beneficiaries; Partners and beneficiaries are involved in the collection and analysis of information; Peoples attitudes to change are as important as physical measures of change; Partners and beneficiaries play a key role in judging performance.1

2.2.2 Holistic and cross-sectoral, Taking an holistic and cross-sectoral approach requires looking beyond the direct outcomes of a project in a particular sector to embrace a wider range of livelihoods impacts: on different assets, on vulnerability and on livelihoods strategies. This may require collection of a range of economic, physical, social and financial data. The unintended, unexpected and intangible impacts are also relevant to Livelihoods M&E. This doesn't mean however that all data on all things is required, but that preliminary data on a wider range of complimentary factors should be sought before prioritising a smaller number of significant factors for more in depth analysis. 2.2.3 Linking micro to macro Many projects and programmes now recognise that changes at the local level are not sufficient to ensure sustainable improvements in livelihoods, and thus they are increasingly engaging in policy and institutional issues. Livelihoods M&E should aim to understand the actions and impacts not only at the local level, but also at the national or policy level, and the linkages between them. This involves looking beyond the household, to impacts in the
1 Blake, B et al (2000) West Africa Sustainable Fisheries Livelihoods: Guidelines for Programme and Project Monitoring and Evaluation. NRI for DFID.

organisational environment (e.g. organisational capability, and culture), and at the broader societal level (e.g. impacts on societal values and attitudes; on the policy and institutional environment). 2.2.4 Dynamic and Sustainable Livelihoods are dynamic and influenced by seasonal and other trends. It is important that the dynamism of livelihoods is captured in Livelihoods M&E by looking at vulnerability, trends and changes over time in relation to the context, rather than just livelihood status. This also has implications for understanding the sustainability aspects of interventions, not just ecologically, but institutionally, socially and economically. Possible knock-on impacts (negative and positive) on non-beneficiaries, or in other geographical locations elsewhere should be explored; or, for example, the long-term sustainability of positive trends within given resource constraints. 2.2.5 Support a process, learning approach Traditionally M&E has primarily been about providing accountability to funders and assessing the achievement of project outcomes. However, Livelihoods M&E can make a further contribution to the development process by building capacity for ongoing learning beyond the life of the project, and for producing information that provides a useful input into current and future planning, policy making and other resource allocation decisions. Hence, emphasis must shift from producing knowledge products (reports of process and output indicators), to a goal of strengthening the learning process. Establishing a learning process requires building the capacity of individuals, and institutionalising a culture within organisations for reflecting, learning, communicating and applying knowledge, (i.e. adapting in response to lessons learned). In this way M&E can be an empowering process, facilitating dialogue and mutual learning, and building trust and transparency between project partners.

2.3 The SL Framework The Sustainable Livelihoods Framework presents the main factors that affect peoples livelihoods, and typical relationships between these. It provides a way of thinking through different influences (constraints and opportunities) on livelihoods, and ensuring

that important factors are not neglected. In particular, the framework:

Provides a checklist of the important issues and sketches out the way these like to one another Draws attention to core influences and processes; and Emphasises the multiple interaction between the various factors which affect livelihoods2 Hence, the framework can be also used as a tool for designing Livelihoods Monitoring

and Evaluation: as a conceptual framework for identifying influences and interactions; as a checklist for designing indicators; and as an aid to analysing, understanding and structuring M&E data. As a conceptual framework the SL framework draws attention to the need to measure changes in the different factors that contribute to livelihoods:

Changes in capital assets Changes in institutional structures and processes Changes in the resilience or vulnerability of livelihoods Changes in livelihood strategies Changes in livelihood outcomes It also draws attention to the relationships and feedback loops between these different

elements, and between macro- and micro-level changes such as:

Relationships between policies, institutions and processes (PIPs) and the vulnerability context. Relationships between access to assets and improved livelihood outcomes. Relationships between national policy or institutional decisions and household level activities.

Hence, indicators can be designed either for measuring positive or negative changes in access to assets, institutional structures and relationships, or livelihoods strategies. Or they may be designed to try and determine linkages between different aspects of change. Whichever of these aims the Livelihoods indicators aim to fulfil, they should share a number of characteristics, that help them to provide the quality and type of information
2 Sustainable Livelihoods Guidance Sheets (www.livelihoods.org) 7

needed for Livelihoods M&E: Box 1. Characteristics of livelihoods indicators3 more outcome focused - address the wider aspects of peoples livelihoods and recognise that outcomes are diverse and go beyond simple quantitative increases in the variable being measured. more process based - as well as seeking to understand the outcomes people seek and achieve, they should look at the quality of the processes by which these outcomes are, or are not reached. more negotiable and open-ended to take account of diverse unpredicted outcomes; different peoples interpretations of success and understandings of impact. looking at intermediate signals - this dynamism in peoples lives necessitates the use of proxy signals so that interim results and changes can be tracked; looking at negative trends as well as positive trends - understanding what went wrong as well as right is important; more context dependent - comparing the effectiveness across projects or programmes in contributing to sustainable improvements in livelihoods is difficult when measures of success differ.

3 Source: Drake, L. (2000) Scoping mission to investigate the development of Livelihoods Indicators and Livelihoods
Monitoring systems for DFID-Bangladesh.

3. How do I implement a Livelihoods approach to M&E? This section looks at some of the practical implications of Livelihoods M&E with respect to its place in the project cycle and the logical framework, methods, and issues of organisational structure. 3.1 How does livelihoods M&E fit in the project cycle? As already mentioned, Livelihoods M&E can take place at the project, the programme and the country strategy level. Figure 1. illustrates how Livelihoods M&E activities fit in to the project cycle. The basic principles of this process hold also for programme and strategy planning, i.e. that monitoring should be seen as an iterative, learning process, with constant feedback loops correcting implementation and operation, as well as lessons feeding back to the design stage of new projects or programmes under identification. Information and lessons generated at both the monitoring, as well as the evaluation stages should be made available to the range of stakeholders involved in the project through appropriate events (e.g. workshops) or media (e.g. posters, video). Sharing of lessons can also take place between sectors, programmes, and even countries. What this figure does not clearly illustrate is that Livelihoods M&E is not intended as something that only feeds into the project cycle. Its goals should stretch beyond the project time frame, aiming to build capacity for continuous learning and the creating of relevant information for policy and planning decision. It should also stretch beyond project boundaries, and seek to understand the wider livelihoods context and trends.

Figure 1. Livelihoods M&E in the Project Cycle

9

0000160482180c11e0ffe0ff0500000004010d0000002d000000f7000003140000000000800000000080000080800000000080008000800000808000c0 0a7b09270a74091d0a6d09120a6709050a6109f7095c09e8095809d8095409c7095009a2094c097b094a09e4088e0856084a09040000002d0101000400 250025000f0013002500210025000f002500250013000f002500250013002500130013002500160025002100250021002100130025002500250013001 0025002500250021002100040000002701ffff030000001e00050000000102ffffff00050000000902000000000700000016046d0bbe1008000800040000 e500b40654302cd0120074000000000000000200c7300000c5300042000000000000000000004802e700d0cf12007f7a6d300e00c730d10000006ed012 06200d0000600b00658000801000000000000ec00a2000f0004f0a2000000e2030af00800000022040000000a000003010bf0600000008000d81ce5008 0000030301000000bf0300000800000010f012000000000003005a020400e90005005a020500c600000011f000000000 0200090000001d062100f00001004503f2000000040000002d010100050000001402080100000500000013020801450

The collection of baseline data is important for accurate analysis of impacts and outcomes (i.e. for before and after comparisons). A Sustainable Livelihoods Analysis should ideally be carried out at the design stage and this would then provide a useful baseline study since it focuses on peoples own reality, and identifies strengths as well as weaknesses. Otherwise a baseline study should be conducted shortly after obtaining project approval. With-without comparisons can also be conducted as well as before-after, to generate more robust findings. The detailed planning and organisation of an M&E system should be dealt with early on in the project implementation phase. Whether this involves setting up a new M&E unit, or working within existing structures, it is likely that training, support and supervision will be required both at the start, and at phased intervals throughout the project. 1. M&E and the Logical Framework The design of the M&E methodology and the writing of the Logical Framework should ideally be done in tandem since the identification of indicators and their means of verification are common to both these activities. The project goal will be verifiable through

impact indicators, the project purpose through the outcome indicators, and the project outputs through process indicators. 3.2 How does livelihoods M&E relate to the Logical Framework? Logframes are often associated with blueprint projects. However, their widespread use means that monitoring activities increasingly have to be structured in this format. In trying to incorporate Livelihoods M&E in a project logframe, it is necessary to take a process approach, i.e. indicators and means of verification should be flexible and updated frequently to incorporate any process changes. The design of the M&E methodology and the writing of the Logical Framework should ideally be done in tandem since the identification of indicators and their means of verification are common to both these activities. A Livelihoods approach may be considered to differ from a conventional approach in that the goal will tend to be a broad livelihoods goal, and Livelihoods M&E is significant in that it aims to ascertain progress towards such broader goals through impact indicators. Livelihoods M&E also measures the achievement of the project purpose through the outcome indicators, and the project outputs through process indicators. The logframe often only outlines a limited number of broad indicators and the means of verification often only refers to sources of verification rather than the methods and tools used. Identification of more detailed indicators, and a range of methods for their verification should nonetheless be decided upon during the design phase or once an M&E unit or team has been established early in the implementation phase (see sections below). The following table summarises parts of the logframe for a Livelihoods project in Andhra Pradesh, India. It illustrates examples of indicators and means of verification including a Project Monitoring and Impact Assessment System. The indicators address impact on a range of assets (e.g. increased income, increased spending on health and education, capacity building), on vulnerability (to drought), on the policy making environment, and on peoples capacity to access to institutions and processes. The feedback and dissemination of M&E findings is made explicit, and will be done by means of a communications strategy and other lesson learning events.

11

Table 3. M&E in the logframe: from the Andhra Pradesh Rural Livelihoods Project4 Narrative Summary Objectively Verifiable Indicators Goal Population below the official Effective and poverty line falls by 30% by sustainable EOP in project districts approaches to eliminate poverty New government guidelines adopted in drought- and schemes which improve prone areas of the delivery of services Andhra Pradesh through participation, equity and convergence adopted Purpose X% of below poverty line Government of households can specify at Andhra Pradesh able least y% increase in incomes to comprehensively as a result of project implement pro-poor interventions by EOP watershed-based sustainable rural 70% of marginal farmers livelihoods report significantly improved approaches in five drought proofing. districts of Andhra Pradesh GoAP develops and implements a Capacity Building strategy on watershed based SRL agenda for the whole state by EOP. Means of Verification Government of Andhra Pradesh (GoAP) below Poverty Line Surveys Policy documents and scheme guidelines; stakeholder workshops and other lesson learning events Stakeholder workshops at PY2 and annual reviews in PY 4, 5, and 7. Project monitoring and impact assessment system developed in PY1. Participatory approaches to PMIAS developed using PRA case studies, small scale sample surveys using special focus groups identified by stakeholder analysis against which to report distributional issues.

Communities, particularly Schedule Caste/Tribe groups, have greater capacity to take up, participate in and influence Government services. Component 1: Watershed-plus based sustainable rural livelihood initiatives 2. Higher return Livelihood options of the Project monitoring and income and poorest common interest impact assessment system employment options groups have been adequately (PMIAS) developed in (both land-based and represented in microplans. PY1. As above. non-land-based) identified and Expenditure on food, health Regular project monitoring pursued through care, education and shelter by data collection on quarterly

4 Source: Adapted from APRLP Logical Framework 1999-2006. APRLP is a Sustainable Livelihoods project that uses watershed management as its entry point. This table has selected a limited number of elements from the complete logframe in order to illustrate impact and output indicators, and a range of participatory and conventional sources of verification.

increased access to Government and other initiatives/schemes by the poorest in the watershed programme in 500 watersheds.

landless and marginal and six monthly basis farmers increased and sustained by EOP. Reviews of micro-plans by MDTs, DPAP and DCBC, Increased and successful PIAs and village based access to a range of GoAP workshops with village poverty alleviation schemes professionals, SHGs, user by Below Poverty Line groups and other WDA households, particularly members. SC/ST and female headed households, (at least X%) by PY2. Component 4: Lesson Learning and policy influence 6. Approaches Participatory M&E, impact DPAP/PSU annual reports, developed in the assessment and planning policy specific studies and project, particularly approaches developed and documentation, stakeholder on non-land-based informing development of workshops, feedback initiatives and other project approaches by PY2. through communications aspects of watershed strategy development and plus, replicated Key lessons and approaches dissemination events. widely. from the project adopted by GoAP in the entire State. Project experience documented and disseminated to all stakeholders using appropriate media GoAP makes policy changes through appropriate instruments in relation to issues which impact on the poor

7. The sectoral policy environment in AP strengthened to ensure greater (& sustainable) impact for women and the poorest and more effective working partnerships between Government and NGOs

GoAP orders/ circulars and documents charting progress of Watershed Development Programme

Stakeholder workshops, Mechanisms for identifying participatory monitoring such issues, and taking policy and evaluation system decisions in relation to them feedback established.

3.3 What institutional arrangements support Livelihoods M&E? Styles of monitoring that involve occasional visits to project sites by consultants or project managers, verbal feedback from field supervisors, or reviewing documentation are limited in the extent to which they get a true picture of what is happening on the ground. If systematic and continuous and participatory monitoring and evaluation is to be carried out during project implementation, and to be sustained throughout operation, then dedicated staff and resources need to be made available for this task.
13

One option is to designate an M&E unit or team, i.e. a distinct administrative entity. Its location may be within the project management unit, with a partner organisation involved in the project (e.g. NGO, research institute), or within the department or ministry if they are the implementing agency. There may be M&E units at each of the following levels: Project: limited to M&E of one or more projects Steering Group Programme: M&E of all projects under in one sectoral or area based programme. Central: M&E of a country programmes progress towards achieving its broader goals. Livelihoods Unit NR/FAR advisors The proposed Project M&E
Co-management team

institutional arrangement for the Bangladesh Livelihoods monitoring system CARE-DFID E - place country-regionBangladesh provides an illustration of one possible model. of DFID and CARE-B DFID place country-regionBangladesh
Field Management Staff

ANR Sector

Box 2. Organisational structure of Bangladesh Livelihoods Monitoring System5 The long-term institutional management and implementation arrangements for the LMS are still under discussion. The following factors are being considered in deciding institutional arrangements: - Objectivity - Sustainability - Cost - Ownership Compatibility with existing institutional structures - Ownership - Consistent/supportive of the wider agenda The establishment of a livelihoods monitoring unit within CARE is the most likely institutional option. This unit would be managed by a co-management team supported by a steering group, and would sit above the two projects. The unit would be staffed by 2 to 3 people with a strong background in rural development, participatory methodologies and monitoring and evaluation skills

HABGE, Go-interfish and other projects

The responsibility of an M&E unit will be: design and organisation of the M&E system; training of staff and others involved; organisation of data collection; co-ordination of data analysis; and presentation and feedback of information. It may consist of anything from just one or two part-time staff, to a full team of professionals, however it is important that a range of relevant stakeholders are involved at all stages, and that long term personnel are involved and trained to ensure sustainability. 3.4 What methods are appropriate for Livelihoods M&E? Livelihoods M&E should be people-centred and participatory, thus it should involve a wide
5 Turton, C. with K. Westley and J. Goulden (2000) Bangladesh: Design of a Livelihood Monitoring System. In Development Ltd. for DFID

range of stakeholders, and draw principally on participatory methods. A complementary balance of both qualitative and quantitative research approaches should be employed. Participatory methods are more effective in identifying intangible outcomes and unforeseen impacts, and can help to give voice to those who are often invisible or ignored, and providing opportunities for discussion and analysis amongst beneficiaries. They have the added benefit of strengthening the capacity of individuals and organisations to have more control in the development process. Some participatory methods used for various aspects of Livelihoods monitoring of wildlife projects in East Africa are listed in Table 4 6. The design of innovative and culturally appropriate methods is encouraged, for instance involving video, photography, dance or drama. Conventional methods have the advantage of generally requiring less time and commitment of both M&E staff and beneficiaries. They can be more effective in gaining a picture of the wider context, and are considered to be more objective. Some conventional methods may involve the use of technical equipment, such as remote sensing, geographic information systems, etc., which can be highly effective for monitoring impacts (e.g. in vegetation cover, flooding) over time, but can have significant cost implications.

6 Ashley, C and K. Hussein (2000) Developing Methodologies for Livelihood Impact Assessment: Experience of the African Wildlife Foundation in East Africa. ODI Working Paper 129.

15

Table 4 Topics and PRA-type tools for livelihood impact assessment7. Topics Activities Current List pros and cons livelihood activites Rank according to: - Contribution to income - Preference - Importance to HH. Discuss. Generate criteria for scoring activities and construct matrix What can be learned Livelihood strategies. Criteria for judging Key activities and assets. Ball park figures for income from different activities. Values other than cash income. Criteria can then be discussed/expanded/ranked. As above but more complex. Focuses on locally-generated criteria (which can then be ranked). Scoring against criteria is easier to visualise for consensus-building and comparing across SH groups. How wildlife enterprise fits into strategies, how it meets livelihood criteria. What needs are, which activities are pursued and why. Which activities have multiple functions. Impacts of project on other livelihood activities

Incorporate the wildlife enterprise in the above. Construct matrix of activities and needs

Seasonali ty Wealth ranking

Scenariobuilding

Current

Construct matrix of positive and negative impacts of WE on other activities Carry out any of the above in Differences between SHs in terms of stakeholder Groups. activities, strategies, and impacts. Construct matrix or discussion Livelihood strategies. Main needs. of seasonality of income, work, Human capital availability. food availability. Carry out wealth ranking of Stakeholder identification participants and explanation of Local criteria for livelihood security criteria Compare with previous wealth How people move in and out of ranking poverty and why Paint picture (verbally or Long-term trends. Long-term literally) of impacts of project positive and negative future in Useful if going on to joint planning. general or resulting from this enterprise Discuss: what are the assets and Should identify livelihood assets, and

7 Source: Ashley, C and K. Hussein (2000) op cit.

assets and resources you currently rely on resources to support the family (building blocks)? How? Constrain Discuss: what are the ts constraints that prevent livelihood improvement? Pros and Lists pros and cons. Rank pros cons of and cons. Identify who bears WE costs and receives benefits. Participat ion in the project Expendit ure of earnings Time-line and trends

relative importance. Encourages influences focus on external

Direct and indirect impacts of project. Priority concerns, significance of impacts. Distribution of impacts between stakeholders. Discuss who does and does not Stakeholder roles. Impacts as participate, why? Discuss how perceived by each. Barriers to participants are selected? participation (external or internal) Rank/matrix of items expenditure. Who decides? of Impact of earnings (e.g. on needs, HH assets) Who benefits Adaptive livelihood strategies and coping strategies. Influence of external policies and organisations. Dynamic processes. Role of internal organisation. Changes in livelihoods over time. Role of external influences. Significance or not of the project as a major influence.

Changes and causes

Construct time line. Discussion of key events and gradual trends. How people coped or adapted? How are they preparing for the next change? Household action, community action. Construct matrix of recent major changes and their causes, then rank the most influential causes of each.

Some relevant questions to be asked in deciding methods might include: Which collection method can provide the needed data in the allotted time? Will the adoption of more than one methods provide the opportunity for cross-checking findings or only result in conflicting and incompatible data? What are the skills of those who will conduct the study in the field? 8 What budget and logistical support can be supplied to the field operations? 3.6 How should I design Livelihoods Indicators? The SL Framework can provide a useful guide and checklist for the identification of indicators and the key linkages between them. The Framework is useful for this purpose in that it highlights the various dimensions of livelihoods that may be affected by project interventions. It helps to overcome some common problems relating to indicator choice, such as: overlooking impacts in unrelated sectors; focussing on material outputs, rather than
8 Blake, B et al (2000) West Africa Sustainable Fisheries Livelihoods: Guidelines for Programme and Project Monitoring and Evaluation. NRI for DFID.

17

impacts on people and policy; measuring income based impacts rather than wider livelihood outcomes; or not considering the possibility that people might change their livelihood strategy altogether. The Bangladesh Livelihoods Monitoring System used the livelihoods framework as a guide for ensuring that the different dimensions of livelihoods were captured. The table below summarises the range of indicators that emerged from field visits and discussions with project staff. The aim is to distil from this full baseline a sub-set of key indicators which will inform them of the general livelihood status of project participants9.

9 Turton, C. with K. Westley and J. Goulden (2000) Bangladesh: Design of a Livelihood Monitoring System. In Development Ltd. for DFID

Table 5. Livelihood indicators for the Bangladesh Livelihoods Monitoring System10. Indicator Vulnerabilit y Seasonality Shocks/stress es Resource trends Assets Land/trees Water Livestock Physical assets Human capital Financial PIPs Local networks Marketing Caste Gender Variables Most difficult time of the year? Food stocks. Dowry; River erosion; cyclone; pest disease attacks; rainfall patterns; illegal possession of land Permanent and seasonal migration; reduced income opportunities.

Owned/rented/leased Access to irrigation facility STWs/DBWs; Number of adult/young cow/buffalo/goat/poultry/Ducks. Owned or shared Housing condition/furniture; Bicycle, radio, TV; agricultural equipment No. in household; old age dependency ratio; Literacy levels; disabled member; female headed. Type of health service used (FWC; private doctor); purchase of prescription Remittances; saving/loan status. Participation in community activity; membership in indigenous organisations; contact with other NGOs; access to financial institutions; access to extension; access to NGO loan Who participates? Nature of marketing private company, middleman, individual initiative, exchange within village For Muslims as well as Hindus? Frequency of women coming together; movement within and outside community; level of control over household decisions. Involved in any conflict with household within the village

Conflict Strategies Income Homestead agriculture; field agriculture; daily field labour; daily sources/ time town labour; selling fodder grass; wholesale business; fruit and allocation vegetable production; rickshaw pulling; short-term migration; poultry rearing; cattle rearing; selling milk in market; small Coping business - fried rice selling strategies Selling land; ornaments; draft animals; tin sheets; trees; utensils; loans; child/women labour; migration to towns; illegal felling Adapting New activities - diversification; migration strategies Number of days sold by gender; contract arrangement advance Labour selling ; wage rate in peak and lean Are you saving? loan/savings use Investment Outcomes Food Number of months from own production. In difficult months
10 Source: Turton, C. et. al (2000) op cit

19

security

can you feed adequately no. of meals/day Number of children in school; Number of years in school Education Use of pesticide/fertiliser; number of trees/household; livestock to Environment land ratio; use of organic matter fuel vs. field; access to Sustainabilit common property resources. Energy use; y Under five wasting; under five stunting; BMI; Incidence of diarrhoea Health Night blindness; Skin disease; medical expenses Expenditure Eid expenditure; Womens Frequency of women coming together; Movement within and empowermen outside community; Level of control over household decisions t Indicators should be selected in discussion and collaboration with project partners and ideally with beneficiary involvement. Such processes involve considerable time and cost investment hence the trade-off between covering a range of livelihoods indicators and ease should be carefully considered. Only measure what is really required, and think strategically about who will use the information and what for. It is important to leave indicators open to capture unexpected issues that may emerge, and there should be a certain degree of flexibility and room for negotiation and adaptation. They should also consider intangible as well as tangible impacts, although these may be harder to measure. Collecting baseline data is also important for indicator data to be of optimal use. This activity needs to be planned early in the project cycle, hence the importance of thinking about M&E right from the project planning stage. It can also be a costly and time consuming process which will have implications for the scope and depth of the M&E activities proposed. Livelihoods M&E is not ultimately concerned with attaching a value to livelihoods outcomes, but rather with understanding whether livelihoods are moving in a positive direction. Hence, it is more relevant to focus on determining trends and direction of change rather than to attach values to change. There is, however, significant value in differentiating impacts between stakeholder groups (e.g. poverty groupings, by gender, etc) in order to see who benefits more, and who less from certain interventions, and to ensure that ultimately the livelihoods of the poor are being addressed. Tabulating direction of change for different stakeholder groups over time may provide useful insights. 3.7 How can Livelihoods M&E data be analysed? Much emphasis should be placed on the stage of analysis of M&E data. There is a tendency to focus more on collecting quantities of data at the expense of good quality analysis. Planning for analysis is important: who will carry out the analysis?; how will it be presented?; who or what is the resulting information intended for? Analysis of information needs to be appropriate to the needs it aims to fulfil. Firstly,

understanding process, understanding outcomes and understanding impacts are all helpful for ongoing managerial learning so that if outcomes and impacts of the intervention do not begin to emerge as envisaged, then corrective action may be taken. Secondly, M&E can produce relevant information that feeds into future planning, policy-making and resource allocation. Finally, lessons from M&E can add to our understanding and validation of the Livelihoods approach and its effectiveness as an approach to poverty reduction. Different lessons may be drawn from the data (and the M&E methodology) to fulfil each of these, and the results targeted at a particular audience perhaps using distinct means of communication in each case. However, analysis of Livelihoods data can have its limitations. There are often difficulties associated with attributing impacts to individual projects. These are potentially compounded when trying to determine the impact of interventions on the livelihoods of the poor. This is particularly the case in Livelihoods M&E as it attempts to: assess the influence of, or influence on macro level changes in policies or institutions and correlate with changes at the household level (e.g. directly attributing improved resilience to changes in health or educational status of individuals, households and target populations on a national scale); assess less easily quantifiable, more subjective livelihood outcomes, (e.g. empowerment, improved well-being); assess changes in the long-term; tries to capture predicted as well as unpredicted livelihood outcomes; progress from outputs to goal level within project/ programme logframes where more assumptions and risks are added11. Such issues should be considered at the design stage, as they may be partially overcome through the use of appropriate research tools. The use of flow diagrams, for example, can help establish the links between impacts and particular project interventions. However, most of these issues require further work in order to understand better how complex livelihoods impacts can be assessed. Presentation and dissemination of the findings of a Livelihoods M&E is a critical stage of the M&E cycle that is commonly forgotten. Careful consideration should be given to the potential users of the information resulting from the M&E, and to appropriate means of communications. Users may include project staff, project beneficiaries, policy makers and other donor agency staff. Means of communication of results might include workshops, publications, video, or meetings with policy makers, amongst others. 3.8 What are the Costs and Benefits of Livelihoods M&E? In any type of M&E activity time, technical capacity and cost constraints are commonly identified as limiting factors. These constraints, and the various options for dealing with them, need to be considered carefully and weighed against the benefits of a comprehensive, sustainable and participatory Livelihoods M&E system. The costs of Livelihoods M&E include:
11 Drake, L. (2000) Scoping mission to investigate the development of Livelihoods Indicators and Livelihoods
Monitoring systems for DFID-Bangladesh.

21

Opening up the design of M&E methods and indicators, as well as collection and analysis of data to include people with different views means that the process will probably take longer and require more compromises. Additional skills requirements include: inter-disciplinary collaboration, strong analytical skills, good facilitation, and more complex data analysis. These may require hiring additional (possibly expatriate) input, with related costs. Other additional resources required might include more time required for planning and executing a comprehensive approach, office space to locate an M&E unit, additional transport, and resources for communication of findings.

Some of these problems can be at least partially overcome. Participatory methods are not always more costly than conventional methods, particularly if conventional methods involve international consultants with high fees, or purchase of technical equipment, e.g. for remote sensing. The costs of staff, training and other resources required to set up an M&E unit may be significant, but such a unit can be based within an existing structure, or its development can be gradual over time. Furthermore, the value of establishing a sustainable, learning-process oriented and peoplecentred M&E system is, of course, considerable. Benefits include, amongst others: Costs saved in halting or redesigning activities that are having unforeseen negative effects; Benefits of a clearer understanding of how the project goals and impacts including how they relate to Sustainable Livelihoods; and the Rewards from having more motivated project staff and beneficiaries who are not afraid to acknowledge difficulties and learn from experience. 3.9 Summary In summary, the focus, principles and framework of the Sustainable Livelihoods approach draw attention to ways in which conventional M&E can more effectively contribute to poverty reduction. The following table summarises some of the emerging characteristics of a Livelihoods approach to M&E. Table 6. Summary of characteristics of Livelihoods M&E View of impact Scope Purpose Levels Responsibility Methods

Broad, holistic and cross-sectoral Direct and indirect, intended and unintended Beyond the project Tracks impacts over time, and beyond life of projects Includes cumulative impacts Learning not policing Learning about project progress and effectiveness Learning about the Sustainable Livelihoods Approach Looks at local level and macro level changes Makes linkages between them Works in partnership Strengthens internal commitment and capacity Participatory

Outputs

Quantitative and qualitative Information for corrective action within projects Information for planning and policy making Capacity built for continuous learning

The implications of these new goals and perspectives for implementation are only now beginning to be explored and tested in the field. As more experiences of Livelihoods M&E are documented and shared further insights and practical lessons can be added to those already outlined in this tool. 4. Links to further information online Eldis hot topic guide to Participatory Monitoring and Evaluation http://nt1.ids.ac.uk/eldis/hot/pme.htm SL guidance sheet 3.4 on M&E http://www.livelihoods.org/info/info_guidanceSheets.html Discussions on Livelihoods Indicators http://www.livelihoods.org/post/Indic-Theme1.html Sustainability Indicators for Natural Resource Management and Policy http://les.man.ac.uk/ses/research/CAFRE/indicators/home1.htm Developing Methodologies for Livelihood Impact Assessment: Experience of the African Wildlife Foundation in East Africa, by Caroline Ashley and Karim Hussein, ODI Working Paper 129: http://www.odi.org.uk/publications/intro.html

23