Monitoring And Evaluation Manual - FSN Network

Monitoring and Evaluation ManualPrepared forADRA InternationalFood Security DepartmentPrepared byTANGO International, Inc.March 2007

Table of ContentsAcronyms . ivPart I. Overview of Monitoring and Evaluation . 11. Introduction . 11.1 Purpose of the Monitoring and Evaluation Manual . 11.2 Using the Manual . 22. Guiding Principles and Elements of Monitoring and Evaluation . 32.1 What is Monitoring and Evaluation? . 32.2 Why Does a Program Need a Monitoring and Evaluation System? . 52.3 General Overview of Basic Elements of a M&E System . 62.4 Monitoring and Evaluation Framework . 72.5 General Overview of Indicators and Indicator Development . 82.6 Managing for Impact. 103. Conceptual Frameworks for Programming. 123.1 Conceptual Frameworks for Project Design . 123.2 The USAID/FFP Expanded Conceptual Framework for Food Security . 15Part II. Key Components and Stages of M&E Systems . 174. Vulnerability or Holistic Assessments . 174.1 Assessment Preparation . 174.2 Defining Assessment Objectives . 204.3 Target Area Selection . 205. Problem Analysis . 215.1 Using Cause and Effect Logic in Project Design . 225.2 Hierarchical Problem Analysis . 235.3 From Problem Analysis to Project Strategy . 246. Program Design and Logical Frameworks . 276.1 Establishing SMART Goals and Objectives . 286.2 Design Principles for Monitoring and Evaluation . 296.3 Logical Frameworks . 296.4 General Logical Frameworks: linking indicators to program design. 306.5 The Logical Framework Matrix (LogFrame). 317. Steps to Setup a Monitoring Systems . 377.1 Six steps to Setup an M&E System . 377.2 Monitoring for Performance and Participation . 407.3 Participatory Monitoring . 417.3 Monitoring the Risk and Vulnerability Context . 438. Establishing Indicators and Performance Targets . 458.1 What are Indicators and Targets . 458.2 Types of Indicators . 468.3 Identifying what and how to measure . 478.4 Characteristics of Ideal Indicators . 488.5 Criteria for selection of sound indicators . 498.6 Outcome and impact indicators. 498.7 Performance targets and benchmarks . 508.9 Approaches to establishing/setting targets . 518.10 Limitations of targets . 519. Sampling: Key Concepts . 529.1 Introduction to Sampling . 529.2 Types of Sampling . 529.3 Key Definitions . 539.4 Sampling Methods/Design . 54ADRA – Monitoring and Evaluation Manuali

9.5 Non-Probability Sampling . 5610. Baseline Surveys and Data Management. 5810.1 Initial considerations in conducting a baseline survey . 5810.2 Methods of data collection . 5910.3 Developing a baseline survey . 6010.4 Data collection process . 6310.5 Data management. 6511. Data Analysis . 6711.1 Data Cleaning. 6711.2 Quantitative Data Analysis . 6811.3 Qualitative Data Analysis . 7011.4 Data Interpretation and Presentation . 7012. Information Sharing . 72Part III. M&E Considerations for Sector-Specific Programming. 7613. M&E Considerations for Food Security Programs . 7613.1 Seasonality and Cross-program Comparability . 7613.2 Food Availability . 7613.3 Food Access . 7713.4 Food Utilization . 7913.5 Market Information . 8014. M&E Considerations for Health and Nutrition. 8014.1 Selection of Indicators . 8014.2 Integrated Disease Surveillance and Nutrition Surveillance Systems . 8214.3 Nutrition . 8315. M&E Considerations for Micro Enterprise Programs . 8416. M&E Considerations for Reproductive Health Programs . 8517. Integrating Cross-Cutting Issues into Data Collection . 8717.1 HIV/AIDS . 8717.2 Gender . 8817.3 Conflict . 88References . 89Annexes . 91Annex 1: Components of a Humanitarian Information System . 92Annex 2: Sampling . 94Annex 4: Food Security Weathervane Indicators . 106Annex 5: Potential Indicators for Use in Health and HIV/AIDS . 113Annex 6: Potential Indicators for Use in Micro & Small Enterprise Development . 114Annex 9: Example of Topical Outline for Use in Qualitative Assessments . 122Annex 10: Qualitative Assessment Tools . 127Annex 11: Example of Community Questionnaire. 129Annex 12: Example of Qualitative Data Matrix . 140Annex 13: Preparatory Tasks for Vulnerability Assessments . 142Annex 14: How to Develop M&E Plan . 148ADRA – Monitoring and Evaluation Manualii

List of FiguresFigure 1: Project Hierarchy. 6Figure 2: Monitoring and Evaluation Framework . 8Figure 3: Indicators and Targets Along With the Project Hierarchy . 13Figure 4: Livelihoods Framework . 13Figure 5: USAID/FPP Expanded Conceptual Framework . 13Figure 6: Hierarchical Problem Tree . 13Figure 7: The Project Cycle . 13Figure 8: Linking the Results Framework to a Logical Framework . 30Figure 9: A Logical Framework with indicators. 31Figure 10: Indicators for LogFrame . 13Figure 11: Partial Results framework for National TB Program . 13Figure 12: Definition of Results Based Management . 13Figure 13: Factors Influence People’s Participation in M&E . 13Figure 14: Key Steps in Creating and Selecting Indicators . 13Figure 15: Transforming Ideas about Quality into Measures for Which Targets Can Be Set . 50Figure 16: Various Methods of Random Sampling . 13Figure 17: Essential Components of a Baseline Survey Plan . 61Figure 18: TIPS for Questionnaire Design . 61Figure 19: Sample Frequency Table . 69Figure 20: Minimum Information Requirements for Food Availability. 77Figure 21: Minimum Information Requirements for Food Access . 77Figure 22: Information Requirements for Food Utilization . 77List of TablesTable 1: Types of Indicators and Purpose. 9Table 2: Common Sources of Secondary Data .18Table 3: Descriptive Information Obtained through Secondary Data Analysis .19Table 4:Generic LogFrame Matrix (WFP) .32Table 5: Objective Hierarchy Links to Monitoring & Evaluation .38Table 6: Difference between traditional evaluation and participatory evaluation .42Table 7: Sharing of Information needs to be based on Information Needs .72ADRA – Monitoring and Evaluation Manualiii

AIDWHOADRA Monitoring and Evaluation ManualAdventist Development and Relief Agency InternationalAuto Immune Deficiency SyndromeBehavior Change CommunicationCenters for Disease ControlControl of Diarrheal DiseaseCommunity Health WorkerChild Survival Collaborations and Resources GroupDiarrheal DiseaseDemographic and Health SurveyDesign Monitoring and EvaluationDevelopment ReliefFood and Nutrition Technical AssistanceFood for PeaceGlobal Alliance for Vaccines and ImmunizationHygiene Improvement FrameworkHealth Facility AssessmentHealth Facility SurveyHealth Information SurveyHuman Immunodeficiency VirusHousehold Livelihood SecurityHealth Management Information SystemsIntegrated Child Health SurveyInformation, Education and CommunicationInternational Fund for Agricultural DevelopmentIntegrated Management of Childhood IllnessesKnowledge, Attitude, PracticeMaternal and Child HealthMonitoring and Evaluation to Assess and Use ResultsMillennium Development GoalMonitoring and EvaluationMicro-finance InstitutionsMultiple Indicator Cluster SurveyManagement Information SystemMicro and Small EnterpriseNongovernmental OrganizationPromotion of Maternal to Child TransmissionPrimary Health CareReproductive HealthTuberculosisUnited Nations Development ProgrammeUnited Nations Children’s FundUnited States Agency for International DevelopmentWorld Health Organizationiv

Part I. Overview of Monitoring and Evaluation1. IntroductionThe Adventist Development and Relief Agency (ADRA) is an independent humanitarianorganization established in 1984 by the Seventh-day Adventist Church for the specific purposeof providing individual and community development and disaster relief. ADRA serves people inover 125 countries regardless of ethnic, political or religious association. ADRA helps those inneed, particularly those most vulnerable such as women, children and senior citizens.ADRA partners with communities, organizations and governments to improve the quality of lifefor millions around the world through a range of core programming areas including foodsecurity, economic development, primary health care, emergency management and basiseducation. ADRA recognizes that current programming in each of these core areas can bestrengthened through the improvement of staff capacity in project monitoring and evaluation.Monitoring and Evaluation (M&E) has become a leading priority for many development andhumanitarian organizations. Advancements in measurement approaches, indicators and targets,performance monitoring and managing for results (impact) have been made in recent years inorder to adequately and effectively evaluate progress and program impact. M&E is essential inorder to design appropriate, effective, measurable programs and projects, and to consistently andeffectively monitor implementation and evaluate the impact of specific activities among targetpopulations.In line with this philosophy and in order to better meet programming goals, ADRA is seeking toenhance its program training with the inclusion of a monitoring and evaluation component.Building ADRA’s capacity to establish effective M&E systems will improve program design andmanagement, and ensure that new projects develop M&E plans appropriate to the particularprogramming and vulnerability context.This manual is directed towards programmers and M&E technical staff within the ADRAnetwork who are responsible for collecting, analyzing and distributing information on programs.It is designed to guide program managers and M&E staff in the establishment and use ofmonitoring and evaluation systems for large programs, specific program components and smallprojects alike. It should serve as a guide to improve understanding of M&E in general andincrease competency in key aspects of practicing M&E in the field.1.1 Purpose of the Monitoring and Evaluation ManualThis manual introduces fundamental concepts and components of M&E. It then presentsdefinitions of the basic components of an effective M&E system and offers guidance foradapting each component to local programming contexts. It also provides key considerations forthe development of appropriate M&E tools within the primary sectors in which ADRA works.Perhaps most importantly, the manual is intended to contribute to the learning environmentwithin ADRA by describing the ways in which a comprehensive M&E system can beconsistently used to inform problem analysis, program design, implementation, monitoring andreporting of evaluation findings.ADRA Monitoring and Evaluation Manual1

The M&E Manual is intended to strengthen the following principal competencies:Understanding conceptual frameworks for program design and planning upon whichmonitoring and evaluation systems will be based;Identifying and distinguishing between the key components of monitoring and evaluationsystems;Understanding the synergistic relationships between program design and management,and M&E systems in order to determine the expected impact and objectives and how theywill be achieved;Knowledge of the various tools and frameworks for M&E design planning andmanagement;Determining appropriate indicators and targets for both implemen

ADRA Monitoring and Evaluation Manual 2 The M&E Manual is intended to strengthen the following principal competencies: Understanding conceptual frameworks for program design and planning upon which monitoring and evaluation systems will be based; Identifying and distinguishing between the key components of monitoring and evaluation .