How-To Note: Preparing Evaluation Reports
NUMBER 1VERSION 1.0NOV 2012HOW-TO NOTEPreparing Evaluation ReportsMonitoring and Evaluation SeriesCreateevaluationreports thatare clear,credible, anduseful.INTRODUCTIONThis Note supplements USAID Automated Directives System (ADS)Chapter 203 and provides current good practice in preparing evaluationreports, the main deliverable for most evaluations. Following thesepractices will help to establish clear expectations for evaluation reportsduring the preparation of evaluation statements of work and the in-briefingof the evaluation team. These practices also serve as a guide for reviewingthe quality of draft evaluation reports submitted by the evaluation team.This Note is also a resource for USAID partners and independentevaluators of USAID programs and projects. An evaluation report templateand sample evaluation report covers are available as additional resources.BACKGROUNDThe most important outcome of an evaluation is that it is used to informdecisions and improve USAID projects and programs. A key factor in usingevaluation findings is having a well-written, succinct report that clearly andquickly communicates credible findings and conclusions, including easy-tounderstand graphics and consistent formatting.How-To Notesare published by theBureau of Policy,Planning and Learningand provide guidelinesand practical advice toUSAID staff andpartners related to theProgram Cycle. ThisHow-To Notesupplements USAIDADS Chapter 203www.usaid.govREQUIREMENTSUSAID’s Evaluation Policy and ADS 203 provide guidance on evaluationreport structure and content, and steps in the process of creating a report.These are listed in Table 1. The report must present a well-researched,thoughtful and organized effort to objectively evaluate a USAID activity,project or program. Findings, conclusions and recommendations must bebased in evidence derived from the best methods available given theevaluation questions and resources available. The evaluation methods,limitations, and information sources must be documented, including byproviding data collection tools and the original evaluation statement of workas annexes to the main report. Finally, the findings should be sharedtransparently and widely, to ensure accountability and to promote learningfrom USAID’s experience.
HOW-TO NOTEPreparing Evaluation ReportsTABLE 1: EVALUATION REPORT REQIREMENT(from the USAID Evaluation Policy and ADS 203)Report should be A thoughtful, well-researched, well-organized, and objectively evaluate what worked, whatdid not, and why.ExecutiveSummaryInclude a 3 to 5 page Executive Summary that provides a brief overview of the evaluationpurpose, project background, evaluation questions, methods, findings, and conclusions.EvaluationQuestionsAddress all evaluation questions in the statement of work.Methods Explain evaluation methodology in detail. Disclose evaluation limitations, especially those associated with the evaluationmethodology (e.g. selection bias, recall bias, unobservable differences betweencomparator groups, etc.).NOTE: A summary of methodology can be included in the body of the report, with the fulldescription provided as an annex.Findings Present findings as analyzed facts, evidence and data supported by strong quantitative orqualitative evidence and not anecdotes, hearsay or people’s opinions. Include findings that assess outcomes and impacts on males and females.Recommendations Support recommendations with specific findings. Provide recommendations that are action-oriented, practical, specific, and define who isresponsible for the action.AnnexesInclude the following as annexes, at minimum: Statement of Work. Full description of evaluation methods. All evaluation tools (questionnaires, checklists, discussion guides, surveys, etc.). A list of sources of information (key informants, documents reviewed, other datasources).Only if applicable, include as an annex Statement(s) of Differences regarding any significantunresolved differences of opinion on the part of funders, implementers, and/or members ofthe evaluation team.Quality ControlAssess reports for quality by including an in-house peer technical review with commentsprovided to evaluation teams.Transparency Submit the report to the Development Experience Clearinghouse (DEC) within threemonths of completion. Share the findings from evaluation reports as widely as possible with a commitment to fulland active disclosure.UseIntegrate findings from evaluation reports into decision-making about strategies, programpriorities, and project design.2
HOW-TO NOTESTEPS IN THE PROCESS1. Define Report Requirements in theEvaluation Statement of Work and FinalWork PlanAll evaluation statements of work (SOW) shouldclearly define requirements and expectations forthe final evaluation report. All of the items inTable 1 must be included as requirements forthe final report. Ensure that all requirements inthe SOW are also included in the final evaluationwork plan that is put in place once theevaluation team is on board. Adjustments can bemade at this time, as long as the minimumrequirements are met, and additions can beincluded such as defining when the first draft willbe due, how many days USAID will have toreview and provide comments, and when thefinal report will be submitted.2. Review First DraftProgram Offices must ensure that evaluationdraft reports are assessed for quality bymanagement and through an in-house peertechnical review and comments provided to theevaluation teams. USAID staff may considerincluding implementing partners and other directstakeholders in the review process. Tools suchas the USAID Evaluation Report Checklist canbe Reports4. Submit to DEC and Share Findings WidelyUSAID Program Offices must ensure thatevaluation final reports (or reports submitted byevaluators to USAID as their final drafts) aresubmitted within three months of completion tothe Development Experience Clearinghouse athttp://dec.usaid.gov. The actual submission canbe done by USAID staff or by the evaluationteam with USAID concurrence (once anopportunity has been provided for USAID orothers to include a Statement of Differences, ifappropriate). In addition to submission to theDEC, USAID should also consider how to sharethe evaluation report widely to facilitate broaderlearning. This could include posting the reporton the USAID mission website, translating asummary into local language, and hostingpresentations of the evaluation findings.5. Use Evaluation Findings to InformDecisionsUSAID must integrate evaluation findings intodecision making about strategies, programpriorities, and project design. While the ProgramOffice in a mission should ensure this happens, itis the responsibility of all USAID staff.3. Final Draft and Statement of DifferencesEvaluation reports are independent products andtherefore the evaluation team leader reviews thecomments and determines which to incorporateinto the final draft. Once the final draft issubmitted to the USAID mission or office, thecontent should not be changed without thepermission of the evaluation team leader.USAID, other funders, implementing partners,and other members of the evaluation team candecide to include a Statement of Differences as anannex to the report, if there are differencesrelated to the evaluation findings orrecommendations.3
HOW-TO NOTEPreparing Evaluation Reportsthe project was started, a description of thebeneficiary population, geographic area of theproject, and the underlying developmenthypothesis, or causal logic, of the project or thebroader program of which the project is a part.If a results framework (for strategies, objectivesor programs) or logical framework (for projects)is available, this should be included here. Forprojects designed under the project designguidance released in 2011, the evaluation teamshould have access to the final Project AppraisalDocument and related annexes (which includes alogical framework and original monitoring andevaluation plans, among other things). Thisinformation provides important context forunderstanding the evaluation purpose, questions,methods, findings and conclusions.CONTENT AND STRUCTUREGeneral StyleWhen writing a report, the evaluation team mustalways remember the primary audience: project andprogram managers, policymakers, and directstakeholders. The style of writing should be easy tounderstand and concise while making sure toaddress the evaluation questions and issues withaccurate and data-driven findings, justifiableconclusions, and practical recommendations.Report Sections and ContentAt a minimum, all reports should include thefollowing sections: Executive Summary (3 to 4pages); Evaluation Purpose and Questions (1 to 2pages); Project Background (1to 3 pages); EvaluationMethods and Limitations (1 to 3 pages, with fullversion provided in an annex); Findings, Conclusionsand Recommendations (15 to 25 pages); and,Annexes. Reports may include additional content,split the sections up differently, or present thesections in a different order. Executive SummaryThe Executive Summary, between three to fourpages in length, should stand alone as anabbreviated version of the report. All content ofthe full report should be summarized, and theExecutive Summary should contain no newinformation. Evaluation Purpose and QuestionsThe evaluation purpose should be clearly definedat the beginning of the report. It should describein about one page or less why the evaluation isbeing conducted now, how the findings areexpected to be used, what specific decisions willbe informed by the evaluation, and who the mainaudiences are for the evaluation report. Theevaluation questions are linked to the purpose,and should be listed here. Good practice is tolimit the questions to three to five that are clear,focused, and that will directly inform specificdecisions. Project BackgroundThis section should summarize the project beingevaluated in one to three pages, including theoriginal problem the project is designed toaddress, any changes that have occurred since Methods and LimitationsThis section should provide a detaileddescription within one to three pages of theevaluation methods and why they were chosen.If more space is needed, additional detailedinformation on the methods should be providedin an annex. The reader needs to understandwhat the evaluation team did and why to makean informed judgment about the credibility ofthe findings and conclusions and the underlyingevaluation design including the data collectionand analysis methods.Evaluation methods should correspond directlyto the questions being asked and shouldgenerate the highest quality and most credibleevidence possible, taking into consideration time,budget and other practical considerations.This section should provide information on allaspects of the evaluation design and methods,including tradeoffs that led to selection ofspecific data collection and analysis methods, adescription of data availability and quality, andsampling strategies (purposive, random, etc.),including how interview subjects or site visitswere selected. Just as important as describingthe evaluation methods is describing anylimitations in data collection and analysis, dataquality, access to data sources, or any otherfactors that may result in bias. To show the4
HOW-TO NOTErelationship between the evaluation questionsand methods, it is useful to include a chart thatlists each evaluation question, the correspondingevaluation method to be used for data collectionand analysis, data sources, sample sizes, andlimitations. Findings, Conclusions andRecommendationsFindings, conclusions, and (if requested in theevaluation statement of work)recommendations, make up the main body ofthe report, synthesizing what was learned duringthe evaluation and presenting it in an easy tounderstand and logical fashion. Findings areempirical facts based on data collected duringthe evaluation and should not rely only onopinion, even of experts. Conclusions synthesizeand interpret findings and make judgmentssupported by one or more specific findings.Recommendations, if applicable, are specificactions the evaluation team proposes be takenby program management that are based onfindings and conclusions. The reader should beable to discern what evidence supports theconclusions and recommendations. Wheneverpossible, data should be presented visually ineasy to read charts, tables, graphs, and maps todemonstrate the evidence that supportsconclusions and recommendations. All graphicsmust have a title, be clearly labeled, and includea caption.Preparing Evaluation Reports AnnexesAll evaluation reports must include the followingas annexes: 1) the Evaluation Statement ofWork, 2) Detailed description of the evaluationdesign and methods, 3) copies of the actual datacollection tools such as survey or interviewquestions, 4) a list of information sources(including documents reviewed, sites visited, andkey informants, assuming they gave permissionto be identified)., and 5) disclosure of anyconflict of interest by including a signedstatement by evaluation team members thatattests to a lack of conflict of interest ordescribes an existing conflict of interest relativeto the project being evaluated. Additionalannexes can be included at the discretion of theevaluation team and USAID, and in some casesimplementing partners, including, if applicable,any Statements of Differences with the evaluationconclusions.5
HOW-TO NOTEFORMAT AND GRAPHIC STANDARDSReminder on USAID Graphic StandardsThe USAID Graphic Standards Manual is available athttp://pdf.usaid.gov/pdf docs/PNADB334.pdf.Evaluation report authors and reviewers should befamiliar with the USAID Graphic Standards and applythem consistently. These include requirements andguidance related to USAID branding, choice oftypography, and color palette.CoverThe cover of an evaluation report should beattractive and provide enough information that areader can immediately understand what wasevaluated. To make evaluation reports distinct fromother types of USAID publications all evaluationreport covers should: Follow USAID Branding and Graphics Standards. Include a title block in USAID light bluebackground color with the word “Evaluation” atthe top and the report title underneath. Thetitle should also include the word “evaluation.” Include the following statement across thebottom: “This publication was produced at therequest of the United States Agency forInternational Development. It was preparedindependently by [list authors and/ororganizations involved in the preparation of thereport].” Feature one high-quality photographrepresentative of the project being evaluated.The photo should be high resolution, visuallysimple, colorful, and in focus. Include a briefcaption on the inside front cover explaining the“who, what, when, where, and why” of thephoto and with photographer credit.Preparing Evaluation ReportsTitle and Title PageWhile titles are determined by the evaluation teamand the USAID Mission or operating unitcommissioning the evaluation, all evaluations will besubmitted to the DEC and therefore titles should beclear to the general reader. A review of recentevaluation titles leads to the following suggestionsfor good practice: Compose a title that is informative, clear andcompelling (e.g., “Improving Community Healthin Fredonia: Evaluation of the USAID/FredoniaCommunity Health Project”). Avoid acronyms and do not use implementingpartner names (e.g., “XYZ LTD Evaluation”). Include the word “evaluation” in the title. Thiswill help the DEC correctly archive thedocument.The report title should be repeated on the titlepage, the first right-hand text page of the report.The title page also includes the subtitle, if any, andthe standard disclaimer for publications by externalauthors: “The author’s views expressed in thispublication do not necessarily reflect the views ofthe United States Agency for InternationalDevelopment or the United States Government.” Itmay also include the date of the report, a shortabstract summarizing the report, or otherinformation.AcronymsKeep the use of acronyms to a minimum, and defineall acronyms used in the report by including anacronym list in the beginning of the reportTable of ContentsThis comes before any content referenced in thetable. Sufficient detail should be provided to guidethe reader through the report, including pagenumbers.6
HOW-TO NOTELengthEvaluation reports should be no more than 30 pagesin length, not including any annexes and three tofour pages for an Executive Summary. This shouldbe sufficient to provide a summary of the evaluationpurpose and approach, key findings, conclusions andrecommendations. Additional detail can be providedas annexes.Preparing Evaluation ReportsCOMPANION PRODUCTSUSAID staff should consider other products toinclude in the evaluation statement of work that cancomplement the report and aid in disseminatingevaluation findings to a broader audience. Thesecould include photos documenting the evaluation, ashort video that combines footage from theevaluation with a summary of the findings, a shortfact sheet, a local language translation of theexecutive summary of the evaluation report, or apresentation via webinar of the evaluation report.Some products are not appropriate to ask as adeliverable from an evaluation team, such as“Success Stories” as this would put the evaluationteam’s objectivity and independence into question.ADDITIONAL RESOURCESThe following resources can be used as samples or templates, or provide more information on evaluationreports and on evaluation in general. Some other resources exist but are out-of-date with current USAIDguidance. Where information differs, the USAID Evaluation Policy and the USAID ADS (Automated Directives System) 200 series take precedence over that in other resources. Evaluation Report Template: t-template Evaluation Cover Samples: t-covers Sample Disclosure of Conflict of Interests Form: terest USAID Graphic Standards Manual: http://pdf.usaid.gov/pdf docs/PNADB334.pdf USAID’s Center for Development Information and Evaluation Publications: Style Guide: Guidelines forProject Managers, Authors, and Editors, December 2001http://pdf.usaid.gov/pdf docs/PNACN266.pdf7
Summary purpose, project background, evaluation questions, methods, findings, and conclusions. Include a 3 to 5 page Executive Summary that provides a brief overview of the evaluation Evaluation Questions Address all evaluation questions in the statement of work. Methods Explain evaluation methodology in detail.
work/products (Beading, Candles, Carving, Food Products, Soap, Weaving, etc.) ⃝I understand that if my work contains Indigenous visual representation that it is a reflection of the Indigenous culture of my native region. ⃝To the best of my knowledge, my work/products fall within Craft Council standards and expectations with respect to
PATHFINDER INTERNATIONAL TOOL SERIES Monitoring and Evaluation – 1 PREPARING A CASE STUDY: A Guide for Designing and Conducting a Case Study for Evaluation Input By Palena Neale, PhD, Senior Evaluation Associate Shyam Thapa, PhD, Senior Monitoring and Evaluation Advisor Carolyn Boyce, MA, Evaluation Associate May 2006
POINT METHOD OF JOB EVALUATION -- 2 6 3 Bergmann, T. J., and Scarpello, V. G. (2001). Point schedule to method of job evaluation. In Compensation decision '. This is one making. New York, NY: Harcourt. f dollar . ' POINT METHOD OF JOB EVALUATION In the point method (also called point factor) of job evaluation, the organizationFile Size: 575KBPage Count: 12Explore further4 Different Types of Job Evaluation Methods - Workologyworkology.comPoint Method Job Evaluation Example Work - Chron.comwork.chron.comSAMPLE APPLICATION SCORING MATRIXwww.talent.wisc.eduSix Steps to Conducting a Job Analysis - OPM.govwww.opm.govJob Evaluation: Point Method - HR-Guidewww.hr-guide.comRecommended to you b
Section 2 Evaluation Essentials covers the nuts and bolts of 'how to do' evaluation including evaluation stages, evaluation questions, and a range of evaluation methods. Section 3 Evaluation Frameworks and Logic Models introduces logic models and how these form an integral part of the approach to planning and evaluation. It also
Evaluation SOW as part of a peer review process, please see the Evaluation Statement of Work Review Template. For guidance on developing an Evaluation SOW, see the Evaluation Statement of Work How-to Note and Template. Evaluation Title: Evaluation SOW Review By: Date: 1. Information about the Strategy, Project, or Activity Evaluated COMMENTS 1.1.File Size: 282KB
The evaluation roadmap presents the purpose of the evaluation, the evaluation questions, the scope of the evaluation and the evaluation planning. The Steering Group should be consulted on the drafting of the document, and they should approve the final content. The roadmap identifies the evaluation
This Project Evaluation Plan Sample is part of the Evaluation Plan Toolkit and is designed to support the associated Evaluation Plan Guide and Evaluation Plan Template. This toolkit is supported with an educational webinar: Program Evaluation Plan Toolkit. The purpose of the Evaluation Plan Toolkit is to su
TAMINCO GROUP NV Pantserschipstraat 207, 9000 Ghent, Belgium Enterprise number 0891.533.631 Offering of New Shares (with VVPR strips attached) and Existing Shares
