Writing An Evaluation Scope Of Work (SOW)
Module 8Writing an EvaluationScope of Work (SOW)
Module ObjectivesBy the end of this module you will be able to:1 Develop most elements of an Evaluation Scope of Work withoutassistance.2 Ensure that the 4 main elements of a SOW are present and of highquality.3 Determine whether there is a good match between evaluation teamcomposition and the evaluation tasks.456 Determine whether a SOW is complete. Review a response to solicitation that includes an evaluation SOW forcompleteness and responsiveness. Understand 5 main differences between SOWs for PEs and IEs.2
ADS References for this ModuleADS ReferenceTopic or IssueADS 203.3.1.5SOW to include methods that generate the highestquality and most credible evidence. SOW to includestrengths and weaknesses of methodsADS 203.3.1.4Written design to include data analysis plan and adissemination planADS 203.3.1.10SOW/design to be shared with country-levelstakeholders and IPs before being finalizedADS 203.3.1.5SOW to include criteria for quality of evaluationreportADS 203.3.1.5Mission Program Office to organize peer review ofSOW Based on the best methodsADS 203.3.1.5Statement of Work (SOW) ProtocolsUSAID How To Note No. 3 Preparing an Evaluation SOW (2013 version)www.ioce.net provides information on evaluation organizations worldwide3
REVIEW: Organizing Questions for Writing a SOW What is the main purpose of this evaluation? (Accountability, Learning,or Other?) What specific information is needed to make a management decision? Who needs the information and why do they need it? What questions must be asked to obtain the required information? What are the data requirements? (Based on level of change expectedfrom the intervention and level of certainty or precision required tomake the decision?) What data is already available? What is the best design given information required and constraints?(Budget, timing, etc.)4
Developing an SOWIt is important to understand precisely what you want toknow from the evaluation and how you will get there beforefocusing on the elements of an SOW. What resources can be mobilized for the evaluation? Is there a decision timeline into which an evaluationmust fit to be useful? Are certain questions and approaches ruled out by timing orresource constraints?(For example, if it is Year 2 of the project, it may be too late toestablish a valid comparison group unless the project is beingphased-in to new districts each project year).5
Utilization is a Goal – Build Toward ItTreat the SOW development process as a first steptowards the utilization of an evaluation. Consult with key stakeholders early to identify their interests andpossible information requirements; Build awareness of the planned evaluation in USAID and amongpartners; Elicit input on the evaluation purpose, questions, and decision-makingschedules from USAID staff and partners.6
Elements of an Evaluation SOWThe Four Main Elements of a good evaluationSOW: Program/Project Information – what, where,when Evaluation Fundamentals – purpose,questions, use and users Technical Requirements – design, methods,staffing/competencies Management Information – schedule, budget,deliverables, POC/roles, logistics7
Elements of an Evaluation SOWLook through Exercise 8-1: Evaluation SOWElements. In your group, focus on one of the4 evaluation SOW elements, and note sub-components of each. E.g. What is included under ProjectInformation? Evaluation Fundamentals?8
Program/Project Information - DescriptionTitle:Start and End Dates: 2006-2010;Budget:Major changes - Have major changes in theprogram/project environment occurred and affectedimplementation or expectations about being able toachieve intended results? Target group Target area - be specific: Explain the situation – don’t force an evaluationteam to discover what you already know.9
Program/Project Information: ExistingInformationInclude program and projectperformance reports andother background materialsas attachments or URLs: Quarterly reports Annual reports Management assessment Previous evaluation10
A map included in a SOW helps explaintarget areas and groupsLaos11
BackgroundImportant background information necessary tounderstand project status, and success or challenges,should also be provided. Context Current status History of the project Institutional background: key decisions madeor upcoming12
State the Development HypothesesDescribe the chain of “if-then” propositions (Theory of Change)that transform project inputs into results.If possible, include one of the following: Narrative description of chain of linked hypotheses Results Framework (Program level) Logical Framework (Project and Activity levels)13
Technical Requirements – EvaluationProcessesDescribe any special characteristics of the evaluationJoint (or Collaborative) Evaluations involve multipleorganizational perspectives. They vary in terms of thedegree to which they are participatory – other than interms of sharing the funding burden.Since the Paris Declaration was signed,some donors have significantly increasedthe share of their evaluations that areundertaken jointly with assisted countries.16
Technical Requirements – EvaluationProcessesParticipatory evaluations draw stakeholders, includingbeneficiaries into the evaluation process – the SOWshould indicate what level of participation is envisioned.Design rigor and degree of stakeholderparticipation are different dimensions.Evaluation Design ContinuumParticipatory evaluations taketime, but can also be rigorous17
Technical Requirements – Data Collectionand Analysis Methods If USAID has prepared an evaluation design thatincludes data collection and analysis methods –include it. If USAID expects the team to propose data collectionand analysis methods, any USAID expectationsabout methods or data need to be specified:*Quantitative or qualitative emphasis inanswers*Comparisons to be made – to norms,benchmarks, non-recipients18
Technical Requirements – Data Collection Specify if you know how large your sample sizeshould be, or if you want the findings to bestatistically significant. This may affect theevaluation team’s proposed design, methods, andbudget!If needed, ask an evaluation methodsexpert, PPL/LER or a local surveyresearcher for assistance19
Technical Requirements – Data AnalysisConsider requiring that an analysis plan include illustrativeversions (empty shells) of the tables and graphs that will beproduced. It is a good way to tell whether all of the data a teamplans to collect are likely to be used.LocationRegion 1Region 2Region 3Region 4TotalAdopted Project TechnologyMenWomen#%#%Did Not Adopt Project TechnologyMenWomen#%#%20
Technical Requirements – DataDisaggregationSex- USAID’s requirement for data disaggregated bysex extends to evaluation - but must be askedfor in an evaluation SOW. Percentage of trainees that were female Percentage of technology adopters thatwere femaleDisaggregation of evaluation data in other ways may be useful: Ethnicity, region, age, education level, orincome level21
Technical Requirements – DataDisaggregationSOW requirements fordisaggregatingdata have implications forconducting an evaluation.Examining the effects of anational program onMacedonia’s ethnic Albanianminority would force a teamto spend time in districtswhere they live.22
Technical Requirements – TeamQualificationsWhat skills and experience must be represented on theevaluation team?Evaluation skills/experience, per ADS 203Technical/Sector skills/experienceSurvey, sampling and statistical skillsRelevant qualitative methods experienceFinancial analysis, cost-effectivenessLanguage skillsTeam management experienceGender balance/skillsCultural sensitivity24
Technical Requirements – Team Size How large a team will be needed? (Consider purpose, type ofevaluation, research design, number of questions, size of project,geographic scale, data requirements, etc.) Across how many different individuals will the skillsrequired be spread?- Does it matter which team members have whichskill sets?- Can the team split up to do the field work ordo skill requirements dictate that the wholeteam visits every project site? Are all skills needed on a full time basis over the lifeof the evaluation?25
Technical Requirements –TeamCompositionUSAID is actively seeking to build evaluation capacity internally.In order to meet this objective, USAID personnel may be askedto participate on evaluation teams. There are three possibleconfigurations: All team members are external Some team members are external, some are USAIDstaff All team members are USAID staffIf consultants are expected to work with USAID staff asevaluation team members, these expectations need to beexplicit and roles need to be clearly defined.26
Technical Requirements – Local PersonnelMany USAID Missions advocate using local personnel onevaluation teams. This has important implications foroverall team composition. Cost: If cost is the issue, can the entire team be local? Evaluation capacity building: If capacity building is thereason, in what evaluation areas is local capacity weakand how could the evaluation process be used to strengthenthat aspect of capacity?www.ioce.net provides information on evaluation organizations worldwide27
Technical Requirements – Short ExerciseYou have been asked to review and approve this Team Compositiondescription for an Evaluation SOW.D. Composition of Review Team9. The Final Evaluation team is expected to consist of members which sets up the team to have agood local understanding, together with specialists who are recognized international experts inthe different disciplines SHOUHARDO engages in. Specifically, the team is expected to consistof experts in ‘food and nutrition security’, ‘economic security’, ‘health and hygiene’,‘governance’, ‘rights based approaches’, ‘women empowerment’, ‘community empowerment’,‘institutional development’, ‘urban’, and ‘disaster management’.10. The team is expected to make a critical analysis of both ‘software’ and ‘hardware’, and howeffective the Program has been to compliment one with the other.11. The team size is expected to be between 5 to 8 strong (with a healthy mix of both expatriateand local team members, as well as an appropriate mix of gender). All the proposed members ofthe team must have a demonstrated track record, and be recognized as seasoned professionalswho can conduct such a complex evaluation with a high degree of proficiency.12. The composition and number of team members has deliberately not been ‘exactly’ defined,leaving it open for bidders to define this. At the time of review of proposals, this will be a criticalassessment area for determination of the winning bid.How will you respond?28
Management Requirements – DeliverablesAsk for everything you want/need in the SOW: Inception report summarizing what is known fromperformance monitoring reports and/or literature reviews Detailed evaluation plan (design, methods, data analysisplan, schedule) for review and approval prior to datacollection – if not already prepared by USAID Post field work and analysis briefing on findings,conclusions and recommendations prior todrafting report If applicable, describe any planned USAID contributions tothese deliverables.29
Management Requirements – DeliverablesAsk for everything you want/need in the SOW: Draft report (5 hard copies and soft copy) Oral briefings (2) Final report (15 hard copy and soft copy) Encrypted flash drive including all instruments and data informats suitable for reanalysis Cloud Space for document sharing/storage30
Management Requirements – Schedule Table Brainstorm (2 minutes): What schedulerelated factors do we need to address in anevaluation SOW? How many ideas can you generate?31
Management Requirements – ScheduleAn evaluation schedule needs to include adequate time for: Detailed Planning – Instrument Development,Pre-testing and Revision Data Collection Data Analysis Report Preparation, Review and Revision32
Management Requirements – ScheduleAn evaluation schedule defines when specific steps inthe evaluation process will occur and when deliverablesare due. Some activities will overlap – a GANTT chart is a goodway to display time periods during which activities occur. Elapsed time is often as important as real time. Elapsedtime is the total period during which an activity takes place,which may be much longer than the number of billabledays involved.33
Management Requirements – ScheduleTasksWeek Week Week Week Week Week Week Week Week123456789Initial MeetingsE-Survey developed/sentField intstrumentsdeveloped/testedNGO headquarters visitsField site visitsData analysisDraft reportFinal report.Sufficient elapsed time is what makes an e-survey and waiting forUSAID comments possible in an otherwise short evaluation period.Allowing international team members to participate in an initial meetingby phone, work on the instruments prior to arrival, and be on site by theend of week two may save resources – without compromising data quality.34
Management Requirements – ScheduleExerciseYou have been asked to review and approve this Evaluation Schedule.E. Timeline13. The Final Evaluation is scheduled to take place over the months of August – October 2011.The expected start date is 10 August 2011. The draft report is to be submitted by 30 September2011, around which time a detailed presentation is to be made to USAID and the implementingpartner organization. USAID will take till 8 October 2011 to provide detailed feedback on thedraft report. The submission date for the Final Report is 17 October 2011.How will you respond?35
Management Requirements – LogisticsA section on logistics in a SOW explains what assistanceUSAID will provide – or may in some situations place limitson when and where an evaluation team can travel. USAID will provide transportation in conflict area. Evaluation team members must stay at USAID approved hotels sothat USAID staff members can attend mini-training on impactevaluation and attend evaluation briefings at that site. USAID will provide facilities for oral briefings to staff and otherstakeholders on evaluation findings36
Evaluation SOWChecklistEvaluation SOW ChecklistStatement of Work Checklist Keyed to USAID’s Evaluation Policyand ADS 203.3.6.3Project or Program to be Evaluated:Person Who Reviewed the SOW: Date of Review:The Evaluation SOW Checklist willhelp you to ensure quality andrealism in evaluation SOWSUsing R-7 and C-16,review the SOW inExercise 8-2.How Well is the SOWElement Addressed1SOW Elementsand Sub-Elements5432Issues Noted bySOW Reviewer1Adherence to General Principles in USAID’s New Evaluation Policy1. Is the SOW developed as part of projectdesign?2, Does the SOW include provisions forcollecting, reviewing or reconstructingbaseline data?3. Does the SOW take employ measures toreduce bias such as contracting evaluationswith third-party contractors?4. Does the evaluation address the mostimportant and relevant questions about projectperformance?5. Does the SOW propose methods thatgenerate the highest quality ad most credibleevidence to answer the key questions?6. Does the SOW take into account ways inwhich to reinforce local evaluation capacityand to use local evaluation specialists?7. Does the SOW include provisions forsharing the findings from the evaluation aswidely as possible with full and activedisclosure?8. Is the SOW clear about requirements for theFinal Evaluation Report following Appendix 1of USAID’s New Evaluation Policy?Identify the activity, project, or approach to be evaluated9. Is the SOW clear and specific about what isto be evaluated, e.g.,1Key: 1 element was not covered at all in SOW; 2 At least one key aspect was not covered; 3 Allaspects were covered at a basic level; 4 Covered all aspects but went beyond basics in at least one waythat is likely to help evaluators; 5 All aspects were covered thoroughly and completely, going beyondbasics in a number of ways which will aid the evaluators.37
PE and IE SOW SimilaritiesAll evaluation SOWs have some common elements,no matter whether they are SOWs for PerformanceEvaluations or Impact Evaluations: PurposeQuestions and expectations about answersDesign and methodsDeliverables, including reportsTeam qualificationsScheduleLogisticsBudgetThere are also some important differences.38
PE and IE SOW Key DifferencesImpact Evaluations – and thus their SOWs – differ fromthose of Performance Evaluations in four main ways: Duration Scope/Focus Importance of Design Interface with the Project Implementation Team39
Review Questions What are the four main elements of a SOW? What should be considered when reviewing a solicitationresponse? What are some main differences between SOWs for PEsand IEs? What should be considered when developing LOE andbudget for an evaluation?40
Scope of Work (SOW) Module Objectives 2 By the end of this module you will be able to: 1 Develop most elements of an Evaluation Scope of Work without assistance. 2 Ensure that the 4 main elements of a SOW are present and of high quality. 3 Determine whether there
Chapter 5 Project Scope Management Chapter 5 Project Scope Management 5.1 Plan Scope Management 5.1 Colect Requirements 5.2 Collect Requirements 5.2 Define Scope 5.3 Define Scope 5.3 Create WBS 5.4 Create WBS 5.4 Verify Scope 5.5 Validate Scope 5.5 Control Scope 5.6 Control Scope PMBOK 5th Edition
Scope 2 market-based. 1. Scope 2 location-based. 1. Scope 3 Purchased goods and services. 2. Capital goods. 2. Fuel and energy-related activities Employee commuting Waste generated in operations Total Scope 1, Scope 2 market-based, Scope 3 GHG intensity (Scope 1 and Scope 2 mtCO2e/sq ft) 3. 1,560 10,523 11,171 121,817 89,963 3,522 3,677 15,388 .
Best Practice Book for IELTS Writing. Table of Contents IELTS Writing 1 IELTS Writing 9 IELTS Writing - Overview 9 IELTS Academic Writing 10 IELTS ACADEMIC WRITING 10 IELTS General Writing 11 IELTS Writing Task General (Task 1) 12 Sample 1 12 Sample 2 12 Sample 3 13 Sa
Academic Writing Quiz xvii Part 1 The Writing Process 1 1.1 Background to Writing 3 The purpose of academic writing 3 Common types of academic writing 4 The format of long and short writing tasks 4 The features of academic writing 6 Some other common text features 6 Simple and longer sentences 7 Writing in paragraphs 8 1.2 Reading: Finding .
camera. The main imaging scope can be virtually any scope suitable for your CCD or DSLR camera. Guide Scope You will need an additional telescope for guiding, referred to as a guide scope. The guide scope is mounted on top of, or beside the main imag-ing scope. Adjustable guide scope tube rings (available from Orion) are
(CCSS) for Writing, beginning in early elementary, will be able to meet grade-level writing goals, experience success throughout school as proficient writers, demonstrate proficiency in writing to earn an Oregon diploma, and be college and career-ready—without the need for writing remediation. The CCSS describe ―What‖ writing skills students need at each grade level and K-12 Writing .
Writing Spectrum is available in these titles for second grade success: Carson-Dellosa Publishing LLC P.O. Box 35665 Greensboro, NC 27425 USA carsondellosa.com SPECTRUM Writing GRADE 2 Focused Practice for Writing Mastery Writing a story Writing to inform Writing an opinion
The evaluation roadmap presents the purpose of the evaluation, the evaluation questions, the scope of the evaluation and the evaluation planning. The Steering Group should be consulted on the drafting of the document, and they should approve the final content. The roadmap identifies the evaluation
