Public Sector Performance- The Critical Role Of Evaluation
Public Disclosure AuthorizedPublic Disclosure AuthorizedPublic Disclosure AuthorizedPublic Disclosure AuthorizedWORLD BANK OPERATIONS ationSelected Proceedings froma World Bank Seminar205321999
WORLD BANK OPERATIONS EVALUATION DEPARTMENTEVALUATION CAPACITY fEvaluationSelectedProceedingsfroma WorldBankSeminarKeithMackayEditorThe WorldBankWashington,D.C.
This Proceedings is part of the OEDProceedingsSeriesproduced by the Operations EvaluationDepartment, Partnerships and Knowledge(OEDPK)group of the WorldBank.The viewsexpressed hereshould not be attributed to the WorldBank or its affiliated organizations.
Contentsv Abbreviations/Acronymsvii Acknowledgmentsix PublicSectorPerformance-The CriticalRoleof EvaluationKeith Mackay1 PART1: The Roleof Evaluationin Development3 WhyBotherAbout ECD?7 The Roleof EvaluationRobert PicciottoMarkBaird13 PART2: The MissingLink in Good Governance15 EvaluationCapacityand the PublicSectorDavidShand1921555962PART3: Experienceof DevelopedCountriesThe Developmentof Australia'sEvaluationSystemEvaluationin the FederalGovernmentof CanadaCommentsCommentsKeithMackayStan DivorskiRayC.RistFrans Leeuw6567728594PART4: Experienceof DevelopingCountriesLessonsfrom CapacityDevelopmentin ZimbabweCommentsMarioMarcelAlain BarbarieStephenBrushettEduardoWiesner97 PART5: TwoPerspectives99 A Viewfrom a WorldBankNetwork101 A Viewfrom USAIDCherylGrayGeraldBritan105 PART6: Where Do WeGo from Here?107 Overviewand ConclusionsElizabethMcAllister112 RecommendedReading113 List of Authorsand DiscussantsContentsiii
and gAgency,Governmentof trategyCenterfor DevelopmentInformationand Evaluation(USAID)Committeeon Bank)Economics&ChiefDevelopmentDepartmentof nomicDevelopmentInstitute (WorldBank)Economicand SectorWorkGovernmentof IndonesiaGovernmentPerformanceand ResultsAct (USgovernment)InternationalBank for Reconstructionand DevelopmentInternationalDevelopmentResearchCentre (Canada)Learningand LeadershipCenter(WorldBank)Monitoringand evaluationNongovernmentorganizationOrganisationfor EconomicCooperationand enditureReformLoanPovertyReductionand ceGroup onResults-basedmanagementSwissAgencyfor Developmentand CooperationTechnicalassistanceUS Agencyfor InternationalDevelopmentWorldBankAbbreviationsand Acronymsv
Acknowledgmentsseminarwas organizedunder the auspicesof LLC,PREMand OED.The joint OED-LLCThe seminar team consistedof KeithMackay(task manager,OEDPK),KathyPeterson,and ErikaMarquina(LLC),with additionalsupport from Patty Rodriguez(OEDPK).This proceedingwas producedin the Partnershipsand KnowledgeGroup (OEDPK)bythe Disseminationand OutreachUnit. The unit is directedby ElizabethCampbell-Page,TaskManager,and includesCarolineMcEuen(editor),KathyStraussand Lunn Lestina(desktopdesign and layout),and artnerships&KnowledgeTaskManager: KeithMackayvii
PublicSectorPerformanceTheCriticalRoleof EvaluationSelectedproceedingsfrom a World BankseminarEditor'sPrefaceThe developmentcommunityand the WorldBank recognizethe importanceof soundgovernanceand the needto support countries'capacitybuildingand institutionaldevelopment.To encouragedebateand explorethe links between public sector performance and the role of evaluation,the WorldBankorganizeda seminarfocusingon theexperiencein developingnational evaluationsystems,or evaluationcapacitydevelopment (ECD),as an aid to better governance,in WashingtonDCin April 1998.The purposeof this publicationis to present selectedpapers plus the transcriptsof keyseminar proceedings.This selectionrepresents only a subset of the broad and rich rangeof issues covered.A main focusof this volumeis national experiencesin developingevaluationsystems.Bypreservingthese experiences,OEDintends to contributeto thegrowing"library"of evaluationcase studies-in other wordsto documentlessonsconcerningwhat worked,what did not, and why.Theselessonswillbe sharedwith WorldBankstaff,other developmentagencies,and developingcountry governments.To furthersupportthis objective,the WorldBankis alsolaunchinga separateseriesof short ECDpapers, the main focusof whichis country case studies.The Priorityfor DevelopingEvaluationCapacityEvaluationpotential can be better understoodby recognizingthe importanceof economic governanceand a sound public sector to national economiccompetitivenessmarketsreward countriesable to manageand screen publicexpenditures,and evaluation offersa tool to help do that.RobertPicciottoarguesthat evaluationis in many wayscentral to the effectivenessofdevelopmentassistance:* that the developmentassistancecommunityhas turned to results-basedmanagement at the project,country and globallevelsand that this approachrequiressolidinstitutional capacityin countries;ix
that partnershipsand coalitionsamongdevelopmentagenciesto help supportcountry programsand institutionsalso require a commonapproach to evaluationand assessment;and* that our growingneed to demonstratethe effectivenessof developmentinterventions to the electoratesof industrial democraciesis evident.eDevelopingnationalevaluationcapacitiesis a meansfor ensuringthat evaluationfindingsareavailableto assistcountriesin threeareas.First,evaluationfindingscan be an importantinputfor governmentdecisionmakingand prioritization,particularlyin the budgetprocess.Second,evaluationassistsmanagersby revealingthe performanceof ongoingactivitiesat the project,programor sectorlevels-it is thereforea managementtoolwhichleadsto leamingandimprovementin the luationresultscanalsobe usedto assessthe performanceof organizationsand institutionalreformprocesses.Third,evaluationdata contributeto entscan be heldaccountablefor the performanceof theiractivities.As eenthesethreeusesof evaluationfindings.The conceptof performanceencompassesthe efficiencyof a project or activity-theabilityto undertake an activityat the minimumcostpossible.It also includeseffectiveness-whether the objectivesset for the activityare being achieved.There are many types of evaluationtools,whichcan be used in a varietyof ways.Thesetools are related-in that they deal with the conceptof performance-but they can leadto confusion,exacerbatedby the differentterminologyemployedby evaluationpractitioners.Regardlessof the terms used: ongoingmonitoringand performanceinformation; projectand program evaluation-ex ante, ongoing/formativeand ex post/summative;performance(or value-for-money)audits; financialauditing,they alladdress performancemeasurement.This broad spectrum of performancemeasurementactivitiesis alsoknownby other genericlabels,such as monitoringand evaluation(M&E).Unlessotherwisestated,the term 'evaluation'is used in this volume.Expertsfrom lyfocusedon projectevaluation),programevaluators(who typicallyhavea broadsocialsciencebackground)andauditors(whoin the pasthaveemphasizedfinancialcompliance)-tendto use differentconceptsbut oftenwith similaror evenidenticalnomenclature.It is littlewonderthat thosenew to evaluationare oftenconfusedby the panoplyofconceptsandjargon.Thisvolumedoesnot addressthe problemof terminologyin any depth,but the readeris alertedto its existenceand is advisedto carefullyinterpretthe messageson whichthesepapersfocus.xKeith Mackay
Performancemeasurementis a valuableexercisenot least becauseit providesanopportunityand a frameworkfor askingfundamentalquestionssuch as: Whatare youtrying to achieve?;Whatdoes"success"looklike?;Howwillyou knowif or when you'veachievedit?As MarkBaird emphasizesin his paper,No publicsectorcan affordto overlookthe importanceof clearlydefiningits objectivesand benchmarks,andchangingthe bureaucraticcultureinto onethat stressesclientserviceand achievementof results. uationnowbecomesa keyinstrumentof goodgovernanceand institutionaldevelopmentwithinour clientcountries.Weallhavea responsibilityto makesurethisfunctionisnurturedand supported,as it has beenwithinour owninstitutions.The supportthat the developmentof evaluationcapacityoffersto broader governance,institutionaldevelopmentand public sector reformis often not fullyappreciated.Linksand commonalitiesare seen in the areas of:* inancialreportingandauditing;* intergovernmentalfiscalrelations,and the extentto whichthey encompassa focuson performance;* commercializationand the private sector deliveryof publicservices.For the privatesectorto be successfulin the deliveryof public services,governmentsshouldhave aclearunderstandingof programobjectives,and they shouldundertakeex ante,ongoingand ex post assessmentsof performance;- formulationof customerservicestandardsby servicedeliveryagencies,andmonitoringthe extent to whichthese standards are actuallyachieved;* civil mentand appraisal-recognizing that individualperformanceis reflected,to some extent, inproject or program performance;* civil servicepolicyadvice,which shouldencompassexistingevaluationfindingsorseek to commissionnew ones;* participationand the 'voice'of civil society,whichincorporatesthe viewsandexpectationsof ordinarycitizensconcerningthe performanceof governmentactivities;and* anti-corruptionefforts,particularlyto orting,strengthenwatchdogagencies,and achievegreatertransparencyin policymakingand implementation.Public Sector Performance-The Critical Role of Evaluationxi
Country ExperienceDevelopedand developingcountriesalike are accumulatinga growingvolumeofexperiencewith national evaluationsystems.Theseproceedingspresent experiencesinChile,Indonesia,Zimbabwe,Australiaand Canada.The experienceof developedcountriesillustratesthe potential links betweennationalevaluationcapacityand goodgovernance,reflectingthe opportunitiesand difficultiesinachievingculturalchangein a government-winning hearts and mindsis a slowbusiness.It also underscoresthe differentdimensionsthat must be developedto achievea robust national evaluationsystemin the areas of demand,supply,and informationinfrastructure.The mainpreconditionfor developinga national evaluationsystemis countrydemand-an evaluationsystem cannotbe effectivelyfoistedon an unwillinggovernment.There are particularrisksif the impetusfor an evaluationsystem is donor-driven;this isnot to say that donors cannottake the lead in "selling"the merits of evaluationsystemsto countries,but rather,that unless and until countriesacceptthe strength of sucharguments,or reach their ownconclusionsabout the merits of evaluation,an evaluationsystem is unlikelyto be sustainable.As StephenBrushettnotes in his paper,two buildingblocksfor effectivedemandincludesensitizingkey stakeholdersto the need for and benefitsfrom evaluation,and buildingawarenessof suitabletechniquesand approaches.Experiencetellsus that the mainbarriersto developingevaluationsystemsin developingcountriesare: poordemandand ownershipin countries;lackof a cultureof accountability(oftenrelatedto ethicsor corruption);absenceof evaluation,accounting,or auditingskillsthereis a needto developthe supplyof theseskillsandsystemsto matchdemandas andwhenit grows;poor qualityof financialand otherperformanceinformation,and of accounting/auditingstandardsand systems;lackof esses;and the needforgreatereffortsto .(Theseissuesare discussedin f the TaskForce,WashingtonDC,1994.)Whatis not yet clearis whethera governmentor forean evaluationsystemcan realisticallybe contemplated.Butwedoknowthat developingan evaluationsystemshouldnot be viewedas a stand-aloneactivity-itxiiKeith Mackay
wouldbe unrealisticto attemptto simply"bolton"an evaluationsystemto an existingstructureof governanceifthe institutionalframeworkand incentivesdo not supportit. Onthenthisis a strongargumentthe otherhand, if the frameworkand incentivesare insufficient,of an evaluationsystemis part of a broaderinitiativetoforensuringthat the developmentdevelopgovernance.This approachrecognizesthe strong synergiesbetweenperformanceand sFactorsExperiencesuggeststhat a number of successfactors(discussedin greater depth in thecountry papers) contributeto the developmentof an evaluationsystem-developing asystem shouldbe pursuedonly if many of these alreadyexistor if there are reasonableprospectsfor creatingthem. One exampleof a successfactor is the role playedby a'champion'ministryor agencyin supporting,encouragingand pushingthe developmentof an evaluationsystem.Bestresultsare seen with powerfuland influentiallead agencies,such as financeor planning ministries(as in Australiaand Canada),or a nationalauditoffice.Indonesiafound it valuableto havethe support of such centralagencies,as wellasa networkof committedsupportersin ministries.An explicitand high-profileevaluation strategycan also be effectivein sellingthe message,as can the support of individualministers or the Cabinetas a whole-particularly via ministerialor s alsoimportant.An perienceindicatesthat it can take at leasta decadeat thelevelto embedsucha systemin a sustainablemanner,to developwhole-of-governmentto makefulluse ofnecessaryskills,and set up tionfindings.Thecounterpartof the needforsustainedgovernmentsupportis theneedfor sustainedsupportfrom zesthe valueof ongoing,activeand visiblesupportof the WorldBankin Indonesia.approach (e.g.,Chile,Indonesia,Canada,Australia)has advanA whole-of-governmenttages in terms of achievingmomentumand helpingto ensurethat laggardministriesendeavorto keep up with leading ministries.This approachmightbe especiallyfeasibleif a major seriesof reformsis being contemplated,such as major changesto publicexpendituremanagement(i.e. budgetaryprocessesand decisionmaking)in response tobudgetaryimperatives.approachmight be unrealisticin a number of developingBut a whole-of-governmentcountries-it can be difficultto achieve.In these countriesa more modestapproachPublicSectorPerformance-The CriticalRole of Evaluationxiii
might be to start with an initialfocus on ongoingperformancemonitoring(in particularsectors or ministriesin order to createa demonstrationeffect),and then seek to applythe approachto other sectors/ministriesand to other performancemeasurement tools(such as evaluation)as opportunitiesare found or created.This type of sequencingimplicitlydepends on perceptionsof satisfactorybenefitsvis-avis the costs of the initial,more modestapproach.Chiletook such an incrementalapproach,followingan initial focuson performanceinformation,whichled to questionsabout program outcomesand causality.Thesetypes of questionscan only be answeredby summativeprogram evaluations.Perhapsthe key messagehere is the need to tailor approachesfor developingevaluationcapacityto suit the circumstancesand opportunitiesin differentcountries.It is definitelynot the case that "one-size-fits-all"The final lesson is that incentivesare crucialto ensuringboth that an evaluationsystemis developedand that evaluationfindingsare actuallyused.In workingwith nationalgovernmentsit is important for those providingtechnicalassistanceto understandtheincentiveframeworksin the country.This involvesconductingan institutional diagnosisas a precursorto provisionof advice,while also ensuringa closedialoguewith thegovernment(such an approachwas followedin Zimbabwe).This diagnosisshould alsoexaminethe extentof evaluationcapacityin a government-it is often the case thatgovernmentsover estimatethe extentand quality of their evaluationactivity.EduardoWiesneremphasizesthat existingincentivesoften work againstevaluation,andthat it is importantto identifywho stands to lose from havinginformationavailableonthe performanceof governmentactivities.Line ministries may perceiveevaluationfindingsas a threat becauseof fears that financeor planningministrieswilluse thefindingsto reducebudget appropriationsor demand accountability.This underscoresthe trade-offbetweencentral ministriesusing findingsto assist budget allocationsor foraccountability,versus managersusing the findingsto improveperformance.Who should be responsiblefor measuringperformance?Impartialoutsiders, or expertinsiders?The former approachstresses objectivityand independence,while the latterstresses expert knowledgeand ownershipof the evaluationresults,whichin turn, islikelyto encouragelearningby managersand their staff.One answer could be to emphasizelearning wherecivil serviceperformanceandxivKeithMackay
capabilitiesare believedto be reasonablygood and the institutionalenvironmentisjudged to be conduciveto improvement.The other option is to emphasizeaccountability especiallywhereperformanceis very poor or corruptionis a problem.In the latter case there are opportunitiesfor measuringperformanceto exposepoorperformanceof the civil serviceand thus to increasethe pressuresfor a more responsivepublic sector.This was carried out in Bangalore,India, for example,wherean NGOconductedsurveysof citizenviews concerningthe quality of governmentservicesandthe extentof corruptionin delivery.There is clearlyscopefor civil society(includingNGOs)to play a role in performancemeasurement.There may alsobe opportunitiesfor bilateraldonors to promote andsupport such activities.Implicationsfor the World Bankand Other DonorsThe developmentof evaluationcapacityis a long-termproposition,not amenableto asimple,two- or three-yearproject approach.Thus sustainedsupport and commitment,and the activeinvolvementof the country governmentis necessary.Developmentagenciesshouldviewthe provisionof technicalassistanceas part of a sustained,ongoingpartnership or dialoguewith the government.The role of donors is important-donorscan help or hinder the developmentof evaluationcapacity.Donorscan helpby providingtechnicalassistance-theiradviceand sharingof lessonsandas canprovidingfundsfor trainingandbuildingevaluationbestpracticecanbe invaluable,systems.Theirparticipationhelpsto buildconfidencewithinthe government.Donorscan impede the developmentof evaluationcapacityby excessive,conflictingormultipledonor requirementsfor evaluation.There is a particulardanger that scarcecountry evaluationcapacitywillbe divertedto satisfydonor requirementsfor theevaluationof developmentassistanceactivities,whichmight not necessarilyalignwiththe greatestareas of benefit from evaluationin the government.Newlendinginstruments,which providemulti-yearfundingfor technicalassistanceactivities,are likelyto be better tailored to meeting countryneeds for long-termcommitmentand supportin this area, as CherylGrayemphasizes.donors havea considerableand growingtrack recordin supporting suchCollectively,Pub
ix Public Sector Performance-The Critical Role of Evaluation Keith Mackay 1 PART 1: The Role of Evaluation in Development 3 Why Bother About ECD? Robert Picciotto 7 The Role of Evaluation Mark Baird 13 PART 2: The Missing Link in Good Governance 15 Evaluation Capacity and the Public Sector David Shand 19 PART 3: Experience of Developed Countries
May 02, 2018 · D. Program Evaluation ͟The organization has provided a description of the framework for how each program will be evaluated. The framework should include all the elements below: ͟The evaluation methods are cost-effective for the organization ͟Quantitative and qualitative data is being collected (at Basics tier, data collection must have begun)
Silat is a combative art of self-defense and survival rooted from Matay archipelago. It was traced at thé early of Langkasuka Kingdom (2nd century CE) till thé reign of Melaka (Malaysia) Sultanate era (13th century). Silat has now evolved to become part of social culture and tradition with thé appearance of a fine physical and spiritual .
On an exceptional basis, Member States may request UNESCO to provide thé candidates with access to thé platform so they can complète thé form by themselves. Thèse requests must be addressed to esd rize unesco. or by 15 A ril 2021 UNESCO will provide thé nomineewith accessto thé platform via their émail address.
̶The leading indicator of employee engagement is based on the quality of the relationship between employee and supervisor Empower your managers! ̶Help them understand the impact on the organization ̶Share important changes, plan options, tasks, and deadlines ̶Provide key messages and talking points ̶Prepare them to answer employee questions
Dr. Sunita Bharatwal** Dr. Pawan Garga*** Abstract Customer satisfaction is derived from thè functionalities and values, a product or Service can provide. The current study aims to segregate thè dimensions of ordine Service quality and gather insights on its impact on web shopping. The trends of purchases have
Chính Văn.- Còn đức Thế tôn thì tuệ giác cực kỳ trong sạch 8: hiện hành bất nhị 9, đạt đến vô tướng 10, đứng vào chỗ đứng của các đức Thế tôn 11, thể hiện tính bình đẳng của các Ngài, đến chỗ không còn chướng ngại 12, giáo pháp không thể khuynh đảo, tâm thức không bị cản trở, cái được
Le genou de Lucy. Odile Jacob. 1999. Coppens Y. Pré-textes. L’homme préhistorique en morceaux. Eds Odile Jacob. 2011. Costentin J., Delaveau P. Café, thé, chocolat, les bons effets sur le cerveau et pour le corps. Editions Odile Jacob. 2010. Crawford M., Marsh D. The driving force : food in human evolution and the future.
Le genou de Lucy. Odile Jacob. 1999. Coppens Y. Pré-textes. L’homme préhistorique en morceaux. Eds Odile Jacob. 2011. Costentin J., Delaveau P. Café, thé, chocolat, les bons effets sur le cerveau et pour le corps. Editions Odile Jacob. 2010. 3 Crawford M., Marsh D. The driving force : food in human evolution and the future.
