Mid-point Evaluation Of The UK Public Health Rapid Support Team (UK-PHRST)

Evaluation key findingsBelow is a summary of the main findings for each evaluation question (EQ), by workstream.1WORKSTREAM 1: Design: Model and StrategyEVALUATION QUESTION 1How appropriate is UK-PHRST’s integrated model andconsortium approach in contributing to improvedoutbreak response? The novel approach of combiningoutbreak response deploymentswith research and capacity buildingis ahead of the curve andconsidered valuable, but itsappropriateness cannot yet be fullyassessed as strategies are stillevolving and implementationlimited. The model effectively utilises anddevelops outbreak responsespecialists across differentdisciplines, and there areoperational benefits to having apermanent team available fordeployment and related research,such as increased internal andexternal knowledge sharing toinform and improve futureoutbreak response. The consortium approach broadensaccess to expertise and to existingconnections and projects in lowand middle-income countries(LMICs), which has allowed UKPHRST to build on existing positiverelationships in LMICs andsupported operationalisation of UKPHRST’s activities and work acrossthe triple mandate.EVALUATION QUESTION 2To what extent are UK-PHRST activities relevant, strategic andappropriate in relation to UK-PHRST programme goals? UK-PHRST’s ability to be strategic inthis first phase of programming hasbeen somewhat constrained bybeing a new entity formed ofinstitutions with very different waysof working, and needing to learnand reflect on its strategy as it hasevolved. There is still a lack of clarity andcohesion around areas of UKPHRST’s approach, particularly inrelation to research and capacitybuilding, within UK-PHRST andacross its stakeholders. Three key areas were identifiedthat need strengthening to ensureUK-PHRST is able to implement itstriple mandate and achieve itsgoals: i) maintaining and developingprocesses to deliver strategicpriorities, ii) building strategicpartnerships to enable delivery ofthe triple mandate model, and iii)ensuring alignment with nationalprocesses such as the Joint ExternalEvaluation (JEE) and associatedNational Action Plan for HealthSecurity (NAPHS).xi

2WORKSTREAM 2: Implementation: Delivery, Process and PartnershipsEVALUATION QUESTION 3How successfully has UK-PHRST been operationalised? UK-PHRST’s activities and outputshave largely been achieved or are ontrack for output milestones. For thefirst 18 months, UK-PHRST was in theinterim set-up phase and the firstdeployment was conducted in April2017. From this point onwards, UKPHRST demonstrates ongoingprogress against activities for all triplemandate areas. While deploymentsand research activities are overallprogressing well, capacity buildingactivities have incurred some delays. UK-PHRST is a highly professional,expert team, who are building astrong reputation for high-qualitywork in outbreak response.Theconsortium has not yet fullymanifested a unified UK-PHRSTidentity, which impacts on bothHowever, the current team model hasstruggled to respond to demandsacross the triple mandate andrequests from external parties. Thishas had negative implications in termsof skills gaps against deploymentdemands, has led to differentialdemands upon individual team hasregularly taken stock of the demandsof the triple mandate model andpartners’ requests, and has madeefforts to address some of the keychallenges, with revised strategiesbeing drafted or revised to identifyUK-PHRST’s priorities moving forward. UK-PHRST’s governance and reportingstructures are perceived by some coreteam members to be complex, andmay contribute to tensions betweenPHE and LSHTM. Governancestructures and ways of working haveensured effective oversight ofresearch and deployment portfolios,but there has been less focus oncapacity building activities. Reasonsincluded an operational need toprioritise deployments and researchactivities during the early stages of theUK-PHRST, which contributed to adelay in establishing capacity buildingpriorities for the programme, alongwith internal UK-PHRST governancearrangements and lack of clarity onorganisational responsibilities forcapacity building. Management andreporting systems have struggled toadapt and provide the necessaryflexibility to deal with the highpressure nature of UK-PHRST’s work,leading to team frustrations whichare further challenged by thedisperse locations and regular travelschedule of key staff. The consortium model has conferredmany benefits for UK-PHRST and is animportant driver of success.Collaboration between the academicpartners has been generally positiveand occurs across the triple mandate,although to differing degrees.Collaboration and coordinationbetween PHE and LSHTM as the mainpartners have been more challengingdue to differences in organisationalculture, management systems and theteam’s disperse physical locations.UK-PHRST has made efforts toaddress these challenges, althoughthe evidence suggests this has notbeen entirely successful, especially interms of internal communicationbetween the consortium partners.The consortium has not yet fullymanifested a unified UK-PHRSTidentity, which impacts on bothinternal and external relationshipsand communication. External communications havehelped UK-PHRST to become morevisible and respected among somekey UK and international GHSstakeholders, including GlobalOutbreak Alert and ResponseNetwork (GOARN) and LMICgovernments where they havedeployed bilaterally. There are somechallenges to externalcommunications due to politicalsensitivities and securityconsiderations around GHSdeployments. There is opportunityduring the current revision of thecommunications strategy to considerthese challenges and improve UKPHRST’s internal joint sense ofidentity to further enhance visibility,ensure that the team are fullyutilised, that the triple mandate canbe fulfilled, and that their work isproperly attributed.xii

2WORKSTREAM 2: Implementation: Delivery, Process and PartnershipsEVALUATION QUESTION 4To what extent does UK-PHRST complement or duplicateother UK ODA health security? Although close collaboration andalignment of activities across HMGGHS actors is widely acknowledgedas important, existing mechanismsat central level do not allow for fullcross-programme learning, and ingeneral do not translate intoeffective communication,coordination and collaboration atcountry level. Similarly, there isfragmentation and lack ofcoordination across the various UKdeployment mechanisms, andopportunities for collaboration toreduce potential duplication ofefforts or inefficiencies are beingmissed.EVALUATION QUESTION 5To what extent has UK-PHRST supported coherent andcollaborative national and international health activitieson response? UK-PHRST operates within acomplex international GHSlandscape and is only one ofnumerous actors supporting LMICsin epidemic preparedness andresponse. UK-PHRST has built onexisting collaborative partnershipsand forged new ones with LMIC,regional and global actors and isseen as a reputable, highly skilledand valuable partner. However,there is still need for increasedawareness and visibility of UKPHRST and continued focus onrelationship building with keystakeholders at all levels.Image credit: UK-PHRST team photoxiii

3WORKSTREAM 3: Performance: Results, Sustainability and AccountabilityEVALUATION QUESTION 6What contribution are UK-PHRST’s deployment, research and capacity buildingoutputs making to achieve programme outcomes? As discussed in our Inception Report, wehave not carried out contribution analysis atmid-point. Moreover, the current UK-PHRSTMonitoring, Evaluation and Learning (MEL)framework is not adequately capturingchanges at the outcome or impact level.Evidence suggests however that UK-PHRSThas made a difference in terms of speed andquality of UK response to outbreaks inparticular. There are also some earlyindications to suggest that, as a result ofUK-PHRST’s more rapid UK deployment,research and capacity building, in somecountries and key supportinginternational partners’ responses tooutbreaks may have been strengthened.In some occasions, external factors suchas politics and national rules andregulations, conflict and insecurity, andlack of a sufficient number of studysubjects have sometimes hinderedcontribution to outcomes.EVALUATION QUESTION 7Are programme outputs and outcomes likely to be sustained?Image credit: c-republic-congo-and Sustainability concerns have not beenadequately embedded in the UK-PHRST’sstrategy or implementation plans. UKPHRST’s relative reduced focus on thecapacity building component in a context oflimited human resources has hamperedprospects for sustainability. There is nosystematic action plan/needs assessmentcoming out of deployments and nosystematic linking up with the InternationalHealth Regulations (IHR) or other capacitybuilding initiatives. There is agreement thatforming long-lasting relationships is key toincreasing the chances of project outcomesbeing sustainable.xiv

3WORKSTREAM 3: Performance: Results, Sustainability and AccountabilityEVALUATION QUESTION 8To what extent has UK-PHRST followed the NAO principles ofeconomy, efficiency and effectiveness and demonstrated VfM? Overall, there is adequate to goodevidence to suggest that appropriateprocesses are in place to ensure thedelivery of VfM, with furtherattention required in some areas. UK-PHRST’s approach to economyhas been assessed as adequate.There have been efforts to ensurethat appropriate procurementprocesses have been implementedto ensure VfM. This has resulted inthe procurement of high-qualityinputs. In terms of efficiency, appropriateprocesses are in place to trackabsorption and measures are beingconsidered to monitor efficiency. Todate, despite some underspend,there has been strong performanceagainst output indicators. As far as effectiveness, a high-leveltheory of change (ToC) is in placewith some evidence to validate thecausal pathways for theachievement of outcomes. There is,however, greater uncertaintyaround capacity building. Equity has been considered in theproject design although there islittle evidence that this has beentranslated into implementationpractices where activities aredesigned to target vulnerablegroups and promote genderequality.EVALUATION QUESTION 9Is UK-PHRST capturing the right data to measure results andensure transparency and how can this be improved? Since developing the ToC for thepurposes of this evaluation, UKPHRST has been through a strategicreview process and further revisionsto the ToC may be required. Monitoring, evaluation and learning(MEL) systems are currently outputfocused and could be strengthenedto better capture evidence andmeasurable outcomes and impact. In terms of transparency, UK-PHRSTmeets self-reporting InternationalAid Transparency Initiative (IATI)transparency standards anddemonstrates improvements from40–59% (fair) in 2017/18 to 60–79%(good) in 2018/19.xv

MID-POINT EVALUATION OF THE UK-PUBLIC HEALTH RAPID SUPPORT TEAM – EXECUTIVE SUMMARYEvaluation conclusionsThe UK-PHRST model is still valid. The idea of combining response, research andcapacity building in a readily deployable team still holds. UK-PHRST appears to bethe only full-time team dedicated to outbreak response with an explicit mandate tocombine deployments with research and capacity building into a single offer in theGHS landscape. Across the board, the model is still seen as unique, pioneering andessential for influencing the outbreak research agenda globally and strengtheningcountries’ ability to respond quickly and effectively, especially considering that theGlobal Preparedness Monitoring Board (GPMB) recently warned that ‘current effortsremain grossly insufficient’ despite the progress made by the internationalcommunity in preparing to face health emergencies 2.Image credit: UK-PHRST DRC Ebola outbreak response team working with localcounterparts to review outbreak data and existing toolsAs discussed in the findings’ session, UK-PHRST has been successful in: Establishing a highly-professional and wellrespected team of experts from well-respectedinstitutions, with valuable existing connections andreputations. In doing that, UK-PHRST havedeveloped positive relationships with GOARN andnational governments, who report improved speedand effectiveness of outbreak response when UKPHRST are deployed. This has contributed to greaterexpert-readiness, albeit mainly at the level ofindividual experts. Mobilising a permanent team focussed on outbreakresponse across the triple mandate, which hashelped to support outbreak-related research. Thishas already contributed to the global evidence base,and has enhanced learning and sharing across UKPHRST and the broader GHS landscape, thuscontributing towards better research readiness. Providing invaluable access to the consortiumpartners’ pre-existing and positive relationshipswith LMIC stakeholders. They have had somesuccess in effectively building on these existingnetworks to identify successful capacity buildingactivities with the potential to contribute to greaterexpert readiness on the ground and potentiallyproviding opportunity for greater sustainability ofoutcomes.2 GPMB. 2019. A world at risk. Annual report on global preparedness for health emergencies. Available at: https://apps.who.int/gpmb/assets/annual report/GPMB annualreport 2019.pdfxvi

The full potential benefits of the model have not materialised yet for a number of reasons: Limited implementation and funding period.Although donor funding tends to be short term,programme outcomes take time to materialise, andeven more so in fields such as research and capacitybuilding. DHSC should bear this in mind whenreflecting on this first phase of the programme andplanning for any subsequent phases. Operationalisation of the concept of the triplemandate is still a work-in-progress. When theprogramme started in 2016, there was no BusinessCase, ToC or logframe in place. A ToC and a logframewere only established ex-poste and the ToC is stillcurrently under revision. The MEL system is still alsounder revision and key strategies, such as theresearch and capacity building strategies, have justrecently been developed. Others, such as asustainability strategy, are still missing. Difficulties working across institutions andcultures. Issues around governance, managementand communication have limited effectivecoordination, collaboration and cross learning Challenges around the capacity and skills of the UKPHRST team relative to the demands of the modeland requirements for individual deploymentswithin the triple mandate model. In a context inwhich contributing to outbreak responses isperceived by most as the primary mission of UKPHRST and limited human resources, involvement infrequent and sometimes repeated deployments(such as in the case of DRC) has resulted in less focuson or delays to the other two components of thetriple mandate, especially capacity building. Modality of deployment: Most requests fordeployment have come from GOARN and this canlimit UK-PHRST’s ability to deliver a more strategic,cross-HMG UK response and/or opportunities toinfluence or ability to integrate research andcapacity building into outbreak response. Weak communication and coordination with otherHMG GHS actors. UK-PHRST’s efforts to work withother HMG GHS programmes within LMICs has sofar had limited success. There is still a need for morecollaboration between UK deployment mechanismsto remove the risk of duplication and to build onsynergies with other HMG GHS programmes,including the PHE IHR Strengthening project. Tension between visibility and recognition againstalignment and coordination with other actorsinvolved in the response. There is still limitedawareness of the UK-PHRST at country level whenthe team is deploying through GOARN. Enhancingvisibility will be dependent on longer-terminvestments in relationship building, particularly atthe country and regional level. MEL systems need strengthening in order tosupport measurement of progress towards desiredoutcomes and support learning and adaptivemanagement. While recognising that research andcapacity building outcomes require time to fullymaterialise, the UK-PHRST needs to revise andstrengthen the way it tracks progress against its ToCto demonstrate and ensure contribution to thedesired long-term changes going forward. Sustainability warrants some special attention.Capacity building has received less focus within thetriple mandate, which poses questions in terms ofthe sustainability of UK-PHRST's outcomes. Equity considerations need to be more routinelyintegrated into project design and decision making.Integrating equity and human rights considerationswithin UK-PHRST’s operations would support greatereffectiveness of interventions.While UK-PHRST remains fairly unique, these issuesare not uncommon in the internationaldevelopment space. The 2018 ICAI review on ‘The UKaid response to global health threats’3 highlighted ‘ageneral need for improvements in cross-governmentcollaboration and communication’. Tensions versusshort term development funding and the challenge ofbuilding sustainability are also well documented4,while countless organ

This is the mid-point evaluation report. The report has been revised upon reception of feedback from UK-PHRST and following a co-creation of recommendations workshop that took place on 17th February 2020. An end-point evaluation report is due in early 2021. The report presents findings and conclusions from the three evaluation