Tegegne et al. BMC Public Health 2018, 18(Suppl SEARCHOpen AccessThe role of supportive supervision usingmobile technology in monitoring andguiding program performance: a case studyin Nigeria, 2015–2016Sisay G. Tegegne1*, Faisal Shuaib2, Fiona Braka1, Pascal Mkanda3, Tesfaye B. Erbeto1, Aron Aregay1,Oyaole D. Rasheed1, Akpan G. Ubong3, Njie Alpha1, Ahmed Khedr1, Mirghani A. Isameldin1, Yared G. Yehushualet1,Charity Warigon1, Usman Adamu2, Eunice Damisa2, Bassey Okposen2, Peter Nsubuga4, Rui G. Vaz5and Wondimagegnehu Alemu1AbstractBackground: Supportive supervision is one of the interventions that fosters program improvement by way ofimparting knowledge and skills to health workers. The basic challenge in supportive supervision is the availabilityof data in real time for timely and effective feedback. Thus, the main objective of this study was to determine thecontribution of real-time data collection during supportive supervision for timely feedback and generation ofevidence for health intervention planning.Methods: We analyzed supportive supervision records collected through handheld devices employing the opendata kit (ODK) platform from July 2015 to June 2016. Supervision was conducted across the country by 592 WorldHealth Organization (WHO) officers. The availability of real-time data and the distance of health facilities to thecommunity were analyzed.Results: During the study period, 90,396 health facilities were supervised. The average time spent during supervisionvaried from 1.53 to 3.78 h across the six geopolitical zones of the country. The average interval between completion ofthe supervisory checklist and synchronization with the server varied from 3.9 h to 7.5 h. The average distance betweenthe health facility and a ward varied from 5 to 24 km.Conclusion: The use of handheld devices for supportive supervision provided real-time data from health facilities tostate and zonal levels for analysis and feedback. Program officers used the findings to rectify process indicators in timefor a better outcome.Keywords: Real-time data, Supportive supervision, Timely feedback, Health facility distanceBackgroundSupportive supervision is an effective strategy for continuously enhancing staff performance. It is carried out withthe focus on using supervisory visits as an opportunity toimprove the knowledge and skills of health staff . Astudy on the rise and fall of supervision in a projectdesigned to strengthen supervision of the integrated* Correspondence: [email protected] Health Organization Country Office, Abuja, NigeriaFull list of author information is available at the end of the articlemanagement of childhood illness in Benin emphasized theimportance of supervision for management purposes. Thestudy suggested that resources should be allocated to promote supervision and to remove the obstacles of supervision .The importance of supportive supervision in publichealth has been linked with improvements in programperformance. A study in Zimbabwe showed that, followingsupervision, overall drug management improved significantly compared with control groups. Similarly, a study inIndia showed that regular supervision of sessions and The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link tothe Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication o/1.0/) applies to the data made available in this article, unless otherwise stated.
Tegegne et al. BMC Public Health 2018, 18(Suppl 4):1317vaccine stores are an important step in ensuring qualityimmunization services [3, 4].Another study conducted in Nigeria, on supportivesupervision as an effective intervention in achievinghigh-quality malaria case management at the primaryhealthcare level, recommended that supportive supervision should be incorporated into existing frameworks forimproving healthcare worker performance .Furthermore, a desk review of publications and grayliterature on community health worker effectivenessshowed that supportive supervision strengthened theproductivity of community health workers .However, there is little evidence on technology-assistedmobile data collection for real-time data transfer duringsupportive supervision. However, evidence has been foundfor the effect of the modality of supervision on its quality.For instance, a study on the use of personal digital assistants for data entry in southern Tanzania showed thatelectronic methods of data collection were efficient fordata entry and merging .Similarly, mobile technology in health interventionshas shown a change in program management in southern Zambia. In another study, mobile phones were usedfor reactive case searches through timely and accuratedissemination of local surveillance information .The importance of timely data availability for promptcase detection and subsequent case management wasfound to be beneficial in a study in Pakistan. The studyshowed that the information exchange using mobilephones improved supportive supervision by enhancingcoordination among health workers for timely casereporting and appropriate follow-up .The experience with a large-scale baseline survey on research conducted on the use of mobile phones as a datacollection tool in South Africa suggests that the system ispreferable to a paper-based approach. The real-time quality control and supervision of data collection that are enabled by the use of a mobile phone-based survey systemmakes this an attractive management option .The added advantage of using mobile phones for collecting data during health facility supervision is theavailability of the global positioning system (GPS) withincurrent generation mobile phones or smartphones.Using GPS information, it is possible to obtain approximate distances between healthcare facilities, which couldinform proper health service provision. A study performed in Kenya on modeling distances traveled to government health services showed a link between theachievement of Millennium Development Goals and access to health facilities . Various studies have alsoshown that knowledge of the actual distance of healthfacilities from each other has policy implications. Astudy in Nigeria showed that the distance to primaryhealthcare facilities was a barrier to proper utilization ofPage 76 of 144Table 1 Average time of supportive supervision verses time tosubmit to server, July 2015 to June 2016, NigeriaZonesAverage time ofAverage time submittedsupervision in the field (h) to server (h)North-central zone 2.424.5Northeast zone2.057.5Northwest zone1.537.3Southeast zone2.764.3South-south zone3.784.2Southwest zone2.573.9National1.995.28health facilities. An assessment of the factors associatedwith utilization of immunization services in the community in a rural setting in Ethiopia showed that among thefactors significantly associated with full immunization ofchildren was the average walking time from home tohealth facilities. In families where the average distance was 1 h the immunization coverage was higher [12–14].We set out to determine the contribution of supportive supervision using mobile technology in providingtimely program information in the polio program inNigeria. We also measured the distance of the healthfacility to be accessed by the communities from the information gathered. We also identified how supportivesupervision contributes to the implementation of othernon-polio health programs in visited facilities.MethodsOutline of study methodWe analyzed supportive supervision findings collected bymobile phones under open data kit (ODK) platform fromJuly 2015 to June 2016. ODK is a free and open-source setof tools. ODK provides an out-of-the-box solution forusers to build a data collection form or survey, collect theTable 2 Proportion of supportive supervision conducted withgovernment counterparts, July 2015 to June 2016, NigeriaMonthProportion of joint supervisionsJuly 201555%August 201554%September 201553%October 201551%November 201551%December 201548%January 201656%February 201654%March 201656%April 201654%May 201654%June 201657%
Tegegne et al. BMC Public Health 2018, 18(Suppl 4):1317Page 77 of 144Fig. 1 Number of supportive supervision to health facilities with the number of AFP cases detected by health workers, July 2015 to June 2016, Nigeriadata on a mobile device, send it to a server, aggregate thecollected data on a server, and extract it in useful formats.The checklist for supportive supervision was uploadedto the mobile phones of World Health Organization(WHO) officers. The officers used this checklist tosupervise health facilities. The system replaced apaper-based data collection method. After data had beencollected at the field level, it was transferred to thecentral server. The data was then downloaded from theserver at the national and zonal level in real time. TheODK platform has inbuilt data analysis for prompt feedback. We measured the time lapse from data capture toserver upload to measure the availability of data foraction in real time. A time stamp was also embedded inthe platform to measure the time taken to perform supportive supervision. We analyzed the maximum, minimum, and average time spent measuring the quality ofsupportive supervision.A GPS coordinate was captured at the beginning andend of supportive supervision to measure the actual geolocation of the facilities visited. Using the GPS coordinates, we mapped the distance of each health facilityfrom the Local Government Area (LGA) headquartersand the nearest ward for the health facility supervised.We also performed trend analysis of the collected information on surveillance and routine immunization overthe study period.Supervision was conducted across the country by 592WHO officers. All officers were provided with mobilephones and given access to download a form for useduring supervision. At the end of each supervisory visit,the officers sent the data to a centrally managed server.Although supervision took place in all health facilitiesbased on surveillance prioritization, we analyzed supervisory data collected from all health facilities that conducted routine immunization in Nigeria during July2015 to June 2016. We also measured the contributionof polio structure in visited facilities for strengtheningroutine immunization.ProceduresA structured questionnaire was developed that consistedof general information on visited facilities, disease surveillance, and routine immunization activities. The questionnaire was pretested in Kano state to verify theaccuracy and flow of the questions and to assess theaverage time required to supervise a health facility. Timemeasurements and GPS coordinates were included inthe checklist. We provided hands-on training for 592WHO field officers on how to download a questionnaireon their mobile phones from a server and on how tosend a completed questionnaire back to a server. Datamanagers in the six geopolitical zones of the country(namely north-central, northeast, northwest, southeast,southwest, and south-south zones) were trained to assistin the process. We also provided training for troubleshooting during the process of data collection.Data analysisAll data sent to the server were downloaded at the zonaland national levels for detailed analysis and feedback.
Tegegne et al. BMC Public Health 2018, 18(Suppl 4):1317Page 78 of 144Table 3 Average distance of health facilities to nearest ward inkilometers by state; analysis of supervisory data using mobilephones, July 2015 to June 2016, NigeriaTable 3 Average distance of health facilities to nearest ward inkilometers by state; analysis of supervisory data using mobilephones, July 2015 to June 2016, Nigeria (Continued)ZoneStateAverage distance accessingfacilities (km)ZoneNorth-central zoneBenue17Zone average7FCT, teauZone averageNortheast Abia7Ebonyi11Enugu12Imo69Akwa Ibom8Bayelsa13Cross River17Delta11Edo17Rivers9Zone averageSouthwest zone8AnambraZone averageSouth-south zone16KadunaZone averageSoutheast zoneAverage distance accessingfacilities (km)Regular feedback was provided to all officers. Furthermore, the ODK platform was customized to produce instant analysis of each question in the supervisorychecklist for onsite feedback. The time taken to administer the entire checklist was also analyzed.18Zone averageNorthwest ltsA total of 90,396 facilities were supervised at least onceduring the study period from July 2015 to June 2016. Ofthe total supervisions conducted based on priority levels,86% were providing routine immunization services. During the supportive supervision, geocoordinates of healthfacilities were captured. The precision of actual healthfacility location captured with the system was within aradius of 100 m.The average time to conduct supervision variedfrom 1.53 to 3.78 h across the six geopolitical zonesof the country. The national average time was 1 h99 min. The average time to send the completedchecklist of supportive supervision to a central serveronce supervision was completed varied from 3.9 to7.5 h. The national average time for submitting thedata to the server as a measure of real time submission was 5.28 h (Table 1).Much of the supportive supervision by WHO officerswas jointly conducted with government counterparts. Asshown in Table 2, the proportion of joint supervisionwith government counterparts was above 50% for eachmonth during the study period. The highest proportionof joint supervision was 57%.As shown in Fig. 1, acute flaccid paralysis (AFP) caseswere detected in health facilities that benefited fromsupportive supervision during the study period. The analysis excluded health facilities that had conducted training during the study period. The only opportunity forthe health workers in those facilities was sensitizationduring supportive supervision. Of the total 6544 AFPcases detected during the study period, 1778 were inhealth facilities by focal persons that were not formallytrained but were supervised. The rest of the cases werefrom focal persons that were trained and supervisedduring the year.The average distance of health facility to a ward variedfrom 5 to 24 km. The maximum was in the northeastgeopolitical zone, while the lowest average distance ofhealth facilities to a ward was in the southwest
Tegegne et al. BMC Public Health 2018, 18(Suppl 4):1317Page 79 of 144Fig. 2 Average time spent on supportive supervision for routine immunization (RI) and surveillance; supportive supervision data 2015–2016,Nigeria. NCZ north-central zone, NEZ northeast zone, NWZ northwest zone, SEZ southeast zone, SSZ south-south zone, SWZ southwest zonegeopolitical zone. The national average distance betweenwards and health facilities was 13 km (Table 3).The average time spent at the national level was71 min for the routine immunization section of the supportive supervision, while for surveillance the averagewas 48 min. Among the six geopolitical zones in thecountry, the lowest average time spent for supportivesupervision was 55 min for routine immunization and37 min for surveillance. The northwest had the lowestrecord of time spent for supervisory support, while thesouth-south zone had the highest record. (Fig. 2).DiscussionWe sought to determine the real-time data availabilityfor timely feedback using mobile phone-assisted supportive supervision. We found that supportive supervision, using a handheld mobile data collection tool, wasperformed on average in 199 min and transferred to acentral level server within 5 h from more than 6000health facilities across the country. The evidence suggests that real-time data were available for action at thenational and zonal levels on the same day that supervision took place in health facilities. Program officers usedthe findings to rectify process indicators in time for abetter outcome.We found out that health workers detected AFP cases,and the detection correlated with the real-time supportive supervision to the reporting health facilities. We disaggregated the data to look at the pattern of reportedAFP cases in facilities that have not received formaltraining but received supervision as the only means ofsensitization to detect cases. The detection of cases waslikely due to sensitization during the supportive supervision of focal persons.Ours is not the only study to demonstrate a correlation between supportive supervision and change in outcome. A case study on supportive supervision onmonitoring and evaluation in Haiti by Marshall andFehringer  showed that the contribution of supportive supervision had a similar result on health workerproductivity. A similar study by Frimpong et al. inGhana  demonstrated that investment in supportivesupervision could help maximize the output of scarcehuman resources in primary healthcare facilities.We also observed that in all six geopolitical zones of thecountry more time was spent in supervising routineimmunization activity than on surveillance. As a result ofthis time spent in supervising routine immunization activities, improvements in process indicators such as a steadyincrease in the availability of updated monitoring chartsfor routine immunization, recording and reporting tools,and availability of supplies for routine immunization wereobserved. This finding is in line with Reaching EveryDistrict (RED) approach where supportive supervision is acomponent for improving routine immunization .Also, this finding could be used for mainstreaming thepolio functions to routine immunization activities.The findings of distances of health facilities to beaccessed by the community could be used to plan healthinterventions to the community. A similar finding wasrecorded in a study conducted in Bangladesh on acost-minimization approach to planning the geographical distribution of health facilities . The study identified the distance to a health center as the mostimportant factor determining utilization.
Tegegne et al. BMC Public Health 2018, 18(Suppl 4):1317One of the limitations of our study was the associationbetween improvement in surveillance and routineimmunization indicators with supportive supervision.Improved program output could also be associated withtraining of health workers and sensitization sessions. Although it is difficult to control all contributing factors,we did exclude all health facilities in the analysis wheretraining for health workers took place 1 year prior to thestudy period.ConclusionsSupervision using mobile technology has contributed toproducing real-time data. The system has been used to locate the actual location of facilities and the distance of ahealth facility from a given point. It helps to give timelyfeedback and plan health interventions properly. The system of using mobile phones could be utilized to monitorprogram implementation for other health interventionssuch as defaulter tracing in routine immunization, sendingalerts for outbreak-prone diseases, investigation of outbreaks, and supervision of any other health interventions.We recommend that a study be conducted to assessthe impact of utilization of inbuilt feedback by supervisors to give real-time feedback to each health facility supervised at the point of supervision.AbbreviationsAFP: Acute flaccid paralysis; GPS: Global positioning system; LGA: LocalGovernment Area; ODK: Open data kit; RED: Reaching every district;WHO: World Health OrganizationAc
RESEARCH Open Access The role of supportive supervision using mobile technology in monitoring and guiding program performance: a case study in Nigeria, 2015–2016 Sisay G. Tegegne1*, Faisal Shuaib2, Fiona Braka1, Pascal Mkanda3, Tesfaye B. Erbeto1, Aron Aregay1,