Youth Peer Assessing The Education Toolkit Quality Of Youth Peer .

nnCommunity involvement. The degree of cooperation between a YPEprogramme and the local community where it operates, including variousstakeholders, is important. Broad community support is critical to programmeproductivity and sustainability because it increases the motivation of youthpeer educators and involvement of parents as well as the responsiveness ofthe programme to the community and its institutions.Technical frameworks. The CDC framework, adapted here for YPEprogrammes, includes four parts: programme design, implementation,management, and responsiveness to the target audience. In this case, thetechnical framework needs to be responsive to the audience from whichthe peer educators are recruited and where activities are conducted. Thetechnical or operational frameworks of YPE programmes have the samerequirements as non-YPE programmes but with the added responsibility ofdirectly involving young people.How to use this tool: the assessment processThe assessment involves three major steps.1. Plan the assessment and assemble and train the team. Careful planningis needed, including identifying the team members and providing training(particularly on interviewing skills).2. Conduct interviews. The assessment team conducts interviews with thevarious stakeholder groups, including the peer educators themselves. Theteam compiles the data and impressions from the interviews into notes thatcan be referenced. This tool includes sample interview questions to be usedwith peer educators, staff and management, parents, and stakeholders.3. Complete checklists and develop report. The team uses the notes andteam meetings to complete the checklists, as well as other information onthe programme that the team gathers. Included in this tool are the eightchecklists, along with a suggested instrument, Summary of Checklist Results.How this tool is used depends on several factors: the goal of the programme,the stakeholders involved, available resources, logistics (e.g., how close thepeer education projects are), the structure of the sponsoring organizations, thecommunities, donor needs, and others. Ideally, an assessment team would taketwo to three days to conduct interviews and complete the checklists. This couldvary depending on the size of the programme, availability of the interviewees,and other logistical issues.Introduction

If a project must reduce the number of interviews in order to shorten theprocess, it is most important to interview the peer educators themselves andthe programme staff and management. But, without interviews with the otherstakeholder groups, a full picture of the programme will not be available.Interviewing parents and community stakeholders will add depth to the findings.The tool can be used as an ongoing monitoring approach, if that is useful for theprogramme. If used as part of a formal evaluation, the data gathered using thistool should be part of a baseline and end-line assessment, which could includeother types of data about the project.Before the assessment team begins its work, a briefing about the assessmentshould be done with the programme and all of its participants, especially thoseindividuals and groups to be interviewed. It may be necessary to notify or obtainpermission from the parents of the peer educators to be interviewed, dependingon their age.The briefing should include the purpose of the assessment and how the resultswill be used (and by whom). The assessment procedures and the types ofquestions to be asked should be discussed. The issues of confidentiality andanonymity need to be explicit. An official letter from a government agency (e.g.,the ministry of health) giving permission for the assessment is necessary inmany countries.Other types of activities related to the assessment are not discussed in this tool.For example, this tool does not offer details on how to conduct programme fieldassessments in general, such as gathering background materials, writing reports,and other necessary steps. An excellent reference on field assessments is theRapid Assessment and Response Technical Guide compiled by the World HealthOrganization.3 Similarly, this tool does not discuss action plans that might betaken as a result of using this tool. A sample action plan format is included inClinic Assessment of Youth Friendly Services by Pathfinder International.4 Assessing the Quality of Youth Peer Education Programmes

Step1Plan Assessment, Assemble and Train TeamA donor, a stakeholder institution, a YPE programme, or an evaluator willidentify the programme(s) for the assessment and a team leader. The team leaderconducts a desk review of programme descriptions, reports, work plans, manuals,and strategy documents, including issues related to organizational structure,operations, stakeholders, and collaborators in the community. If the team leaderis not based in the area where the assessment is to be done, a pre-assessment visitto the programme site is highly recommended to gather programme details andintroduce the assessment to project staff, peer educators, and stakeholders.The ideal assessment team would be composed of adults and youth experiencedin youth peer education and in working with youth, with two adults and twoyoung people of mixed sexes. This demographic mix can help facilitate opendiscussions about youth-adult and gender perspectives. The team will need to betrained to use the interview guides and checklists (see Steps 2 and 3).The team members should have the basic skills needed for such an assessment,including local language fluency, interviewing skills, note-taking and reportwriting abilities, and related professional experience. Also, they must betrained in and adhere to ethics regarding confidentiality and special issues forinterviewing youth. The content of the interviews can contain personal views orsensitive information that must remain confidential. The sharing of informationgained in interviews is not only unethical but can cause damage to individuals ora programme. The integrity and objectivity of the team is crucial to the success ofthe assessment.Depending on the skills and experience of the team members, training couldrange from one to three days. At the least, all members need to be familiar withthe interview guides, the checklists, and the basic structure of the programmesbeing assessed. During the training, the team should review the interview guides(see Step 2, below) and adapt them to the local programme and its environment.Other training and planning may involve role plays between adult and youthteam members to help them work as a team during interviews, backgroundsessions on the topics covered by the checklists (such as youth-adultpartnerships), and clearly developed approaches for recording notes from thesessions. The training should be interactive with periods of discussion andfeedback, conducted by the team leader or a consultant.Plan Assessment, Assemble and Train Team

Step2Conduct InterviewsThe assessment team conducts interviews with the various groups supporting theprogramme, including the peer educators themselves. The interviewers should beexperienced in interviewing and know the local culture. The teams will work inpairs during the interviews (an interviewer and a note-taker, ideally one youth andone adult). Skilled note-taking is important because the notes will be used later forthe checklist ratings.Group or individual interviews can be used. Group interviews require less timeand personnel and will generally suffice for nearly all groups, except for interviewswith key staff such as programme coordinators, trainers, and management.Responses from these groups tend to be more open and frank when interviewedwithout the presence of peer educators or their supervisors. Some stakeholdersmay have to be interviewed individually because of distances involved. Parentscan be interviewed as a group if this approach can be coordinated. Peer educatorsshould be interviewed in small, interactive groups of no more than 10 people. Inmany cultures it may be necessary to interview them in both mixed-sex andsingle-sex groups.Many items on the checklists may be considered too direct or personal to ask.During the piloting of the checklists, four interview guides were developed tohelp interviewers ask about such personal or difficult questions. The four guidesare organized according to the group to be interviewed: peer educators, staff andmanagement, parents, and stakeholders. The guides for the peer educators andstaff and management are organized to a large extent by the checklist themesbecause comments from these two groups relate to most of the themes. The two forparents and stakeholders are more general about their involvement with the peereducators and the programme’s operation. The guides are the first instrumentin this publication.How the team uses the interview guides will vary, depending on the experience ofthose being interviewed and the type of issue under discussion. Some participantsmay respond to direct questions while others will need to be gently probed.Questions about gender roles or youth-adult partnerships may be more contentious,generating more opinions than straightforward questions with specific answers,such as the number of education sessions held per week.10Assessing the Quality of Youth Peer Education Programmes

How to use the interview guidelinesThe interview team should read the interview guidelines thoroughly before conducting the interviews.Then, depending on the education, experience, language, time allotted, and other issues, the teamshould adjust the questions so that they form the basis for a discussion. These questions are notmeant to be used as a questionnaire. The questions are written in a style to be understood by theinterview team. They will need to be adjusted for the interviewees – particularly for peer educatorswith less experience with issues such as gender equity and youth-adult partnerships.In some cases, the assessment team may ask interviewees to give a rating of 1 to 5to a specific item on a checklist. In other cases, the interview approach will needto be more circumspect, with indirect and then probing questions. The interviewguides are not structured as a questionnaire with the items on the checklist to becompleted as one would complete a survey. So, the interview team will need tosort through the notes from the interviews and use the material as appropriate toanswer the checklist items.The perspectives of young people and adults can be very different and evencontrary. The piloting process demonstrated the importance of having both ayouth and adult team member present during all interviews. This allowed theteams to cross-reference their perspectives during the ratings and to switchinterviewer/note-taker roles if necessary. In some cases, peer educators mayfeel too intimidated or respectful towards adult interviewers to give honestanswers; having a youth as part of the interviewer team encourages moreaccurate responses.Peer educators should be interviewed in both mixed-sex and same-sex groups.Many of the issues addressed in the checklists are directly or indirectly relatedto gender issues. Depending on the group and the culture, this can lead toconsiderable debate among the young people or even silence. If genderissues have not been addressed in the programme, young people may lack thevocabulary to discuss the subject and need to be probed. The piloting processrevealed that when the interviewer and interviewee are of the same sex,especially for gender-related questions, more helpful responses resulted.Below are recommendations regarding the subgroups to be interviewed for eachof the checklists. Note that the term ‘programme staff’ and ‘programme staffConduct Interviews 11

and management’ are similar but not necessarily interchangeable. Some YPEprogrammes are attached to larger organizations. In that case, interviewing themanagement of the larger organization may be important.nStakeholder Cooperation ChecklistStakeholders, programme staff and management, peer educatorsnParental Involvement ChecklistParents, peer educators, programme staffnYouth Involvement ChecklistPeer educators, programme staff and managementnYouth-Adult Partnerships ChecklistPeer educators, programme staff and management, parents, stakeholdersnPeer Educator Cooperation ChecklistPeer educators, programme staffnGender Equity and Equality ChecklistPeer educators, parents, programme staffnCommunity Involvement ChecklistStakeholders, peer educators, programme staff and managementnTechnical Frameworks ChecklistProgramme staff and management, peer educators, stakeholdersBecause team members will probably not be able to interview all the peereducators, parents, or stakeholders in a programme, they will need to selectinterviewees using a sampling strategy. There are several methods for samplingthe interviewees; these are not covered in this manual (see the WHO assessmenttool mentioned on page 8).5 However, there are strategic considerations whensampling YPE programmes.The peer educator sample should include:n Recently recruited as well as trained and active youth, i.e., not only ‘oldtimers’n Females and males proportional to the programme itselfn Youth representing age, ethnic, and geographical diversity of the programme12Assessing the Quality of Youth Peer Education Programmes

The stakeholder sample should include organizations and individuals that:n Work actively with a programme and have an investment in its successn Work directly with the peer educators, especially schools, clinics, andyouth centersn Are community opinion leaders and decision-makers such as faith leaders,traditional leaders, and governmental decision-makers, such as ministerialrepresentativesThe sample of parents should include those who are active or invested in theprogramme as well as those representing the same community diversity as thepeer educators.The interview teams should share the results of the interviews and lessons learnedduring daily debriefings with the entire team. The team leader is central to thisprocess and needs to take responsibility for ensuring that debriefings take place.All of the interviews with the various stakeholders need to be completed beforebeginning the step of completing the checklists.7. Youth-initiated and directed6. Adult-initiated, shared decisions with youth5. Consulted and informed4. Assigned but informed3. Tokenism2. Decoration1. ManipulationConduct Interviews13Degrees of ParticipationHart’s Ladder6 is aconceptual framework theteam might use in interviewswith peer educators. Theinterview team should befamiliar with the concepts inthis ladder and find ways toincorporate these ideas intothe discussions regardingprogramme managementand staff.8. Youth-initiated, shared decisions with adultsNon-participationHart’s Ladder ofParticipation

Step3Complete Checklists and Develop ReportThe team members will need to determine a process for moving from theinterviews to completing the checklists. The team will need to review the resultsof the interviews (both verbally and by sharing notes), compare and discussfindings from the different groups, and make the final rating for the items on thechecklists.Different groups may have different views on the same issue. For example, peereducators, programme staff, and stakeholders may have different opinions aboutthe quality of youth-adult partnerships in a programme. These different opinions,along with the other information from the assessment, need to be considered bythe team when discussing the items in the Youth-Adult Partnerships Checklist.A five-point scale used for rating each checklist item can be classified as:1-2 Low3 Medium4-5 HighA low of 1 and a high of 5 should only be used in extreme cases. In mostassessments, the ratings should be based on a three-point scale where 2 indicateslow, 3 medium, and 4 high.The checklist items have a small space for notes that can be used for importantcomments in the final draft of the checklists. Longer comments that provide abackground or justification for the rating should be provided separately andnumbered according to the checklists, i.e., checklist 2, item 5. Such commentsare highly valuable to the organization requesting the assessment and theprogrammes. The comments should always be provided when available.The not applicable (N/A) column is marked if, for some reason, the item couldnot be rated. For instance, the item may not be applicable to the programme orthe interviewee(s) did not provide adequate responses.The results from the assessment will provide valuable information for everyoneinvested in the programme. They need to be reported with clarity, in order toprovide guidance for those involved to make improvements. Where there aresubstantial differences in perspectives among the various groups interviewed,the notes should address these differences. Differences in scores may simplyreflect different points of view. Or, they may reflect a deeper, more systemic lack14Assessing the Quality of Youth Peer Education Programmes

of cooperation among those most involved in the programme. Perhaps the mostimportant overall quality to measure is the degree of cooperation among thesegroups and their support for the programme and its goals.The completed checklists can be compiled into a Summary of Checklist Results.A final score is determined for each of the eight checklist areas by totalingthe rating number for all items and dividing by the number of items rated. Forthe Technical Frameworks Checklist, the scores are calculated and reportedfor each subcategory, i.e., programme design, implementation, management,and responsiveness. This summary, together with other information gatheredduring the assessment, can serve as the basis for the overall conclusions andrecommendations.A suggested outline for a final report follows:nnnnBackground information. This section provides the dates of the assessment,number of individuals interviewed in each category, and names andaffiliations of the assessment team members. In some cases, it may beappropriate to include names and affiliations of those interviewed, such askey community stakeholders, donors, and management. It is important tomaintain confidentiality in interviews, especially with the peer educators andstaff, so including names of those interviewed should be done only with theirapproval and after weighing carefully any benefit from doing so.Programme description. The programme description provides backgroundinformation gained from programme documentation, preliminary visits, andthe assessment, including the goals of the programme, the issues addressed,and the context in the community. It summarizes the programme strategies interms of target population, objectives, strategic approaches, delivery systems,and peer education recruitment and training. It identifies the collaboratingpartners, stakeholders and gate keepers, and donors. It also coversmanagement issues such as the position of the YPE programme within a largerorganizational structure.Assessment results. This section includes the summary of the checklist results.It might also include the full checklists and selected notes for backup support,depending on the scope and length of the report.Recommendations. This section addresses improvements needed, suggestionsfor further action, resources needed, potential obstacles, and person(s)responsible for actions.Complete Checkl

is part of a Youth Peer Education Toolkit, developed in collaboration with the United Nations Population Fund (UNFPA), for the Y-PEER (Youth Peer Education Network) Programme. YouthNet is a five-year programme designed to improve reproductive health and prevent HIV among young people. The YouthNet team is led by FHI and includes CARE USA and RTI