Evaluating Community Projects: A Practical Guide

Evaluating community projectsA practical guideMarilyn Taylor, Derrick Purdue, Mandy Wilson and Pete WildeThese guidelines were initially developed as part of the JRF Neighbourhood Programme.This programme is made up of 20 community or voluntary organisations all wanting toexercise a more strategic influence in their neighbourhood. The guidelines were originallywritten to help these organisations evaluate their work. They provide step-by-step adviceon how to evaluate a community project which will be of interest to a wider audience.ContentsAll about evaluation2What is evaluation?2Why evaluate?2Key principles of evaluationHow to evaluate: a step-by-step approach3Step 1: Review the situation3Step 2: Gather evidence for the evaluation4Step 3: Analyse the evidence6Step 4: Make use of what you have found out6Step 5: Share your findings with others7Annex A: Reviewing the situation (example)8Annex B: Reviewing the situation (record sheet)10Annex C: Evaluation planning table11

All about evaluationWhat is evaluation?Put simply, evaluation by members of a project ororganisation will help people to learn from theirday-to-day work. It can be used by a group ofpeople, or by individuals working alone. It assessesthe effectiveness of a piece of work, a project ora programme. It can also highlight whether yourproject is moving steadily and successfully towardsachieving what it set out to do, or whether it ismoving in a different direction. You can thencelebrate and build on successes as well as learnfrom what has not worked so well.Why evaluate?Although evaluation may seem like an unnecessaryadditional task if you are already short of timeand resources, it can save you both time andresources by keeping participants focused on, andworking towards, the ultimate goal of the project.If necessary, it can refocus activity away fromunproductive or unnecessary work.Evaluation can help you to: learn from your experience; record what you have learnt, and share itwith other stakeholders; check your progress; check whether what you are doing is stillwhat local people want or need; identify strengths and weaknesses in yourproject; create a basis for future planning; demonstrate whether you have used yourresources – time and money – effectively; explain to funders, and others involved inyour work, what you have achieved and howsuccessful it is.2Evaluating community projectsKey principles ofevaluationEvaluation is most effective when: it is a continuous (not just one-off) processinforming planning and delivery as theproject develops; it involves all those with an interest in theproject in defining the questions they wantanswered; it uses imaginative and creative approaches,which engage those involved; it helps projects to be more accountable tothe wider community; it is used to challenge discriminatory andoppressive policies and practice, and toovercome inequality and disadvantage; it highlights and celebrates successes andachievements; it encourages an honest appraisal ofprogress, so that you can learn from whathasn’t worked as well as what has.

How to evaluate: a stepby-step approachStep 1: Review the situationEvaluating a project is like taking a journey. Firstyou need to plan your journey – you need todecide where you want to get to; then work outhow you are going to get there, and what you needto equip yourself with for the journey; you will alsoneed to look out for the signposts and milestonesthat will tell you whether you are on the right roadand whether you are likely to get there on time.When starting to evaluate a community project, itis helpful to begin by writing down how you aregoing to tackle each stage of this journey.Annex A (pages 8-9) is a table reflectingthe range of community empowermentissues being faced by projects in the JRFNeighbourhood Programme. It was producedto help projects think about what they weretrying to achieve,Annex B (page 10) is a blank, expandedversion of the same table. It can be used byany project, tackling any issue. It provides theframework for your own evaluation process.The best way of using it is to bring together asmall group – say project staff, some board orcommittee members and any other key players– to discuss it and fill it in. It is a way ofgathering the views of other people involvedin the project, to see if their expectations aresimilar or different.To complete Annex B you will need to ask yourselfthe following questions:What problems are you trying to solve?Before you can evaluate your progress, you needto be clear about what you are trying to achieve.Community projects usually want to solve one ormore local problem. What are these problems?What would make change happen?What needs to change if the problem is to besolved, and why? Think about what might havecaused the problem, and what needs to happen tomake a difference.How do you plan to make change happen?What can you do to solve the problem? Look atyour answer to the last question and think aboutwhat your project can do. Try to be as precise aspossible. Break your broad or long-term goalsdown into short- and medium-term goals. Whatactions are you going to take to achieve thesegoals? What resources will you need?What results do you want to see?How will you know when you have achieved yourshort-, medium- and long-term goals? What willbe different about the original problem at theend of the project, if you are successful? Try tobe as precise as possible about what you want toachieve. The achievements should all be related tothe original problem you wanted to work on.How can you measure progress?In order to know whether you are on the righttrack to achieve your goals and how far youhave got, you will need to decide on a few keyquestions, and collect evidence to answer them.What will tell you whether you are on the righttrack? What signs, milestones or ‘indicators’should you be looking for?Evaluating community projects3

Step 2: Gather evidence for the evaluationEstablish a baselineThis is a key part of the evaluation process. Youneed to find evidence to show whether or not youhave reached the goals you set yourself in Annex B(be they short-, medium- or long-term).Be as clear as you can about where you arestarting from. If you know where you are startingfrom, it will be easier to assess the distanceyou have travelled during the project. Whatinformation do you already have? Make sure youhave collected together in a central file materialthat will be useful to you later.Annex C (page 11) will help you to plan whatinformation you will need to gather in orderto judge the project’s progress, and so supportdecisions about what needs to change.For each measure of progress (indicator), it willbe important to find out what information youhave already, what additional information youwill need to collect, what methods you will useto gather it, who will take responsibility forcollecting the information, and the timescale.Different types of evidence should be collected: numbers (for example, the number of peopleyou have reached, the number of peoplegetting work, the number of good stories inthe press, changes in crime levels); people’s opinions, views and experiences(for example, people’s stories about theirexperience on the programme, photos of thearea ‘before and after’, people’s views onwhether they think they have more power); who has benefited and who has not.Relevant paperwork for creating a baseline mightinclude: the programme proposal, past researchreports and any statistical data you have on thearea, press reports and, if relevant, minutes of keypreliminary meetings.Set up systems to gather data on a regularbasisClarify what information you will require tohelp you to answer or explore the questionsset out in Step 1. Think about the records youwill be collecting anyway as part of your projectmanagement that will provide some of theinformation you need.Relevant paperwork that should be gathered on aregular basis might include: action plans and workprogrammes, minutes of meetings, reviews andreports, and records of attendance at groups andevents.Decide what additional information youwill gather specifically for the evaluation,and how you will gather itThe following menu lists ways of gatheringinformation. You should choose two or threemethods in order to get a rounded picture. Also,different ways of gathering information will suitdifferent projects. Which you choose to use, andhow much time you spend gathering data, shouldby tailored to the capacity of your organisation.4Evaluating community projects

Ways of gatheringadditional information:a menu A questionnaire surveyA questionnaire survey can be used to find outmore about the views and experiences of users,the wider community, agencies, etc. Use tickboxes or questions that can be answered with ayes or no if you want to survey a lot of people,or ask a lot of questions. Questions that allowpeople to say more than just yes or no will giveyou more detailed information, but they takelonger to fill in, a lot more time to analyse, andfewer people will fill them in. Responses toquestionnaires are often low so think aboutoffering a prize. In-depth interviewsIt is usually best to limit the number of in-depthinterviews to those people whose involvementwith the project gives them particular insightsor valuable experience – but try to talk to arange of people who are likely to have differentperspectives and views on your project. Feedback formsYou can find out whether people have foundyour training and other events useful by askingthem to fill in a short form. Ask them, forexample, what they found most and least useful;what they might do differently as a result; whatcould be improved. Focus groups and round tablesA ‘focus group’ gathers together about halfa dozen people who are broadly similar (forexample, they are all single parents with youngchildren) to discuss themes or questions youwant to address in the evaluation. A ‘roundtable’ discussion is a similar idea, which bringstogether people with different perspectives(for example, teenage parents, teachers, healthvisitors). DiariesAsk key people to keep diaries of theirinvolvement with the project. Press reportsGather and review press reports on the area (forexample, you could see whether positive reportsabout the area are increasing). ObservationTake photographs of your area over time, to seeif you can observe any changes. Observe whocontributes to meetings or comes to your centre,and see whether this changes over time. Thiswill give you an idea of which types of peopleyou are reaching (men, women, younger, older)and which of these types of people are playing amore confident role in the project. Case studiesIn order to make the evaluation manageable, youmight want to pick a few pieces of work (casestudies) to explore in detail, rather than tryingto explore everything. Pick pieces of work thatillustrate your main objectives. Evaluation workshops and reviewmeetingsHold special workshops/review meetings ofpeople who are involved in your project and usepictures, photographs or models, as well as thespoken word, to get feedback from participants.Evaluating community projects5