Progress In Reading Assessment (PiRA)

Progress in ReadingAssessment (PiRA)Second editionInterim Manualfor Autumn tests – Years 3–6 (updated)Colin McCarty & Kate Ruttle

Contents1 Introduction3This interim manual for PiRA AutumnWhat is PiRA?Why use PiRA?Measuring and following progress using PiRAProgression in reading demand in PiRA2 Administering the PiRA tests7When to testGroup sizeTimingPreparationAdministering the tests3 Answers and mark schemes8Marking the answersFinding the total raw scoreObtaining other scoresAnswers and mark scheme: PiRA 3 AutumnAnswers and mark scheme: PiRA 4 AutumnAnswers and mark scheme: PiRA 5 AutumnAnswers and mark scheme: PiRA 6 Autumn4 Obtaining and interpreting test scores21Summative measuresDiagnostic and formative interpretation5 Standardised score tables27Age-standardised scoresStandardised scoresCopyright 2015 Hodder and Stoughton Ltd.12/10/152

1IntroductionThis interim manual for PiRA AutumnThis free teacher guidance provides everything you need to administer and mark the Autumntests.More extensive teacher guidance, including information relating to the PiRA Spring and Summertests will be provided in the full PiRA Manuals – Stages 1 and 2, which will be published in January2016, together with the PiRA tests for Spring and Summer. The PiRA Manuals will also include thefollowing information, to assist you when using PiRA across the whole school year: Diagnostic and formative informationPupil profile sheets for each term, to enable you to review patterns of strengths andweaknesses across the yearFurther information about interpreting and analysing resultsTechnical information about the standardisation and equating studyAnswers and mark schemes for the PiRA Spring and Summer tests.In the meantime, should you have any queries about using the PiRA Autumn tests, please emailassessment@hodder.co.uk.What is PiRA?This is the second edition of Progress in Reading Assessment (PiRA). It provides a termlystandardised assessment of a pupil’s reading attainment, plus a profile of reading skills, whichhelps you identify those pupils who may need further teaching and practice, as well asenabling you to celebrate success. Separate tests are available for the spring and summer termsin Reception and for each of the autumn, spring and summer terms in Years 1–6. PiRA is designedfor whole-class use, with pupils of all abilities.This new edition retains all the texts of the original edition, but about 20 per cent of the questionshave been replaced so that it fully matches the 2014 National Curriculum and the content domainassessable elements of the reading test framework. The new questions have been written toreflect the styles of questions exemplified in the test framework.The tests provide thorough coverage of the new National Curriculum Programme of Study for eachyear. This has been assured by systematically sampling Key Stage 1 and 2 performance descriptorsfor English using the 2015 reading test framework and being informed by the sample material forthe Key Stage 1 and Key Stage 2 national tests.A sophisticated equating exercise was carried out for each termly test between January andJuly 2015 (comprising over 5,000 pupils), to ensure that the marks pupils obtain on each new PiRAtest are equated to the marks they gained in the original tests. This enables continuity of data forschools using the new edition of PiRA. Further details of this process will be provided in thetechnical chapter of the full published manual.Copyright 2015 Hodder and Stoughton Ltd.12/10/153

Why use PiRA?PiRA provides reliable summative information. For example: It uniquely provides three carefully designed tests for each year, enabling you to follow theprogress of your pupils from term to term, as well as year to year throughout primary school.Marks have been calibrated onto the Hodder Scale, which is an independent measure ofprogress throughout primary. This enables you to follow progress term by term and compareprogress to national norms. It allows you to predict what score pupils should obtain insubsequent terms and so set meaningful targets.PiRA also has a diagnostic capability, enabling you to investigate the strengths and weaknesses ofyour pupils’ reading skills. PiRA tests give you five distinct types of information to inform you ofeach child’s, class’s and cohort’s progress: age-standardised scores, which take into account a pupil’s chronological age so that you cansee how a pupil’s performance compares with other pupils of the same age;standardised scores, which show the standard score for a year cohort;reading age for a quick at-a-glance-reference;performance indicators to help you make a judgement against the year-related expectationsof the 2014 National Curriculum;the Hodder Scale, which is an independent measure of progress throughout primary.To enable you to use the information in a diagnostic/formative way, total scores can be brokendown into the following distinct aspects of reading.Reading analysisComprehensionInferenceLanguage, structure and presentationContent domainreference2b, 2c2d, 2e2a, 2f, 2gDetails of the content domain are given in the mark schemes, along with the reading analysis,which defines the relevant area of the curriculum.PiRA systematically assesses pupils’ reading of different text types or genres – fiction, variousforms of non-fiction, poetry and playscripts – in line with national guidelines across the primaryphase.You can also examine the performance of pupils on each question. Using the percentage of pupilsthat answered each question correctly in the national standardisation (technically, the facilityvalue), you can easily compare the performance of your own pupils with those in the nationalsample. You will find facility values alongside every question in the mark scheme for the test.Measuring and following progress using PiRAThe PiRA test results have been statistically linked from term to term and year to year to enableyou to track or predict progress through the whole primary phase, using the Hodder Scale. Thisenables you to monitor strengths and weaknesses and compare in detail individual patterns ofperformance against the norms and patterns for the term or year.Copyright 2015 Hodder and Stoughton Ltd.12/10/154

The Hodder Scale acts as a common ‘spine’ on which all of the PiRA tests across the whole primaryphase are plotted. It provides the statistical basis for predicting pupil progress and futureattainment, based on the termly performance data of over 15,000 pupils nationally. The HodderScale is presented in Table 4.2.Profiling performance by categoryThe reading category that the question focuses on is indicated by a code letter beneath each markbox. (We have used the abbreviations ‘comp’ for comprehension – literal understanding andretrieval from text; ‘inf’ for making inference – including prediction from text; and ‘lsp’ forlanguage, structure and presentation – understanding structure and purpose of text.) If you wishto profile the pupil’s performance, add up the number of correct answers the pupil has obtained ineach coded category and record these in the boxes at the bottom of the grid on the front cover.You can then compare the pupil’s performance to the national averages from the equating study,which are provided in a table beneath the mark scheme for each test.Progression in reading demand in PiRAEach test has been designed to ensure there is a steady progression in the demand of both thereading texts and the questions.We have evaluated the demand of each PiRA question, taking the DfE performance descriptorsinto consideration, as well as the ‘facility’ for each question (see below) and our own professionaljudgement. More detail will be provided about this in the full published manuals.Performance indicatorsWe have provided an algorithmically derived set of thresholds that indicate where the child is withrespect to the National Curriculum performance descriptors, by applying the summer standard forthe whole year to a test being taken in the autumn or spring.In the mark schemes, we have provided the facility for each question, which is the percentagesuccess of children who took the test in the standardisation. (Note: a high facility indicates an easyquestion and a low facility a difficult question.) It is possible to set thresholds drawing on theperformance of children in the equating standardisation trials, by using the facilities of thequestions and applying a reasonable set of expectations (presented in the table below). Pleasenote that these thresholds apply to the curriculum age-expectation for complete years. Therefore,in autumn they will be seen as very challenging and hard to achieve, but any child achieving theexpected standard in autumn will be doing very well.This information is given at the end of the mark scheme for each year. It links the performance(facilities) to the language most commonly used to describe children’s performance (for example,‘working towards’, ‘emerging’), based on our test-development experience.Table 1.1: Thresholds for performance indicatorsWorking towardsEmergingExpectedExceeding90–100% success(i.e. facility)60–89% success20–59% success0–19% successCopyright 2015 Hodder and Stoughton Ltd.12/10/155

Our thinking is: to be at the ‘expected’ standard of performance, for example, children need to get90 per cent of the ‘emerging’ questions of that term’s test correct, over 60 per cent of the‘expected’ questions correct, all of the ‘working towards’ but none of the ‘exceeding’ questions.(The thresholds for ‘emerging’ and ‘exceeding’ will be explained in the full published manual.)There is no official definition or golden rule that gives us this. However, many years of working intest development and analysing related data have given us the considerable experience to createalgorithms to help inform thresholds. The critical factor is that the rule we use has to have a senseof reasonableness, that is to say it is based on our experience, which has been informed bydiscussions with a number of headteachers and literacy co-ordinators who concur with ourthinking and methodology for creating a data-driven way to evolve thresholds.Copyright 2015 Hodder and Stoughton Ltd.12/10/156