Review Of Standards In GCSE Mathematics - GOV.UK

Review of Standards in GCSEMathematics2004 and 2008 April 2012Ofqual/12/5154

Review of Standards in GCSE Mathematics: 2004 and 2008ContentsExecutive summary . 3Findings . 3Section 1: Introduction . 4Context. 4Methodology . 5Provision of assessment materials and student work . 5The review team . 6Analysis of the specifications and assessment materials . 7Analysis of student performance . 7Section 2: Subject demand in GCSE mathematics . 8Overview . 8Findings . 10Assessment objectives . 10Specification content . 10Schemes of assessment . 10Time demand . 11The effects of the increased move towards modularity. 11Options . 12Question papers . 13Allocations of grades to questions . 14Percentage allocation of questions . 15Grade boundaries . 15Tiering . 16Ofqual 20121

Review of Standards in GCSE Mathematics: 2004 and 2008Coursework . 16Section 3: Standards of performance . 18Overview . 18Findings . 18Recommendations . 18Appendix A: Provision of assessment materials and student work at GCSE and GCElevels for Ofqual’s archive (annual inclusion and standards reviews) . 20Section 1: Specification of requirements . 20Section 2: Student work . 21Appendix B: Schemes of assessment . 22Appendix C: Assessment objectives . 25Appendix D: GCSE specifications reviewed . 27Appendix E: GCSE scripts reviewed . 28Appendix F: Availability of specification materials for the purposes of this review . 29Appendix G: Student achievement by grade . 30Appendix H: Review team . 31Ofqual 20122

Review of Standards in GCSE Mathematics: 2004 and 2008Executive summaryThe Office of Qualifications and Examinations Regulation (Ofqual) undertakes arolling programme of reviews across high-profile GCSE and GCE A level subjects tomonitor whether standards in assessment and student performance have beenmaintained over time.This report details the findings for GCSE mathematics in the years 2004 and 2008.The previous review for this subject compared the years 1999 and 2004. The findingswere published in a report in 2006, which is available on our website gcse-a-level.pdf .The study compared subject specifications, assessment materials and student workfrom the five organisations awarding this qualification in the years being reviewed(AQA, CCEA, Edexcel, OCR and WJEC) by collecting the views of a number ofsubject specialists.Findings The major change that affected all GCSE mathematics examinations between2004 and 2008 was a move from a three-tier examination system of foundation,intermediate and higher tiers to a two-tier system, comprising foundation andhigher only. These changes had a significant effect on the demand of theexamination by changing the balance of questions focused on each grade. The spread of grades to be covered in each tier increased and in someawarding organisations this resulted in a rise of structuring within questions. Inaddition question design showed an increasing trend towards structuring ofquestions. Both factors made examinations less demanding over time. The increasing numbers of centres entering students for specifications withmodular examinations highlighted a mixed effect on demand. OCR’s modularassessment design minimised the effect of the changes and allowed standardsto be maintained over time, whereas AQA’s modular design (also available in2004) fragmented the assessment and increased structuring in questions,making the examinations less demanding. The layout of question papers, the language used and the clarity of graphs anddiagrams had all improved over the time period reviewed, providing a betterquality assessment in mathematics.Ofqual 20123

Review of Standards in GCSE Mathematics: 2004 and 2008Section 1: IntroductionContextIn his Review of Qualifications for 16–19 Year Olds (1996), Lord Dearing madeseveral recommendations to ensure that “there is a basis and accepted procedure for monitoring and safeguarding standards over time”. In the same year, the SchoolCurriculum and Assessment Authority (SCAA), one of our predecessors, and theOffice for Standards in Education, Children’s Services and Skills (Ofsted) jointlyrecommended that there should be:a rolling programme of reviews on a five-year cycle to ensure examination demands andgrade standards are being maintained in all major subjects. (Standards in PublicExaminations 1975 to 1995, page 4, 1996)As a result of these recommendations we, in collaboration with the WelshGovernment and the regulator in Northern Ireland (Council for the Curriculum,Examinations and Assessment (CCEA)), introduced a programme to investigatestandards in GCE A level and GCSE examinations by systematically collecting andretaining assessment materials and student work to enable standards reviews tocover two or more years.The Apprenticeships, Skills, Children and Learning Act 2009 formalised our role inundertaking such reviews by including a statutory objective “to secure that regulatedqualifications indicate a consistent level of attainment (including over time)”.We report on our work in meeting this objective. And we use our findings to informdevelopments in qualification and subject criteria to support meeting this objective inthe future. In our reviews we: analyse the nature of the requirements that different assessments make onstudents compare the levels of performance required for a particular grade in differentassessments consider how these two elements relate to each other.In 2004 there were 743, 899 students taking the GCSE mathematics specificationsbeing reviewed. In 2008 the number was 743,833. A detailed breakdown of studententry numbers and cumulative percentage pass rates can be found in Appendix G.Ofqual 20124

Review of Standards in GCSE Mathematics: 2004 and 2008Our immediate predecessor, the Qualifications and Curriculum Authority (QCA), mostrecently conducted a standards review in GCSE mathematics, using materials from1999 and 2004. The findings were published in a report in 2006, which is available onour website at se-a-level.pdf .MethodologyStandards reviews examine different specifications within a qualification, theassociated assessment instruments and samples of student work by collating andanalysing the views of a number of subject specialists. The following sections of thisreport detail how we collect and process this information. In these reviews, demandis measured against that of the other specifications under review and includesconsideration of: specification-level factors such as assessment objectives, content and structure assessment-level factors such as what content is assessed, the weighting ofeach component and how the assessments are marked student performance-level factors, including how the students responded to theassessments and the grades they received as a result.The demand of an assessment or qualification can be defined in a variety of waysand is linked to the purpose of the qualification. It is related to the: amount and type of subject knowledge required to be assimilated complexity or number of processes required of the students, the extent to whichthe students have to generate responses to questions from their ownknowledge or the extent to which resources are provided level of abstract thinking involved extent to which the students must devise a strategy for responding to thequestions.Provision of assessment materials and student workEach of the five awarding organisations offering the qualifications being reviewed(AQA; CCEA; Edexcel; OCR and WJEC) was asked to provide specificationmaterials for GCSE mathematics (from the specification with its largest entry insummer 2008).Ofqual 20125

Review of Standards in GCSE Mathematics: 2004 and 2008Details of our requirements for the provision of assessment materials and studentwork for review are given in Appendix A and, in summary, include: the current specification all associated question papers final mark schemes the 2008 chief examiner’s report and grade boundaries, overall and by unit(both raw and scaled) mark distributions, grade descriptors and assessment grids any other information that was routinely supplied to centres all the assessment work carried out by a sample of students whose final gradelay at or near the judgemental grade boundaries for the qualification beinganalysed.The equivalent materials that were collected and retained for the previous reviewwere retrieved from our archive of assessment materials and student work.Full details of the materials supplied by awarding organisations can be found inAppendix E and Appendix F.The review teamWe contracted 13 experts in GCSE mathematics to undertake the review. Thesereviewers were sourced through: a subject-expert recruitment exercise carried out by us in November 2008,advertised via The Times Educational Supplement and our website andnewsletter nominations made by awarding organisations involved in the review nominations made by subject associations and other learned bodies invited toparticipate in the review.A full list of reviewers can be found in Appendix H.We contracted a lead reviewer, specification reviewers and script reviewers. (Allnominees from awarding organisations and subject associations were scriptreviewers.)Ofqual 20126

Review of Standards in GCSE Mathematics: 2004 and 2008Analysis of the specifications and assessment materialsThe lead reviewer and specification reviewers (specification review team) analysedthe awarding organisations’ materials, using a series of forms which can be found viathe comparability page on our website at icles/23-%20comparability .These analyses are designed to describe the demand of the specification. Eachreviewer analysed a subset of the specifications available, so that there were at leastthree different views on each specification. The lead reviewer then produced a reportwhich brought together the views of the reviewers on each of the awardingorganisations’ specifications. The specification review team was given theopportunity to discuss the lead reviewer’s conclusions at a follow-up meeting. Thesefindings are presented in Section 2 of this report.Analysis of student performanceTo assess student performance, all reviewers were brought together for a two-daymeeting to analyse student scripts (pieces of student work supplied by the awardingorganisations). This process is referred to as a script review. The meeting startedwith a briefing session to make sure that all the reviewers had a commonunderstanding of the methodology and the judgement criteria.The scripts were organised into packs for consideration during the review. Packswere organised by grade: A/B, C/D and F/G for GCSE. (Other grades are calculatedarithmetically after the former grade-boundary marks have been set during theawarding process carried out by awarding organisations.) Reviewers were asked tomake qualitative comments on the work they saw. For example, they were asked tocomment on whether they thought the work provided by the awarding organisationshad demonstrated the required level of knowledge and skill to warrant the grade thatthe work had received.Ofqual 20127