Report On The Quality Assurance Of The Examinations And .
December 20092009Report on the Quality Assuranceof the Examinations and Assessmentfor Vocational Education and Training37 General Van Ryneveld Street, Persequor Technopark, PretoriaTelephone: 27 12 349 1510 Fax: 27 12 349 1511 info@umalusi.org.zawww.umalusi.org.zaCouncil for Quality Assurance inGeneral and Further Education and TrainingCouncil for Quality Assurance inGeneral and Further Education and Training
REPORT ON THE QUALITY ASSURANCEOF THE EXAMINATIONS AND ASSESSMENTFOR VOCATIONAL EDUCATION AND TRAINING2009PUBLISHED BY:Council for Quality Assurance inGeneral and Further Education and Training37 General Van Ryneveld Street, Persequor Technopark, PretoriaTelephone: 27 12 349 1510 Fax: 27 12 349 1511 info@umalusi.org.za
COPYRIGHT 2009 UMALUSI COUNCIL FOR QUALITYASSURANCE IN GENERAL AND FURTHER EDUCATIONAND TRAINING. ALL RIGHTS RESERVED.
ContentsFOREWORDvACRONYMSviSECTION ONE1OVERVIEW OF THE REPORT21.Background22.Purpose of the report33.Scope of the report3SECTION TWO: QUALITY ASSURANCE OF NCV ASSESSMENT4CHAPTER ONE: MODERATION OF QUESTION 75.Findings6.Areas of good practice107.Areas for improvement118.Conclusion158CHAPTER TWO: MODERATION OF INTEGRATED SUMMATIVE ASSESSMENT TASKS roach205.Findings216.Areas of good practice237.Areas for improvement238.Conclusion26CHAPTER THREE: MODERATION OF INTERNAL CONTINUOUS ASSESSMENT roach305.Findings306.Areas of good practice347.Areas for improvement358.Conclusion38iii
CHAPTER FOUR: MONITORING OF 4.Approach425.Findings426.Areas of good practice457.Areas for improvement468.Conclusion48CHAPTER FIVE: MODERATION OF roach525.Findings536.Areas of good practice567.Areas for improvement568.Conclusion58CHAPTER SIX: STANDARDISATION OF roach595.Findings606.Areas for improvement617.Conclusion61SECTION THREE: QUALITY ASSURANCE OF NATED EXAMINATIONS621.Introduction632.Scope and Approach633.Findings654.Conclusion68SECTION FOUR: iv
ForewordUmalusi has since its inception been involved in the quality assurance of vocational education andtraining qualifications that are certified by this statutory body, namely the National TechnicalEducation (NATED) Report 190/191 and the National Certificate Vocational (NCV) programmes.Umalusi by virtue of its Act, undertakes to quality assure these national qualifications and does sothrough a rigorous process involving reporting on each of the assessment processes andprocedures.By the end of 2009 the roll out of the NCV qualification, which was introduced in 2007, had reachedlevel 4 – the final exit level for this qualification. While this may be viewed as an accomplishment forthe vocational education trajectory in South Africa, it has also identified challenges for stakeholdersto embrace.During 2009 certain processes of all the examinations conducted for the afore-mentionedqualifications were quality assured. The major emphasis was, however, on the NCV programmes atlevels 2, 3, and 4 as the NATED Report 190/191 programmes are being phased out. While Umalusiwould normally have targeted level 4 of the NCV (as the qualification exit point), they respondedto the request from the National Department of Education to quality assure levels 2, 3, and 4. In anintegrated approach, samples from each level were quality assured. Samples were interrogated byUmalusi to ensure that the overall standard and quality of the examinations for these qualificationsare beyond reproach. This was accomplished through the following quality assurance processes: Moderation of question papers, including the Integrated Summative Assessment Tasks; Monitoring and moderation of internal continuous assessment; Monitoring of the different phases of the examinations; Moderation of marking; and Standardisation of marks according to agreed statistical and educational principles.Substantial progress has been made with the improvement of certain systems and processes for thesuccessful implementation and quality assurance of the NCV qualification, but there areunfortunately still many challenges. The quality assurance of the practical component of thisqualification coupled with the inability to find and/or retain suitably qualified and experienced staffto offer and assess the qualification pose particular challenges. It is of utmost importance thatsolutions to these challenges be found.Based on the reports received from Umalusi's team of external moderators and monitors, as well asthe deliberations and conclusions of its Assessment Standards Committee, Umalusi Councilconcluded that the 2009 assessment processes were conducted in a fair and reliable manner, thatthe results are valid, and that there is currently no report of any irregularity that would compromisethe credibility of the 2009 Vocational Education and Training assessments.Through its quality assurance processes, Umalusi will continue to ensure that the quality, integrity,and credibility of the Vocational Education and Training assessments are improved andmaintained, and will continue in its endeavor towards a world-class assessment system in this sector.Prof J D Volmink21 December 2009v
MpuNATEDNCNCVNICNQFNSCNWOBEPDEUMALUSI-VETWC-Chief Directorate: National Examinations and MeasurementDepartment of Higher Education and TrainingEastern Cape ProvinceFurther Education and TrainingFree State ProvinceGauteng ProvinceKwaZulu-Natal ProvinceInternal Continuous AssessmentIntegrated Summative Assessment TaskLevel 2Level 3Level 4Limpopo ProvinceMpumalanga ProvinceNational Technical EducationNorthern Cape ProvinceNational Certificate VocationalNational Intermediate CertificateNational Qualifications FrameworkNational Senior CertificateNorth West ProvinceOutcomes Based EducationProvincial Department of EducationQuality Council for Quality Assurance in General and FurtherEducation and TrainingVocational Education and TrainingWestern Cape Provincevi
Section One1
Overview of the report1. BACKGROUNDThe amended General and Further Education and Training Quality Assurance Act (No. 58 of 2001)as well as the recently enacted National Qualifications Framework Act (No. 76 of 2008) assignquality assurance to the Quality Councils. Umalusi, as one of the quality councils, executes thisfunction through, inter alia: Setting standards; Monitoring and reporting on the adequacy and suitability of qualifications and standards; Quality assurance of exit point assessments; Certification of learner achievements; Quality promotion amongst providers; and Accreditation of private providers of education, training and assessment.Umalusi quality assures the assessments of the following VET qualifications: The National Certificate Vocational (NCV) levels 2, 3 and 4; and The NATED 190/191 which are currently being phased out, namely,oThe National Technical Certificates, N2 and N3;oThe National Intermediate Certificate (NIC); andoThe National Senior Certificate (NSC).This report focuses on the quality assurance of the assessment of the above qualifications for theyear 2009. Umalusi reports on the standard of the Vocational Education and Training (VET)assessment on an annual basis. In this regard, Umalusi reports on each of the quality assuranceprocesses and procedures, which jointly allow Umalusi to make an evaluative judgement on thecredibility of the assessments. These processes ensure that all aspects of the assessment aremoderated and monitored against prescribed criteria thus ensuring that standards are maintainedand improved.The Evaluation and Accreditation Unit of Umalusi monitored the assessment systems of the NCV atnational, provincial and college levels. This report incorporates findings from the Evaluation andAccreditation Report as appropriate.Six National Technical Certificate (NATED Report 190/191) examinations were conducted duringMarch, April, June, August, and November 2009. The Natural Sciences examinations wereconducted during April, August, and November 2009 and the Business and General Studiesexaminations were conducted during March, June, and November 2009.The NCV level 2 and 3 supplementary examinations were conducted during February / March 2009and the level 2, 3, and 4 examinations were conducted during November 2009.Umalusi quality assured sampled aspects of each of the above mentioned NATED190/191examinations and the NCV assessments. This included the Internal Continuous Assessment(ICASS) of a sample of the NCV subjects that are currently moderated by Umalusi. This processinvolved the monitoring and moderation of ICASS at selected sites (during August 2009) andcentralised moderation of a sample of portfolios of evidence and portfolios of assessment atprovincial level (during October 2009).2
The Integrative Summative Assessment Tasks (ISATs) of a sample of programmes and subjects wereevaluated and the conduct monitored / moderated during September / October 2009. Thisinvolved monitoring the conduct of the ISATs and/or the moderation of the completed tasks.Examinations in this sector are written at different kinds of centres, namely, public Further Educationand Training (FET) colleges, private FET colleges, Correctional Services centres, and schools. Themarking of scripts takes place externally at designated centralised marking centres in the case ofN3, NSC, and NCV level 4; and at designated provincial / college marking centres in the case ofNCV level 2 and 3.Umalusi judges the quality and standard of assessment in vocational education and training bydetermining the: Level of adherence to policy in implementing assessment related processes; Efficiency and effectiveness of systems, processes, and procedures for the conduct ofassessment; Cognitive challenge, weighting of content, and presentation of examination questionpapers and ISATs; Quality of the marking of examination answer scripts and ICASS; and Quality of assessment and conduct of the ISAT.2. PURPOSE OF THE REPORTThe purpose of this document is to report on Umalusi's quality assurance of the 2009 VETassessments with respect to the following: The quality and standard of question papers and ISATs; The quality and standard of scoring / marking the assessments; The efficiency and effectiveness of systems and processes for the conduct of the VETassessments; The identification of good practice and areas for improvement of assessment; and The moderation of marks during the standardisation process.The salient findings from the external moderators' and monitors' reports are analysed and used tomake judgements on the standard of the VET assessments during the standarisation process.3. SCOPE OF THE REPORTThis report covers all the 2009 VET assessments and reports on the quality assurance of assessmentprocesses used by Umalusi to ensure that the assessment was of the required standard.This section of the report provides an overview of the quality assurance processes used by Umalusito ensure that the assessments meet the required standards. It furthermore outlines the purpose andscope of the report. Section two covers the quality assurance of the NCV. Section three covers thequality assurance of the NATED 190/191 qualifications namely, Natural Sciences, as well as theBusiness and General Studies examinations.Section two and three capture the salient findings with respect to each of the quality assuranceprocesses, identify good practice, and highlight areas requiring improvement.3
Section TwoQuality assurance of NCV assessment4
Chapter 1Moderation of question papers1. INTRODUCTIONThe examination question paper is central to the assessment of knowledge, skills, attitudes, andcognitive abilities of a learner at a given level. In the case of the NCV, the Department of HigherEducation and Training (DHET) is responsible for the setting and the administration of questionpapers, while Umalusi, through a rigorous moderating exercise, attests to the standard and qualityof the question papers sampled in its moderation process.Umalusi externally moderates question papers to ensure that the papers are fair, valid, and reliableand are of an acceptable standard in terms of criteria which have been nationally accepted.Umalusi judges the quality and standard of question papers by determining the level of adherenceto policy for the setting of question papers, the cognitive challenge of the question papers, theappropriateness and weighting of content in the question papers in accordance with the subjectand assessment guidelines, and the quality of presentation of the question papers.2. PURPOSEThe purpose of this chapter is to report on the validity and quality of the 2009 examination questionpapers for the NCV levels 2, 3 and 4 (including the March 2009 supplementary papers) in terms ofthe sample moderated. The report aims to: provide an indication of the sample size in terms of subjects and levels; give an overview of the crucial findings relating to the quality and standard of the questionpapers that were externally moderated; highlight both areas of good practice and areas for improvement; and include recommendations that will enhance the quality of future question papers.3. SCOPEThis report covers the findings of the moderation process on the NCV examination papers of thesubjects written during the February / March 2009 supplementary examinations and the November2009 examinations.The table below provides a list of the subjects and levels that were sampled and moderated during the2009 moderation process:Table 1: Moderation sample – subjects and levelsNo.MarchNCV SUBJECTS MODERATED 2009NovemberLevel 2 Level 3 Level 2 Level 3 Level 41234Advertising and PromotionAfrikaans 1st Additional LanguageAnimal ProductionApplied Accounting5
No.MarchNCV SUBJECTS MODERATED 2009NovemberLevel 2 Level 3 Level 2 Level 3 Level 45Applied Engineering Technology6Automotive Repair and Maintenance7Business Practice8Civil Construction 9303132333435363738394041424344454647Client Service and Human RelationsConstruction Carpentry and Roof WorkConstruction MaterialsConstruction PlanningConstruction Plant and EquipmentConstruction PlumbingConstruction SupervisionDrawings, Setting Out, Quantities, and CostingEconomic EnvironmentElectrical Principles and PracticeElectronic Control and Digital ElectronicsEngineering Graphics and DesignEngineering FabricationEngineering Fabrication Sheet Metal WorkEngineering FundamentalsEngineering Practice and MaintenanceEngineering ProcessesEngineering SystemsEngineering TechnologyEnglish 1st Additional LanguageEntrepreneurshipFarm Planning and MechanisationFinancial ManagementFitting and TurningFood PreparationHospitality GenericsIntroduction to Information SystemsIntroduction to Systems DevelopmentIsixhosa 1st Additional LanguageLife Orientation (Paper 1)Management PracticeMarketingMathematical LiteracyMathematicsNew Venture CreationOffice Data ProcessingOffice PracticePhysical SciencePrinciples of Computer Programming6
No.MarchNCV SUBJECTS MODERATED 2009NovemberLevel 2 Level 3 Level 2 Level 3 Level 448495051525354Project ManagementScience of TourismSoil ScienceSustainable Tourism in South AfricaSustainable Tourism in SA and Regional TravelSystems Analysis and DesignTourism OperationsTotal Subjects at each level (note somesubjects may have two papers – not includedin this total)4029252829Note:1.2.All the listed subjects were moderated on some level.Where the subject has two papers, each paper was moderated except for Life Orientation where thecomputer paper (Paper 2) was not moderated.3.Due to changes made by the DHET in the moderation model, Umalusi was unable to moderate all thepapers originally planned. The change from on-site moderation to off-site moderation did notfacilitate immediate contact between examiners and internal and external moderators and resultedin tight time constraints for the process.4. APPROACHUmalusi utilises subject experts to externally moderate a sample of NCV level 2, level 3, and level 4question papers (outlined above). The criteria used to moderate the question papers cover thefollowing aspects: The technical aspects relating to the presentation of the question papers and memoranda; The internal moderation in terms of purposeful efficiency for quality assurance; The coverage of content, learning, and skills outcomes of the subjects; The balance of cognitive skill demands of the types of questions; The consistency of mark distribution and allocation according to cognitive skill demand andtype of question; The relevance and correctness of the marking memoranda especially in terms of theaccuracy and the standard of marking; The language and bias in terms of universality of interpretation; The predictability and level of innovation in the questions; The adherence of the paper to Subject Guideline and Assessment documents in terms ofweightings and spread of question types and cognitive demand as demonstrated by theassessment frameworks / grids used in paper setting; and The evaluative judgement of external moderators on their overall impression of the standardof the paper in terms of the validity, reliability, and suitability of the paper to assess on thelevel it purposed to evaluate.7
The original on-site moderation model was designed to provide opportunities for clarifyingexchanges between the assessment personnel of the DHET and Umalusi external moderators. Theintended exchanges in these meetings were to allow for the: Explanation of Umalusi's external moderation process with regard to the quality andstandard of national examination papers; Discussion of issues emanating from previous moderation of examination papers and focuson the moderation of the 2009 examination papers; Establishment of mutual consensus on the format, content, and cognitive demands of theexamination papers; and Finalisation of the assessment framework / grid and memorandum for each paper.After the first meeting on 24 May 2009 at the DHET, the plan was changed to an off-site moderationmodel. This decision was intended to assist the process but resulted in delays and the necessity ofcontingency plans that were not ideal.Furthermore, time became so critical that, instead of Umalusi's moderators finally signing off papersto ensure that all recommendations had been effected, the DHET Management took responsibilityfor the signing off of the NCV November 2009 question papers.5. FINDINGSThe findings of the moderation of the sample of question papers by Umalusi moderators arepresented both statistically and in expanded notation.In 2009, 84 March Supplementary papers and 103 November papers were moderated. Statisticsindicated the following information about the approval status of question papers after the firstUmalusi moderation: Overall, between the March and November 2009 examinations there was a drop of 5% ofpapers approved on first moderation. Further, between March and November 2009 there was an increase of 4% of those rejectedon first moderation and an increase of 6% in those requiring resubmission for a subsequentmoderation. During 2009 the percentage of level 2 papers approved on first moderation was 7% higherthan those of level 3. In the same year the percentage of level 3 papers approved on first moderation was 9%higher than level 4.From the above the following trends are evident: Overall the skills involved in the setting of fair, valid, and reliable examinations for nationalassessment purposes still need to be refined and improved. The statistics might be interpreted to mean that the experience of examiners in setting level2 papers assisted in achieving a
Six National Technical Certificate (NATED Report 190/191) examinations were conducted during March, April, June, August, and November 2009. The Natural Sciences examinations were conducted during April, August, and November 2009 and the Business and General Studies examinations
May 02, 2018 · D. Program Evaluation ͟The organization has provided a description of the framework for how each program will be evaluated. The framework should include all the elements below: ͟The evaluation methods are cost-effective for the organization ͟Quantitative and qualitative data is being collected (at Basics tier, data collection must have begun)
Silat is a combative art of self-defense and survival rooted from Matay archipelago. It was traced at thé early of Langkasuka Kingdom (2nd century CE) till thé reign of Melaka (Malaysia) Sultanate era (13th century). Silat has now evolved to become part of social culture and tradition with thé appearance of a fine physical and spiritual .
Dr. Sunita Bharatwal** Dr. Pawan Garga*** Abstract Customer satisfaction is derived from thè functionalities and values, a product or Service can provide. The current study aims to segregate thè dimensions of ordine Service quality and gather insights on its impact on web shopping. The trends of purchases have
̶The leading indicator of employee engagement is based on the quality of the relationship between employee and supervisor Empower your managers! ̶Help them understand the impact on the organization ̶Share important changes, plan options, tasks, and deadlines ̶Provide key messages and talking points ̶Prepare them to answer employee questions
On an exceptional basis, Member States may request UNESCO to provide thé candidates with access to thé platform so they can complète thé form by themselves. Thèse requests must be addressed to esd rize unesco. or by 15 A ril 2021 UNESCO will provide thé nomineewith accessto thé platform via their émail address.
Chính Văn.- Còn đức Thế tôn thì tuệ giác cực kỳ trong sạch 8: hiện hành bất nhị 9, đạt đến vô tướng 10, đứng vào chỗ đứng của các đức Thế tôn 11, thể hiện tính bình đẳng của các Ngài, đến chỗ không còn chướng ngại 12, giáo pháp không thể khuynh đảo, tâm thức không bị cản trở, cái được
Food outlets which focused on food quality, Service quality, environment and price factors, are thè valuable factors for food outlets to increase thè satisfaction level of customers and it will create a positive impact through word ofmouth. Keyword : Customer satisfaction, food quality, Service quality, physical environment off ood outlets .
critical issues the University has established a Quality Assurance Directorate, which is mandated to develop a Quality Assurance Framework and a Quality Assurance Policy. The Quality Assurance Framework would clearly spell out the Principles, Guidelines and Procedures for implementing institutional quality assurance processes.
