083743 LTJ 15-38

083743 LTJ 15-38.qxd1822/12/079:06 AMPage 18The key to successPublic English Testing System (PETS), the Cambridge BusinessEnglish Certificate (BEC), and the WSK – an examination to selectprofessionals for study and training overseas.The College English Test (CET) is a large-scale standardized testadministered nationwide by the National College English TestingCommittee on behalf of the Higher Education Department of theMinistry of Education (MoE) in China. It aims at measuring theEnglish proficiency of college/university undergraduate students inaccordance with the College English Teaching Syllabus (NationalCollege English Syllabus for Non-English Majors, 1999). The CETis a test battery, which is comprised of the CET Band 4 (CET-4), theCET Band 6 (CET-6), and the CET – Spoken English Test (CETSET). The CET-4 and CET-6 were first administered in 1987, and areadministered twice a year afterwards, in January and June. The CET-4and CET-6 are criterion-related norm-referenced tests (Jin, 2005;Yang and Weir, 2001). In this sense, the test criteria are based onthe National College English Syllabus by the Higher EducationDepartment of the Ministry of Education to guide the English teaching at university level. The test scores are based on 100 points, ofwhich a score of 60 points constitutes a passing grade. The reportedscore of the CET conveys two pieces of information. First, it indicates whether a candidate has met the requirements of the CollegeEnglish Teaching Syllabus (National College English Syllabus forNon-English Majors, 1999). Second,, it indicates the percentile position of a candidate in the norm group, which consists of over 10 000college/ university students from six top universities in China.In order to meet the needs of China’s economic reform and opendoor policy, the CET-SET was included in the test battery in 1999. TheCET-SET is available to students who have passed the CET-4 with ascore of 80 or above or the CET-6 with a score of 75 or above. TheCET-6 is available to students who have passed the CET-4, and havetaken the optional College English Course of Bands 5–6. Certificatesare issued by the Higher Education Department of the Ministry ofEducation, P. R. China, to those who qualify on the test. Since itsinception in 1987, the CET has attracted an increasing number of candidates every year. In the 2005 academic year, more than 9.58 millionstudents in China took the test (Jin, 2005). The CET is reported to havemaintained high reliability and validity (Jin, 2005; Yang and Weir,2001) and has a set of standardized procedures in the administrationand the interpretation of raw scores (Yang and Jin, 2000).Consequently, the stakes associated with the CET are extremelyhigh. In most colleges and universities, the CET-4 certificate is oneDownloaded from http://ltj.sagepub.com at QUEENS UNIV LIBRARIES on February 17, 2008 2008 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.

083743 LTJ 15-38.qxd22/12/079:06 AMPage 19Liying Cheng 19of the graduation requirements for students to obtain their academicdegree. The CET certificate is also an asset for university graduateswho want to stand a better chance in the job market. Students’ performance in the CET-4 also affects the evaluation of teachers, theirpromotions, and even merit awards. At the institution level, the passing rate on the CET is often regarded as one of the criteria to judgethe prestige of a university. Therefore, the CET has exerted a hugeamount of influence, reportedly negative, on English language teaching and learning at the tertiary level in China since its first administration (Han et al., 2004; Zhang, 2003). Many language educators andresearchers have started to investigate various aspects of the test,hoping to bring some insights into the improvement and innovationof the test. This will be discussed in more detail later.The National Matriculation English Test (NMET) is the universityentrance test of English for the whole country. The purpose of the testis to make inferences about candidates’ English language ability,which is used in university admission decisions together with thescores from university entrance tests in another five or six secondaryschool subjects. A student needs to take tests in five or six subjectsdepending on the requirements of the type of the university for whichhe/she applies. Chinese, mathematics, and English are three compulsory subjects for all candidates regardless of their choice of university. The NMET is taken annually on 8 June by millions of secondaryschool graduates who wish to gain entrance to Chinese universitiesand colleges. The number of candidates varies each year. It was aboutthree million in the early 1980s and increased to 9.5 million in 2006(http://www.sina.com.cn). The test is about two hours long. It isadministered by the municipal, county, and provincial Bureau ofEducation at testing centres across the entire country (Cheng and Qi,2006). The NMET, introduced in 1985, is one of the three compulsorytests in the university entrance examination battery in China. It is anorm-referenced standardized test with a major function of selectingsecondary school graduates for institutions of higher education. TheNMET is a high-stakes English test in China, which influences howsociety evaluates the schools, and how schools, parents, and studentsevaluate the teachers. Apart from its selection purpose, the NMETwas used to bring positive washback to English teaching and learningat the secondary level (Qi, 2003; 2004; 2005).For English majors in Chinese colleges and universities, the Test forEnglish Majors (TEM) is an important test. The TEM assesses thelanguage performance of English majors and is administrated by theNational Advisory Commission on Foreign Language Teaching inDownloaded from http://ltj.sagepub.com at QUEENS UNIV LIBRARIES on February 17, 2008 2008 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.

083743 LTJ 15-38.qxd2022/12/079:06 AMPage 20The key to successHigher Education (NACFLT) in China. Another purpose of the test isto promote English teaching and learning for English majors. TheTEM is a criterion-referenced test. Students’ performance is evaluatedagainst the criteria stipulated by the teaching syllabus (Zou, 2003).The test consists of two levels: TEM-4 administered at the end of the2nd year, and TEM-8 at the end of the 4th year in their undergraduateprogram. The Graduate School Entrance English Examination (GSEEE)is another entry test administered at the national level once every yearfor entrance into graduate schools at all Chinese universities. TheGSEEE is administered by the National Education ExaminationsAuthority of the Ministry of Education in China.Apart from these tests in institutional settings, there are some noncredential English tests in China. The Public English Testing System(PETS) is probably the largest in scale among them. It was developed in 1999 by the Chinese National Education ExaminationsAuthority (NEEA) with assistance from the University of CambridgeLocal Examinations Syndicate (UCLES). This test is a non-credentialtest, which is open to all English learners, with no restriction onage, profession or academic background. It aims to promote Englishlearning nationwide. It provides assessment and certification of communicative English language skills in reading, writing, listening,and speaking at five levels of competence from Level 1 to Level 5.Another public test is the Cambridge Business English Certificate(BEC), another collaborative program between the NEEA and UCLES,designed to test English language ability used in the business context.The test was introduced to China in 1993 with three levels – BECPreliminary, BEC Vantage, and BEC Higher. Individual learners whowish to obtain a business-related English language qualification sit forthis test. BEC certificate is widely recognized by foreign companiesand enterprises in China.In addition, every year, the Chinese government provides fundingfor professionals to study and/or receive training outside China. Apartfrom their professional qualifications, these professionals (non-foreignlanguage majors) are selected based on a language proficiency test –WSK (an acronym from Chinese pinyin: wai-yu-shui-ping-kao-shi 䇁 ᑇ㗗䆩 ) administered by the National Education ExaminationsAuthority (NEEA) of Ministry of Education. WSK provides tests ofcommunicative competence in five languages: English, French,German, Japanese, and Russian.Compared with the academic tests, the non-credential tests of Englishreceive less attention from language educators and researchers.Many more empirical studies are found in academic journalsDownloaded from http://ltj.sagepub.com at QUEENS UNIV LIBRARIES on February 17, 2008 2008 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.

083743 LTJ 15-38.qxd22/12/079:06 AMPage 21Liying Cheng 21investigating tests such as the CET and the NMET – the two largescale high-stakes national tests.III Empirical studies in language testingAs mentioned above, examinations have played an important socialand educational role in China. The promotion of an effective Englishtesting system has thus been of great importance in China. Particularlyfor the past 15 years, language educators and researchers in Chinahave devoted much effort to issues in language testing. Due to theresearch tradition in China with its focus on knowledge dissemination,a fair number of the published articles on language testing in Chineseacademic journals are review articles or state-of-the-art articles synthesized by known researchers in the field of language testing inChina. These publications offer an insight into the kind of discussions on language testing and the theories and models of languagetesting that have been introduced to Chinese academics and researchers(see Han, 1995, 2003). However, due to the space limitation, I haveonly reported empirical studies here. A fair number of empiricalstudies have been conducted in China and published in Chinese academic journals. Only a few have been published in academic journalsoutside China. These studies are primarily in the areas of test validity, testing of speaking, test-taker characteristics; and test washback,which represent the main issues in language testing within the Chinesecontext.1 Test validityThe issue of test validity has drawn a fair amount of attention fromChinese language testing researchers. Much has been discussed aboutthe validity of a widely used test format in China – multiple-choicequestions. Considering the huge population of millions of test-takerseach year in China, many of the language tests – designed and administered in China as mentioned above – adopt large numbers of multiplechoice questions in order to save time and manpower that wouldotherwise be involved in grading the test papers. Sun (2000) conducted a study to evaluate the test items in a language proficiency testfor specific purposes. Each test item was examined in three aspects:item difficulty, passing rate, and discrimination. The results were usedto validate an item, and to determine whether the distractors for eachitem are overly distracting or whether the test-takers had the requiredDownloaded from http://ltj.sagepub.com at QUEENS UNIV LIBRARIES on February 17, 2008 2008 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.

083743 LTJ 15-38.qxd2222/12/079:06 AMPage 22The key to successlanguage knowledge. This study has implications for both test constructors as a basis for improving the items, and for teachers to drawattention to aspects of language knowledge that may have been neglected in their teaching.Wang (1996) conducted a comparative study of multiple-choiceand true–false questions to evaluate the effectiveness of the latter asa test format. A test with 60 multiple-choice questions and a test with60 true–false questions were administered to three groups of around30 students representing different levels of language proficiency.It was shown that the efficiency of true–false test was 1.18 timesgreater than that of the multiple-choice tests and that language proficiency level did not influence the time taken to complete the test.However, both the reliability and discrimination indices of thetrue–false test were lower than those of the multiple-choice test. Theresearcher concluded that the true–false test format can be used forclassroom assessment, but not in a large-scale standardized test.It is commonly believed among teachers and students in Chinathat students do not need to read carefully or even comprehend passages to pass a test, and that multiple-choice reading comprehensiontests do not accurately indicate students’ actual reading comprehension ability. Based on this assumption, Cheng and Gao (2002) explored the extent to which Chinese university students rely on readingpassages in answering multiple-choice reading comprehension tests.This study examined the reading test scores of groups of Chineseuniversity students taking the standardized multiple-choice readingcomprehension test of the College English Test in China undertwo different sets of testing conditions. Sample CET test papers wereused for this study. Under the first testing condition, the test performance was compared between one group of students who wereallowed to do the test as in a normal reading comprehension testingsituation and another which was allowed to read the passages onlyonce and then do the multiple-choice questions without going backand forth with the reading passages. The second condition comparedthe test performance of two groups of students in a with- or withoutpassage situation. The findings show firstly that students performbetter when they are not allowed to go back and forth between thequestions and passages when compared with a normal readingcomprehension test situation and second, that passage comprehension is relevant to reading comprehension test performance (i.e.students need to read the passages in order to answer most questionscorrectly). However, in the second condition, even without theassociated passages, students still achieved scores above chanceDownloaded from http://ltj.sagepub.com at QUEENS UNIV LIBRARIES on February 17, 2008 2008 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.

083743 LTJ 15-38.qxd22/12/079:06 AMPage 23Liying Cheng 23level (39.91% and 28.89% respectively on MC items with fourchoices) with some test items showing a mean item difficulty of 0.94and 0.88. This suggested that the guessing was prevalent among theparticipants in this study.Zhou (2004) conducted a comparability study of two national tests –CET-6 (for non-English majors) and TEM-4 (for English majors) interms of test-takers, test scores, and test content. The results show modest covariance of the two tests with a Pearson correlation coefficient of0.712 ( p 0.01). The two types of test-takers were quite similar toeach other, with the only obvious difference being in the hours ofEnglish instruction they had received. The two tests are related, both intheir power to measure language ability, and in terms of test methods.The researcher concluded that either test could serve the purpose ofmeasuring language proficiency. This study provided insights andposed questions about the validity of the two big English tests in China.2 Testing speakingThe development of oral testing only started in the 1990s in China,where the tradition of language testing had solely focused on readingand writing. The Cambridge Business English Certificate (BEC) wasintroduced to China in 1993 with an oral component. The oral test in theTest for English Majors (TEM) battery started in 1994. The NationalMatriculation English Test – Oral Subtest (NMETOS) was formallyintroduced in 1995 in three provinces in China (see Li and Wang, 2000).The College English Test – Spoken English Test (CET-SET) was introduced in 1999 (see Jin, 2000b; Huang, 1999; Yang, 1999). Languagetesting researchers in China have conducted a number of empiricalstudies to investigate various issues concerning oral test development.Li and Wang (2000) reported on the development of the NationalMatriculation English Test – oral subtest. They discussed the many limitations including the huge size of the test candidature and the rigid limitation on human and time resources in China. Nevertheless the validityof the NMETOS format was asserted on the strength of its beinga message-based test of spoken interaction achieving a balance betweencontrol and spontaneity in the required spoken output, a union of theanalytical and the holistic approach to rating, and a combination ofthe single-examiner method in test administration and a doublemarking method for scoring. The authors argued that ‘the NMETOS isnot just a test that suits the conditions and meets the needs of China. Inthe wider context of language testing, it can claim to be a successfulinnovation in mass-scale oral testing’ (Li and Wang, 2000: 160).Downloaded from http://ltj.sagepub.com at QUEENS UNIV LIBRARIES on February 17, 2008 2008 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.

083743 LTJ 15-38.qxd2422/12/079:06 AMPage 24The key to successLi (1999) reported on the development of the CET-Spoken EnglishTest, which was based on Bachman and Palmer’s (1996) theoreticalframework of language test design. The first task of the test development was to specify the target language use domain and language usetasks for the test. A survey was conducted of the needs for speakingabilities in different work settings, with an attempt to find out thefuture real-life language use domain for target test-takers. The testdevelopers realized that the survey, because of its scale, was far frombeing complete in terms of specifying all the possible future languageuse domains for college/university students in China, and that thismight result in bias in test task design against certain sub-groups ofthe test-taker population. In addition, the requirements of the CollegeEnglish Teaching Syllabus with regard to speaking abilities were alsotaken into consideration. To choose suitable task types for the test asthe next step in test development, another two surveys were administered to college/university students and their English teachers. Thesesurveys explored the current speaking abilities of the students, and theprevalent oral activities in their classrooms. It also investigated characteristics of Chinese college/university students, such as age, educational background, aptitude, and attitudes towards speaking tests, asit was believed that these might influence test performance. Theresearcher stated that the development of the test was never an easytask and further research is needed. It is a daunting task to find aneffective way to assess the speaking ability of a large population ofnon-English major students in China. This is one of the reasons thatcurrently CET-SET is only accessible to a smaller number of students(those who passed the CET-4 with a score of 80 or above or the CET-6with a score of 75 or above) given the time and cost involved in thetest administration and rater training.He and Dai (2006) conducted a corpus-based investigation intothe validity of the CET-SET. They examined the degree of interaction among candidates in the group discussion task with respect toa set of interactional language functions (ILFs) to be assessed. Theirresults showed a low degree of interaction among candidates in thistask. The researchers discussed a variety of factors that may explainthe low degree of interaction and they suggest that ‘the inadequateelicitation of ILFs from the candidates may well pose a problem formeasuring their speaking ability in terms of the ability to engage incommunicative interaction’ (p. 393).In an attempt to find an alternative way of conducting a large-scalespeaking test, Xiong, Chen, Liu, and Huang (2002) carried out anexperimental study on a semi-direct oral test, or ‘Recording OralDownloaded from http://ltj.sagepub.com at QUEENS UNIV LIBRARIES on February 17, 2008 2008 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.

083743 LTJ 15-38.qxd22/12/079:06 AMPage 25Liying Chen

18 The key to success Public English Testing System (PETS), the Cambridge Business English Certificate (BEC), and the WSK – an examination