Measurement Error Studies
NATIONAL CENTER FOR EDUCATION STATISTICSMeasurementError Studiesat the National Centerfor Education StatisticsU.S. Department of EducationOffice of Educational Research and ImprovementNCES 97-464
U.S. Department of EducationRichard W. RileySecretaryOffice of Educational Research and ImprovementRamon C. CortinesActing Assistant SecretaryNational Center for Education StatisticsPascal D. Forgione, Jr.CommissionerThe National Center for Education Statistics (NCES) is the primary federal entity for collecting,analyzing, and reporting data related to education in the United States and other nations. It fulfills acongressional mandate to collect, collate, analyze, and report full and complete statistics on the conditionof education in the United States; conduct and publish reports and specialized analyses of the meaningand significance of such statistics; assist state and local education agencies in improving their statisticalsystems; and review and report on education activities in foreign countries.NCES activities are designed to address high priority education data needs; provide consistent, reliable,complete, and accurate indicators of education status and trends; and report timely, useful, and highquality data to the U.S. Department of Education, the Congress, the states, other educationpolicymakers, practitioners, data users, and the general public.We strive to make our products available in a variety of formats and in language that is appropriate to avariety of audiences. You, as our customer, are the best judge of our success in communicatinginformation effectively. If you have any comments or suggestions about this or any other NCES productor report, we would like to hear from you. Please direct your comments to:National Center for Education StatisticsOffice of Educational Research and ImprovementU.S. Department of Education555 New Jersey Avenue, NWWashington, DC 20208July 1997The NCES World Wide Web Home Page address ishttp://www.ed.gov/NCES/Suggested CitationU.S. Department of Education. National Center for Education Statistics. Measurement Error Studies atthe National Center for Education Statistics, NCES 97-464, by Sameena Salvucci, Elizabeth Walter,Valerie Conley, Steven Fink, and Mehrdad Saba. Project Officer: Steven Kaufman. Washington, DC:1997.Contact:Steven Kaufman(202) 219-1337For free single copies of this publication, call the National Data Resource Center at (703) 845-3151 orsend a FAX request to (703) 820-7465.2
PrefaceThis report provides an overview of a selection of measurement error studies conductedon National Center for Education Statistics (NCES) surveys. Its intent is not to offer newanalysis of program data, rather it summarizes information from internal memoranda,working papers, and adjudicated reports. The purpose of this report is to illustrate thediversity of NCES efforts in this area.The emphasis of our review is on reinterview studies, but other types of empirical studiesof measurement error including “multiple indicators” studies, record check studies, andcognitive studies will also be described. The report is not meant to be comprehensive, butcan be used as a reference for types of items within surveys at NCES that have beenexamined, the techniques used for measuring and controlling measurement error, as wellas the results of the examinations.235
AcknowledgmentsThe authors wish to extend grateful thanks to all of those who contributed time,knowledge, and efforts to this report. National Center for Education Statistics (NCES)staff were enormously supportive. Our project officer, Steve Kaufman, provided guidancefrom beginning to end, but Dan Kasprzyk and Marilyn McMillen, Senior TechnicalAdvisor, also reviewed several versions of the report. Project officers (past and current)for all of the surveys we discuss in this report helped us locate material, answered ourquestions, and reviewed their respective sections: from the Data Development andLongitudinal Studies Group, C. Dennis Carroll, Aurora D’Amico, Paula Knepper, andAndrew G. Malizio, and from the Surveys and Cooperative System Group, Dan Kasprzyk,Kathryn Chandler, and Peggy Quinn, Peter Stowe, and Linda Zimbler. Sue Ahmed, ChiefStatistician of NCES, Statistical Standards and Services Group, chaired the adjudicationcommittee. Ralph Lee, Data Development and Longitudinal Studies Group, reviewed theentire report. Irwin Schreiner, Bureau of the Census, Michael Brick, Westat, and SylviaFisher, Bureau of Labor Statistics, also reviewed the entire report, and all providedextensive and useful comments. The work was sponsored by the Surveys and CooperativeSystems Group.Among Synectics staff, principal thanks goes to Dhiren Ghosh, who spent many hoursworking through the statistical models and formulas. Mindy Reiser, Steven Wenck, andGeorge Carter also read and criticized various portions of the report. Kevin Jones createdthe figures.Table of Contents236
Preface . iiiAcknowledgments . ivAcronyms. xiChapter 1 Introduction and Overview . 1Measurement Errors in the Context of Other Survey Errors . 1Conceptual Structure of Survey Errors . 3Languages of Measurement Error . 5Empirical Estimation of Survey Measurement Error. 9Chapter 2 Profile of NCES Reinterview Studies. 21Characteristics of NCES Reinterview Studies. 22Methodological Issues in NCES Reinterview Studies . 24Lessons Learned from NCES Reinterview Studies . 26Detailed Findings from NCES Reinterview Studies . 27Chapter 3 Reinterview Studies: Simple Response Variance and Response Bias. 31Simple Response Variance Model . 31Response Bias Model. 36GDR, NDR, and IOI Results in this Report. 39National Household Education Survey (NHES) . 41NHES:91 Early Childhood Education Reinterview. 43NHES:93 School Readiness and School Safety and Discipline Reinterviews. 50NHES:95 Adult Education Reinterview . 60Review of NHES Reinterviews . 63NHES:95 Adult Education Bias Study . 64Recent College Graduates (RCG:91) Reinterview Study . 68Schools and Staffing Survey (SASS) Reinterview Studies . 73School Administrator Reinterviews (SASS 1987-88 and 1990-91) . 75School Reinterviews (SASS 1987-88 and 1990-91) . 78Teacher Reinterviews (SASS 1987-88 and 1990-91). 82Summary Comparison of the SASS 1987-88 and 1990-91 Reinterviews. 89Library Survey Reinterview Study (SASS 1993-94). 90Teacher Followup Survey (TFS) Reinterview Studies . 95TFS 1989-89 Reinterview Study. 95TFS 1991-92 Reinterview Study. 97Comparing the 1991-92 Model to the 1988-89 Methodology. 102Baccalaureate and Beyond Longitudinal Study (B&B:93/94) Field TestReinterview Study. 104Chapter 4 Reinterview Studies: Reliability and Validity . 111Categories of Reliability Measurements. 115237
Beginning Postsecondary Students (BPS) Longitudinal Study ReinterviewStudies . 115BPS:90/92 First Followup Field Test Reliability Reinterview. 116BPS:90/92 First Followup Full Scale Reliability Reinterview. 120BPS:90/94 Second Followup Field Test Reliability Reinterview . 123National Postsecondary Student Aid Study (NPSAS) Field Test ReinterviewStudies . 131NPSAS:1992-93 Field Test Reliability Reinterview. 131NPSAS:96 Field Test Reliability Reinterview. 134National Study of Postsecondary Faculty (NSOPF) Reinterview Study . 139Chapter 5 “Multiple Indicators” Studies . 147High School and Beyond (HS&B) . 149Methodology and Design of HS&B Base Year Quality of Responses Study . 149Summary of Results for the HS&B Study . 151National Education Longitudinal Study of 1988 (NELS:88). 164Methodology and Design of NELS:88 Base Year Quality of Responses Study . 164Summary of Results for the NELS:88 Study . 167National Study of Postsecondary Faculty (NSOPF). 172Chapter 6 Record Check Studies. 179Record Check Model. 180Findings from NCES Record Check Studies. 183Recent College Graduates (RCG) . 183Schools and Staffing Survey (SASS). 186Chapter 7 Cognitive Studies . 195Cognitive Research Methodology . 195Cognitive Research Results. 196Beginning Postsecondary Students (BPS) Longitudinal Survey . 197National Household Education Survey (NHES) . 201Schools and Staffing Survey (SASS). 205How to Count Teachers, Part 1: Teacher List Validation Study (TLVS) . 205How to Count Teachers, Part 2: SASS Teacher Listing Form Study . 208SASS Public School 1991-92 Field Test Questionnaire . 210Chapter 8 Summary. 217References. 220AppendixA Summary of a Reinterview Study on Mode Effects for the 1990-91 Schoolsand Staffing Survey (SASS) School Survey (1990-91) . 233List of TablesTable 1.Cognitive laboratory research methods currently being used in the United238
Table 2.Table 3.Table 4.Table 5.Table 6.Table 7.Table 8.Table 9.Table 10.Table 11.Table 12.Table 13.Table 14.Table 15.Table 16.Table 17.Table 18.Table 19.Table 20.Table 21.Table 22.Table 23.Table 24.Table 25.States to study the question-answering process. 15Reinterview sample size, sampling percentage of original sample, responserate, and primary study purpose for studies. 22Interview by reinterview table. 35Uses of reinterview statistics, by type of reinterview response. 391991, 1993, and 1995 NHES components and reinterview studies. 41NHES:91 ECE reinterview study . 43NHES:91 average rank of subject area measurements for preprimary andprimary school children items . 45NHES:91 average rank of subject area measurements for preprimaryschool children items . 46NHES:91 average rank of subject area measurements for primary schoolchildren items. 47NHES:93 School Readiness (SR) and School Safety & Discipline (SS&D)reinterview studies . 50NHES:93 average rank of subject area measurements for the SchoolReadiness (SR) reinterview . 51NHES:93 average rank of subject area measurements for the SchoolSafety & Discipline (SS&D) reinterview of parents of 3rd through 5thgraders. 54NHES:93 average rank of subject area measurements for the SS&Dreinterview of parents of 6th through 12th graders. 55NHES:93 average rank of subject area measurements for the SS&Dreinterview of 6th through 12th graders. 56NHES:95 Adult Education (AE) reinterview study. 60NHES:95 items identified for further consideration in the AdultEducation (AE) reinterview. 62NHES:95 Adult Education (AE) reinterview bias study . 66RCG:91 reinterview study . 69RCG:91 average rank of reinterview subject area measurements. 71Schools and Staffing Survey (SASS) components and selected reinterviewstudies. 74SASS School Administrator Survey reinterview study . 751990-91 SASS School Administrator Survey average rank of reinterviewsubject area measurements . 77SASS School Survey reinterview study . 791990-91 SASS School Survey average rank of reinterview subject areameasurements. 80SASS Teacher Survey reinterview study. 83239
Table 26. 1990-91 SASS Teacher Survey average rank of reinterview subject areameasurements. 84Table 27. Comparison of 1987-88 and 1990-91 SASS Teacher Survey summaryof IOI results. 87Table 28. Summary of 1987-88 and 1990-91 SASS reinterview response varianceresults . 90Table 29. SASS Library Survey reinterview study. 91Table 30. 1993-94 SASS Library Survey average rank of reinterview subject areameasurements. 93Table 31. 1988-89 TFS reinterview study . 96Table 32. 1988-89 TFS summary IOI results. 96Table 33. 1991-92 TFS reinterview study . 98Table 34. 1991-92 TFS current teachers: average rank of reinterview subject areameasurements. 99Table 35. 1991-92 TFS former teachers: average rank of reinterview subject areameasurements. 100Table 36. Before reconciliation GDRs: 1988-89 versus 1991-92 . 103Table 37. B&B:93/94 field test reinterview study. 105Table 38. B&B:93/94 field test reinterview subject areas and items withdisagreement levels greater than 20 percent . 108Table 39. Measures of association used to compute reliability in the BPS,NPSAS, and NSOPF field test and full scale reinterview studiesincluded in this report. 112Table 40. BPS:90/92 first followup field test reliability reinterview study . 117Table 41. BPS:90/92 first followup field test reliability reinterview studymedian subject area measurements. 117Table 42. BPS:90/92 first followup full scale reliability reinterview study . 120Table 43. BPS:90/92 full scale reinterview median subject area measurements . 121Table 44. BPS:90/94 second followup field test reliability reinterview study. 124Table 45. BPS:90/94 second followup field test reliability reinterview studymedian subject area measurements. 125Table 46. NPSAS:1992-93 field test reliability reinterview study. 131Table 47. NPSAS:1992-93 field test reliability reinterview study median subjectarea measurements . 133Table 48. NPSAS:96 field test reliability reinterview study. 135Table 49. NPSAS:96 field test reliability reinterview study median subject areameasurements. 136Table 50. NSOPF-93 field test reliability reinterview study. 140Table 51. “Multiple Indicators” studies source of data, sample size, number ofmatched pairs, and match rate . 148Table 52.
239 States to study the question-answering process .15 Table 2. Reinterview sample size, sampling percentage of original sample, response
Min Longitude Error: -67.0877 meters Min Altitude Error: -108.8807 meters Mean Latitude Error: -0.0172 meters Mean Longitude Error: 0.0028 meters Mean Altitude Error: 0.0066 meters StdDevLatitude Error: 12.8611 meters StdDevLongitude Error: 10.2665 meters StdDevAltitude Error: 13.6646 meters Max Latitude Error: 11.7612 metersAuthor: Rafael Apaza, Michael Marsden
We can overcome these medication errors by educating physicians, nurses regarding the areas where medication errors are more prone to occur. Key words: Medication error, Prescribing error, Dispensing error, Administration error, Documentation error, Transcribing error, EPA (Electronic prior authorization), Near miss, Missed dose. INTRODUCTION
Physics 215 - Experiment 1 Measurement, Random Error & Error analysis σ is a measure of the scatter to be expected in the measurements. If one measured a large number of
LG Air Conditioning Multi F(DX) Fault Codes Sheet Macedo - November 2007 - 8 - Fault Code 07 On Multi split systems, the first unit switched on is the cool heat master, the master tells the condensing unit what to do. If the condenser is in heating and any slave is set to cooling a CH07 fault code will appear.File Size: 863KBPage Count: 19Explore furtherLG AC Error Codes and Troubleshootingacerrorcode.comLG AC Error Code Solution Inverter Air Conditioner HVAC .helpdeskminority.comHow-to & Tips: Error Codes - Room Air Conditioner LG .www.lg.comFix Lg Art Cool Error Code Ch 07 (Solved)cdbug.orgRecommended to you b
LAMPIRAN 9 Perhitungan Reliabilitas Angket Uji Coba Self Efficacy. Error! Bookmark not defined. LAMPIRAN 10 Skor Angket Uji Coba Self Efficacy. Error! Bookmark not defined. LAMPIRAN 11 Kisi-kisi Angket Self Efficacy.Error! Bookmark not defined. LAMPIRAN 12 Angket Self Efficacy.Error! Bookmark not defined.
TEST 1 SECTION 1 TASK 1 - SPELLING No. Error Correction No. Error . . sea. ) ) )
1. Describe the role that biostatistics serves in biomedical and public health research. 2. Describe the basic principles and practical importance of random variation, systematic error, sampling error, measurement error, hypothesis testing, type I and type II errors, and con dence l
JIS are possible. Moreover, the 6240B is capable of high-precision contact resistance measurement that cancels thermal EMF generated on metal contact surfaces. Source and Measurement Function The source and measurement functions are selectable from voltage source, current source, voltage measurement, cur-rent measurement and resistance measurement.
