Issued March 2004 Census 2000 Topic Report No. 12 Content .

Executive SummaryThe purpose of this report is tosynthesize results from Census2000 evaluations and other assess4ments of the quality of census dataon population and housing charac4teristics and to make recommenda4tions for planning the 2010 cen4sus.The formal studies of content anddata quality included in the Census2000 Testing, Experimentation,and Evaluation Program includefive studies covering— 5nonresponse and imputation forthe 100 percent census items, 5response variance for mostitems included in Census 2000,and 5consistency of information fromselected census items with simi4lar information from other datasources.The analysis included here also isbased on a set of tabulations prepared for the National Academy ofSciences, Committee on NationalStatistics. These tabulations showinformation on allocation for miss4ing responses for the Census 2000sample (long form) questionnairein comparison with the 1990Census and the Census 2000Supplementary Survey. Alsoincluded is some limited informa4tion on the quality of Census 2000coding operations.Evaluation report B.1.b, Analysis ofItem Nonresponse Rates for the100 Percent Housing andPopulation Items from Census2000, provides the base set ofinformation on the quality of cen4U.S. Census Bureausus data. Consideration of patterns of nonresponse is critical forimproving question design, train4ing, and procedures and therebyimproving quality in future census4es. Nonresponse rates for the 100percent items in Census 2000 werefairly low overall, ranging from 1.1to 4.1 percent.In report B.1.a, Analysis ofImputation Rates for the 100Percent Person and Housing UnitData Items from Census 2000,imputation refers to assignmentand allocation. The data showthat, as expected, patterns ofimputation varied in a manner sim4ilar to nonresponse rates.Imputation rates ranged from alow of 2.0 percent for sex to 5.1percent for age and 5.5 percent fortenure. Imputation rates are high4er than nonresponse rates shownin these studies partially becauseof universe differences. Also,responses that were not meaning4ful or that were not consistent withother information for the personwere treated as blank and wereimputed. Short forms had some4what lower imputation rates thanthe same items on census longforms for all items except race.The biggest difference betweenshort and long form imputationrates was for the item on housingtenure, which was most likelyrelated to the design of the longform questionnaire.Except for the item on Hispanicorigin, self response question4naires had lower imputation ratesthan enumerator filled question4naires.Questionnaires received via theInternet or Telephone Question4naire Assistance had quite lowimputation rates. These forms ofresponse required a proactiveeffort on the part of the householdand, therefore, are likely to reflecta segment of the population highlymotivated to participate in thecensus.The tabulations prepared for theNational Academy of Sciencesshow that only a few sample itemsin Census 2000 had lower alloca4tion rates than in 1990. For one ofthese, the item on length of serv4ice in the Armed Forces, a ques4tionnaire design change made toimprove reporting appears to havebeen successful.Many sample items in Census2000 had at least double the rateof allocation that occurred in the1990 Census. At least part of thisincrease can be attributed to pro4cedural changes between 1990and 2000.The Census 2000 SupplementarySurvey was included as part ofCensus 2000 to determine whethersuch an independent sample sur4vey (the American CommunitySurvey) with essentially the samecontent as the census sample formcould be accomplished successful4ly during the same time period asthe census itself. One mightexpect the quality of these data tobe better than the Census 2000sample data since the exclusivefocus of the supplementary sampleand the American CommunitySurvey is to provide high qualitymeasures of social, economic andContent and Data Quality in Census 2000 1

housing variables. This expecta4tion was realized in terms of allo4cation rates.The purpose of the ContentReinterview Survey was to measurethe consistency of response toquestions asked in the census.Response variance can result froma number of factors, but highmeasures of variability usuallyindicate that the question needsimprovement, the concept is diffi4cult to measure in a setting that isprimarily self response, and/orthat the respondent was unable toprovide the information desired.The measures of simple responsevariance from the ContentReinterview Survey also reflectmethodological limitations of thereinterview and differencesbetween the Census 2000 andContent Reinterview Survey meth4ods.For the long form census questionsevaluated in the ContentReinterview Survey, the index ofinconsistency (measure ofresponse variance) ranged fromquite low levels under ten to somevery high levels over 75. Somequestions with extensive and criti4cal government uses like race, abil4ity to speak English, and selectedincome types had rather highmeasures of inconsistency.Possible reasons behind responsevariability include the subjectivenature of some questions, theeffect of rare occurrences on thecalculation of the index, recallproblems, and privacy concerns.To those individuals planning the2010 Census, questions with seri4ous quality problems signal theneed for further work in questiondesign, procedures, and conceptclarification. To users of Census2000 data, quality problemsshould be a warning to use datafrom these items with caution.2 Content and Data Quality in Census 2000A Census 2000 auxiliary assess4ment study compared employ4ment, income, and poverty datafrom Census 2000 to such datafrom the Current PopulationSurvey. This was conducted by theCensus Bureau in partnership withthe Bureau of Labor Statistics.Since the Current PopulationSurvey is designed specifically toprovide the measures of employ4ment and income at the nationaland state levels for the federalgovernment, these measures areviewed as standards against whichthe census results are compared.The differences between the twodata sources are such that addi4tional work is needed to explainthe reasons behind them. Someanswers may be forthcoming fromStudy B.7, not completed in timefor this report. That study match4es individual responses from theCurrent Population Survey with theresponses for the same person inCensus 2000.U.S. Census Bureau

1. IntroductionThis report summarizes key resultson content and data quality fromthe Census 2000 Testing,Experimentation, and EvaluationProgram. It also includes assess4ments of data quality from tabula4tions and reports not part of theformal program.U.S. Census BureauThe formal studies include analy4ses of nonresponse and imputationfor the 100 percent census items,response variance for most censusdata items, and consistency ofselected census information withsimilar data from other sources.The other assessments cover allo4cation for missing responses forsample (long form) Census 2000data items, comparison of sampledata completeness with the 1990Census and the Census 2000Supplementary Survey, and limitedinformation on the quality ofCensus 2000 coding operations.Content and Data Quality in Census 2000 3

This page intentionally left blank.

2. BackgroundCensus 2000 differed from the1990 Census in a number of waysthat directly or indirectly affectedthe quality of the resulting statis4tics. This report focuses on thequality of the characteristics dataproduced from the census, not thecompleteness of the populationcounts. The studies summarizedherein cannot directly measure theeffect of any one factor on thequality of a particular statistic orset of numbers but it is often pos4sible to theorize about such rela4tionships.Among changes designed to affectcensus content results werechanges to the wording, format orplacement of questions between1990 and 2000. For example, thequestion on Hispanic origin wasplaced before the question on racein an effort to improve the leveland quality of response. Detailson such changes are contained in anumber of formal documents ontesting done in preparation forCensus 2000, which are cited inthe References section of thisreport.U.S. Census BureauThere were also changes with apossible indirect effect on the content quality. For the first time inCensus 2000, names of personsentered on the questionnaire werecaptured in computer-readableform and could be used in process4ing. This was done primarily toimprove the population counts butit also allowed the "assignment" ofsex for individuals for whom thischaracteristic was not reported.short as possible and allowedOther procedural changes inCensus 2000 (as compared to1990) also had potential impact oncontent quality. For example, in1990 there was an operationdesigned to improve the complete4ness of information on the samplequestionnaires through a follow-upinterview for questionnaires withexcessive blanks. There was nosimilar operation in Census 2000.This may have served to increasenonresponse for questions on the2000 sample long form.the Census 2000 SupplementaryIn an attempt to encourage mailresponse in Census 2000 the basicmail questionnaire was made asthe C2SS for the same contentroom to record information foronly six household members.Since it was often difficult toobtain the information for personsin larger households, this too mayhave increased nonresponse forthe questions.Perhaps the most important content-related improvement inCensus 2000 was the inclusion ofSurvey (C2SS). The C2SS was con4ducted to demonstrate the feasibil4ity of collecting long form informa4tion at the same time as, but in aseparate process from the census.It used the American CommunitySurvey questionnaire and proce4dures and covered 1,203 countiesnationwide surveying 58,000households each month. As aresult, we have a comparison ofresults from the Census 2000 andareas. This will be a critical plan4ning tool for the 2010 Census.Content and Data Quality in Census 2000 5

This page intentionally left blank.

3. ScopeThe primary purpose of this topicreport is to summarize the findingsand recommendations from theformal studies of content and dataquality included in the Census2000 Testing, Experimentation,and Evaluation Program. Thesestudies include Census 2000 Evaluation B.1.a,Analysis of Imputation Rates forthe 100 Percent Person andHousing Unit Data Items fromCensus 2000 Census 2000 Evaluation B.1.b,Analysis of Item NonresponseRates for the 100 PercentHousing and Population Itemsfrom Census 2000U.S. Census Bureau Census 2000 Evaluation B.5,Census 2000 ContentReinterview Survey: Accuracy ofData for Selected Population andHousing Characteristics asMeasured by Reinterview Census 2000 Evaluation B.7,Current Population Survey (CPS) Census 2000 Match Study (notavailable at the time this reportwas prepared) Census 2000 AuxiliaryAssessment, ComparingEmployment, Income, andPoverty: Census 2000 and theCurrent Population SurveyThe analysis also is based on a setof tabulations prepared for theNational Academy of Sciences,Committee on National Statistics,showing information on allocationfor missing responses for theCensus 2000 sample (long form)questionnaire in comparison withthe 1990 Censu

Randall J. Neugebauer of the Planning, Research and Evaluation Division; Violeta Vazquez of the Decennial Management Division; and Frank A. Vitrano (formerly) of the Planning, Research and Evaluation Division. The Census 2000 TXE Program was coordinated by the Planning, Research and Evaluation Division: Ruth Ann