Health Information National Trends Survey (HINTS) 2007 .

Health Information NationalTrends Survey (HINTS) 2007FINAL REPORTAuthors:David Cantor, PhDKisha Coa, MPHSusan Crystal-Mansour, PhDTerisa Davis, MPHSarah Dipko, MSRichard Sigman, MSFebruary 2009Prepared for:National Cancer Institute6120 Executive BoulevardBethesda, MD 20892-7195Prepared by:Westat1650 Research BoulevardRockville, MD 20850

Table of ContentsChapter123PageIntroduction .1-11.11.2Background .Mode of HINTS 2007.1-11-1Pretesting Methods and Results .2-12.12.2Testing of Advance Materials.Pilot Studies .2-12-22.2.12.2.2RDD Pilot Study.Mail Pilot Study .2-32-5Instrument Development .3-13.1Questionnaire Development .3-13.1.13.1.2Working Groups .Question Tracking System .3-13-2CATI Instrument Cognitive Testing.Mail Questionnaire Development .3-33-33.3.13.3.23.3.3Mail Cognitive Testing: Round 1.Mail Cognitive Testing: Round 2.Mail Cognitive Testing: Round 3.3-43-53-5Final Instruments .3-6RDD Study Design and Operations .4-14.1Sample TS 2007 Final ReportSize of RDD Sample .Stratification by Mailable Status .Subsampling of Screener Refusals.i

Contents (continued)ChapterPage4.24.35Summary of RDD Operations.4-34.2.14.2.24.2.3Staffing and Training .Advance Materials .Calling Protocol .4-34-44-4Findings From the CATI Operations 4-94-104-114-124-144-14Weekly Reports .Administration Times .Average Calls per Case.Cooperation Rates and Refusal Conversion .Results of Hispanic Surname Coding .Data Retrieval .Imputation .Interview Data Processing.Mail Study Design and Operations .5-15.1Sample Selection.5-15.1.15.1.2Sampling Frame for Address Sample .Selection of Main-Survey Address Sample .5-15-3Mail Survey Operations .5-45.2.15.2.2Questionnaire Mailing Protocol .Interactive Voice Response (IVR) Experiment .5-45-5Findings From the Mail -125-125.25.3HINTS 2007 Final ReportWeekly Reports .Telephone Contacts .IVR Experiment Results.Survey Processing .Imputation .ii

Contents (continued)Chapter67PageCombined Data Set and Accompanying Metadata .6-16.16.26.3Combining Data Sets.Codebooks .Metadata Development .6-16-16-2Sample Weights and Variance Estimation Overview .7-17.17.27.37.4Overview of Sample Weights.Variance Estimation Methodology for HINTS 2007 .Base Weights .Nonresponse Adjustment .7-17-27-47-57.4.17.4.2RDD Screener Nonresponse Adjustment .RDD Extended Interview NonresponseAdjustment .Address-Sample Nonresponse Adjustment .Replicate Nonresponse Adjustment .7-6Calculation of Composite Weights.Calibration Adjustments .7-97-97.6.17-107.4.37.4.47.57.68Control Totals .7-67-87-9Response Rates .8-18.1RDD Sample.8-18.1.18.1.28.1.3RDD Screener Response Rate.RDD Extended Interview Response Rate .RDD Overall Response Rate.8-28-48-4Address-Sample Response Rate.8-58.2.18.2.28.2.3Address-Sample Household Response Rate.Within Household Response Rate .Overall Response Rate.8-58-68-6References.R-18.2HINTS 2007 Final Reportiii

Contents (continued)AppendixesPageARDD Pilot Study Letters and Introductions .A-1BRDD Main Study Advance Letter.B-1CRDD Information Request Letter .C-1DRDD Screener Refusal Conversion Letter.D-1ERDD Extended Refusal Conversion Letter.E-1FSample of Production Report by Release Group .F-1GSample Weekly TRC Report From NCI .G-1HMail Advance Letters, Cover Letters, and Postcards .H-1IDecisions for Combining CATI and Mail Data .I-1TablesPage2-1RDD pilot test sample size.2-32-2Incentive/mail mode treatment combinations.2-52-3Mail pilot field period schedule .2-62-4Household-level response rates by incentive and mail method.2-72-5Average proportion of questionnaires returned per household .2-74-1Unweighted RDD sample by mailable status .4-24-2Unweighted RDD sample results by mailable status.4-64-3Weekly TRC production: Completed cases by week .4-84-4Total screener level of effort: Number of call attempts by result .4-9HINTS 2007 Final Reportiv

Contents (continued)Tables4-5PageTotal extended (CATI) level of effort: Number of call attemptsby result .4-10Residential, cooperation, refusal conversion, and response ratesand yield by mailable stratum, for screener and extendedinterviews .4-114-7Data retrieval calls .4-135-1Mail survey schedule and protocol .5-55-2Household cooperation in the mail survey .5-65-3Household response by week .5-75-4Household response by mailing and strata .5-75-5IVR calls .5-105-6Live interviewer prompt calls.5-115-7Household response by treatment in IVR experiment .5-118-1Weighted estimates of percentages of residential telephonenumbers that are residential in the HINTS 2007 RDD sample .8-3Screener response rate calculations for the HINTS 2007 RDDsample .8-3Extended interview response rate calculations for HINTS 2007RDD sample .8-48-4Overall response rate calculations for HINTS 2007 RDD sample.8-48-5Household response rate calculations for the HINTS 2007address sample .8-5Weighted within-household response rate calculations forHINTS 2007 address sample .8-64-68-28-38-6HINTS 2007 Final Reportv

8-7Overall response rate calculations for HINTS 2007 addresssample .HINTS 2007 Final Reportvi8-6

Introduction1The National Cancer Institute’s (NCI’s) Health Information National Trends Survey (HINTS)collects nationally representative data about the U.S. public's use of cancer-related information. Thisstudy, increasingly referenced as a leading source of data on cancer communication issues, wasdeveloped by the Health Communication and Informatics Research Branch (HCIRB) of theDivision of Cancer Control and Population Sciences (DCCPS) as an outcome of NCI’sExtraordinary Opportunity in Cancer Communications. HINTS strives to: provide updates onchanging patterns, needs, and information opportunities in health; identify changing healthcommunications trends and practices; assess cancer information access and usage; provideinformation about how cancer risks are perceived; and offer a test-bed to researchers to investigatenew theories in health communication. HINTS data collection is conducted every 2-3 years in orderto provide trends in the above areas of interest. This report presents a summary of the third roundof HINTS data collection known as HINTS 2007.1.1 BackgroundThe first round of HINTS, administered in 2003, used a probability-based sample, drawing onrandom digit dialing (RDD) telephone numbers as the sample frame of highest penetration at thattime. Due to an overall decline in RDD rates, the second cycle of HINTS, HINTS 2005, includedembedded methodological experiments to compare data collected by telephone with data collectedthrough the Internet. In addition, the field study explored the impact of various levels of incentiveson response rates. Unfortunately, providing respondents with an Internet alternative, a monetaryincentive for nonresponse conversion, and having an operations priority on nonresponse conversionwere not successful in reducing the impact of falling response, and the overall response rate forHINTS 2005 was lower than expected.1.2 Mode of HINTS 2007In an effort to address dropping RDD response rates, NCI turned to work done at the Centers forDisease Control and Prevention (CDC) on the Behavioral Risk Factor Surveillance System (BRFSS).HINTS 2007 Final Report1-1

Introduction1BRFSS data collection has recently included experiments with mail surveys and mixed mode datacollection (mail and telephone). Recent research by Link and colleagues (2008) suggests that use of amail survey, with appropriate followup, can achieve a higher response rate than RDD alone. Oneexperiment (Link & Mokdad, 2004) found that a mail survey led to significantly more responses thana web survey (43% vs. 15%), and that a mail survey with a telephone followup produced asignificantly higher response rate than a RDD telephone survey (60% vs. 40%).Following the model provided by BRFSS, HINTS 2007 used a dual-frame design that mixed modesin a complementary way. One frame was RDD, using state-of-the-art procedures to maximize theresponse rate. The second frame was a national listing of addresses available from the United StatesPostal Service (USPS). This list is relatively comprehensive (Iannacchione et al., 2003) and includesboth telephone and nontelephone households. These households were administered a mail survey.The study was designed to complete 3,500 interviews with the RDD and 3,500 from the USPSframe. National estimates were developed by combining the two frames using a compositeestimator.There are a number of advantages of this dual-frame design. One is that using two modes offers thepotential for improving coverage over a design that exclusively relies on RDD. In addition tolandline telephone users, the use of the USPS frame also allows for the coverage of mobile-onlytelephone users and those without a telephone. This directly addresses the increasing difficulty RDDsurveys have with reaching those who do not regularly use a landline telephone. There is also thepossibility of improved measurement for a number of characteristics (e.g., those subject to socialdesirability bias). Moving to a dual frame leaves open the opportunity to implement other modes inthe future if they are found to be appropriate.Link and Mokdad (2004) report that unit response rates between the two modes for theirexperiment with the BBRFSS were generally equivalent. An important issue discussed was thetendency for mail respondents to have characteristics associated with higher socioeconomic status,such as higher income, majority race, and higher education. This finding is consistent with otherstudies that have examined characteristics of nonrespondents to mail surveys (e.g., Hauser, 2005).The design of the HINTS mail survey was developed to maximize response rate while minimizingthe potential for nonresponse bias. In addition, experiments with incentives and delivery methodswere conducted in an attempt to decrease the different nonresponse bias patterns that emerge formail surveys (i.e., lower response rates by levels of education and minority status).HINTS 2007 Final Report1-2