How Much Does Adding Cell Phone Sample Reduce Demographic Biases? A Case Study d 2011 IFD&TC Scottsdale, AZ Presented by Brian Harnisch Wyoming Survey & Analysis Center (WYSAC), University of Wyoming Outline Questions of interest Change in prevalence of cell phone-only households Demographic characteristics of adults living in CPO HHs Wyoming Crime Victimization Survey, 2011 Moving forward Questions of interest… How much does adding g cell p phone sample p to g general population surveys reduce the potential for demographic biases? Can adding cell phone sample help avoid the need ffor h heavy weighting, i hti quota t sampling, li etc., t while hil remaining a cost effective option? The Changing Landscape National Health Interview Survey (NHIS) Estimates of the prevalence of cell phone-only households produced and released by the National Center for Health Statistics (NCHS) and the Center for Disease Control (CDC) Wireless Substitution: State-level Estimates from the NHIS (Blumberg & Luke, 2011) Wireless Subsitution: Early Release of Estimates from the NHIS (Blumberg & Luke, 2010) The Changing Landscape More than a quarter of American households (26.6%) are now cell phone-only households 8x increase in the prevalence of cell phone-only HHs in the last 6 years At the same time, the number of landline-only homes continues to decrease The Changing Landscape 80.0% 70.0% 60.0% 58.1% 50.0% Landline households with a wireless telephone (Dual) 40.0% Landline households without a wireless telephone (LLO) 30 0% 30.0% 26.6% Wireless-only households (CFO) 20.0% 10.0% 12.9% 0.0% (Blumberg & Luke, 2011) The Changing Landscape Significant State diff differences… Adults living in CFO HHs, b state by 35.2% in AR 22.3% in WY 12.8% in RI Compared p to 26.6% nationally (Figure via Blumberg & Luke, 2010) Don’t forget about cell phone-mostly households! Add in the households with LLs that receive almost all of their calls on a cell and some states now have over 50% of all adults that are largely reachable only b cell by ll phone h 52.8% CPM adults in Texas 35 4% CPM adults in Wyoming 35.4% 24.9% CPM adults in South Dakota General demographics of CPO HHs 5 51.3% 3 of all adults aged g 25-29 5 9 lived in CPO HHs The same group accounts for 39.8% of all CPO adults 26.2% of men lived in CPO, 23.7% of women But, the gender distribution is split evenly when looking only at cell phone only HH adults Adults living in poverty more likely to be CPO Hispanic adults more likely to be living in CPO HHs than non non-Hispanic Hispanic Wyoming Crime Victimization Survey, 2011 Statewide telephone survey • Funded through the BJS State Justice Statistics (SJS) project. • Simple within-household random adult selection for LL sample, no selection for cell sample • 70% Landline sample (listed) • 30% Cell phone sample • Why listed? In a perfect world, we would prefer RDD and Cell Proposed a listed LL and cell phone dual-frame instead of a single frame RDD sample in an effort to meet budget requirements. Cost/benefit / trade-off Raw Completion Rates Wave Completion Rate Calendar Days in field 1 25.20% 29 2 23 80% 23.80% 25 3 22.60% 18 4 20.80% 15 5 20 40% 20.40% 10 6 19.60% 8 7 17.00% 5 Wtd Avg 22 60% 22.60% Cell 8 5.60% 30 Cell 9 5.90% 22 Cell 10 Cell 10 5.80% 11 Wtd Avg 5.70% Completions 1,451 ,45 total completions p 171 from cell sample (11.8%), 1,280 from LL (88.2%) 196 9 completed p on cell p phone ((13.4%) 34 ) 2.1% of completions from LL sample on cell phone 97.7% of completions from cell sample on cell phone Duall users not screened d out 17 minute average length No N iincentives ti offered ff d Typically very difficult to get completions male and/or from younger respondents in WY Completions totals by telephone status Telephone Status Wyoming Wyoming Estimates (NHIS) LL Sample Cell Sample Survey Total CPO 22 3% 22.3% 0 4% 0.4% 50 3% 50.3% 6 3% 6.3% CPM 13.1% 12.7% 17.5% 13.2% Dual‐use 22.0% 47.0% 25.1% 44.4% LLM 5.9% 28.0% 7.0% 25.5% LLO 35.5% 12.0% ‐‐ 10.6% 98.8% 100.0% 100.0% 100.0% Results by telephone status Telephone Status Wyoming Wyoming Estimates (NHIS) LL Sample Cell Sample Survey Total CPO 22 3% 22.3% 0 4% 0.4% 50 3% 50.3% 6 3% 6.3% CPM 13.1% 12.7% 17.5% 13.2% Dual‐use 22.0% 47.0% 25.1% 44.4% LLM 5.9% 28.0% 7.0% 25.5% LLO 35.5% 12.0% ‐‐ 10.6% 98.8% 100.0% 100.0% 100.0% Gender results Gender WY Adult Pop. Est. American CPO HHs LL Sample Cell Sample Survey Total Male 50.7% 50.9% 44.0% 59.0% 45.8% Female 49.3% 49.1% 56.0% 41.0% 54.2% 100.0% 100.0% 100.0% 100.0% 100.0% Age group distribution results Age WY Adult Pop. Est. American CPO HHs LL Sample Cell Sample Survey Total 18‐24 years 14.5% 20.7% 1.3% 12.1% 2.5% 25‐34 years 18.2% 33.0% 5.7% 20.6% 7.4% 35‐44 years y 15.6% 18.9% 9.6% 24.2% 11.3% 45‐64 years 35.6% 23.7% 44.8% 30.9% 43.2% 65 years and over 16.2% 3.7% 38.6% 12.1% 35.5% 100 0% 100.0% 100 0% 100.0% 100 0% 100.0% 100 0% 100.0% 100 0% 100.0% Educational attainment results Educational Attainment WY Adult Pop. American CPO LL Sample Est. HHs Cell Sample Survey Total High school graduate or less 42.0% 43.3% 29.8% 34.1% 30.3% Some college/AA/Tech 37.3% 32.0% 35.7% 32.4% 35.3% College graduate/Graduate school 20.8% 24.7% 34.5% 33.5% 34.4% 100.0% 100.0% 100.0% 100.0% 100.0% Hispanic or Latino Hispanic or Latino, any race(s) spa c o at o, a y ace(s) WY Adult Pop. American CPO LL Sample Sa p e E Est. HH HHs Cell Sample Ce Sa p e Survey T l Total Yes 7.5% 19.4% 3.7% 7.6% 4.2% No 92.5% 80.6% 96.3% 92.4% 95.8% 100.0% 100.0% 100.0% 100.0% 100.0% Adults living with children under age 18 Living with children under 18 WY Adult Pop. American CPO LL Sample Est Est. HHs Cell Sample Survey Total Yes 31.5% 40.9% 21.7% 43.9% 24.3% No 68.5% 59.1% 78.3% 56.1% 75.7% 100.0% 100.0% 100.0% 100.0% 100.0% A few by telephone status - Age Age WY Adult Pop. Est. American CPO HHs CPO/CPM Dual‐use HH LLO/LLM 18‐24 18 24 years years 14.5% 20.7% 8.0% 1.3% 1.2% 25‐34 years 18.2% 33.0% 20.4% 6.3% 1.8% 35‐44 years 15.6% 18.9% 19.7% 13.2% 4.6% 45‐64 years 35.6% 23.7% 39.8% 53.0% 32.7% 65 years and over 16.2% 3.7% 12.0% 26.2% 59.7% 100.0% 100.0% 100.0% 100.0% 100.0% A few by telephone status - Gender Gender Male Female WY Adult Pop. WY Adult Pop American CPO American CPO Est. HHs 50.7% 50.9% CPO CPM Dual‐use LLM LLO 46.2% 54.7% 45.7% 42.8% 42.2% 57.2% % 57.8% % 49.3% % 49.1% % 53.8% % 45.3% % 54.3% % 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Other differences – our respondents Respondents p from our cell p phone sample p more likelyy to have any internet connection (88%) More likely to have a high-speed internet connection (76.1%) Of those with high-speed connections, respondents f from th the cell ll sample l were nott more likely lik l tto prefer f to have taken the survey over the internet than those from the listed LL sample. sample What does this all mean? So,, how much cell phone p sample p would it have taken to match the actual demographic distributions in WY? Is it worth trying to reduce the potential need for heavy weighting h i hti b by adding ddi more and d more cell ll phone sample up front? Eliminate simple weighting? Would take 6x more cell completions p to hit true WY adult gender distribution without weighting. Initial cell sample increases to over 18k necessary from the original 3k, 3k holding LL sample constant at 7k. 7k Would take approximately 36x more initial cell phone sample to bring to hit true WY adult age distribution without weighting. Now 108k cell sample required from original 3k, holding LL sample constant at 7k. Conclusions Understand the state-level CPO estimates - and plan p accordingly. Who is the population of interest? Cost/benefit for client to pay for a population subgroup of little interest? Non Non-general general pop pop. surveys HHs with teens? 25-29 year olds? Hispanic/Latino? Renting vs. owning? Unrelated roommates? Etc. Better position yourself for acceptable weighting procedures….. Conclusions Weighting of dual-frame dual frame telephone surveys will continue to be a hot topic. Try and anticipate demographic shortcomings and supplement with additional cell phone sample - leading to less aggressive weighting procedures and more reliable results. Brian Harnisch Assistant Research Scientist W Wyoming i Survey S & Analysis A l i C Center t University of Wyoming [email protected] 307 766 6103 307-766-6103 Lookout Lake, Near Laramie, WY Thank h k You!! © Brett B Deacon D References Blumberg SJ, Luke JV, Ganesh N, et al. Wireless substitution: State-level estimates from the National Health Interview Survey, January 2007–June 2010. National health statistics reports; no 39. Hyattsville, MD: National Center for Health Statistics. 2011. Blumberg SJ, Luke JV. Wireless substitution: Early release of estimates from the National Health Interview Survey, January–June 2010. National Center for Health Statistics. December 2010. Available from: http://www.cdc.gov/nchs/nhis.htm. 2010 Census Redistricting Data (Public Law 94-171) Summary File, Tables P1, P2, P3, P4, H1. 2005-2009 American Community Survey 5-Year Estimates
© Copyright 2024