Feasibility of Using Web to Survey at a Sample of Addresses: a UK ESS experiment Ana Villar City University London MANY THANKS TO NatCen team Alison Park (UK ESS NC) Maya Agur Alun Humphrey City University London team Rory Fitzgerald Lizzy Gatrell Yvette Prestage www.europeansocialsurvey.org RESULTS ARE VERY PRELIMINARY AND SHOULD BE TAKEN WITH CARE; DATA WERE DELIVERED ONLY TWO WEEKS AGO AND WE ARE STILL IN THE PROCESS OF CLEANING THE DATA. www.europeansocialsurvey.org Study Design and Goals Co-funded by the ESS (EC) and the ESRC Feasibility of implementing a web-F2F survey using an address-based sample • Careful contact and communication design • Within-household random self-selection • Monetary incentives Compare to ESS main stage F2F Sequential: Web then F2F (N = 500) Web only (N = 1,500) www.europeansocialsurvey.org Experiments Within-household random self-selection: • Asking household member with next/last birthday to go to URL and complete survey • Household informant fills out roster, respondent is randomly selected Incentive upon completion (choice between evoucher and postal voucher): • £15 • £35 www.europeansocialsurvey.org Contact Protocol Web stage (full sample): • • • • Pre-notice letter Invitation letter with £5 voucher and instructions leaflet Reminder letter Upon completion Thank you + Incentive F2F stage (mixed-mode subsample): • Of the 500 sampled units from the subsample, those who did not reply to web survey were contacted at home by an interviewer following ESS protocol www.europeansocialsurvey.org Pre-notice letter Advance notification of invitation Survey topic Advance notification of unconditional incentive Thank you www.europeansocialsurvey.org Invitation/Cover letter Reminder of previous letter What the ESS is about; “opportunity to have a say” Legitimacy – NatCen/ESRC Scarcity; refer to leaflet Enclosed incentive and conditional incentive Thank you, signature, confidentiality assurance www.europeansocialsurvey.org Instructions Leaflet: Household Roster Survey topic Who is the respondent How to + incentive URL and unique ID Smartphone not allowed Privacy assurance, password, thank you www.europeansocialsurvey.org Instructions Leaflet: Last/Next Birthday Survey topic Who is the respondent How to + incentive URL and unique ID Smartphone not allowed Privacy assurance, password, thank you www.europeansocialsurvey.org Household roster Step 1 – Number of adults www.europeansocialsurvey.org Household roster Step 2 – Listing all adults www.europeansocialsurvey.org Household roster Step 3 – Confirmation of list www.europeansocialsurvey.org Household roster Step 4a – Computer selects HH informant www.europeansocialsurvey.org Household roster Step 4b – Computer selects another HH member www.europeansocialsurvey.org PRELIMINARY FINDINGS www.europeansocialsurvey.org Response Rates (RR1) 80% 70% 60% 55% 50% 39% 40% Web Web + F2F 30% 21% 20% 10% 0% F2F Response Rates www.europeansocialsurvey.org (RR1) Response Rates by Selection Method HH ROSTER (N = 1,000) BIRTHDAY (N = 1,000) Followed link NA NA Completed roster NA NA 21.1% 20.4% 20.1% 20.0% 19.2% 19.1% Started (Q1) Completes + Partials Completed Web Survey www.europeansocialsurvey.org Response Rates by Incentive £15 (N = 1,000) £35 (N = 1,000) Followed link NA NA Completed roster NA NA 19.4% 18.7% 18.5% 21.7% 20.9% 20.7% Started (Q1) Completes + Partials Completed Web Survey www.europeansocialsurvey.org No Interaction Effect www.europeansocialsurvey.org Compliance with instructions (n = 112) 30% not applicable – single households or only adult 50% comply 20% do not comply Rest a combination of refusals, break-offs, impossible to tell So far number of “cheaters” is similar for both last and next birthday www.europeansocialsurvey.org Respondent Behaviour Average duration: paradata not yet available Typical number of sessions: 1 About half started before the first reminder was sent www.europeansocialsurvey.org Gender by Mode 90% 80% 70% 60% 57% 60% 54% 50% F2F 40% Web Web + F2F 30% 20% 10% 0% www.europeansocialsurvey.org Female Age by Mode 50% 44% 45% 40% 42% 36% 34% 35% 29% 30% 30% 27% 25% F2F 21% 20% 15% 10% Web Web + F2F 11%10% 8% 10% 5% 0% www.europeansocialsurvey.org 15-24 25-44 45-64 65 or more Years of Education by Mode 60% 49% 50% 42% 40% 30% 26% 27% 22% 22% 29% 24% 22% F2F Web Web + F2F 20% 10% 7%6% 3% 11% 5% 6% 0% Less than 9 9-12 years 13-15 years 16-19 years 20 or more www.europeansocialsurvey.org years years Job Situation by Mode 80% 71% 70% 60% 54% 53% 50% F2F Web Web + F2F 40% 30% 20% 10% 0% Has a paid job www.europeansocialsurvey.org Household Income by Mode 45% 39% 40% 35% 30% 35% 31% 31% 27% 30% 25% 25% 24% F2F 22% Web 20% 17% 15% 10% 6% 5% 2% 5% 4% 1% 0% www.europeansocialsurvey.org Low Medium High DK Refused Web + F2F Political Interest by Mode 45% 40%41% 38% 40% 35% 33% 30% 27% 30% 25% 23% 20% 15% 15%16% 13% 12%12% 10% 5% 0% Very interested Quite www.europeansocialsurvey.org interested Hardly interested Not at all F2F Web Web + F2F Conclusions and Lessons Learned It is feasible to get a substantial number of people to complete an online survey There seem to be some differences in sample composition by mode Costs and time necessary to program survey tend to be largely underestimated www.europeansocialsurvey.org Thank you [email protected] www.europeansocialsurvey.org Income by Mode (without item NR) 45% 40% 35% 30% 41% 39% 40% 34% 32% 28% 31% 28% 27% 25% F2F 20% Web Web + F2F 15% 10% 5% 0% www.europeansocialsurvey.org Low Medium High Job Situation by Mode 80% 71% 70% 60% 53% 54% 50% 47% 46% F2F 40% 30% 29% 20% 10% 0% www.europeansocialsurvey.org Has a paid job No paid job Web Web + F2F Gender by Mode 70% 57% 60% 50% 40% 43% 60% 54% 46% 40% F2F Web Web + F2F 30% 20% 10% 0% Male www.europeansocialsurvey.org Female Economic Hardship by Mode 90% 81% 81% 80% 70% 70% 60% 50% F2F 40% Web 30% 30% 20% 19% Web + F2F 19% 10% 0% www.europeansocialsurvey.org Difficulties Coping or Comfortable Urbanicity by mode 90% 80% 79% 81% 74% 70% 60% 50% F2F 40% Web 30% 20% 21% 26% 19% 10% 0% www.europeansocialsurvey.org Urban Web + F2F Rural www.europeansocialsurvey.org Number of completes by date (roster) 50 45 40 Reminder sent 35 30 25 20 15 10 5 0 www.europeansocialsurvey.org Number of completes by date (birthday) 45 40 35 Reminder sent 30 25 20 15 10 5 0 www.europeansocialsurvey.org
© Copyright 2024