The Economic and Cognitive Costs of Annoying Ads
The Economic and Cognitive Costs of Annoying Ads Daniel G. Goldstein Siddharth Suri Fernando Diaz • Microsoft Research, New York City R. Preston McAfee • Microsoft Corporation Matthew Ekstrand-Abueg • Northeastern University Questions of the day • How can online labor markets be used for field studies? • What are the economic and cognitive costs of annoying ads? Field studies in online labor markets • Mechanical Turk • Widely used as subject pool (much like the early ones of Reips, Birnbaum, Baron, Göritz, Goldstein, etc.) • However, MTurk was created as a bona fide labour market • Work as DV. Dropout as a good thing. • Compensating differential • Amount needed to pay someone to compensate for risk or unpleasantness • Innovative method of Toomim et al for testing user interfaces Publisher: New York Times Citibank paid New York Times Display Ad Advertiser: Citibank Display advertising • Online advertising about as large as broadcast TV • Display: $13 billion USD in 2013 • 33% of online advertising revenue • Display expected to overtake sponsored search ads on mobile devices Good Ads Bad Ads Mobile annoying ads Tension " Advertisers pay publishers to display annoying ads " Annoying ads cost publishers page views through user abandonment Impacted parties " Publishers " Lost traffic " Apparent desperation: Warning of service quality " Users " Interfere with content consumption. 200 million AdBlock downloads on Firefox alone. " Advertisers " Appear disreputable " Undermines money-to-burn signal Research questions " What makes ads annoying? " What are the economic costs of annoying ads? " What are the cognitive costs of annoying ads? Related Work " Yoo & Kim 2005, Burke et al 2005 " Moderate animation increases brand recognition and attitude " Heavy animation decreases brand recognition and attitude " Goldfarb & Tucker 2011 " Ads that are intrusive or match content increase intent to purchase " Ads that are both decrease attention decrease intent to purchase " Calculating compensating differentials, Toomim et al 2011 Preparatory study: Identifying annoying ads Ranking Ads by Annoyingness " 72 animated ads, 72 static ads generated from their final frame " Users randomly shown either the static or animated version " First users previewed all ads Ranking Ads by Annoyingness " 72 animated ads, 72 static ads generated from their final frame " Users randomly shown either the static or animated version " First users previewed all ads " Then rated each on a 1-5 scale of annoyingness Ranking Ads by Annoyingness " 72 animated ads, 72 static ads generated from their final frame " Users randomly shown either the static or animated version " First users previewed all ads " Then rated each on a 1-5 scale of annoyingness " Finally, commented on why Preparatory study participants " Conducted on Amazon Mechanical Turk " $0.25 flat rate, $0.02 per ad rated " 141 participants in 18 hours " Restricted to US workers with 95% approval rating Identifying Good & Bad Ads “bad ads” “good ads” " Animated ads are most annoying, static are least " Used 10 most/least annoying ads in study 2 The Impact of Animation " Animation caused ads to be rated as substantially more annoying What Makes Ads Annoying? Study 1: The economic cost of annoying ads" " Measuring the " Compensating Differential " Task: users classify email, can stop any time " Randomly assigned into " 3 pay conditions: .2, .4, .6 cents per email " 3 ad conditions: no ads, good ads, bad ads " Treatment not revealed until acceptance " Circumvented ad blockers " Ads were randomly selected from the set of good or bad ads " New ads with each email " Emails from the Enron data set Study 1 participants " Conducted on Amazon Mechanical Turk " $0.25 flat rate, 0.2, 0.4, 0.6 cents/email " 1223 workers in two weeks " Restricted to workers with 90% approval rating Compensating Differential: " Bad Ads to No Ads 68 " Bad ads caused 12.7 fewer impressions compared to no ads " .2 cents per impression 52 yields an additional ~16.6 impressions " To get an additional 12.7 impressions need to pay 0.2 * 12.7/16.6 = .153 40 50 65 85 Calculation • If you get 16.6 additional impressions for every .2 cents you pay, how much does it cost to make up the 12.7 impressions lost to annoying ads? 𝑋 (16.6 𝑖𝑚𝑝𝑟𝑒𝑠𝑠𝑖𝑜𝑛𝑠/.2 𝑐𝑒𝑛𝑡𝑠 )= 12.7 𝑖𝑚𝑝𝑟𝑒𝑠𝑠𝑖𝑜𝑛𝑠 𝑋 =12.7 𝑖𝑚𝑝𝑟𝑒𝑠𝑠𝑖𝑜𝑛𝑠(.2 𝑐𝑒𝑛𝑡𝑠/16.6 𝑖𝑚𝑝𝑟𝑒𝑠𝑠𝑖𝑜𝑛𝑠 ) 𝑋 =.153 𝑐𝑒𝑛𝑡𝑠/ 𝑖𝑚𝑝𝑟𝑒𝑠𝑠𝑖𝑜𝑛 𝑋 =$1.53 / 1𝑘 𝑖𝑚𝑝𝑟𝑒𝑠𝑠𝑖𝑜𝑛𝑠 =$1.53 𝐶𝑃𝑀 Compensating Differential: " Bad Ads to Good Ads 68 " Bad ads caused 9.5 fewer impressions compared to good ads " .2 cents per impression 52 62 yields an additional ~16.6 impressions " To get an additional 9.5 impressions need to pay 0.2 * 9.5/16.6 = .115 " $1.15 CPM 40 47 81 Robustness " Data are skewed: " Many people categorized a few emails " Few people categorized a lot " Means drop as observations are excluded reflecting skew, annoyingness effect remains " Good ads and no ads yield same number of impressions Accuracy " Enron data set has emails categorized as spam or ham Treatment Accuracy Bad Ads Good Ads No Ads 90% ±1 92% ±1 93% ±1 " Statistically significant difference between bad ads and good ads/no ads Study 2: The cognitive cost of annoying ads" " Mousetracking • Mousetracking correlates with eye tracking • Chen, Anderson, & Sohn, 2001; Guo & Agichtein, 2010; Huang, White & Buscher, 2012 • Mousetracking proxies for user interest • Steiger & Reips, 2010; Willemsen & Johnson, 2010; Navalpakkam, & Churchill, 2012 Study 2 participants " Conducted on Amazon Mechanical Turk " $0.50 flat rate, $0.10 per question answered " 2,840 participants Task Good or Bad Ad Manipulation check • 42% listed ad as annoying in “bad” condition • 5% did so in “good” condition Good ad fixations Bad Ad Fixations Mousetracking metrics • Fixations of the mouse in an area • Distance mouse travels in an area • Number of entrances of the mouse into an area • Time mouse spends in an area Mousetracking metrics:" Ad area Measure Bad Ads Good Ads P-Val (log OLS) Fixations 4.45 (0.67) 1.57 (0.26) <.001 Distance 182.6 (7.9) 157.9 (8.0) .003 Entrances 1.31 (0.05) 1.13 (0.05) .004 Time (ms) 1873 (321) 1101 (186) .001 Mousetracking metrics:" Text area Measure Bad Ads Good Ads P-Val (log OLS) Fixations Distance Entrances Time (ms) 135.7 (9.07) 128.0 (9.05) .047 1,492 (55.20) 1,570 (66.60) .677 1.51 (.06) 1.50 (.07) .322 38,268 (1,207.00) 36,367 (1,085.00) .596 Mouse reading • Rated by 3 judges • About 7% incidence of mouse reading in good and bad conditions, no significant difference Accuracy • Accuracy was significantly lower in “bad” condition compared to the good ad condition (2.5 percentage point delta, 6 percentage points for answer at bottom of page) Summary of mousetracking studies • Annoying ads do capture user attention • Annoying ads have negative affect on accuracy • However, absence of evidence that people read differently when annoying ads present. Why? • Could be depletion, however, in Experiment 1, the accuracy difference between the bad and good ad conditions held constant. • Our hypothesis is that annoying ads make people care less about what they’re reading or the questions they are answering Conclusions & Future Work" " Conclusions " Animation has a causal effect on annoyance " Annoying ads hamper accuracy in cognitive tasks " Annoying ads cause dropout " Compensating differential of bad ads to no ads/good ads is at least $1 CPM, which would price some ads out of the market " Good ads lead to about the same number of impressions as no ads " We would lose money by running annoying ads Conclusions " Mousetracking shows that people attend to annoying ads and they the impact accuracy " However, they don’t appear to affect reading behavior and may impact accuracy for motivational reasons Research Agenda " Towards a psychologically informed display ad pricing scheme " Display ads should be priced in terms of time in view instead of impressions (Economics and Computation, 2011) " Two short ads are better than one long ad (Economics and Computation, 2012) " What is the cost of annoying ads? (Journal of Marketing Research, 2014) Thank you!" "
© Copyright 2024