vue WELCOME ON BOARD FLIGHT 196, DESTINATION This
vue Canadian Publications Mail Agreement #40033932 New and Improved! New columns! New features! WELCOME ON BOARD FLIGHT 196, DESTINATION SAMPLE SIZE: This Is Your Captain Speaking The Seven Habits of Highly-Stressed Survey Researchers Questionnaire Design Levels of Measurement and Statistical Method Choices Master Crafting Qualitative Basics FACE OFF MARKET RESEARCH: Essential or Pointless? the magazine of the Marketing Research and Intelligence Association JANUARY/FEBRUARY 2014 vue JANUARY/FEBRUARY 2014 VUE MAGAZINE IS PUBLISHED BY THE MARKETING RESEARCH AND INTELLIGENCE ASSOCIATION TEN TIMES A YEAR SPECIAL FEATURE 10 Welcome On Board Flight 196, Destination Sample Size: This Is Your Captain Speaking Kathryn Korostoff FEATURES 14 The Seven Habits of Highly-Stressed Survey Researchers Jeffrey Henning 18 Questionnaire Design Levels of Measurement and Statistical Method Choices David S. Dobson 20 Master Crafting Qualitative Basics Kendall Nash 24 FACE OFF – Market Research: Essential or Pointless? Katie Clark and Mike MacLeod COMMENTARY 4 Editor’s Vue 6 Letter from the President 8 Message from the Interim Executive Director INDUSTRY NEWS 26 MRIA Designations 27 Qualitative Research Registry (QRR) 28 Research Registration System (RRS) 29 People and Companies in the News 32 Chapter Chat COLUMNISTS 34 Ask Dr. Ruth 34 Bright-Eyed 35 La Belle Vue 35 Off the Deep End ADDRESS The Marketing Research and Intelligence Association L’association de la recherche et de l’intelligence marketing 94 Cumberland Street, Suite 601 Toronto, ON M5R 1A3 Tel: (416) 642-9793 Toll Free: 1-888-602-MRIA (6742) Fax: (416) 644-9793 Email: [email protected] Website: www.mria-arim.ca PRODUCTION: LAYOUT/DESIGN LS Graphics Tel: (905) 743-0402, Toll Free: 1-800-400-8253 Fax: (905) 728-3931 Email: [email protected] CONTACTS CHAIR OF PUBLICATIONS, EDITOR-IN-CHIEF Annie Pettit PhD, Chief Research Officer, Conversition (416) 273-9395 [email protected] MANAGING EDITOR Anne Marie Gabriel, MRIA [email protected] ASSOCIATE EDITOR Fiona Isaacson [email protected] Interested in joining the Vue editorial team? Contact us at [email protected] 2014 ADVERTISING RATES Frequent advertisers receive discounts. Details can be found by going to: www.mria-arim.ca/advertising/vue.asp Please email [email protected] to book your ad. The deadline for notice of advertising is the first of the previous month. All advertising material must be at the MRIA office on the 5th of the month. Original articles and Letters to the Editor are welcome. Materials will be reviewed by the Vue Editorial Team. If accepted for publication, they may be edited for length or clarity and placed in the electronic archives on the MRIA website. The opinions and conclusions expressed in Vue are those of the authors and are not necessarily endorsed by the Marketing Research and Intelligence Association. Publishing Date: January/February © 2014. All rights reserved. Copyright rests with the Marketing Research and Intelligence Association or the author. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise without the prior written permission of the Marketing Research and Intelligence Association or the author. All requests for permission for reproduction must be submitted to MRIA at [email protected]. RETURN UNDELIVERABLE CANADIAN ADDRESSES TO The Marketing Research and Intelligence Association L’Association de la recherche et de l’intelligence marketing 2600 Skymark Avenue, Bldg 4, Unit 104, Mississauga, Ontario L4W 5B2 Canadian Publications Mail Agreement #40033932 ISSN 1488-7320 COMM E NTARY / CO MME NTAIR E Ed i t o r’s Vue Annie Pettit I feel like a kid in a candy store! Everything in this Vue is new, bright and shiny. First, I’m delighted to welcome Fiona Isaacson as our new Associate Editor and Chapter Chat curator. I’d also like to welcome six new columnists for 2014. Brian Singh will rant and rave, and get your blood boiling. Ric Hobbs will jab you in the funny bone with the droll business that is marketing research. Dylan Cody will remind you what it’s like to be bright-eyed and bushy-tailed, fresh out of school. Susan Abbott will reinvigorate the quallie inside every one of us. Julie Fortin va partager des paroles de sagesse de la belle province de Québec. And lastly, Ruth Corbin (“Dr. Ruth”) will be our wise sage, though she claims she will only answer legal questions… What else is new? This issue marks the first appearance of “Face Off,” a fierce debate between two researchers who may not agree with the point of view they’ve been assigned to argue. Every issue will also introduce you to a researcher you may have never met before. Find out how they came to be in research and then feel free to share your stories with them via email. And, look for a research cartoon in every issue. We’re all passionate about research, so why not have a little fun along the way? I hope you love this issue, and the coming year of Vue. I invite you to write a letter to the editor and tell us what you like and don’t like, or share your point of view on an article. Or, create your own cartoon and send it to us. Maybe you’ll show up in our new “Letters to the Editor” page debuting in the next issue. Cheers to all things new! J e suis comme une enfant dans une confiserie : tout dans cette livraison de Vue me semble aussi intéressant qu’irrésistible. Je souhaite d’abord souhaiter la bienvenue à Fiona Isaacson, notre nouvelle rédactrice adjointe et la modératrice du clavardage régional. Nous accueillons avec ce numéro six nouveaux chroniqueurs et chroniqueuses pour l’année 2014 : Brian Singh, qui ne manquera probablement pas d’en « énerver » certains; Ric Hobbs, un chercheur marketing doublé d’un humoriste; Dylan Cody, un jeune fraîchement diplômé qui nous offrira la perspective d’un débutant dans le secteur; Susan Abbott, qui se fera l’apôtre de l’analyse qualitative; Julie Fortin, qui nous fera profiter de sa sagesse; et l’avocate Ruth Corbin, notre « Dr. Ruth » à nous, qui rappelle au passage qu’elle ne traitera que de sujets juridiques. D’autres nouveautés? Bien sûr ! Le lancement de “Face Off ”, par exemple, une page qui propose un débat entre deux chercheurs sur un sujet d’intérêt au secteur. Chaque numéro vous fera également connaître un chercheur qui vous est probablement inconnu. Vous êtes invité à découvrir comment il ou elle en est arrivé(e) à devenir chercheur et de partager avec lui (ou elle), via courriel, vos propres anecdotes. Enfin, et en prime, une caricature originale sur la recherche vous attendra dans chaque nouveau numéro. Qui a dit que les chercheurs marketing n’avaient pas le sens de l’humour? J’espère que vous apprécierez cette livraison de Vue et tous les numéros qui suivront en 2014. Écrivez-nous et dites-nous ce qui vous a plu et ce que vous avez moins aimé. Laissez-nous savoir ce que vous pensez et exprimez votre opinion au sujet de nos articles et chroniques. Vous pouvez aussi nous expédier une caricature de votre cru. Qui sait? Peut-être reconnaîtrezvous votre contribution dans le prochain numéro de Vue et sa nouvelle section, « Lettres à la rédaction ». Vive la nouveauté ! Annie Pettit PhD, Chief Research Officer / Directrice de la recherche, Peanut Labs Editor-in-Chief, Vue / Rédactrice en chef, Vue • Email: [email protected] • (416) 273-9395 • t @LoveStats Please share your opinions about Vue articles and columns, or submit your cartoons and infographics to the Editor. La rédactrice vous invite à lui faire parvenir directement vos commentaires, opinions, caricatures ou infographies. 4 vue | JANUARY/FEBRUARY 2014 vue | JANUARY/FEBRUARY 2014 5 COMM E NTARY / COMME NTAIR E Letter from the President Anastasia Arabia Happy New Year! I am very excited about the opening of our new MRIA office in downtown Toronto. Most MRIA members, even board members, never visited our office in Mississauga. There are many practical reasons why being downtown is the right decision for the MRIA, but my favourite is that the new office will be more user-friendly and accessible to our members. At our new location, we will be able to hold board meetings, courses, CMRP exams, seminars, and so on, and will no longer have to rent expensive hotel space. Also, out-of-town MRIA members can drop in, have a coffee, use the WiFi, or make a call. This will be especially useful for out-of-town members, like me, who have a few hours before a meeting, or event, and would relish some space to get some work done. The MRIA staff is also very excited about being more closely connected and getting some face time with members, all made possible by our opendoor philosophy. It’s nice to put faces to names and actually shake hands with people you have communicated with for years by email and phone. Look for news of our grand opening in the coming weeks. On a very personal note, I would like to pay a special tribute to Hank Goertzen, CMRP, who passed away suddenly in December. Hank was an exceptional researcher and mentor, and a wise and well-respected senior researcher in Alberta. He will be dearly missed. Bonne année ! Le déménagement du siège social de l’ARIM au centre-ville de Toronto me gonfle d’enthousiasme. En effet, peu de membres de l’ARIM, même membres du conseil, venaient nous rendre visite au bureau de Mississauga. Plus encore que les nombreuses raisons pratiques pour ce déménagement, ce sont la convivialité et l’accessibilité du nouveau siège social à nos membres qui motivent mon enthousiasme. Nous pourrons ainsi accueillir « chez nous » les réunions du conseil, les examens du programme PARM, des séminaires et autres événements, tout en économisant sur la location, onéreuse, de salles d’hôtels. Nous invitons les membres de l’extérieur de Toronto – j’en suis – de nous y rendre visite quand ils seront de passage et de se prévaloir des téléphones, connexion wifi et café qui y seront disponibles. Des espaces de travail ont également été prévus pour ceux qui ont quelques heures à écouler avant une réunion. Les employés de l’ARIM ont d’ailleurs bien hâte de rencontrer tous les membres qui profiteront de notre politique « porte ouverte » pour venir leur serrer la pince. Il est toujours plaisant d’associer enfin des visages aux voix que nous avons entendues au téléphone et aux courriels qui nous ont été expédiés. Plus de nouvelles au sujet de l’ouverture officielle du bureau suivront dans les semaines qui viennent. Enfin, je ne peux passer sous silence le décès de Hank Goertzen PARM, qui nous a quitté soudainement au mois de décembre. Hank était un chercheur hors pair et un mentor exceptionnel, un homme sage et respecté, tout particulièrement en Alberta, sa province de résidence. Tu vas nous manquer, Hank. Anastasia Arabia, Partner / Partenaire, Trend Research Inc. President, Marketing Research and Intelligence Association / Présidente, L’Association de la recherche et de l’intelligence marketing Email: [email protected] • 780-485-6558 ext./poste 2003 6 vue | JANUARY/FEBRUARY 2014 MRIA 2014 CONFERENCE SHERATON CAVALIER AND DELTA BESSBOROUGH SASKATOON | SASKATCHEWAN The MRIA 2014 National Conference is proud to announce that we have secured Jeffery Hayzlett as a keynote speaker. Jeffrey is a global business celebrity and host of “C-Suite with Jeffrey Hayzlett” on Bloomberg Television. He also appeared as a guest judge on NBC’s Celebrity Apprentice with Donald Trump for three seasons. Special thanks to Itracks in assisting to secure Jeffery as a speaker. The national conference “Dig Deeper and Discover” will be held from June 8 – 10, 2014 in Saskatoon, Saskatchewan, Canada’s fastest-growing metropolis. Join us...as a Delegate, Sponsor, or Exhibitor. Follow us on Facebook, LinkedIn and the conference website (http://conference2014.mria-arim.ca/news/index.php). PINNACLE SPONSOR PLATINUM SPONSORS Insightrix_logo.pdf 1 11/13/2013 1:03:57 PM C M Y CM MY CY CMY K GOLD SPONSOR BRONZE SPONSOR EXHIBITORS OTHER SPONSOR COMM E NTARY / COMME NTAIR E Message from the Interim Executive Director John Ball, CMRP 8 Happy New Year to all! There can be no clearer indication of a really new year than one that starts at the MRIA with a message of great appreciation and recognition of all members who have graciously shared their time and insights through the online qualitative research we are conducting to help us better understand member needs, and attitudes towards the MRIA and its Chapters. Those who have been able to join in the conversation about the association have shared their views candidly, and provided solid ideas to improve the association moving forward. Une bonne année à tous ! J’en profite pour remercier tous les membres qui, au cours de la dernière année, ont donné de leur temps et partagé avec nous leurs idées dans le cadre de la recherche qualititative que nous avons menée auprès des membres pour mieux cerner leurs besoins et leur attitudes vis-à-vis l’ARIM et ses sections régionales. Ceux qui ont participé au dialogue au sujet de l’association ont parlé franchement et ils ont avancé de solides idées quant aux moyens à prendre pour l’améliorer. We are hearing about how to improve our advocacy for the industry and its practice amongst the general public, improve our relations with our corporate members who are tasked with adhering to our highest standards, and how to improve our enforcement practices to make sure that any deficiencies are kept in check. Relevons parmi ces suggestions des façons de rehausser nos pratiques avec le grand public et notre représentation du secteur. Il nous a aussi été dit que nous devions améliorer les rapports avec les sociétés membres qui doivent se conformer à nos normes, très strictes, de même que l’application des règlements en cas de contravention aux normes. We also know how to take another look at our educational platform with fresh eyes, and to consider improving access to effective training and build in new content (new technology, client-side specific) regardless of whether it comes from us or somewhere else. You have even been able to help steer the development of our approach to organizing our upcoming National Conference in June of this new year we are entering. We know what issues are important to you, and what will make this conference a good value for you, and also how to augment this conference with new Les participants ont aussi demandé que nous passions en revue notre programme de formation, que nous en facilitions l’accès et que nous proposions de nouveaux modules, quelle que soit leur source, notamment à l’égard des nouvelles technologies et du côté client. Les membres nous ont également aidé à préparer le congrès national qui aura lieu au mois de juin et nous ont fait part des sujets qu’ils souhaitent aborder. Ils ont aussi proposé de recourir à des webinaires pour élargir l’auditoire du congrès. Les participants ont aussi souligné l’importance du lobbying auprès des vue | JANUARY/FEBRUARY 2014 commentar y commentaire events and webinars to increase our connections to our members. We’ve also heard how important you feel it is that we continue to lobby for the industry with various levels of government to build the profile of the marketing research profession. We also understand the value of how to bring new life to the MRIA and our industry, by increasing the number of student members in our association. While we have been moving forward on all of these fronts all along, we understand better now how and where to turn our attention for 2014. We are more aware than ever of the challenges of connecting and creating meaning for those in our various Chapters across the country, each with their own unique challenges. Our Chapters represent the face of the MRIA for many of you, and we are using this research to assist our Chapter leadership to stay the course or expand our local offerings based on the feedback you have provided. This past year has been one of considerable change and we can only expect this to continue through 2014 with a keener sense of what is working and not working from the perspective of our members and where we should set our sights. Thank you for the opportunity your feedback has provided to enable us to make positive change a priority. I am remembering fondly when we initially reached out to the public during our Voxpop campaign in the early days of the association with the following promise, “You talk, we listen, things improve.” I thank you for allowing us the opportunity to walk this talk with you. autorités gouvernementales et de l’affirmation publique de notre profession, la recherche marketing. Nous avons aussi été priés d’apporter un nouveau souffle à l’ARIM et au secteur en recrutant davantage de membres étudiants. Bien que nous faisions déjà des progrès sur tous ces fronts, nous savons maintenant vers quoi exactement tourner nos efforts en 2014 et comment procéder. Nous savons ainsi que nous devons établir des rapports plus étroits et plus productifs avec les sections régionales, qui font face, chacune d’elles, à des défis qui leur sont particuliers. Par ailleurs, nombre d’entre vous ne connaissent l’ARIM que par leur section régionale. Nous travaillerons donc sans relâche pour maintenir le cap sur le rehaussement et pour proposer des initiatives régionales, fondées sur votre rétroaction. Plusieurs choses ont bien changé en 2013. Il en sera sans doute de même en 2014, maintenant que nous savons mieux ce que les membres apprécient, ce qu’ils aiment moins, et quelles orientations ils souhaitent que nous adoptions. Merci à tous de votre rétroaction, qui continue de nous aider à apporter les changements qui s’imposent. Tout ça me rappelle le slogan-promesse que nous avions utilisé, peu après la création de l’ARIM, dans le cadre de la campagne Voxpop, qui visait le grand public : Vous parlez, nous écoutons, les choses s’améliorent. Il est tout aussi pertinent aujourd’hui. Merci de nous avoir donné cette occasion de « faire marcher nos bottines plutôt que nos babines ». John Ball, CMRP, Interim Executive Director /Directeur général intérimaire Marketing Research and Intelligence Association / L’Association de la recherche et de l’intelligence marketing Email: [email protected] • (905) 602-6854 ext./poste 8724 vue | JANUARY/FEBRUARY 2014 9 Welcome On Board Flight 196, Destination Sample Size: This Is Your Captain Speaking Would any of us ever board a plane if we didn’t feel assured that the people commanding that metal capsule hurtling us through the air at 500 mph know what they’re doing? Kathryn Korostoff O Of course, we’re also aware of something called “autopilot.” We all accept the fact that our highly qualified pilot may be judiciously using the autopilot function. But we like to think that it’s not being blindly trusted – that, should the autopilot fail, our human pilot will be perfectly capable of landing the old-fashioned way and in the right place. Imagine if you asked the pilot for flight information and were told, “I don’t know. The autopilot works fine, though.” You’d want off that plane! As marketing researchers, we are the pilots of our survey’s journey. It’s up to us to plot the journey and navigate the path, to reach the right destination, not whisk clients away on an unintended trip to Peoria. Yes, we have tools that automate key tasks – but we still need to know how to do the work ourselves. Why, you ask? First, so we can catch the odd cases of autopilot failure. Second, so that when our clients ask us for an explanation, our answer is a tad more credible than, “Because the software says so.” Before Takeoff: Calculating Sample Sizes Here’s a simple one-question quiz. What do you think the correct answer is? 10 vue | JANUARY/FEBRUARY 2014 Figure 1: A Simple Quiz Most people select “B,” the size of the target population. hich of the following is used W when calculating the sample size needed to ensure a specific survey project has reliable data (using a formula, not a “calculator”)? A. E xpected response rate of target population B. Size of target population C. Variability of target population D. All of the above While sample size calculators use that, the correct answer is “C,” the population’s variability. The reason sample size calculators use population size is because it is a proxy for variability. It’s an assumption. And in some markets, a big one. There is only one thing wrong with that: it creates the illusion that all markets are the same. Whether you are researching attitudes about cars, beauty products or high-end computers – if your desired level of reliability is the same in all cases, you get the same recommended sample size from the calculators. That really doesn’t make sense when you think about it. Some target customer groups have more variability than others. For example, in my experience: SP ECIA L FEAT URE • I n a survey on home electronics brands, variability in brand perceptions may be fairly low. That is, most people have common perceptions about what are the “premium” brands, what brands are “low cost-low frills,” what brands are “reliable,” etc. Variability is fairly low. 1. 2. 3. 4. • I n a survey on retail shopping behaviours, variability may be comparatively high. Some people are luxury shoppers, some are discount seekers. Some shop as a social experience, some are all about efficiency. That’s a lot of variability. How reliable does my data need to be? This was tackled in our formula – when we decided (in our example) that we want to be within +/- one minute of the average. In the real world, we may adjust that up or down to fit our goals and budget. Both of these examples are studies with general consumers. But we would likely see a lot more variability in the second study, than the first – and that means we would need a larger sample size too. Recall the old adage: If everyone had the same opinion, we’d only need a sample size of one. The converse is also true: The more variability in opinions, the larger the sample size we need to make sure our data is representative of the target population. So what is the formula? There are variations, but one I often use is: Where Z is typically 1.96, for a 95 per cent confidence interval, H is the desired precision level. And Sigma is the… you guessed it…variability (standard deviation). So let’s work through an example: Imagine you are doing a study on online shopping. You are asking people to self-report how much time they spend price comparing online, before they make a purchase decision. Assume you know from past studies that you expect the variability in answers to be eight minutes (that is, that the standard deviation based on past reported data is eight minutes). Further, assume that you want your new study to be precise within one minute. That is, you want to be confident that whatever result you get for the average time spent price comparing online is accurate in a +/- one minute range. Here’s the math: Your sample size in this scenario is 246 (rounded up, since we do not want to survey a fraction of a person – that’s a little marketing research humour…ha ha). In reality, if I am studying a market or topic for the first time, I may have no estimate of the standard deviation. In those cases, I usually make a conservative estimate, and then revise after a pre-test. And yes, I do pre-tests. Pre-Flight Checklist So if there is a formula, aren’t we done? Not necessarily. The formula gives us a starting place, the “N” we need to be reasonably confident in our data. But other considerations may cause us to increase our sample size beyond that point. There are five key questions you need to ask yourself before “takeoff ” – sort of a pre-flight checklist for sample size planning: How reliable does my data need to be? What type of data analysis am I planning? Is there a “size of universe” constraint? What is my sample source? What type of data analysis am I planning? For some types of data analysis, we simply need more data. Some researchers who do a lot of conjoint studies like to have at least 100 responses per segment (for example, if they are doing a study and will want to compare results by six geographies, they will want at least n=600). For many projects, sample size needs can be exploded just by crosstabs. I may be planning a study where I will need to analyze data by several different types of banners: four ranges of age, five of income level, two for gender, and maybe four for education level. So even if my calculated sample size is 400, I may decide I want more than that to have some breathing room to have enough data to compare my subgroups. Some researchers prefer at least 30 per sub segment, others prefer 50. Is there a “size of universe” constraint? I may have calculated my needed sample size as 300, but what if I am doing research with a really hard to find population? Well, that raises a complex issue. Such populations can be very contrived – and may not have the attribute of actually being a random population or one with a normal distribution. In such cases, I have a professional obligation to explain this to my clients and advise accordingly. What is my sample source? I need to know what my sample source is. What is my sampling frame? Is it large enough to support my sample size goals? Or, is there a risk factor here that I need to manage? Mid-Flight Turbulence: Two Sticky Points Being clear about sample size calculations will help you in two common, and sometimes uncomfortable, situations. First, the research cynic. At some point in your career, you will have this experience: ou are presenting the final results to a group. Some of your Y results may be a little controversial, or perhaps just a tad unexpected. Someone will raise their hand and challenge your data. It is easier for them to challenge your data than to accept that their worldview may be flawed. In these cases, they will either question your sample source or question your sample size. You need to be able to defend both. Objectively. Then there are people who have a magic sample size number in their heads. I have had several clients in my career who assume 1,000 is “a good number.” Hey, it’s a lovely number – and I have done plenty of studies with 1,000 or more vue | JANUARY/FEBRUARY 2014 11 SPEC IA L F EAT U R E completes. But in some cases it is really overkill, and I hate to waste my client’s money. So how do we navigate through these winds of opinion? With facts. In both cases, I find it best to educate my client a little: let them know that the sample size was based on a specific method and explain the basics of how and why. Teaching them, even just a little, makes them much more comfortable not only with my sample size recommendation – but with my dedication to serving their best interests. Know Your Flight Plan Specify your sample size goal before you actually start data collection. This needs to be done early on for three reasons: • T o get your sample source lined up, make sure you have access to enough qualified people to achieve your sample size (given some assumptions about response rate and list quality). This might seem painfully obvious, but poor planning here can really derail a project; as a marketing researcher, you risk losing credibility with clients if your schedule slips a week because your sampling plan was poorly vetted. • T o set realistic expectations with your clients. You don’t want them making sample size assumptions, since sample size can have a huge impact on budget. • T o put correct quota controls in place in your questionnaire. For example, if you are paying incentives, you want to make sure you stop collecting data when you hit your sample size goal. Calculating your sample size yourself, without a calculator, gives you the discipline of really thinking about your project. Should your “passengers” wonder where they are at any given moment, you’ll be able to tell them and explain why. Happy Landings: Safe on Solid Ground The success of most marketing research projects is judged on a few criteria. When I teach Market Research Project Management, I say that the success of a project is often judged on seven common criteria: 12 vue | JANUARY/FEBRUARY 2014 1. 2. 3. 4. 5. 6. 7. Project completed on time Project completed on budget Project results clearly align with defined project objectives Sample size and quota distributions met Reporting/deliverables meet professional standards Ethics standards adhered to Research results are deemed actionable by clients While item no. 4 is clearly sample size related, clear and correct sample size planning can also impact items 1, 2, 3 and 7 as well. Doing these calculations “the old-fashioned way” will deepen your understanding of your project, prevent autopilot failure, and protect you against mid-flight turbulence and the occasional grouchy passenger. Sample size: THE SHORT ANSWER For some consumer studies, directional data of 100 responses can be used for quick hits. We label such results “directional,” and sometimes that is fine when time and budget dictate. However, if you want data that can be used to represent a population, then a larger sample size is preferred. In consumer marketing research there are common levels of variability that we’re likely to encounter. Based on this, many marketing researchers will use rules of thumb: a sample size of 400 to 500 completes will often give us solid data. But add in some extremes in population variability or sub-group analysis, and you may need to go higher. Knowing the rules is key. So is knowing the logic that informs them. Kathryn Korostoff is the president of Research Rockstar LLC, the only independent company dedicated to market research training. Over the past 20 years, Kathryn has personally directed over 600 primary research projects and published over 100 articles in various magazines, including Quirk’s Marketing Research Review and Alert! magazine. She has been a featured speaker at American Marketing Association, ESOMAR, and Marketing Research Association events. She can be reached at KKorostoff@ ResearchRockstar.com or on Twitter via @ResearchRocks. ANNOUNCING OUR KEYNOTES! Gail Livermore, TARGET Merrill Dubrow, M/A/R/C® Research Naomi Henderson, Riva The strength of qualitative research lies in its ability to make deep connections with people – connections which enrich our understanding of their desires, values and opinions. PLAN NOW for 2014! Registration Open! Sponsorship Opportunities Available! Visit our website and enter the Contest http://qrdconference2014.mria-arim.ca/news/index.php Platinum Sponsor Gold Sponsors Reliable, Actionable Consumer Insights by Design.TM Silver Sponsors Bronze Sponsors Marketing Sponsor Heintzman Research Limited Survey Sampling International - SSI vue | JANUARY/FEBRUARY 2014 13 FEAT UR E The Seven Habits of Highly-Stressed Survey Researchers Jeffrey Henning Pretty busy here. Where I used to carefully write a questionnaire, run a pre-test, run a pilot and then launch – now I have to get it all right the first time. Where I used to think carefully about the results over a few weeks, discuss them with co-workers, let them percolate and come up with an original analysis – now I have to bang them out and move on to the next project. S Surveys have become commodities. That means I fall back on bad habits, because I have to get to the next survey. Here are the bad habits I can’t escape any more than skipping a soda at breakfast, or watching one more late-night TV show rather than getting eight hours of sleep, or checking Twitter when I really should be writing that next questionnaire. 1. Writing leading questions First drafts of questionnaires are often full of leading questions. You just write the question from your viewpoint, without taking the time to think about it in a more objective fashion. And since a lot of first drafts are last drafts these days, off you go. A leading question suggests the answer you’re looking for and often unintentionally reflects your bias. As a result, the answers to leading questions overstate the actual support for the item being researched. When you’re in a hurry, it’s hard to step back and be objective. Leading questions can be subtle, or obvious. An obvious leading question, taken from an event survey, is “How likely is it that you will attend the 2014 conference at our new, low entry prices?” A subtler example, used in the text Marketing Research by Alvin C. Burns and Ronald F. Bush, is “Should people be allowed to protect themselves from harm by using Mace as self-defense?” The survey sponsor’s viewpoint is clear – yes, yes, they should. 14 vue | JANUARY/FEBRUARY 2014 Or, take “How much do you think you can save by buying online?” A neutral question would have asked if online prices were the same, higher or lower than offline alternatives, and then would have asked by how much. Like many bad habits, this one is addressed by being more deliberative and investing more time. In this case, have a third party review the questionnaire, or spend the effort thinking about a neutral alternative to leading questions. 2. Using the Likert scale I was recently sent an employee satisfaction questionnaire where the first 80 questions all used a Likert scale (“strongly agree,” “agree,” “neutral,” “disagree,” “strongly disagree”). I envied the economy of effort – pretty easy to write such a beast. I hope they were paid by the question. But, like leading questions, Likert scales skew the results. Over 100 separate studies have demonstrated acquiescence bias – the tendency for respondents to agree, regardless of the content of the statement. In their landmark paper, “Comparing Questions with Agree/Disagree Response Options to Questions with Construct-Specific Response Options,” Willem E. Saris, Melanie Revilla, Jon A. Krosnick and Eric M. Shaeffer share three reasons why respondents are so darn agreeable: FEAT URE • Out of a sense of politeness • O ut of respect for the survey author – in effect, deferring to “expert” judgment • From mental exhaustion with the survey (“satisficing”) We survey researchers hate to acknowledge it, but Likert scales are as outdated a technique as smoking the same brand of cigarettes as your doctor. For instance, here are three questions from an employee satisfaction survey: 1. I am satisfied with my overall compensation. Strongly agree, agree, neutral, disagree, strongly disagree. 2. I am often so involved in my work that the day goes by very quickly. Strongly agree, agree, neutral, disagree, strongly disagree. 3. C ommunication between senior leaders and employees is good in my organization. Strongly agree, agree, neutral, disagree, strongly disagree. The better approach, which will be less mentally taxing for respondents and produce more accurate results, is to use what Saris et al., call “construct-specific responses”: basically, measure the dimension of interest using an appropriate scale, which will most likely vary from question to question. Here are the same three questions rewritten to use the appropriate scales: 1. H ow satisfied are you with your overall compensation? Not at all satisfied, slightly satisfied, moderately satisfied, very satisfied, completely satisfied. 2. H ow frequently are you so involved in your work that the day goes by very quickly? Never, rarely, sometimes, often, always. 3. H ow is communication between senior leaders and employees in your organization? Very poor, poor, fair, good, excellent. 3. Laying out grids “Hey,” you probably just realized, “I can reorganize that battery of Likert scale questions into groups of grid questions using more appropriate scales!” Congratulations, you just replaced that bag of potato chips you were munching on while watching late-night TV with a bag of pretzels. Tell yourself they’re better. Go ahead. Then lick all the salt out of the bottom of the bag. Survey Sampling International (SSI) has conducted many studies on the negative effects of grids. In one of its more recent research-on-research initiatives, presented at the Council of American Survey Research Organizations (CASRO) 2013 Online Research Conference, SSI discovered that respondents completed a grid almost 20 times faster than they completed the same material when asked as a battery of regular questions. They took half a second per item in a grid, but eight to 10 seconds per question. Additionally, seven per cent of respondents fell for an inattention trap, compared to four per cent in the question battery. And questions about job satisfaction were more likely to correlate to willingness to recommend their job in the battery of questions than in the grid (.56 correlation vs. .48 correlation), indicating that the data was higher quality. Grids simply encourage respondents to respond too quickly. Re-engineering a grid requires thinking of alternative approaches, such as choose-many lists, MaxDiff scales, drilldowns and so on. But a lazy approach, that produces superior results, is to simply break up that grid into separate questions. 4. Reporting averages everywhere Finally, here’s a bad habit where you can be even lazier following the good habit. Traditionally, if you need a summary statistic, you report the average. The magic of the median, compared to the average, is that it automatically corrects for outliers. For instance, in a recent survey with 105 responses, household income ranged from $5,000 to $205,000. The average was $45,857. The median, however, was just $35,000. Another example, from a study on app purchasing behaviour, had the median amount spent on apps since the mobile device was purchased at $15, but the average at twice that: $30.53. Why was the average so high? One per cent of respondents (eight individuals) spent $300 or more dollars, and two individuals spent $600 and $970, respectively. Now, if you want to make the average work you have to ask yourself if these outliers are unreasonable – using the median spares you from having to make such judgment calls. The average and the median are summary statistics, meant to describe a range of values. The median is more representative of all the values than the average when data is skewed asymmetrically. Even when data is distributed symmetrically, the median is still accurate. If you are going to summarize the results with just one number, the median is more representative. 5. Discussing cross-tabulations I was playing with the banner report of a new-to-me survey system. The default option simply reported the cross-tabs. You had to change a setting to report the results that were statistically significant. The recent research obsession with storytelling is easily sated by just reporting the cross-tab results: “25 per cent of men would rather watch hockey than have sex, compared to just 19 per cent of women” – there’s a statement that can provide some nice story angles, even if the results are statistically the same given the sample size. Again and again, I see publicists write up survey results while ignoring statistical significance altogether. vue | JANUARY/FEBRUARY 2014 15 FEAT UR E 6. R eporting only sampling error Most methodological descriptions simply refer to the margin of sampling error. For studies with convenience samples, the summary might caveat it by saying, “If this were a probability survey, the margin of error would be...” Similarly, if this magazine article were printed on gold tablets, it would very valuable. Here’s some language you can borrow for your next survey: As this was not a probabilitybased sample, calculating the theoretical margin of sampling error is not applicable. However, as with probability surveys, it is important to keep in mind that the results are estimates, and typically vary within a narrow range around the actual value that would be calculated by interviewing everyone in a population. Again, as with probability surveys, on occasion the results from a particular question will be completely outside a typical interval of error. Many types of survey error can limit the accuracy of generalizing to a population. Throughout the research process, we sought to consider and minimize total survey error, including coverage error, non-response error, measurement error, mode effects, and post-survey error. We are confident that the information gathered from this survey can be used to make important business decisions. 7. Spending inadequate time on analysis Sorry, I don’t have time to think about these bad habits anymore. Off to the next survey. Well, after I quickly check Twitter. Jeffrey Henning, PRC, is president of Researchscape International, http:// researchscape.com, a market research firm providing custom surveys to small businesses. He coined the use of the #MRX hashtag on Twitter when he should have been writing a customer-satisfaction survey. You can procrastinate by following him via @jhenning. 16 vue | JANUARY/FEBRUARY 2014 vue | JANUARY/FEBRUARY 2014 17 FEAT UR E Questionnaire Design Levels of Measurement and Statistical Method Choices David S. Dobson Picture this: You’ve completed a marketing research survey using a probability sample, and your client asks if you can do a regression analysis; how would you answer? As long as you have the relevant data, you can perform any type of analysis. This article lays down some simple guidelines on how to plan your data analysis, and ensure that you are using the appropriate levels of measurement in your questionnaire. F First, you should develop your analysis plan, because knowing what data you need will ensure you ask the appropriate survey questions. Next, you should form your research hypotheses, to help identify what statistical tests to perform. Finally, develop a tab-plan by specifying the banner points to answer the research questions. Questionnaire Design To develop a reasonable and clear research objective, you need to define the research problem of the project. Ensure that your questionnaire is designed to answer your research objectives. Exploratory research, such as focus groups and literature reviews, will help define what questions should be asked in your survey. Dawn Iacobucci and Gilbert A. Churchill Jr., (2010) outline the following procedures for designing a questionnaire: Step 1: Specify what information will be sought Step 2: Determine type of questionnaire and method of administration Step 3: Determine content of individual questions Step 4: D etermine form of response to each question Step 5: Decide on question wording Step 6: Decide on question sequence Step 7: Determine physical characteristics of questionnaire Step 8: Re-examine and revise questionnaire Step 9: Pre-test the questionnaire, revise where needed 18 vue | JANUARY/FEBRUARY 2014 All the steps of questionnaire design are important, but pre-testing the questionnaire is the most important. A pre-test can determine whether a respondent understands the questions and is able to give unbiased responses. Levels of Measurement In data analysis, the characteristic of a question is referred to as a variable. For example, in the question “What is your gender?” the variable is “gender.” There are four levels of measurement of a variable, also known as scales of measurement: Nominal level data: These are survey responses that provide data on a respondent’s identity. For example, a respondent’s gender, or the city of his/her residency, is a nominal scale variable. These variables might have numerical codes for classification, but the number has no hierarchical order. Generally, questions with nominal scale responses are easy to answer. Respondents do not have to think too hard to answer questions about their gender or the city they live in. Ordinal level data: These are survey responses that can be ranked in order; for example, age categories, (under 25, 25–34, 35–64, 65+) and customer satisfaction ratings (superior, good, average, poor). Data on such variables can be classified by their relative rank. Interval level data: These are survey responses that provide numerical values which can be ranked, with the difference between two numbers being meaningful. An example is the question: “Please indicate how likely you are FEAT URE to attend a special event, on a scale of 0 to 10, with 0 being very unlikely, and 10 being very likely?” An interval scale has an equal interval between each increment; for example, the difference between “0” and “1” is the same as the difference between “9” and “10.” Different measurement scales can be used to measure the same question. Knowledge of different types of measurement scales will help you decide what scale to use for your specific questions. Thus, the scales used in survey questions will determine the type of data collected. Depending on the data type, the data analysis and data interpretation will vary. Ratio level data: Ratio level data is similar to interval level data, but it possesses a true zero point; therefore, the ratio between two numbers can be calculated. An example is the question: “On average, how much time do you spend on social media sites per day?” It is possible that some respondents may spend four hours and some respondents may spend two hours on such sites. Unlike 0 on an interval scale, 0 on a ratio scale represents the absence of the value. Thus, a person spending four hours on social media sites is spending twice as much time as the one who spends two hours on such sites. You must be mindful of the survey respondents. Will they be able to handle the level of difficulty when you try to use different measurement scales in your survey? Nominal and ordinal scale questions tend to be easy to answer, whereas interval and ratio scale questions are not. It is important to remember that as the difficulty of answering questions increases, the quality of the data decreases. Choice of Statistical Methods If probability sampling methods are used, the levels of measurement are important in data analysis because the appropriate statistical method depends on the type of data. Nominal and ordinal scales provide categorical data, whereas interval and ratio scales provide numerical data. The statistical method used depends on the type of data. Listed below are some common data analysis techniques: Categorical Data Analysis: Categorical data can be easily tabulated or cross-tabulated. Modes or proportions of categorical variables can be calculated as summary statistics. For statistical tests, the z-test can be used for the difference between two proportions, and the chi-squared test can be used for the difference among more than two proportions. Numerical Data Analysis: Numerical data is the highest level of measurement; with it, you can measure the magnitude of a response. The mean and the standard deviation can be computed with numerical data. For the statistical techniques, the t-test for comparing the means, analysis of variance, covariance, correlations and linear regressions, can be performed with numerical data. Advanced techniques such as the k-means clustering, factor analysis, and discriminant analysis, can be performed with numerical data. Numerical data can be re-coded into categories, and, as such, can be treated as categorical variables. It is important to remember that the probability sample is the only sample that allows you to use statistical methods. Statistical methods cannot be used to generalize data from a non-probability sample. The above mentioned data techniques will have no meaning if the respondents are not selected randomly. As this is an article, the information provided is brief. If you are interested in learning more and delving further into this topic, there are a number of good books on business statistics and marketing research methodology; I have listed some on my reference list. References Melvin Crask, Richard J. Fox and Roy G. Stout, Marketing Research: Principles and Applications. (New Jersey: Prentice Hall, 1995). David S. Dobson, “Significance Testing: How to compare the statistics of two independent samples,” Imprints (PMRS), September 2003, 26–27. David S. Dobson, “Does Sample Size Matter?” Imprints (PMRS), January 2001, 4–8. David S. Dobson, “The ABC of Data Analysis,” Imprints (PMRS), June 2000, 48–53. David F. Groebner, Patrick W. Shannon and Phillip C. Fry, Business Statistics (9th ed.). (Pearson, 2013). Dawn Iacobucci and Gilbert A. Churchill Jr., Marketing Research: Methodological Foundations. (10th ed.). (Cengage Learning, 2010). David M. Levine, Mark L. Berenson, Timothy C. Krehbiel and David F. Stephan, Statistics for Managers using MS Excel. (6th ed.). (Pearson, 2010). Conclusion Douglas A. Lind, William G. Marchal and Samuel A. Wathen, Basic Statistics for Business and Economics. (6th ed.). (McGraw-Hill, 2007) All analysis, whether basic or advanced, should be done while always keeping in mind your research objectives. It is important to keep focused on this, as it is easy to get distracted if you do not know the purpose of your analysis. Having a data analysis plan, formulating the research hypothesis, and a tab plan will help you get on the right track to addressing your research objectives. David S. Dobson is an instructor at the School of Business at the University of the Fraser Valley in B.C. He can be reached at [email protected]. vue | JANUARY/FEBRUARY 2014 19 FEAT UR E Master Crafting Qualitative Basics Kendall Nash Exhilarating and Evolving! The past five years of Qualitative Research have been nothing if not fastpaced and ever changing. When you encounter those spirited qualitative practitioners and they say it’s an exciting time to be in qualitative research, they mean it. Leaps in technology and blurs in the Qual-Quant paradigm have invited researchers into the most exciting opportunities alongside our clients. It’s easy to let it all carry you away into a happy land of blissful possibility. In fact, I often now have clients asking me to conduct research using methods that I have worked tirelessly to have others accept over the years. I should be happy. Well, I’m glad to have the door open if the method is, in fact, what I know will yield optimal learning, but in other cases I receive a request for a round peg in a square hole. Bells and whistles don’t always mean better. Sometimes I truly need to simply sit one-onone with a consumer and unpack their story through a very rudimentary dialogue. The truth is that when things move quickly, the foundational details for insightful learning from the consumer often become fuzzy – and occasionally lost in the shuffle. So, let’s hit pause on the lightning-speed evolution of what we do, and talk about how we ensure that what we do is done well. Understand the business decision at hand Remember the game of telephone where you whispered a phrase from friend to friend until it was hysterically diluted by the time the last person repeated it out loud? Unfortunately, it’s all too often the case that the objectives of a research initiative are articulated and then passed through a series of filters that undoubtedly add their own twist until you end up with a listed objective that sounds about right, but is missing something of value. Whenever possible, ask your clients (internal and external) to tell you about their goals of the research. Nothing replaces hearing from the mouths of your clients about what they need and expect the research to be able to do for their business. And then there’s the unfortunate scenario of a team uncertain of the real objectives themselves. The overall value and effectiveness of the research is completely shaped by the clarity of the objectives, so if the research goals and how the research will 20 vue | JANUARY/FEBRUARY 2014 be used are not clear to all parties, the entire team needs to spend some time getting on the same page. Clearly understanding the goals of the research so that you can effectively probe during an interview truly makes or breaks the research. Get the right people Anyone who has done qualitative research for any amount of time has more stories than they’d like to share about the interviews or groups they’ve conducted in which the respondents weren’t the “right people.” In some instances, the respondents have been blatantly unqualified, but in other cases, the respondent may have fit the profile, technically speaking, but didn’t capture the essence of who it was we needed to really learn from. Enter the recruiter. Developing strong relationships with solid recruiters who implement best practices and recruit with integrity is one of the most important determining factors to setting up a project for success. You can create an amazing discussion guide and utilize a dynamic moderator, but if the people in the room don’t fit the bill, it’s all for nothing. However, a good recruiter can only do so much if they don’t have a strong screener from which to work. It’s critical to spend adequate time designing a screener that effectively gets the “right” respondents into the research. At times, the speed of the project launch requires turning around a screener within a couple of hours. I’d argue that we all need to give the screener the attention it deserves, as a slight miswording or omission of any given question may lead us to the wrong respondent who can’t teach us what we desire to learn. Take the time to build a screener that works. Ask the right questions Understandably, a lot of time is spent on determining the data collection method itself. However, sufficient time is not always taken at that same point to strategically vet how the team will elicit feedback from the consumer. So let’s refocus, even if only briefly, on a few techniques in which every qualitative researcher should strive to excel. Mastery of these techniques will ensure a qualitative consultant is able to elicit deep learning from respondents, beyond the surface comments of convenience and perhaps even awareness. While skilled moderators are able to apply a multitude of techniques at the right occasion, there are three in which all qualitative practitioners should be well-versed: Nominal Group Techniques, Laddering, and Projective Techniques. FEAT URE Nominal Group Techniques: Respondents in a group setting are asked to independently write down their responses, opinions, or ideas, and rationale before responses are discussed and captured as a group. This technique helps keep respondents accountable for their personal opinions and can reduce some of the challenges of group influence. Effective qualitative consultants leverage this technique when appropriate, and can use it to gather a wider range of opinions and ideas. It helps to establish a voice for everyone participating in the research, despite individual personality. There is a range of variations of this technique, and the ease of utilizing it in many online settings has presented one strong benefit of online tools. Laddering: This primarily refers to an in-depth, one-on-one interviewing technique used to develop an understanding of how consumers translate a product’s attributes with respect to self. Means-End Chaining, (the basic questioning structure for laddering), focuses on the linkages between the attributes that exist in the products (the “means”), the consequences for the consumer provided by the attributes, and the personal values (the “ends”) that the consequences reinforce. Understanding how attributes of a specific brand are personally relevant is the basis for the development of messaging strategies. Now, in English? It’s taking face-value responses and going much deeper by digging to find the higher-level need-states that may exist. A woman may say she prefers a spicy cracker, but what if you could find out from her that the reason she really wants it is because the stronger taste makes her eat less, so that she doesn’t gain weight, so that she has a better figure, and ultimately feels good about herself? It’s a better conversation. Projective Techniques: This includes a wide range of activities and lines of questioning that offer an indirect way of eliciting learning from the respondent. Through these techniques, the researcher is able to understand the underlying feelings, needs, and opinions, by offering respondents a way to express things that are either uncomfortable to say, or may be less conscious thoughts or decisions. It helps to put the respondents in the moment, or in a situation through which stories and examples can provide context as to why and how someone felt the way they did. These tools allow the respondent to more easily articulate something that they are more familiar with, compared to something very abstract, or nebulous things like a brand, which has many layers or levels. Examples of more commonly used projective tools include: personification exercises, situational drawings, storytelling, scripting activities and collaging. There are a host of other approaches for which a qualitative practitioner should be familiar, but the techniques covered here are widely applicable across a range of industries and have the potential to deliver powerful learning to the research team. But know that sometimes the right answer is even simpler than all these specialized tools – ask a question and truly listen to the answer. Get creative There is no guarantee in qualitative research except to expect the unexpected. Qualitative projects are completely customized initiatives. A meaningful and fruitful exercise in one group may fall flat in another. We may be testing stimulus in the form of wireframes or prototypes with some groups, and asking other respondents to try on underwear. The craziness is what keeps a lot of us in love with what we do, but it’s not for the faint of heart. Be prepared to get creative – come up with creative solutions to strange problems, be willing to try new techniques at a moment’s notice, and be creative in how you approach the process, as well as the ways you ask the client team members to contribute to the learning. Deliver the results in a meaningful way Now that we have thoughtfully planned and executed solid research, we must turn our attention to delivering it to our clients in a way that has an impact on their business. Consider form and function when it comes to reporting on qualitative insights. It’s not enough to merely tell them what you heard. The power is in the analysis – looking for patterns of similarities, as well as points of dissonance. Where do the client’s opportunities really lie? How does the learning from the research have the potential to deeply impact their current or future business? The answer is insights-driven reporting. It’s taking the face value responses from the research and marrying them with peripheral knowledge of the product, service, or brand, as well as emotional context, to help your clients make sense of it all. Reports live on to tell the story of our research, long after our foggy memories lose track of things we never thought we’d forget. So, take the time to make a lasting report that tells the story. A report that is: visually appealing so that people will actually want to read it; structured and written in a way that the client culture can really consume; concise enough that upper level management can get the gist with one perusal; and makes that leap from insight to action by providing recommendations for the clients. Never underestimate the power of an effective and meaningful report. To be a true partner for your client, you must provide guidance in navigating these projects with excellence in each step of the process. So, make sure you clearly understand the research goals, help identify and recruit the right people, truly consult about the questioning techniques used to elicit rich learning, offer creativity to the process, and deliver impactful and meaningful reports. Do these well and your client won’t be able to let you go. Kendall Nash is the senior qualitative consultant, Decision Sciences, at Burke, Inc. in Cincinnati, Ohio, where she conducts and oversees hundreds of in-person and online qualitative projects. She serves as the president of the Qualitative Research Consultants Association (QRCA) and teaches the “Next Generation Qualitative Tools” course through the Burke Institute. vue | JANUARY/FEBRUARY 2014 21 THANK YOU TO OUR SPONSORS, SPEAKERS AND DELEGATES SPONSORS PLATINUM An Aimia Company GOLD BRONZE REAL PEOPLE. REAL SOLUTIONS. REAL DATA SILVER SPEAKERS Susan Abbott, CMRP Anne Crassweller Tom de Ruyck Caroline Fletcher Frank Graves, MA, FMRIA Jeffrey Henning Chris Long Bernie Malinoff, CMRP Eric Meerkamper Mark Michelson Grant Miller Rudy Nadilo Annie Pettit We thank everyone who attended and look forward to welcoming you at future MRIA events ! 22 vue | JANUARY/FEBRUARY 2014 HELLO my name is Tracy Rideout St. John’s, Newfoundland & Labrador MQO Research • • • • • When I was ten, I wanted to be: A speech pathologist My first job was: A cashier at A&W A researcher I admire is: Kerry Bodine Why I became a market researcher: After finishing my Bachelor’s in Psychology, I was unsure about my career path. I decided to pursue a Master of Applied Social Psychology degree which focuses on research methodology, program evaluation, attitude measurement and advanced statistics. After completing this program, it was almost a natural step for me to enter the industry. CMRP date: August 2013 vue | JANUARY/FEBRUARY 2014 23 This is what happens when two researchers are told to argue an opinion they might not agree with. MARKET RESEARCH: ESSENTIAL OR POINTLESS? What does the business environment look like without market research? Picture this: • Companies develop products without any feedback and miss the mark in meeting their customers’ needs. Katie Clark • Companies don’t listen to their customers’ issues and miss the opportunity to jump in and fix problems before the customer defects to a competitor. • Companies don’t investigate the competitive landscape and risk being blindsided by a competitor’s new product or service, losing customers and dollars. It’s a dismal picture, isn’t it? Now imagine a world where market research is a requirement for all companies : businesses MUST conduct market research. I’d like to propose a brave new world where it is an absolute requirement that companies conduct market research; that research becomes hard-wired into the DNA of a business. Imagine a new startup business immediately saying “We need research,” in the same breath as “We need an accountant” and “We need office space”! Some, even in our industry, might argue that businesses do not need to conduct market research (cue the overused Steve Jobs quote). Some companies may argue that their product “is what it is,” and their business contracts are solid for the next few years; rendering research and new product development “unnecessary.” Others may argue that their company knows best about what products to develop and produce; that consumers don’t know what they need so why even ask them. 24 vue | JANUARY/FEBRUARY 2014 Steve Jobs once famously said, “We were the group of people who were going to judge whether it was great or not. We weren’t going to go out and do market research.” And, of course, we all interpreted that as Steve Jobs doesn’t do market research. Steve Jobs was correct. Consumers don’t know what they Mike MacLeod want. At least collectively, they don’t know what the next major leap should be in the development of a product. And they are even more inept when describing what should be included in a product that creates an entirely new category. Truly innovative products and concepts are usually so far away from the current state that it requires the ability to incorporate changes that cannot even be comprehended by most. Jobs also said, “How can I possibly ask someone what a graphics-based computer ought to be when they have no idea what a graphics-based computer is? No one has ever seen one before.” Consider 2,000 consumers on a panel, balanced to match the census for age, gender and ethnicity. No survey could possibly be written that would result in data supporting the development of the newest, most innovative, cutting-edge product, regardless of the category. What questions would you have asked this panel that would have led to the development of the Mac? Or, how about a dozen focus group participants, sitting around a conference table, eating deli sandwiches for a couple of hours? There is no moderator in the industry that could have described the iPod to these people 14 years ago. And even if FAC E OF F These companies may rail against a market research mandate. My counter-argument? They have a very narrow definition of market research! For example, now may not be the right time for extensive consumer research to develop a new product. But what if your competitor is launching a product? Competitive research in this situation would be key! Resting on your laurels with your existing product? Consider some secondary research on your own data to determine patterns that may lead to new thinking about how to increase sales for those existing products. Or perhaps a study on how customers are using those products and how that use has changed over time? Juicy insights abound! We’ve all heard the horror stories of companies that have launched products or services without research only to watch those products quickly fail. We’ve winced when we hear about companies who continue to ignore the voice of their customer. Now, really, who wants to live in a world where market research isn’t conducted? Show me a spectacularly successful company that has ignored all facets of market research: from secondary to primary, from pricing to ethnography, from new product development to the Voice of the Customer! Research can take many forms, so there’s really no need for companies to fight against the “businesses MUST conduct market research” requirement. If companies embrace market research as a necessity and not just a “nice to have,” the opportunities for the kinds of research that can be conducted are endless. Research can be primary or secondary, gathering new data, or finding new ways to look at existing data. Startups can focus on researching the marketplace they’re getting into and competitive products. Companies that are struggling with accurately positioning and pricing a product might look to an in-depth conjoint study. Companies hoping to more deeply understand customers’ experience with their product might do deep dive ethnography. That said, in no way am I recommending companies do throwaway research – doing research just because it’s mandated. Instead, find the right kind of research for the right phase of the company or product that can help the business improve and be competitive. A requirement to mandate research would be a blessing – how can it be a bad thing to require companies to listen to their customers? Pray tell why shouldn’t it be a requirement that companies hard-wire the voice of the customer into their organization’s DNA? Should market research be a requirement for all companies? Yes. Let’s make it happen! Now, who’s with me? Katie Clark is the senior research manager at Diversified Business Communications and active in the social media space blogging and tweeting (@InsightsGal) about research. The opinions expressed here are her own and not those of her employer. Katie was assigned this “side” of the argument to argue. While mandated research might be a stretch in the current business environment, she believes that the most successful companies are those that have made research a priority. one could, how many of these respondents could understand how an iPod would be better than their portable CD player? What’s an MP3? I can’t see it? I have to download it? Where do I keep it? How do I pay for it? What’s iTunes? Now, how about a small handful of designers with a narrow focus, led by a leader fixated on a singular mission? A group such as this, combining their time and effort with a superb imagination and the drive to provide something innovative is what is needed to develop the next big thing. They should have zero concerns about the opinions of the masses who have pondered the topic but for a mere few moments. Consumers that understand and appreciate the results of such a group can become product evangelists. Some consumers won’t get it, and others still may hate what is produced. But that’s better than feeling that it’s just OK. Maybe it’s a little better, or a little worse, than something else already on the market. Something that is easily replaced. Something that no one cares about. When you listen to customers, (existing or potential ones), the result is product development that is evolutionary – but definitely not revolutionary. That may be fine if your goal is to create something that is marginally better than last year’s model or if you’re looking to compete directly with an existing competitor. This will help you get a sliver of an already existing pie. Alternatively, if you are looking to revolutionize your existing category, or, better yet, create a brand new product category, then run quickly from your market research supplier and don’t look back. No amount of research with writers and typists in the 1970s would tell anyone to build word processors as a replacement for the typewriter. Henry Ford may have said, “If I asked people what they wanted they would have said faster horses!” Of course, that statement is correct. What would your answer have been if he stopped your horse and buggy to ask about your transportation needs? Even if he could describe the automobile to you, what would you do if it breaks down? Where would you drive it with the terrible road conditions? And where are you going to get gas? The idea sounds silly. We should just stick with horses. Of course, that’s ridiculous. Ford knew better than to even ask such questions. He made his own decisions about what he would produce, how it would be produced, and the features it would or would not have. He would have been much less successful, and had minimal impact on the world, if he catered to the thoughts of the rank and file. So yes, Steve Jobs was correct. Market research doesn’t help with innovation because people don’t know what they want. Not until you show it to them. Mike MacLeod has been on the leading edge of the marketing research industry for 22 years. His focus is to maximize data quality by addressing the issue from all sides. He is currently senior account director at Lightspeed Research. A transplanted Canadian, he currently resides in Northeast Pennsylvania. Mike tweets at @MikeMacLeod. vue | JANUARY/FEBRUARY 2014 25 INDUSTRY N EW S Marketing Research and Intelligence Association L’Association de la recherche et de l’intelligence marketing CMRP – PARM CERTIFIED MARKETING RESEARCH PROFESSIONAL – CMRP PROFESSIONNEL AGRÉÉ EN RECHERCHE MARKETING – PARM SIGNIFIES HIGH LEVEL OF KNOWLEDGE AND CAPABILITY IN MARKETING RESEARCH THEORY AND PRACTICE, AND ADHERENCE TO RIGOROUS ETHICAL STANDARDS SET OUT IN MRIA’S CODE OF CONDUCT AND GOOD PRACTICE. A CMRP has the ability to: 4.1 4.2 4.3 4.4 4.5 4.6 4.7 Apply Core Knowledge* Identify Issues Analyze Integrate Knowledge and Issues Make a Professional Judgment Respond to Users’ Needs Effectively Communicate *The Core Knowledge of the CMRP includes: I. Professional Practice II. Marketing Research Design III. Statistical Methods IV. Questionnaire Design V. Qualitative Marketing Research VI. Market Intelligence & Competitive Intelligence VII. Marketing Management Please see Core Curriculum Competencies document for details. A copy of this document is available online: http://mria-arim.ca/education/cmrp-certification GOLD SEAL CERTIFIED CORPORATE RESEARCH AGENCY INDICATES THAT A CORPORATE RESEARCH AGENCY MEMBER: • H as successfully completed an independent, third party Certification Review focused on how effectively the Agency meets MRIA standards, particularly those related to quality control practices, obligations to clients and obligations to respondents. The Certification Reviewer has found the Research Agency to be in compliance with MRIA Standards in all material respects; • Operates as a business in Canada and derives at least 75% of its revenues from the provision of marketing and survey research or business intelligence services * to other companies or organizations; * (Includes consulting services, data collection, data analysis, and adding value to information) • S ubscribes to the Code of Standards and Ethical Practices of the MRIA; • Registers its surveys with MRIA’s Research Registration System; • Has been operating as a business in Canada for at least 24 consecutive months. Please see the MRIA website for a complete list of Gold Seal Agencies: http://mria-arim.ca/directories/gold-seal-members BASIC CORPORATE RESEARCH AGENCY INDICATES THAT A CORPORATE RESEARCH AGENCY MEMBER: • O perates as a business in Canada and derives at least 75% of its revenues from the provision of marketing and survey research or business intelligence services* to other companies or organizations; * (Includes consulting services, data collection, data analysis, and adding value to information) • Subscribes to the Code of Standards and Ethical Practices of the MRIA; • Registers its surveys with MRIA’s Research Registration System. For more information on MRIA Corporate Memberships and on the CMRP designation, visit us at www.mria-arim.ca Pour de plus amples renseignements sur l’adhésion corporative et sur les sociétés de recherche Sceau d’or, visitez http://mria-arim.ca/fr/adhesion/adhesion-corporative/sous-categories 26 vue | JANUARY/FEBRUARY 2014 IND U STRY NEWS QUALITATIVE RESEARCH REGISTRY (QRR) In accordance with federal privacy laws, MRIA’s Qualitative Research Registry (QRR), or Registre de la recherche qualitative (RRQ) in French, was created to provide an ongoing, user-friendly vehicle for tracking those who do not want to be contacted or should not be contacted for qualitative research studies. QRR is a comprehensive do not call list of those who have recently participated in qualitative research studies, those who have asked not to be contacted further, and those felt by recruiters and moderators to be best served by not being contacted. These respondents are marked as “do not call” in accordance with established MRIA Standards. All field and full-service companies are encouraged to submit a list of their qualitative respondents for entry into the QRR system each month, including those who do not wish to be contacted. Participating firms will receive monthly updates of respondents to be screened from qualitative recruitment samples. QRR works effectively to increase the quality and integrity of the qualitative research process, by serving as a control to ensure respondents are not contacted more frequently than is necessary. However, the ability of the system to function effectively is directly related to the co-operation received from firms who provide recruitment services. If you are a full service research firm or field supplier that is currently participating in the Qualitative Research Registry program – thank you very much and keep up the good work! If you are not currently participating, please get involved! If you are interested in submitting to QRR, please visit the MRIA website at http://mria-arim.ca/about-mria/qualitative-researchdivision/qualitative-research-registry for further explanation and guidance on how to submit qualitative research participants’ names, along with the required electronic forms. THE FOLLOWING COMPANIES HAVE SUBMITTED NAMES TO QUALITATIVE RESEARCH REGISTRY JULY 2013 ONTARIO QUEBEC Barbara C. Campbell Recruiting Consumer Vision CRC Research Dawn Smith Field Management Services Inc. I & S Recruiting Ipsos Reid Nexus Research Quality Response Research House Inc. Ipsos Reid WEST Barbara C. Campbell Recruiting CRC Research Ipsos Reid Trend Research Qualitative Research Registry submissions should be sent to: [email protected] Submission templates and payment forms can be found at http://mria-arim.ca/about-mria/qualitative-research-division/qualitative-research-registry-fees Rules of Conduct and Good Practice for Members of the Marketing Research and Intelligence Association (2007), Section C Rules Specific to the Conduct of Qualitative Research: 20. R ecruiters should provide accurate data to the Qualitative Research Registry, where such exists, on a consistent basis and check all respondents against the Registry. 21. M oderators buying recruiting services should give primary consideration to recruiting agencies which submit to the Qualitative Research Registry, where such a service exists, on a regular and ongoing basis. vue | JANUARY/FEBRUARY 2014 27 INDUSTRY N EW S RESEARCH REGISTRATION SYSTEM Since 1994, the RRS has allowed respondents to verify the legitimacy of a research project; helped legislators and regulators differentiate between legitimate survey researchers and unscrupulous telemarketers, phishers and scammers; and protected the industry from unnecessary and unwanted regulation. MRIA’s Research Registration System (RRS) has long been a cornerstone self-regulatory mechanism for the marketing, survey and public opinion research and market intelligence industry in Canada. Combined with other self-regulatory initiatives such as our Code of Conduct and Good Practice and our Charter of Respondent Rights, the RRS has paid huge dividends in protecting the industry’s positive reputation and good name with Canadians. All Gold Seal and Basic Corporate Research Agency members of the Association are obligated to register all of their research projects with the RRS, and Client-Side Corporate members are encouraged to require their agency suppliers to do so. MRIA’s Research Agency Council provides strategic, policy-level oversight of the Research Registration System, and receives aggregate data-only on the System’s performance. Questions about the Research Registration System should be addressed to Erica Klie, Member Services & Events Coordinator, at 1-888-602-6742 or 905-602-6854, ext. 8727 or [email protected] or, in her absence, Interim Executive Director, John Ball, CMRP at ext. 8724 or [email protected]. The following companies have registered research projects with the Research Registration System during JULY 2013: GOLD SEAL CORPORATE RESEARCH AGENCIES Academica Group Advitek Inc. BBM Analytics BBM Canada Blue Ocean Contact Centers Campaign Research Canadian Viewpoint Inc. Cido Research Consumer Vision Ltd. EKOS Research Associates Inc. Greenwich Associates Harris/Decima Inc. Head Count Hotspex Inc. Ifop North America Ipsos Reid Maritz Research Canada MBA Recherche MD Analytics Inc. MQO Research Nanos Research NRG Research Group Opinion Search Inc. Research Dimensions Research House Inc. Tele-Surveys Plus / Télé-Sondages Plus The Logit Group Inc. TNS Canada (Canadian Facts) Trend Research Inc. BASIC CORPORATE RESEARCH AGENCIES Barbara C. Campbell Recruiting Inc. (BCCR Inc.) Goss Gilroy Inc. Nexus Market Research Inc. SmartPoint Research Inc. INDIVIDUAL MEMBER ORGANIZATION Illumina Research Partners Rules of Conduct and Good Practice For Members of the Marketing Research and Intelligence Association (2007): Section A (5) Members must uphold the MRIA Charter of Respondent Rights. Charter of Respondent Rights, Article 2 You can verify that the research you have been invited to participate in is legitimate in one of two ways. You can either obtain a registration number and the MRIA’s toll-free telephone number for any research registered in the MRIA’s Research Registration System or you can obtain the contact information of the research director who is conducting the study. http://mria-arim.ca/about-mria/research-registration/research-registration-overview 28 vue | JANUARY/FEBRUARY 2014 IND U STRY NEWS n To read more news online, or to submit your “People and Companies in the News,” s imply fill out our online form at http://mria-arim.ca/news/people-and-company-news. n The Vue editorial team reserves the right to select and edit your submission for appearance in Vue. n MRIA is neither responsible for the accuracy of this information nor liable for any false information. Bruce Anderson Joins Abacus Data as Chairman Abacus Data is excited to announce that Bruce Anderson has joined Abacus Data as Chairman, starting December 1, 2013. He will be responsible for providing strategic advice to Abacus Data clients and helping Abacus grow its business across Canada and North America. Bruce has over 30 years’ experience in the market research sector and is also a partner at I2 Issues and Ideas Advertising. Website: http://www.abacusdata.ca Research and Incite Appoints Mary Logan, CMRP, to President Research and Incite are pleased to announce the appointment of Mary Logan to the position of President. Mary has been a guiding force at Research and Incite since 2000 and most recently held the position of Executive Vice President. An active member of the research industry for 25 years, Mary has extensive experience of both the North American and European markets. A professional member of the MRIA with CMRP status, she holds an honours degree in Social Psychology from the University of Sussex, U.K. Website: http://www.riconsultants.com Hank Goertzen, CMRP Obituary published in The Saskatoon StarPhoenix on Dec. 13, 2013: It is with great sadness that the family of Hank Goertzen announces his sudden passing on Tuesday, December 10, 2013 at the age of 59 years. Hank is survived by his loving wife, Linda; sons, Michael (Heather), Kyle, and Sean (Allie); his daughters through marriage, Tracy (Todd), Keri, and Robin (Brian); five grandchildren, Jaimee, Zane, Kai, Lauren, and Hudson; sisters, Erna (Wilf), Frances, and Ruth (Alan); and many other extended family. ESOMAR Applauds Recent Arrests in Speak Asia Survey Fraud Case Following an almost three-year investigation, Indian authorities have arrested the masterminds behind the Speak Asia survey fraud. In 2011, ESOMAR, along with MRSI had actively advocated strict action against the company, which had been allegedly running the pyramid fraud multi-level marketing schemes that encouraged participants to pay large sums of money to become members of Speak Asia where they would receive rewards for completing research questionnaires/surveys. Website: www.esomar.org MDC Partners Buys Majority Stake in Luntz Global Marcoms network MDC Partners has acquired a majority partnership interest in language and communications research specialist Luntz Global, thereby extending its own Consumer Insights and Analytics practice. Terms of the deal were not disclosed.The new partnership will help extend MDC Partners’ Consumer Insights division, which develops data and analytics capabilities for use by MDC companies wishing to analyze clients’ integrated digital and multi-channel media campaigns. Website: www.luntzglobal.com Leger Metrics Launches Text Analytics Tool Canadian firm Leger Metrics has launched a solution to automatically analyze text responses from open-ended survey questions, social media, and anywhere else customers leave comments about their brand experiences. Leger Metrics was known as Agility Metrics, prior to its acquisition by marketing research firm Leger in 2012. The division specializes in voice-of-the-customer (VoC) and customer experience management (CEM) solutions, and offers continuous customer feedback and real-time insights. Website: www.legermetrics.com Nielsen Names New Marcoms Leader Nielsen has appointed former Edelman exec Katie Burke as Executive Vice President of Marketing and Communications, responsible for the firm’s internal and external communications, content development, digital marketing and industry relations. Burke joins with nearly 20 years’ experience. Most recently, she served as the Global Chair of Public Affairs at PR giant Edelman, prior to which she held several major roles in U.S. politics, including Director of Television News in the White House. Website: www.nielsen.com Murdoch Buys Social Media News Agency Storyful Rupert Murdoch’s News Corp group is paying EUR 18 million (C$24.6 million) for Dublin-based social media news agency Storyful, People and Companies in the News sponsored by: vue | JANUARY/FEBRUARY 2014 29 INDUSTRY N EW S which offers a combination of journalistic expertise and proprietary technology, including social media dashboards and other real-time discovery tools and feeds. Storyful was set up in 2010 by former foreign correspondent Mark Little. The company’s technology allows clients to integrate video into their news or advertising via online and mobile platforms, and to monitor social media conversations and sentiment. So far in 2013, verified user-generated videos managed by Storyful generated 750 million views for its partners. Websites: www.newscorp.com and www.storyful.com achieved online virality, or were identified as memorable by industry associations and experts. Using its “ComMotion” ad testing approach, BrainJuicer predicted the in-market efficiency of advertising from the emotional response it evokes, classifying advertising according to its “emotional pull” and awarding it a star rating. According to Chief Juicer and CEO John Kearon, only 5% of the ads tested globally achieve a 5-star rating, (the U.S. norm is 2-star) – in his view, all brands should be focused on using advertising for emotional brand building. Website: www.brainjuicer.com FTC Approves New Imperium Approach for COPPA In the U.S., the Federal Trade Commission (FTC) has given its approval to a new method for obtaining parents’ verifiable consent for online collection and use of children’s personal information, developed by technology firm Imperium, whose ChildGuardOnline will offer a series of “challenge” questions asking for information “not commonly available or typically found in a person’s wallet” and “difficult for someone other than the individual to whom the information pertains to answer correctly’”, in order to verify that the person giving consent is in fact a parent. This ensures compliance with COPPA, the Children’s Online Privacy Protection Act. A month ago the FTC rejected a method put forward by AssertID based on social graph verification. Imperium is online at www.imperium.com Thanks to www.lexology.com for some of the above. IMS Health Plans IPO Pharma research giant IMS Health Holdings has filed a registration with the U.S. Securities and Exchange Commission (SEC) for an Initial Public Offering (IPO) on the New York Stock Exchange. Rumours about the likelihood of an IPO have been circulating for some time. An initial maximum offering price of $100 million is said to be a “placeholder” until the price and number of shares are settled. Some analysts suggested the amount raised would be up to $1 billion. IMS says it will use the proceeds of the IPO to repay long-term debt and for “general corporate purposes.” The company reports revenue up 4 per cent to $1.87 billion in the nine months to September 30, 2013. Website: www.imshealth.com BrainJuicer Debuts FeelMore50 ‘Emotional Ad’ Measure Online research specialist BrainJuicer has introduced the FeelMore50 – a ranking of the “50 most effective emotional ads” in the U.S. Last year, the firm tested more than 200 U.S. ads that had won awards, 30 vue | JANUARY/FEBRUARY 2014 EVENTS MARK YOUR CALENDAR February 19 Montreal Chapter: DéjeunerConférence Le service à la clientèle à l’ère du numérique Montreal, QC March 4 Canadian Marketing Association (CMA) and MRIA, Client Experience (CX) Conference Toronto, ON February 20 Ottawa Chapter: Driving Innovation in Mobile Survey Research and Meet the Marketing and Business Intelligence Research Class (MBIR) of 2013–14 Ottawa, ON June 8–10 MRIA National Conference 2014 – Dig Deeper & Discover Saskatoon, SK February 21 MRIA Qualitative Research Conference 2014 – Creating Connections Toronto, ON June 8 MRIA Annual General Meeting Saskatoon, SK vue | JANUARY/FEBRUARY 2014 31 BC CHAPTER ALBERTA CHAPTER November’s Design and Deliver Better Presentations workshop, hosted by Lauren Isaacson, went “swimmingly,” reports the BC Chapter. There were five engaged and active participants coming from both the client and agency sides. After discussing best practices, a real word research example was used to demonstrate how to best present complex data. The Alberta Chapter Board of Directors is sad to say farewell to Fay Poholko, a board member since 2007 and membership chair for the past six years. Says Fay, “The best thing about being a board member is getting to know a wide range of people in the industry.” We’d like to thank Fay for her contributions over the years and wish her well – she will be missed! In 2014, the BC Chapter wants to do more skills sharing. If you have a research technique or professional expertise that you want to share with your colleagues, please contact chapter president, Adam DiPaula ([email protected]). QUEBEC CHAPTER Qualitative Research Day On November 29, the MRIA’s Quebec Chapter invited its members and friends to take part in “Qual Day” to shed some light on various innovative approaches to qualitative research. Approximately 30 researchers came together to learn from six fascinating speakers. Part of the conference was dedicated to new technologies meant to support traditional qualitative research, including online research communities and social media. Journée sur la recherche qualitative Le 29 novembre, le chapitre québécois de l’ARIM conviait ses membres et amis à une « Journée Quali » pour faire la lumière sur diverses approches innovantes en recherche qualitative. Pour l’occasion une trentaine de chercheurs se sont réunis pour entendre nos six passionnants conférenciers. Il fut notamment question de nouvelles technologies en appui à la recherche qualitative traditionnelle, dont les communautés de recherche en ligne et les médias sociaux. Barry Watson, president of Environics Research Group, was one of six speakers at the Quebec Chapter’s “Qualitative Research Day” on November 29. About 50 people attended the Quebec Chapter’s Holiday Party held at the Smoking Vallée, on December 5. Professor André Richelieu from Laval University was the featured speaker at this year’s party. 32 vue | JANUARY/FEBRUARY 2014 CHA P TE R CHAT OTTAWA CHAPTER The MRIA Ottawa Chapter ended 2013 with “Public Opinion Polling: Is there a future for the discipline?” hosted by Steve Kiar CMRP, president of Phoenix Strategic Perspectives. Panelists – Joan Bryden (The Canadian Press), Éric Grenier (ThreeHundredEight.com), John Wright (Ipsos Global Affairs) and Doug Anderson (Harris/Decima) – discussed the realities our industry faces, including the role of industry professionals, and the media, in reporting accurate, dependable and informative research. There was quite a bit to talk about during the post-discussion wine and cheese. Approximately 40 people, including industry professionals, academia, and members of the media, attended the discussion. Panelist John Wright Ipsos Public Affairs Panelist Joan Bryden The Canadian Press Key recommendations included: 1. Improve the partnership between the industry and the media, bordering on full disclosure; 2. Encourage both industry practitioners and media to police published results; 3. Continue to embrace and promote research on research; 4. Develop working relationships with collaborative interests; including academia to keep the industry engaged; and, 5. Encourage the media to be careful about printing polling results focused on hype rather than on the value to the public. Panelist Doug Anderson Harris/Decima Panelist Moderator Eric Grenier Steve Kiar ThreeHundredEight.com Phoenix Strategic Perspectives TORONTO CHAPTER Approximately 40 people attended the Ottawa Chapter’s “Public Opinion Polling: Is there a future for the discipline?” panel discussion on November 29. T he Toronto Chapter held its holiday party at Buonanotte on December 12. Members and guests are welcome at all MRIA events: Check our online calendar at http://mria-arim.ca/events-awards/calendar for more information on all events and how to register. Members receive emails directly with event updates, so please check your inboxes for instructions on how to register for all upcoming events! vue | JANUARY/FEBRUARY 2014 33 COLUMNISTS Ask Dr. Ruth Dylan Cody Insightrix Research Inc. Ruth Corbin, CMRP CorbinPartners Inc. A column addressing questions received by the Chair of the Standards Portfolio. ______________________________ Dear Dr. Ruth: A major competitor has published polling results in one of Canada’s national newspapers, quoting a margin of error of plus or minus 3%, based on an Internet survey of its panel members. Isn’t it contrary to MRIA standards to attach a margin of error to Internet panel surveys? Should I be registering a formal complaint against the company? (Should I stop dating the CEO?) Troubled in Toronto ______________________________ Dear Troubled: The Marketing Research and Intelligence Association publishes a Code of Conduct for Members at http:// mria-arim.ca/sites/default/uploads/files/ MRIAConduct-Dec2007REV2010.pdf. Section 9(iii), the most recent statement of guidelines concerning the reporting of margins of error, states as follows: “Members must not present research results with greater confidence than the data warrant. Instead, as responsible professionals, members must point out the relevant limitations of the research. This includes...refraining from making statements about margins of sampling error on population estimates when probability samples are not used.” 34 Bright-Eyed vue | JANUARY/FEBRUARY 2014 The guideline refers to the characteristic of the sample (being a probability sample versus a non-probability sample) rather than explicitly referring to the online methodology. In the case of ambiguity or absence of comment in the MRIA guidelines for a given topic of interest, ESOMAR guidelines are to be treated as applicable to Canadian practice. While MRIA has a formal complaint process, you may first want to check for other relevant facts, including any appropriate disclaimers attached to the results, and whether any misunderstanding was introduced by the newspaper writer rather than your MRIA competitor. Thanks for writing. p.s. (If you proceed with a formal complaint against the company, that may take care of your concern about dating the CEO.) ______________________________ Please address any questions about standards or evidentiary issues to Dr. Ruth at rcorbin@corbinpartners. com. If your question is published, your identity will not be revealed unless you so choose. I spent the better part of my early 20s studying at various universities and in different departments. Although my years there were engaging and thoughtprovoking, the focus was primarily on theory while little weight was placed on practical application. Despite this emphasis, the years I spent in higher education are nothing compared to the challenges I now face upon entering the world of market research. With a graduate degree under my belt, I felt relatively immune to the challenges I would face in any industry I entered. Through my education, I learned to think critically, solve problems, and communicate fluently. In fact, I thought that my first “real” job would be a piece of cake. When I entered the market research industry, it felt like a whirlwind. I was hired by a boutique firm and was quickly exposed to many aspects of the business. I was challenged not only to be detail oriented and complete specific tasks, but also to connect each element of the business and envision how all the parts fit together to make the entire company function. After almost two years in, I am still piecing it all together. Despite the many obstacles that stand before me, I always have the support of my colleagues. Academia can leave one with a lone wolf feeling, always needing to solve things individually, and the team environment was something new to me. I had to learn to trust my colleagues and their abilities to get job after job done as a team. Do I regret not pursuing a career with fewer challenges that may have been a better fit with my education? Not at all; I am learning to think quickly with a practical, client-oriented focus which is a far cry from the ivory tower of academia. Off the Deep End Julie Fortin SOM inc. (Québec) Brian Singh, CMRP zinc tank Ah, les répondants… On ne peut vivre sans eux, mais il faut certainement apprendre à vivre avec eux! Bien sûr, il y a le répondant qui collabore de son plein gré, celui qu’on espère à chaque appel. Dans cette catégorie, il y a le convaincu, celui qui accepte avec joie (oui, oui, ça existe!) de répondre à une « étude » (pas un sondage, non, c’est péjoratif!). Il y a aussi celui qui se cherche un tas d’excuses pour ne pas participer (ex. : je dois sortir les poubelles/le chien/ la belle-mère ou encore « Ma téléréalité commence à l’instant »), mais qui finit par accepter, à court d’arguments. Il y a le « bavard », qui ne peut s’empêcher de commenter chacune de ses réponses, voire les questions ellesmêmes, faisant ainsi exploser la durée de l’entrevue (tout en se plaignant qu’elle soit trop longue!). Il y a l’éternel indécis, celui qui n’a jamais assez d’information pour offrir une réponse tranchée. Il y a le « pressé », qui accepte de faire une entrevue de 12 minutes… à condition qu’elle ne dure que 5 minutes. À force d’interrompre l’intervieweur, elle durera au final 15 minutes, top chrono! Il y a le « relayeur », soit celui qui consulte systématiquement son conjoint(e) avant d’émettre une opinion. Plus triste, il y a celui pour qui un sondage téléphonique constitue un antidote à l’ennui, ou encore la seule façon d’exprimer son point de vue. Et il y a les autres, les « refus ». Certains refusent poliment, fermement, courtoisement, mais d’autres le font grossièrement, cavalièrement voire brutalement. D’aucuns nous accusent même de harcèlement et, menaçants, invoquent la sacrosainte « do-notcall-list »… qui ne s’applique pas aux sondages d’opinion! Quoi qu’il en soit, je lève mon chapeau à tous les intervieweurs et les intervieweuses; c’est un métier beaucoup plus difficile qu’il n’y paraît. Can we stop smoking our own crack? Like many across the world, I have observed the ongoing saga of Toronto’s Mayor Rob Ford – from when the news of the crack video broke, to the present allegations of drinking and driving, communicating with criminals, and using other illicit drugs. But the signs of a problem were there long in advance – namely drunken appearances in public. Amidst allegations and confessions, his ongoing behaviour indicates denial and stubbornness. There has clearly been a problem for a long time, but there’s also been little will, or drive, to correct poor and ill-advised behaviour. Oddly enough Mayor Ford’s behaviour is analogous to some of our own in the marketing research industry. For many years, there were signs of emerging data sets and new players entering the industry – players that had no connection to our business and were quickly cutting into our territory. We felt that the realm of insight was firmly in our grasp and our methodologies were safe. We too had an addiction to our past behaviours, and denied the mounting evidence of the looming threats delivered via client DIY research and evolving technology platforms that provide insight more accurately, cheaply and timely than was traditionally available. Like Rob Ford, we may be in the same proverbial hole – in a place that is hard to admit where we are and without a clear map to safety. This is a real challenge for an industry that was supposed to have all the answers. On the flip side, to steal a thought from Ford: Are we asking ourselves the right questions? One key question is “What is our appetite for risk?” Risk is something that contravenes the mindset of our industry – our existence is predicated on our ability to mitigate risk and, hence, we have become risk-averse ourselves. Illicit drugs aside, risk is a good thing – it is times like this when the risk of our survival depends upon our revival, evolution and ability to constantly innovate. We need to adjust our worldview to this reality. While many in the industry have begun innovating and incorporating updated methodologies aligned with evolving client demand, many of us are still smoking the same crack. We tend to seek solutions from within our industry – and from some of the same folks who got us here in the first place. Admittedly, some are masters of re-invention, but there are many who evolved kicking and screaming. Change is hard when you are the master of your domain. Thus, in trying to address our ongoing problems I have noted that researchers are listening to innovation from each other’s own resurgence experiences, and rarely stepping outside of our industry to listen to the level of disintermediation that is going on in the insight arena. Bizarrely, this is analogous to Mayor Ford listening to his brother and his close circle of advisors. It does appear to be untimely advice. So how do we grapple with our own form of addiction? One that has served us well from door-to-door, to mail, telephone through to online? Go to where the innovation is happening. Spend time with the tech community, which is using new forms of data analysis and collection and remixing methodologies. Go to a mobile meet up — they are monthly in a city where you live (or nearby). Go to different conferences. Go beyond what we do within the marketing research arena, go to the fringes of your interests and see how people are doing things differently. Confront the traditional and expand your horizons. You’d be surprised how welcome our craft is among insight innovators who had little knowledge of our industry. vue | JANUARY/FEBRUARY 2014 COLUMNISTS La Belle Vue 35 MRIA INSTITUTE OF PROFESSIONAL DEVELOPMENT VISIT MRIA-ARIM.CA/EDUCATIONMRIA FOR A FULL COURSE LISTING AND A DETAILED Institute for DESCRIPTION OF EACH COURSE. Professional Development Please go online to mria-arim.ca/portal to register. GET $100 OFF the regular price by registering before the early bird (approximately 4 weeks before the course starts). Registration closes approximately 1 week prior to course start. Institut de développement professionnel de l’ARIM Looking BEYOND basic knowledge requirements? MRIA offers the following specialized or advanced areas of marketing research courses taught in-class. Categorical Data Analysis March 18, 2014 MRIA Institute for Conjoint Analysis: Design, Analysis and Reporting that LeadsInstitute to BetterforMarketing Strategy Professional Development développement Create Winning Research Presentations and Reports thatInstitut Deeplyde Connect with your professionnel Audience Professional Development April 22, 2014 May 13, 2014 Measuring Customer Satisfaction, Loyalty and Retention* February 20, 2014 Measuring Customer Satisfaction: Advanced April 17, 2014 Metrics Madness: Is Your Client’s Digital Marketing Working? April 17, 2014 Semiotics, Introduction: How Symbols, Packaging & Advertising Communicate April 8, 2014 SPSS: Analyzing Survey Data: An Introductory Workshop April 23, 2014 SPSS: Analyzing Survey Data: An Advanced Workshop April 24-25, 2014 http://mria-arim.ca/education/in-class-learning/professional-development-courses *Ces cours aussi disponibles en français http://mria-arim.ca/fr/formation Courses covering the core knowledge requirements of the Certified Marketing Research Professional (CMRP) are as follows: 11 of these 12 core courses are available anytime online Course Next in-class session Register by 401-Online Research, Best Practices and Innovations 204-Qualitative Marketing Research 403-Advanced Qualitative Marketing Research 302-Market Intelligence 301-Competitive Intelligence, Mystery Shopping, and Benchmarking 201-Marketing Research Design: An Applied Course 101-Introduction to Marketing Research 102-Ethical Issues and Privacy in Marketing Research 202-Questionnaire Design 203-Marketing Research Statistics & Data Analysis* 303-Marketing Management for Researchers 402-Advanced Analysis Techniques February 26, 2014 March 4, 2014 March 5, 2014 April 3, 2014 April 29, 2014 September 30, 2014 November 4, 2014 Fall 2014 Fall 2014 - Spring 2015 Fall 2014 - Spring 2015 Fall 2014 - Spring 2015 Fall 2014 - Spring 2015 January 29, 2014 February 7, 2014 February 5, 2014 March 6, 2014 April 1, 2014 September 2, 2014 October 2014 online available now online available now online available now online available now Fall 2014 - Spring 2015 http://mria-arim.ca/education/online-learning/online-learning-faq Marketing Research and Intelligence Association l’Association de la recherche et de l’intelligence marketing
© Copyright 2024