1. science
2. mathematics
3. statistics

vue
Canadian Publications Mail Agreement #40033932
New and
Improved!
New columns!
New features!
WELCOME ON BOARD
FLIGHT 196, DESTINATION
SAMPLE SIZE: This
Is Your Captain
Speaking
The Seven Habits
of Highly-Stressed Survey
Researchers
Questionnaire Design
Levels of Measurement and
Statistical Method Choices
Master Crafting
Qualitative Basics
FACE OFF
MARKET RESEARCH:
Essential or Pointless?
the magazine of the
Marketing Research
and Intelligence
Association
JANUARY/FEBRUARY 2014
vue
JANUARY/FEBRUARY 2014
VUE MAGAZINE IS PUBLISHED BY THE
MARKETING RESEARCH AND INTELLIGENCE
ASSOCIATION TEN TIMES A YEAR
SPECIAL FEATURE
10
Welcome On Board Flight 196,
Destination Sample Size:
This Is Your Captain Speaking
Kathryn Korostoff
FEATURES
14 The Seven Habits of Highly-Stressed Survey Researchers
Jeffrey Henning
18 Questionnaire Design
Levels of Measurement and Statistical Method Choices
David S. Dobson
20 Master Crafting Qualitative Basics
Kendall Nash
24 FACE OFF – Market Research: Essential or Pointless?
Katie Clark and Mike MacLeod
COMMENTARY
4 Editor’s Vue
6 Letter from the President
8 Message from the Interim Executive Director
INDUSTRY NEWS
26 MRIA Designations
27 Qualitative Research Registry (QRR)
28 Research Registration System (RRS)
29 People and Companies in the News
32 Chapter Chat
COLUMNISTS
34 Ask Dr. Ruth
34 Bright-Eyed
35 La Belle Vue
35 Off the Deep End
ADDRESS
The Marketing Research and Intelligence Association
L’association de la recherche et de l’intelligence marketing
94 Cumberland Street, Suite 601
Toronto, ON M5R 1A3
Tel: (416) 642-9793
Toll Free: 1-888-602-MRIA (6742)
Fax: (416) 644-9793
Email: [email protected]
Website: www.mria-arim.ca
PRODUCTION: LAYOUT/DESIGN
LS Graphics
Tel: (905) 743-0402,
Toll Free: 1-800-400-8253
Fax: (905) 728-3931
Email: [email protected]
CONTACTS
CHAIR OF PUBLICATIONS, EDITOR-IN-CHIEF
Annie Pettit PhD, Chief Research Officer, Conversition
(416) 273-9395
[email protected]
MANAGING EDITOR
Anne Marie Gabriel, MRIA
[email protected]
ASSOCIATE EDITOR
Fiona Isaacson
[email protected]
Interested in joining the Vue editorial team?
Contact us at [email protected]
2014 ADVERTISING RATES
Frequent advertisers receive discounts. Details can be
found by going to: www.mria-arim.ca/advertising/vue.asp
Please email [email protected] to book your ad.
The deadline for notice of advertising is the first of
the previous month.
All advertising material must be at the MRIA office
on the 5th of the month.
Original articles and Letters to the Editor are welcome. Materials will be
reviewed by the Vue Editorial Team. If accepted for publication, they may
be edited for length or clarity and placed in the electronic archives on
the MRIA website.
The opinions and conclusions expressed in Vue are those of the authors
and are not necessarily endorsed by the
Marketing Research and Intelligence Association.
Publishing Date: January/February © 2014.
All rights reserved. Copyright rests with the Marketing Research and
Intelligence Association or the author.
All rights reserved.
No part of this publication may be reproduced, stored in a
retrieval system or transmitted in any form or by any means, electronic,
mechanical, photocopying, recording or
otherwise without the prior written permission of the
Marketing Research and Intelligence Association or the author.
All requests for permission for reproduction must be submitted
to MRIA at [email protected].
RETURN UNDELIVERABLE CANADIAN ADDRESSES TO
The Marketing Research and Intelligence Association
L’Association de la recherche et de l’intelligence marketing
2600 Skymark Avenue, Bldg 4, Unit 104,
Mississauga, Ontario L4W 5B2
Canadian Publications Mail Agreement #40033932
ISSN 1488-7320
COMM E NTARY / CO MME NTAIR E
Ed i t o r’s Vue
Annie Pettit
I
feel like a kid in a candy store! Everything in this Vue is
new, bright and shiny.
First, I’m delighted to welcome Fiona Isaacson as our
new Associate Editor and Chapter Chat curator.
I’d also like to welcome six new columnists for 2014. Brian
Singh will rant and rave, and get your blood boiling. Ric
Hobbs will jab you in the funny bone with the droll business
that is marketing research. Dylan Cody will remind you
what it’s like to be bright-eyed and bushy-tailed, fresh out of
school. Susan Abbott will reinvigorate the quallie inside every
one of us. Julie Fortin va partager des paroles de sagesse de
la belle province de Québec. And lastly, Ruth Corbin (“Dr.
Ruth”) will be our wise sage, though she claims she will only
answer legal questions…
What else is new? This issue marks the first appearance of
“Face Off,” a fierce debate between two researchers who may
not agree with the point of view they’ve been assigned to
argue.
Every issue will also introduce you to a researcher you
may have never met before. Find out how they came to be in
research and then feel free to share your stories with them via
email.
And, look for a research cartoon in every issue. We’re all
passionate about research, so why not have a little fun along
the way?
I hope you love this issue, and the coming year of Vue. I
invite you to write a letter to the editor and tell us what you
like and don’t like, or share your point of view on an article.
Or, create your own cartoon and send it to us. Maybe you’ll
show up in our new “Letters to the Editor” page debuting in
the next issue.
Cheers to all things new!
J
e suis comme une enfant dans une confiserie : tout
dans cette livraison de Vue me semble aussi intéressant
qu’irrésistible.
Je souhaite d’abord souhaiter la bienvenue à Fiona Isaacson,
notre nouvelle rédactrice adjointe et la modératrice du
clavardage régional.
Nous accueillons avec ce numéro six nouveaux chroniqueurs
et chroniqueuses pour l’année 2014 : Brian Singh, qui ne
manquera probablement pas d’en « énerver » certains; Ric
Hobbs, un chercheur marketing doublé d’un humoriste;
Dylan Cody, un jeune fraîchement diplômé qui nous offrira la
perspective d’un débutant dans le secteur; Susan Abbott, qui se
fera l’apôtre de l’analyse qualitative; Julie Fortin, qui nous
fera profiter de sa sagesse; et l’avocate Ruth Corbin, notre
« Dr. Ruth » à nous, qui rappelle au passage qu’elle ne traitera
que de sujets juridiques.
D’autres nouveautés? Bien sûr ! Le lancement de “Face
Off ”, par exemple, une page qui propose un débat entre deux
chercheurs sur un sujet d’intérêt au secteur. Chaque numéro
vous fera également connaître un chercheur qui vous est
probablement inconnu. Vous êtes invité à découvrir comment
il ou elle en est arrivé(e) à devenir chercheur et de partager avec
lui (ou elle), via courriel, vos propres anecdotes.
Enfin, et en prime, une caricature originale sur la recherche
vous attendra dans chaque nouveau numéro. Qui a dit que les
chercheurs marketing n’avaient pas le sens de l’humour?
J’espère que vous apprécierez cette livraison de Vue et tous
les numéros qui suivront en 2014. Écrivez-nous et dites-nous
ce qui vous a plu et ce que vous avez moins aimé. Laissez-nous
savoir ce que vous pensez et exprimez votre opinion au sujet
de nos articles et chroniques. Vous pouvez aussi nous expédier
une caricature de votre cru. Qui sait? Peut-être reconnaîtrezvous votre contribution dans le prochain numéro de Vue et sa
nouvelle section, « Lettres à la rédaction ».
Vive la nouveauté !
Annie Pettit PhD, Chief Research Officer / Directrice de la recherche, Peanut Labs
Editor-in-Chief, Vue / Rédactrice en chef, Vue • Email: [email protected] • (416) 273-9395 • t @LoveStats
Please share your opinions about Vue articles and columns, or submit your cartoons and infographics to the Editor.
La rédactrice vous invite à lui faire parvenir directement vos commentaires, opinions, caricatures ou infographies.
4
vue | JANUARY/FEBRUARY 2014
vue | JANUARY/FEBRUARY 2014
5
COMM E NTARY / COMME NTAIR E
Letter from the President
Anastasia Arabia
Happy New Year!
I am very excited about the opening of our new
MRIA office in downtown Toronto. Most MRIA
members, even board members, never visited our
office in Mississauga. There are many practical
reasons why being downtown is the right decision
for the MRIA, but my favourite is that the new
office will be more user-friendly and accessible to
our members. At our new location, we will be able
to hold board meetings, courses, CMRP exams,
seminars, and so on, and will no longer have to
rent expensive hotel space. Also, out-of-town
MRIA members can drop in, have a coffee, use
the WiFi, or make a call. This will be especially
useful for out-of-town members, like me, who
have a few hours before a meeting, or event, and
would relish some space to get some work done.
The MRIA staff is also very excited about being
more closely connected and getting some face time
with members, all made possible by our opendoor philosophy. It’s nice to put faces to names
and actually shake hands with people you have
communicated with for years by email and phone.
Look for news of our grand opening in the coming
weeks.
On a very personal note, I would like to pay a
special tribute to Hank Goertzen, CMRP, who
passed away suddenly in December. Hank was an
exceptional researcher and mentor, and a wise and
well-respected senior researcher in Alberta. He will
be dearly missed.
Bonne année !
Le déménagement du siège social de l’ARIM au
centre-ville de Toronto me gonfle d’enthousiasme.
En effet, peu de membres de l’ARIM, même
membres du conseil, venaient nous rendre
visite au bureau de Mississauga. Plus encore
que les nombreuses raisons pratiques pour
ce déménagement, ce sont la convivialité et
l’accessibilité du nouveau siège social à nos
membres qui motivent mon enthousiasme. Nous
pourrons ainsi accueillir « chez nous » les réunions
du conseil, les examens du programme PARM,
des séminaires et autres événements, tout en
économisant sur la location, onéreuse, de salles
d’hôtels. Nous invitons les membres de l’extérieur
de Toronto – j’en suis – de nous y rendre visite
quand ils seront de passage et de se prévaloir des
téléphones, connexion wifi et café qui y seront
disponibles. Des espaces de travail ont également
été prévus pour ceux qui ont quelques heures
à écouler avant une réunion. Les employés de
l’ARIM ont d’ailleurs bien hâte de rencontrer tous
les membres qui profiteront de notre politique
« porte ouverte » pour venir leur serrer la pince.
Il est toujours plaisant d’associer enfin des visages
aux voix que nous avons entendues au téléphone
et aux courriels qui nous ont été expédiés. Plus
de nouvelles au sujet de l’ouverture officielle du
bureau suivront dans les semaines qui viennent.
Enfin, je ne peux passer sous silence le décès
de Hank Goertzen PARM, qui nous a quitté
soudainement au mois de décembre. Hank était
un chercheur hors pair et un mentor exceptionnel,
un homme sage et respecté, tout particulièrement
en Alberta, sa province de résidence. Tu vas nous
manquer, Hank.
Anastasia Arabia, Partner / Partenaire, Trend Research Inc.
President, Marketing Research and Intelligence Association / Présidente, L’Association de la recherche et de l’intelligence marketing
Email: [email protected] • 780-485-6558 ext./poste 2003
6
vue | JANUARY/FEBRUARY 2014
MRIA 2014
CONFERENCE
SHERATON CAVALIER AND DELTA BESSBOROUGH
SASKATOON | SASKATCHEWAN
The MRIA 2014 National Conference is proud to announce that we have secured Jeffery Hayzlett as a keynote
speaker. Jeffrey is a global business celebrity and host of “C-Suite with Jeffrey Hayzlett” on Bloomberg Television.
He also appeared as a guest judge on NBC’s Celebrity Apprentice with Donald Trump for three seasons. Special
thanks to Itracks in assisting to secure Jeffery as a speaker.
The national conference “Dig Deeper and Discover” will be held from June 8 – 10, 2014 in Saskatoon, Saskatchewan,
Canada’s fastest-growing metropolis.
Join us...as a Delegate, Sponsor, or Exhibitor.
Follow us on Facebook, LinkedIn and the conference website (http://conference2014.mria-arim.ca/news/index.php).
PINNACLE SPONSOR
PLATINUM SPONSORS
Insightrix_logo.pdf 1 11/13/2013 1:03:57 PM
C
M
Y
CM
MY
CY
CMY
K
GOLD SPONSOR
BRONZE SPONSOR
EXHIBITORS
OTHER SPONSOR
COMM E NTARY / COMME NTAIR E
Message from the Interim Executive Director
John Ball, CMRP
8
Happy New Year to all! There can be no clearer
indication of a really new year than one that
starts at the MRIA with a message of great
appreciation and recognition of all members who
have graciously shared their time and insights
through the online qualitative research we are
conducting to help us better understand member
needs, and attitudes towards the MRIA and its
Chapters. Those who have been able to join in
the conversation about the association have shared
their views candidly, and provided solid ideas to
improve the association moving forward.
Une bonne année à tous ! J’en profite pour
remercier tous les membres qui, au cours de la
dernière année, ont donné de leur temps et partagé
avec nous leurs idées dans le cadre de la recherche
qualititative que nous avons menée auprès des
membres pour mieux cerner leurs besoins et
leur attitudes vis-à-vis l’ARIM et ses sections
régionales. Ceux qui ont participé au dialogue au
sujet de l’association ont parlé franchement et ils
ont avancé de solides idées quant aux moyens à
prendre pour l’améliorer.
We are hearing about how to
improve our advocacy for the
industry and its practice amongst
the general public, improve our
relations with our corporate
members who are tasked with
adhering to our highest standards,
and how to improve our
enforcement practices to make
sure that any deficiencies
are kept in check.
Relevons parmi ces suggestions des
façons de rehausser nos pratiques
avec le grand public et notre
représentation du secteur. Il nous
a aussi été dit que nous devions
améliorer les rapports avec les
sociétés membres qui doivent se
conformer à nos normes, très strictes,
de même que l’application des
règlements en cas de contravention
aux normes.
We also know how to take another look at
our educational platform with fresh eyes, and to
consider improving access to effective training and
build in new content (new technology, client-side
specific) regardless of whether it comes from us
or somewhere else. You have even been able to
help steer the development of our approach to
organizing our upcoming National Conference in
June of this new year we are entering. We know
what issues are important to you, and what will
make this conference a good value for you, and
also how to augment this conference with new
Les participants ont aussi demandé que nous
passions en revue notre programme de formation,
que nous en facilitions l’accès et que nous
proposions de nouveaux modules, quelle que soit
leur source, notamment à l’égard des nouvelles
technologies et du côté client. Les membres nous
ont également aidé à préparer le congrès national
qui aura lieu au mois de juin et nous ont fait part
des sujets qu’ils souhaitent aborder. Ils ont aussi
proposé de recourir à des webinaires pour élargir
l’auditoire du congrès. Les participants ont aussi
souligné l’importance du lobbying auprès des
vue | JANUARY/FEBRUARY 2014
commentar y
commentaire
events and webinars to increase our connections to
our members. We’ve also heard how important you
feel it is that we continue to lobby for the industry
with various levels of government to build the
profile of the marketing research profession. We
also understand the value of how to bring new life
to the MRIA and our industry, by increasing the
number of student members in our association.
While we have been moving forward on all of
these fronts all along, we understand better now
how and where to turn our attention for 2014.
We are more aware than ever of the challenges
of connecting and creating meaning for those
in our various Chapters across the country, each
with their own unique challenges. Our Chapters
represent the face of the MRIA for many of you,
and we are using this research to assist our Chapter
leadership to stay the course or expand our local
offerings based on the feedback you have provided.
This past year has been one of considerable
change and we can only expect this to continue
through 2014 with a keener sense of what is
working and not working from the perspective of
our members and where we should set our sights.
Thank you for the opportunity your feedback
has provided to enable us to make positive
change a priority. I am remembering fondly when
we initially reached out to the public during
our Voxpop campaign in the early days of the
association with the following promise, “You talk,
we listen, things improve.” I thank you for allowing
us the opportunity to walk this talk with you.
autorités gouvernementales et de l’affirmation
publique de notre profession, la recherche
marketing. Nous avons aussi été priés d’apporter
un nouveau souffle à l’ARIM et au secteur en
recrutant davantage de membres étudiants.
Bien que nous faisions déjà des progrès sur
tous ces fronts, nous savons maintenant vers
quoi exactement tourner nos efforts en 2014 et
comment procéder. Nous savons ainsi que nous
devons établir des rapports plus étroits et plus
productifs avec les sections régionales, qui font
face, chacune d’elles, à des défis qui leur sont
particuliers. Par ailleurs, nombre d’entre vous
ne connaissent l’ARIM que par leur section
régionale. Nous travaillerons donc sans relâche
pour maintenir le cap sur le rehaussement et pour
proposer des initiatives régionales, fondées sur
votre rétroaction.
Plusieurs choses ont bien changé en 2013. Il en
sera sans doute de même en 2014, maintenant que
nous savons mieux ce que les membres apprécient,
ce qu’ils aiment moins, et quelles orientations ils
souhaitent que nous adoptions. Merci à tous de
votre rétroaction, qui continue de nous aider à
apporter les changements qui s’imposent. Tout
ça me rappelle le slogan-promesse que nous
avions utilisé, peu après la création de l’ARIM,
dans le cadre de la campagne Voxpop, qui visait
le grand public : Vous parlez, nous écoutons, les
choses s’améliorent. Il est tout aussi pertinent
aujourd’hui. Merci de nous avoir donné cette
occasion de « faire marcher nos bottines plutôt que
nos babines ».
John Ball, CMRP, Interim Executive Director /Directeur général intérimaire
Marketing Research and Intelligence Association / L’Association de la recherche et de l’intelligence marketing
Email: [email protected] • (905) 602-6854 ext./poste 8724
vue | JANUARY/FEBRUARY 2014
9
Welcome On Board Flight 196,
Destination Sample Size:
This Is Your Captain Speaking
Would any of us ever board a plane if we didn’t
feel assured that the people commanding that
metal capsule hurtling us through the air at
500 mph know what they’re doing?
Kathryn Korostoff
O
Of course, we’re also aware of something called “autopilot.”
We all accept the fact that our highly qualified pilot may be
judiciously using the autopilot function. But we like to think
that it’s not being blindly trusted – that, should the autopilot
fail, our human pilot will be perfectly capable of landing the
old-fashioned way and in the right place. Imagine if you asked
the pilot for flight information and were told, “I don’t know.
The autopilot works fine, though.” You’d want off that plane!
As marketing researchers, we are the pilots of our survey’s
journey. It’s up to us to plot the journey and navigate the path,
to reach the right destination, not whisk clients away on an
unintended trip to Peoria. Yes, we have tools that automate key
tasks – but we still need to know how to do the work ourselves.
Why, you ask? First, so we can catch the odd cases of
autopilot failure. Second, so that when our clients ask us for an
explanation, our answer is a tad more credible than, “Because
the software says so.”
Before Takeoff: Calculating Sample Sizes
Here’s a simple one-question quiz. What do you think the
correct answer is?
10 vue | JANUARY/FEBRUARY 2014
Figure 1: A Simple Quiz
Most people select “B,” the size of the target population.
hich of the following is used
W
when calculating the sample size
needed to ensure a specific survey
project has reliable data (using a
formula, not a “calculator”)?
A. E xpected response rate of target
population
B. Size of target population
C. Variability of target population
D. All of the above
While sample size calculators use that, the correct answer is
“C,” the population’s variability.
The reason sample size calculators use population size
is because it is a proxy for variability. It’s an assumption.
And in some markets, a big one. There is only one thing
wrong with that: it creates the illusion that all markets are
the same. Whether you are researching attitudes about cars,
beauty products or high-end computers – if your desired
level of reliability is the same in all cases, you get the same
recommended sample size from the calculators.
That really doesn’t make sense when you think about it.
Some target customer groups have more variability than others.
For example, in my experience:
SP ECIA L FEAT URE
• I n a survey on home electronics brands, variability in brand
perceptions may be fairly low. That is, most people have
common perceptions about what are the “premium” brands,
what brands are “low cost-low frills,” what brands are
“reliable,” etc. Variability is fairly low.
1.
2.
3.
4.
• I n a survey on retail shopping behaviours, variability may be
comparatively high. Some people are luxury shoppers, some
are discount seekers. Some shop as a social experience, some
are all about efficiency. That’s a lot of variability.
How reliable does my data need to be? This was tackled in
our formula – when we decided (in our example) that we want
to be within +/- one minute of the average. In the real world,
we may adjust that up or down to fit our goals and budget.
Both of these examples are studies with general consumers.
But we would likely see a lot more variability in the second
study, than the first – and that means we would need a larger
sample size too.
Recall the old adage: If everyone had the same opinion, we’d
only need a sample size of one.
The converse is also true: The more variability in opinions,
the larger the sample size we need to make sure our data is
representative of the target population.
So what is the formula? There are variations, but one I often
use is:
Where Z is typically 1.96, for a 95 per cent confidence
interval, H is the desired precision level. And Sigma is the…
you guessed it…variability (standard deviation).
So let’s work through an example: Imagine you are doing a
study on online shopping. You are asking people to self-report
how much time they spend price comparing online, before they
make a purchase decision. Assume you know from past studies
that you expect the variability in answers to be eight minutes (that
is, that the standard deviation based on past reported data is eight
minutes). Further, assume that you want your new study to be
precise within one minute. That is, you want to be confident that
whatever result you get for the average time spent price comparing
online is accurate in a +/- one minute range. Here’s the math:
Your sample size in this scenario is 246 (rounded up, since
we do not want to survey a fraction of a person – that’s a little
marketing research humour…ha ha).
In reality, if I am studying a market or topic for the first
time, I may have no estimate of the standard deviation. In
those cases, I usually make a conservative estimate, and then
revise after a pre-test. And yes, I do pre-tests.
Pre-Flight Checklist
So if there is a formula, aren’t we done? Not necessarily. The
formula gives us a starting place, the “N” we need to be
reasonably confident in our data. But other considerations may
cause us to increase our sample size beyond that point. There
are five key questions you need to ask yourself before “takeoff ”
– sort of a pre-flight checklist for sample size planning:
How reliable does my data need to be?
What type of data analysis am I planning?
Is there a “size of universe” constraint?
What is my sample source?
What type of data analysis am I planning? For some types of
data analysis, we simply need more data. Some researchers who
do a lot of conjoint studies like to have at least 100 responses
per segment (for example, if they are doing a study and will
want to compare results by six geographies, they will want at
least n=600).
For many projects, sample size needs can be exploded just
by crosstabs. I may be planning a study where I will need to
analyze data by several different types of banners: four ranges
of age, five of income level, two for gender, and maybe four for
education level. So even if my calculated sample size is 400,
I may decide I want more than that to have some breathing
room to have enough data to compare my subgroups. Some
researchers prefer at least 30 per sub segment, others prefer 50.
Is there a “size of universe” constraint? I may have calculated
my needed sample size as 300, but what if I am doing research
with a really hard to find population? Well, that raises a
complex issue. Such populations can be very contrived –
and may not have the attribute of actually being a random
population or one with a normal distribution. In such cases, I
have a professional obligation to explain this to my clients and
advise accordingly.
What is my sample source? I need to know what my sample
source is. What is my sampling frame? Is it large enough to
support my sample size goals? Or, is there a risk factor here that
I need to manage?
Mid-Flight Turbulence: Two Sticky Points
Being clear about sample size calculations will help you in two
common, and sometimes uncomfortable, situations.
First, the research cynic. At some point in your career, you
will have this experience:
ou are presenting the final results to a group. Some of your
Y
results may be a little controversial, or perhaps just a tad
unexpected. Someone will raise their hand and challenge your
data. It is easier for them to challenge your data than to accept
that their worldview may be flawed. In these cases, they will
either question your sample source or question your sample
size. You need to be able to defend both. Objectively.
Then there are people who have a magic sample size number
in their heads. I have had several clients in my career who
assume 1,000 is “a good number.” Hey, it’s a lovely number
– and I have done plenty of studies with 1,000 or more
vue | JANUARY/FEBRUARY 2014
11
SPEC IA L F EAT U R E
completes. But in some cases it is really overkill, and I hate to
waste my client’s money.
So how do we navigate through these winds of opinion?
With facts. In both cases, I find it best to educate my client a
little: let them know that the sample size was based on a specific
method and explain the basics of how and why. Teaching
them, even just a little, makes them much more comfortable
not only with my sample size recommendation – but with my
dedication to serving their best interests.
Know Your Flight Plan
Specify your sample size goal before you actually start data
collection. This needs to be done early on for three reasons:
• T
o get your sample source lined up, make sure you have
access to enough qualified people to achieve your sample
size (given some assumptions about response rate and list
quality). This might seem painfully obvious, but poor
planning here can really derail a project; as a marketing
researcher, you risk losing credibility with clients if your
schedule slips a week because your sampling plan was poorly
vetted.
• T
o set realistic expectations with your clients. You don’t want
them making sample size assumptions, since sample size can
have a huge impact on budget.
• T
o put correct quota controls in place in your questionnaire.
For example, if you are paying incentives, you want to make
sure you stop collecting data when you hit your sample size
goal.
Calculating your sample size yourself, without a calculator,
gives you the discipline of really thinking about your project.
Should your “passengers” wonder where they are at any given
moment, you’ll be able to tell them and explain why.
Happy Landings: Safe on Solid Ground
The success of most marketing research projects is judged
on a few criteria. When I teach Market Research Project
Management, I say that the success of a project is often judged
on seven common criteria:
12 vue | JANUARY/FEBRUARY 2014
1.
2.
3.
4.
5.
6.
7.
Project completed on time
Project completed on budget
Project results clearly align with defined project objectives
Sample size and quota distributions met
Reporting/deliverables meet professional standards
Ethics standards adhered to
Research results are deemed actionable by clients
While item no. 4 is clearly sample size related, clear and correct
sample size planning can also impact items 1, 2, 3 and 7 as
well. Doing these calculations “the old-fashioned way” will
deepen your understanding of your project, prevent autopilot
failure, and protect you against mid-flight turbulence and the
occasional grouchy passenger.
Sample size: THE SHORT ANSWER
For some consumer studies, directional data of 100 responses
can be used for quick hits. We label such results “directional,”
and sometimes that is fine when time and budget dictate.
However, if you want data that can be used to represent a
population, then a larger sample size is preferred. In consumer
marketing research there are common levels of variability
that we’re likely to encounter. Based on this, many marketing
researchers will use rules of thumb: a sample size of 400 to
500 completes will often give us solid data. But add in some
extremes in population variability or sub-group analysis, and
you may need to go higher.
Knowing the rules is key. So is knowing the logic that informs
them.
Kathryn Korostoff is the president of Research Rockstar LLC, the
only independent company dedicated to market research training.
Over the past 20 years, Kathryn has personally directed over
600 primary research projects and published over 100 articles
in various magazines, including Quirk’s Marketing Research
Review and Alert! magazine. She has been a featured speaker
at American Marketing Association, ESOMAR, and Marketing
Research Association events. She can be reached at KKorostoff@
ResearchRockstar.com or on Twitter via @ResearchRocks.
ANNOUNCING OUR KEYNOTES!
Gail Livermore, TARGET
Merrill Dubrow, M/A/R/C® Research
Naomi Henderson, Riva
The strength of qualitative research lies in its ability to make deep connections with people –
connections which enrich our understanding of their desires, values and opinions.
PLAN NOW for 2014! Registration Open!
Sponsorship Opportunities Available!
Visit our website and enter the Contest http://qrdconference2014.mria-arim.ca/news/index.php
Platinum Sponsor
Gold Sponsors
Reliable, Actionable Consumer Insights
by Design.TM
Silver Sponsors
Bronze Sponsors
Marketing Sponsor
Heintzman Research Limited
Survey Sampling International - SSI
vue | JANUARY/FEBRUARY 2014
13
FEAT UR E
The Seven Habits
of Highly-Stressed Survey Researchers
Jeffrey Henning
Pretty busy here. Where
I used to carefully write
a questionnaire, run a
pre-test, run a pilot and
then launch – now I
have to get it all right
the first time. Where I
used to think carefully
about the results over
a few weeks, discuss
them with co-workers,
let them percolate
and come up with an
original analysis – now
I have to bang them
out and move on to the
next project.
S
Surveys have become commodities. That
means I fall back on bad habits, because I
have to get to the next survey.
Here are the bad habits I can’t escape any
more than skipping a soda at breakfast, or
watching one more late-night TV show rather
than getting eight hours of sleep, or checking
Twitter when I really should be writing that
next questionnaire.
1. Writing leading questions
First drafts of questionnaires are often full of
leading questions. You just write the question
from your viewpoint, without taking the time
to think about it in a more objective fashion.
And since a lot of first drafts are last drafts
these days, off you go.
A leading question suggests the answer
you’re looking for and often unintentionally
reflects your bias. As a result, the answers to
leading questions overstate the actual support
for the item being researched.
When you’re in a hurry, it’s hard to step
back and be objective. Leading questions can
be subtle, or obvious. An obvious leading
question, taken from an event survey, is “How
likely is it that you will attend the 2014
conference at our new, low entry prices?”
A subtler example, used in the text
Marketing Research by Alvin C. Burns and
Ronald F. Bush, is “Should people be allowed
to protect themselves from harm by using
Mace as self-defense?” The survey sponsor’s
viewpoint is clear – yes, yes, they should.
14 vue | JANUARY/FEBRUARY 2014
Or, take “How much do you think you
can save by buying online?” A neutral
question would have asked if online prices
were the same, higher or lower than offline
alternatives, and then would have asked by
how much.
Like many bad habits, this one is addressed
by being more deliberative and investing
more time. In this case, have a third party
review the questionnaire, or spend the effort
thinking about a neutral alternative to leading
questions.
2. Using the Likert scale
I was recently sent an employee satisfaction
questionnaire where the first 80 questions all
used a Likert scale (“strongly agree,” “agree,”
“neutral,” “disagree,” “strongly disagree”). I
envied the economy of effort – pretty easy to
write such a beast. I hope they were paid by
the question.
But, like leading questions, Likert scales
skew the results. Over 100 separate studies
have demonstrated acquiescence bias – the
tendency for respondents to agree, regardless
of the content of the statement. In their
landmark paper, “Comparing Questions
with Agree/Disagree Response Options to
Questions with Construct-Specific Response
Options,” Willem E. Saris, Melanie Revilla,
Jon A. Krosnick and Eric M. Shaeffer share
three reasons why respondents are so darn
agreeable:
FEAT URE
• Out of a sense of politeness
• O
ut of respect for the survey author – in effect, deferring to
“expert” judgment
• From mental exhaustion with the survey (“satisficing”)
We survey researchers hate to acknowledge it, but Likert
scales are as outdated a technique as smoking the same brand of
cigarettes as your doctor.
For instance, here are three questions from an employee
satisfaction survey:
1. I am satisfied with my overall compensation. Strongly agree,
agree, neutral, disagree, strongly disagree.
2. I am often so involved in my work that the day goes by
very quickly. Strongly agree, agree, neutral, disagree, strongly
disagree.
3. C
ommunication between senior leaders and employees
is good in my organization. Strongly agree, agree, neutral,
disagree, strongly disagree.
The better approach, which will be less mentally taxing for
respondents and produce more accurate results, is to use what
Saris et al., call “construct-specific responses”: basically, measure
the dimension of interest using an appropriate scale, which will
most likely vary from question to question. Here are the same
three questions rewritten to use the appropriate scales:
1. H
ow satisfied are you with your overall compensation? Not
at all satisfied, slightly satisfied, moderately satisfied, very
satisfied, completely satisfied.
2. H
ow frequently are you so involved in your work that the
day goes by very quickly? Never, rarely, sometimes, often,
always.
3. H
ow is communication between senior leaders and
employees in your organization? Very poor, poor, fair, good,
excellent.
3. Laying out grids
“Hey,” you probably just realized, “I can reorganize that battery
of Likert scale questions into groups of grid questions using
more appropriate scales!”
Congratulations, you just replaced that bag of potato chips
you were munching on while watching late-night TV with a
bag of pretzels. Tell yourself they’re better. Go ahead. Then lick
all the salt out of the bottom of the bag.
Survey Sampling International (SSI) has conducted
many studies on the negative effects of grids. In one of its
more recent research-on-research initiatives, presented at
the Council of American Survey Research Organizations
(CASRO) 2013 Online Research Conference, SSI discovered
that respondents completed a grid almost 20 times faster than
they completed the same material when asked as a battery of
regular questions. They took half a second per item in a grid,
but eight to 10 seconds per question. Additionally, seven per
cent of respondents fell for an inattention trap, compared to
four per cent in the question battery. And questions about
job satisfaction were more likely to correlate to willingness to
recommend their job in the battery of questions than in the
grid (.56 correlation vs. .48 correlation), indicating that the
data was higher quality. Grids simply encourage respondents to
respond too quickly.
Re-engineering a grid requires thinking of alternative
approaches, such as choose-many lists, MaxDiff scales,
drilldowns and so on. But a lazy approach, that produces
superior results, is to simply break up that grid into separate
questions.
4. Reporting averages everywhere
Finally, here’s a bad habit where you can be even lazier
following the good habit. Traditionally, if you need a summary
statistic, you report the average. The magic of the median,
compared to the average, is that it automatically corrects for
outliers. For instance, in a recent survey with 105 responses,
household income ranged from $5,000 to $205,000. The
average was $45,857. The median, however, was just $35,000.
Another example, from a study on app purchasing behaviour,
had the median amount spent on apps since the mobile device
was purchased at $15, but the average at twice that: $30.53.
Why was the average so high? One per cent of respondents
(eight individuals) spent $300 or more dollars, and two
individuals spent $600 and $970, respectively. Now, if you
want to make the average work you have to ask yourself if these
outliers are unreasonable – using the median spares you from
having to make such judgment calls.
The average and the median are summary statistics, meant to
describe a range of values. The median is more representative
of all the values than the average when data is skewed
asymmetrically. Even when data is distributed symmetrically,
the median is still accurate. If you are going to summarize
the results with just one number, the median is more
representative.
5. Discussing cross-tabulations
I was playing with the banner report of a new-to-me survey
system. The default option simply reported the cross-tabs.
You had to change a setting to report the results that were
statistically significant.
The recent research obsession with storytelling is easily sated
by just reporting the cross-tab results: “25 per cent of men
would rather watch hockey than have sex, compared to just 19
per cent of women” – there’s a statement that can provide some
nice story angles, even if the results are statistically the same
given the sample size. Again and again, I see publicists write up
survey results while ignoring statistical significance altogether.
vue | JANUARY/FEBRUARY 2014
15
FEAT UR E
6. R
eporting only sampling error
Most methodological descriptions
simply refer to the margin of sampling
error. For studies with convenience
samples, the summary might caveat it
by saying, “If this were a probability
survey, the margin of error would be...”
Similarly, if this magazine article were
printed on gold tablets, it would very
valuable.
Here’s some language you can borrow
for your next survey:
As this was not a probabilitybased sample, calculating the
theoretical margin of sampling error
is not applicable. However, as with
probability surveys, it is important
to keep in mind that the results are
estimates, and typically vary within
a narrow range around the actual
value that would be calculated by
interviewing everyone in a population.
Again, as with probability surveys, on
occasion the results from a particular
question will be completely outside a
typical interval of error.
Many types of survey error can
limit the accuracy of generalizing to a
population. Throughout the research
process, we sought to consider and
minimize total survey error, including
coverage error, non-response error,
measurement error, mode effects, and
post-survey error. We are confident
that the information gathered from this
survey can be used to make important
business decisions.
7. Spending inadequate time
on analysis
Sorry, I don’t have time to think about
these bad habits anymore. Off to the
next survey. Well, after I quickly check
Twitter.
Jeffrey Henning, PRC, is president of
Researchscape International, http://
researchscape.com, a market research
firm providing custom surveys to small
businesses. He coined the use of the #MRX
hashtag on Twitter when he should have
been writing a customer-satisfaction
survey. You can procrastinate by following
him via @jhenning.
16 vue | JANUARY/FEBRUARY 2014
vue | JANUARY/FEBRUARY 2014
17
FEAT UR E
Questionnaire
Design
Levels of Measurement and Statistical Method Choices
David S. Dobson
Picture this: You’ve
completed a marketing
research survey using
a probability sample,
and your client asks if
you can do a regression
analysis; how would you
answer? As long as you
have the relevant data,
you can perform any
type of analysis. This
article lays down some
simple guidelines on
how to plan your data
analysis, and ensure
that you are using the
appropriate levels of
measurement in your
questionnaire.
F
First, you should develop your analysis
plan, because knowing what data you need
will ensure you ask the appropriate survey
questions. Next, you should form your research
hypotheses, to help identify what statistical
tests to perform. Finally, develop a tab-plan
by specifying the banner points to answer the
research questions.
Questionnaire Design
To develop a reasonable and clear research
objective, you need to define the research
problem of the project. Ensure that your
questionnaire is designed to answer your
research objectives.
Exploratory research, such as focus groups
and literature reviews, will help define what
questions should be asked in your survey. Dawn
Iacobucci and Gilbert A. Churchill Jr., (2010)
outline the following procedures for designing a
questionnaire:
Step 1: Specify what information will be sought
Step 2: Determine type of questionnaire and
method of administration
Step 3: Determine content of individual questions
Step 4: D
etermine form of response to each
question
Step 5: Decide on question wording
Step 6: Decide on question sequence
Step 7: Determine physical characteristics of
questionnaire
Step 8: Re-examine and revise questionnaire
Step 9: Pre-test the questionnaire, revise where
needed
18 vue | JANUARY/FEBRUARY 2014
All the steps of questionnaire design are
important, but pre-testing the questionnaire is
the most important. A pre-test can determine
whether a respondent understands the
questions and is able to give unbiased responses.
Levels of Measurement
In data analysis, the characteristic of a question
is referred to as a variable. For example, in
the question “What is your gender?” the
variable is “gender.” There are four levels of
measurement of a variable, also known as scales
of measurement:
Nominal level data: These are survey responses
that provide data on a respondent’s identity.
For example, a respondent’s gender, or the
city of his/her residency, is a nominal scale
variable. These variables might have numerical
codes for classification, but the number has no
hierarchical order. Generally, questions with
nominal scale responses are easy to answer.
Respondents do not have to think too hard to
answer questions about their gender or the city
they live in.
Ordinal level data: These are survey responses
that can be ranked in order; for example, age
categories, (under 25, 25–34, 35–64, 65+) and
customer satisfaction ratings (superior, good,
average, poor). Data on such variables can be
classified by their relative rank.
Interval level data: These are survey responses
that provide numerical values which can
be ranked, with the difference between two
numbers being meaningful. An example is the
question: “Please indicate how likely you are
FEAT URE
to attend a special event, on a scale of 0 to 10, with 0 being
very unlikely, and 10 being very likely?” An interval scale has
an equal interval between each increment; for example, the
difference between “0” and “1” is the same as the difference
between “9” and “10.”
Different measurement scales can be used to measure the
same question. Knowledge of different types of measurement
scales will help you decide what scale to use for your specific
questions. Thus, the scales used in survey questions will
determine the type of data collected. Depending on the data
type, the data analysis and data interpretation will vary.
Ratio level data: Ratio level data is similar to interval level
data, but it possesses a true zero point; therefore, the ratio
between two numbers can be calculated. An example is the
question: “On average, how much time do you spend on social
media sites per day?” It is possible that some respondents may
spend four hours and some respondents may spend two hours
on such sites. Unlike 0 on an interval scale, 0 on a ratio scale
represents the absence of the value. Thus, a person spending
four hours on social media sites is spending twice as much time
as the one who spends two hours on such sites.
You must be mindful of the survey respondents. Will they
be able to handle the level of difficulty when you try to use
different measurement scales in your survey? Nominal and
ordinal scale questions tend to be easy to answer, whereas
interval and ratio scale questions are not. It is important
to remember that as the difficulty of answering questions
increases, the quality of the data decreases.
Choice of Statistical Methods
If probability sampling methods are used, the levels of
measurement are important in data analysis because the
appropriate statistical method depends on the type of data.
Nominal and ordinal scales provide categorical data, whereas
interval and ratio scales provide numerical data. The statistical
method used depends on the type of data. Listed below are
some common data analysis techniques:
Categorical Data Analysis: Categorical data can be easily
tabulated or cross-tabulated. Modes or proportions of
categorical variables can be calculated as summary statistics. For
statistical tests, the z-test can be used for the difference between
two proportions, and the chi-squared test can be used for the
difference among more than two proportions.
Numerical Data Analysis: Numerical data is the highest level
of measurement; with it, you can measure the magnitude
of a response. The mean and the standard deviation can be
computed with numerical data. For the statistical techniques,
the t-test for comparing the means, analysis of variance,
covariance, correlations and linear regressions, can be
performed with numerical data. Advanced techniques such
as the k-means clustering, factor analysis, and discriminant
analysis, can be performed with numerical data. Numerical data
can be re-coded into categories, and, as such, can be treated as
categorical variables.
It is important to remember that the probability sample
is the only sample that allows you to use statistical methods.
Statistical methods cannot be used to generalize data from a
non-probability sample. The above mentioned data techniques
will have no meaning if the respondents are not selected
randomly.
As this is an article, the information provided is brief. If you
are interested in learning more and delving further into this
topic, there are a number of good books on business statistics
and marketing research methodology; I have listed some on my
reference list.
References
Melvin Crask, Richard J. Fox and Roy G. Stout, Marketing Research:
Principles and Applications. (New Jersey: Prentice Hall, 1995).
David S. Dobson, “Significance Testing: How to compare the statistics
of two independent samples,” Imprints (PMRS), September 2003,
26–27.
David S. Dobson, “Does Sample Size Matter?” Imprints (PMRS),
January 2001, 4–8.
David S. Dobson, “The ABC of Data Analysis,” Imprints (PMRS),
June 2000, 48–53.
David F. Groebner, Patrick W. Shannon and Phillip C. Fry, Business
Statistics (9th ed.). (Pearson, 2013).
Dawn Iacobucci and Gilbert A. Churchill Jr., Marketing Research:
Methodological Foundations. (10th ed.). (Cengage Learning, 2010).
David M. Levine, Mark L. Berenson, Timothy C. Krehbiel and David
F. Stephan, Statistics for Managers using MS Excel. (6th ed.). (Pearson,
2010).
Conclusion
Douglas A. Lind, William G. Marchal and Samuel A. Wathen, Basic
Statistics for Business and Economics. (6th ed.). (McGraw-Hill, 2007)
All analysis, whether basic or advanced, should be done while
always keeping in mind your research objectives. It is important
to keep focused on this, as it is easy to get distracted if you do
not know the purpose of your analysis. Having a data analysis
plan, formulating the research hypothesis, and a tab plan will
help you get on the right track to addressing your research
objectives.
David S. Dobson is an instructor at the School of Business at the
University of the Fraser Valley in B.C. He can be reached at
[email protected].
vue | JANUARY/FEBRUARY 2014
19
FEAT UR E
Master
Crafting
Qualitative Basics
Kendall Nash
Exhilarating and Evolving! The past
five years of Qualitative Research
have been nothing if not fastpaced and ever changing. When
you encounter those spirited
qualitative practitioners and they
say it’s an exciting time to be in
qualitative research, they mean
it. Leaps in technology and blurs
in the Qual-Quant paradigm have
invited researchers into the most exciting
opportunities alongside our clients. It’s easy to
let it all carry you away into a happy land of blissful possibility.
In fact, I often now have clients asking me to conduct research
using methods that I have worked tirelessly to have others
accept over the years. I should be happy. Well, I’m glad to
have the door open if the method is, in fact, what I know will
yield optimal learning, but in other cases I receive a request for
a round peg in a square hole. Bells and whistles don’t always
mean better. Sometimes I truly need to simply sit one-onone with a consumer and unpack their story through a very
rudimentary dialogue.
The truth is that when things move quickly, the foundational
details for insightful learning from the consumer often become
fuzzy – and occasionally lost in the shuffle. So, let’s hit pause
on the lightning-speed evolution of what we do, and talk about
how we ensure that what we do is done well.
Understand the business decision at hand
Remember the game of telephone where you whispered a
phrase from friend to friend until it was hysterically diluted by
the time the last person repeated it out loud? Unfortunately, it’s
all too often the case that the objectives of a research initiative
are articulated and then passed through a series of filters that
undoubtedly add their own twist until you end up with a listed
objective that sounds about right, but is missing something
of value. Whenever possible, ask your clients (internal and
external) to tell you about their goals of the research. Nothing
replaces hearing from the mouths of your clients about what
they need and expect the research to be able to do for their
business.
And then there’s the unfortunate scenario of a team uncertain
of the real objectives themselves. The overall value and
effectiveness of the research is completely shaped by the clarity of
the objectives, so if the research goals and how the research will
20 vue | JANUARY/FEBRUARY 2014
be used are not clear to all parties, the entire team needs to spend
some time getting on the same page. Clearly understanding the
goals of the research so that you can effectively probe during an
interview truly makes or breaks the research.
Get the right people
Anyone who has done qualitative research for any amount
of time has more stories than they’d like to share about
the interviews or groups they’ve conducted in which the
respondents weren’t the “right people.” In some instances, the
respondents have been blatantly unqualified, but in other cases,
the respondent may have fit the profile, technically speaking,
but didn’t capture the essence of who it was we needed to really
learn from. Enter the recruiter. Developing strong relationships
with solid recruiters who implement best practices and recruit
with integrity is one of the most important determining factors
to setting up a project for success. You can create an amazing
discussion guide and utilize a dynamic moderator, but if the
people in the room don’t fit the bill, it’s all for nothing.
However, a good recruiter can only do so much if they don’t
have a strong screener from which to work. It’s critical to spend
adequate time designing a screener that effectively gets the
“right” respondents into the research. At times, the speed of
the project launch requires turning around a screener within a
couple of hours. I’d argue that we all need to give the screener
the attention it deserves, as a slight miswording or omission of
any given question may lead us to the wrong respondent who
can’t teach us what we desire to learn. Take the time to build a
screener that works.
Ask the right questions
Understandably, a lot of time is spent on determining the data
collection method itself. However, sufficient time is not always
taken at that same point to strategically vet how the team will
elicit feedback from the consumer. So let’s refocus, even if
only briefly, on a few techniques in which every qualitative
researcher should strive to excel. Mastery of these techniques
will ensure a qualitative consultant is able to elicit deep
learning from respondents, beyond the surface comments of
convenience and perhaps even awareness.
While skilled moderators are able to apply a multitude of
techniques at the right occasion, there are three in which all
qualitative practitioners should be well-versed: Nominal Group
Techniques, Laddering, and Projective Techniques.
FEAT URE
Nominal Group Techniques: Respondents in a group setting
are asked to independently write down their responses,
opinions, or ideas, and rationale before responses are discussed
and captured as a group. This technique helps keep respondents
accountable for their personal opinions and can reduce some
of the challenges of group influence. Effective qualitative
consultants leverage this technique when appropriate, and can
use it to gather a wider range of opinions and ideas. It helps
to establish a voice for everyone participating in the research,
despite individual personality. There is a range of variations
of this technique, and the ease of utilizing it in many online
settings has presented one strong benefit of online tools.
Laddering: This primarily refers to an in-depth, one-on-one
interviewing technique used to develop an understanding of
how consumers translate a product’s attributes with respect to
self. Means-End Chaining, (the basic questioning structure for
laddering), focuses on the linkages between the attributes that
exist in the products (the “means”), the consequences for the
consumer provided by the attributes, and the personal values
(the “ends”) that the consequences reinforce. Understanding
how attributes of a specific brand are personally relevant is
the basis for the development of messaging strategies. Now, in
English? It’s taking face-value responses and going much deeper
by digging to find the higher-level need-states that may exist.
A woman may say she prefers a spicy cracker, but what if you
could find out from her that the reason she really wants it is
because the stronger taste makes her eat less, so that she doesn’t
gain weight, so that she has a better figure, and ultimately feels
good about herself? It’s a better conversation.
Projective Techniques: This includes a wide range of activities
and lines of questioning that offer an indirect way of eliciting
learning from the respondent. Through these techniques,
the researcher is able to understand the underlying feelings,
needs, and opinions, by offering respondents a way to express
things that are either uncomfortable to say, or may be less
conscious thoughts or decisions. It helps to put the respondents
in the moment, or in a situation through which stories and
examples can provide context as to why and how someone felt
the way they did. These tools allow the respondent to more
easily articulate something that they are more familiar with,
compared to something very abstract, or nebulous things like
a brand, which has many layers or levels. Examples of more
commonly used projective tools include: personification
exercises, situational drawings, storytelling, scripting activities
and collaging.
There are a host of other approaches for which a qualitative
practitioner should be familiar, but the techniques covered here
are widely applicable across a range of industries and have the
potential to deliver powerful learning to the research team.
But know that sometimes the right answer is even simpler
than all these specialized tools – ask a question and truly listen
to the answer.
Get creative
There is no guarantee in qualitative research except to expect
the unexpected. Qualitative projects are completely customized
initiatives. A meaningful and fruitful exercise in one group may
fall flat in another. We may be testing stimulus in the form of
wireframes or prototypes with some groups, and asking other
respondents to try on underwear. The craziness is what keeps
a lot of us in love with what we do, but it’s not for the faint
of heart. Be prepared to get creative – come up with creative
solutions to strange problems, be willing to try new techniques
at a moment’s notice, and be creative in how you approach the
process, as well as the ways you ask the client team members to
contribute to the learning.
Deliver the results in a meaningful way
Now that we have thoughtfully planned and executed solid
research, we must turn our attention to delivering it to
our clients in a way that has an impact on their business.
Consider form and function when it comes to reporting on
qualitative insights. It’s not enough to merely tell them what
you heard. The power is in the analysis – looking for patterns
of similarities, as well as points of dissonance. Where do the
client’s opportunities really lie? How does the learning from
the research have the potential to deeply impact their current
or future business? The answer is insights-driven reporting. It’s
taking the face value responses from the research and marrying
them with peripheral knowledge of the product, service, or
brand, as well as emotional context, to help your clients make
sense of it all.
Reports live on to tell the story of our research, long after
our foggy memories lose track of things we never thought we’d
forget. So, take the time to make a lasting report that tells the
story. A report that is: visually appealing so that people will
actually want to read it; structured and written in a way that
the client culture can really consume; concise enough that
upper level management can get the gist with one perusal;
and makes that leap from insight to action by providing
recommendations for the clients. Never underestimate the
power of an effective and meaningful report.
To be a true partner for your client, you must provide
guidance in navigating these projects with excellence in each
step of the process. So, make sure you clearly understand the
research goals, help identify and recruit the right people, truly
consult about the questioning techniques used to elicit rich
learning, offer creativity to the process, and deliver impactful
and meaningful reports. Do these well and your client won’t be
able to let you go.
Kendall Nash is the senior qualitative consultant, Decision
Sciences, at Burke, Inc. in Cincinnati, Ohio, where she conducts
and oversees hundreds of in-person and online qualitative projects.
She serves as the president of the Qualitative Research Consultants
Association (QRCA) and teaches the “Next Generation Qualitative
Tools” course through the Burke Institute.
vue | JANUARY/FEBRUARY 2014
21
THANK YOU TO OUR
SPONSORS, SPEAKERS AND DELEGATES
SPONSORS
PLATINUM
An Aimia Company
GOLD
BRONZE
REAL PEOPLE. REAL SOLUTIONS. REAL DATA
SILVER
SPEAKERS
Susan Abbott, CMRP
Anne Crassweller
Tom de Ruyck
Caroline Fletcher
Frank Graves, MA, FMRIA
Jeffrey Henning
Chris Long
Bernie Malinoff, CMRP
Eric Meerkamper
Mark Michelson
Grant Miller
Rudy Nadilo
Annie Pettit
We thank everyone who attended
and look forward to welcoming you at future MRIA events !
22 vue | JANUARY/FEBRUARY 2014
HELLO
my name is
Tracy Rideout
St. John’s,
Newfoundland & Labrador
MQO Research
•
•
•
•
•
When I was ten, I wanted to be: A speech pathologist
My first job was: A cashier at A&W
A researcher I admire is: Kerry Bodine
Why I became a market researcher: After finishing my
Bachelor’s in Psychology, I was unsure about my career path.
I decided to pursue a Master of Applied Social Psychology
degree which focuses on research methodology, program
evaluation, attitude measurement and advanced statistics.
After completing this program, it was almost a natural step
for me to enter the industry.
CMRP date: August 2013
vue | JANUARY/FEBRUARY 2014
23
This is what happens when two researchers are told
to argue an opinion they might not agree with.
MARKET RESEARCH: ESSENTIAL OR POINTLESS?
What does the business environment
look like without market research?
Picture this:
• Companies develop products without
any feedback and miss the mark in
meeting their customers’ needs.
Katie Clark
• Companies don’t listen to their
customers’ issues and miss the
opportunity to jump in and fix
problems before the customer defects
to a competitor.
• Companies don’t investigate the competitive landscape and
risk being blindsided by a competitor’s new product or
service, losing customers and dollars.
It’s a dismal picture, isn’t it?
Now imagine a world where market research is a
requirement for all companies : businesses MUST conduct
market research. I’d like to propose a brave new world where
it is an absolute requirement that companies conduct market
research; that research becomes hard-wired into the DNA
of a business. Imagine a new startup business immediately
saying “We need research,” in the same breath as “We need an
accountant” and “We need office space”!
Some, even in our industry, might argue that businesses do
not need to conduct market research (cue the overused Steve
Jobs quote). Some companies may argue that their product “is
what it is,” and their business contracts are solid for the next
few years; rendering research and new product development
“unnecessary.” Others may argue that their company knows
best about what products to develop and produce; that
consumers don’t know what they need so why even ask them.
24 vue | JANUARY/FEBRUARY 2014
Steve Jobs once famously said, “We
were the group of people who were
going to judge whether it was great
or not. We weren’t going to go out
and do market research.” And, of
course, we all interpreted that as
Steve Jobs doesn’t do market research.
Steve Jobs was correct.
Consumers don’t know what they
Mike MacLeod
want.
At least collectively, they don’t know
what the next major leap should be in the development of a
product. And they are even more inept when describing what
should be included in a product that creates an entirely new
category. Truly innovative products and concepts are usually
so far away from the current state that it requires the ability
to incorporate changes that cannot even be comprehended by
most.
Jobs also said, “How can I possibly ask someone what a
graphics-based computer ought to be when they have no idea
what a graphics-based computer is? No one has ever seen one
before.”
Consider 2,000 consumers on a panel, balanced to match
the census for age, gender and ethnicity. No survey could
possibly be written that would result in data supporting the
development of the newest, most innovative, cutting-edge
product, regardless of the category. What questions would you
have asked this panel that would have led to the development
of the Mac?
Or, how about a dozen focus group participants, sitting
around a conference table, eating deli sandwiches for a couple
of hours? There is no moderator in the industry that could have
described the iPod to these people 14 years ago. And even if
FAC E OF F
These companies may rail against a market research mandate.
My counter-argument? They have a very narrow definition of
market research!
For example, now may not be the right time for extensive
consumer research to develop a new product. But what if your
competitor is launching a product? Competitive research in
this situation would be key! Resting on your laurels with your
existing product? Consider some secondary research on your
own data to determine patterns that may lead to new thinking
about how to increase sales for those existing products. Or
perhaps a study on how customers are using those products and
how that use has changed over time? Juicy insights abound!
We’ve all heard the horror stories of companies that have
launched products or services without research only to watch
those products quickly fail. We’ve winced when we hear about
companies who continue to ignore the voice of their customer.
Now, really, who wants to live in a world where market
research isn’t conducted? Show me a spectacularly successful
company that has ignored all facets of market research: from
secondary to primary, from pricing to ethnography, from new
product development to the Voice of the Customer!
Research can take many forms, so there’s really no need for
companies to fight against the “businesses MUST conduct
market research” requirement. If companies embrace market
research as a necessity and not just a “nice to have,” the
opportunities for the kinds of research that can be conducted
are endless. Research can be primary or secondary, gathering
new data, or finding new ways to look at existing data.
Startups can focus on researching the marketplace they’re
getting into and competitive products. Companies that are
struggling with accurately positioning and pricing a product
might look to an in-depth conjoint study. Companies hoping
to more deeply understand customers’ experience with their
product might do deep dive ethnography.
That said, in no way am I recommending companies
do throwaway research – doing research just because it’s
mandated. Instead, find the right kind of research for the right
phase of the company or product that can help the business
improve and be competitive.
A requirement to mandate research would be a blessing
– how can it be a bad thing to require companies to listen to
their customers? Pray tell why shouldn’t it be a requirement
that companies hard-wire the voice of the customer into their
organization’s DNA?
Should market research be a requirement for all companies?
Yes.
Let’s make it happen! Now, who’s with me?
Katie Clark is the senior research manager at Diversified Business
Communications and active in the social media space blogging
and tweeting (@InsightsGal) about research. The opinions
expressed here are her own and not those of her employer.
Katie was assigned this “side” of the argument to argue. While
mandated research might be a stretch in the current business
environment, she believes that the most successful companies are
those that have made research a priority.
one could, how many of these respondents could understand
how an iPod would be better than their portable CD player?
What’s an MP3? I can’t see it? I have to download it? Where do
I keep it? How do I pay for it? What’s iTunes?
Now, how about a small handful of designers with a narrow
focus, led by a leader fixated on a singular mission? A group
such as this, combining their time and effort with a superb
imagination and the drive to provide something innovative
is what is needed to develop the next big thing. They should
have zero concerns about the opinions of the masses who have
pondered the topic but for a mere few moments.
Consumers that understand and appreciate the results of
such a group can become product evangelists. Some consumers
won’t get it, and others still may hate what is produced. But
that’s better than feeling that it’s just OK. Maybe it’s a little
better, or a little worse, than something else already on the
market. Something that is easily replaced. Something that no
one cares about.
When you listen to customers, (existing or potential ones),
the result is product development that is evolutionary – but
definitely not revolutionary. That may be fine if your goal is
to create something that is marginally better than last year’s
model or if you’re looking to compete directly with an existing
competitor. This will help you get a sliver of an already existing
pie.
Alternatively, if you are looking to revolutionize your
existing category, or, better yet, create a brand new product
category, then run quickly from your market research supplier
and don’t look back. No amount of research with writers and
typists in the 1970s would tell anyone to build word processors
as a replacement for the typewriter.
Henry Ford may have said, “If I asked people what they
wanted they would have said faster horses!” Of course, that
statement is correct. What would your answer have been if he
stopped your horse and buggy to ask about your transportation
needs? Even if he could describe the automobile to you, what
would you do if it breaks down? Where would you drive it with
the terrible road conditions? And where are you going to get
gas? The idea sounds silly. We should just stick with horses.
Of course, that’s ridiculous. Ford knew better than to even
ask such questions. He made his own decisions about what he
would produce, how it would be produced, and the features
it would or would not have. He would have been much less
successful, and had minimal impact on the world, if he catered
to the thoughts of the rank and file.
So yes, Steve Jobs was correct. Market research doesn’t help
with innovation because people don’t know what they want.
Not until you show it to them.
Mike MacLeod has been on the leading edge of the marketing
research industry for 22 years. His focus is to maximize data
quality by addressing the issue from all sides. He is currently senior
account director at Lightspeed Research. A transplanted Canadian,
he currently resides in Northeast Pennsylvania. Mike tweets at
@MikeMacLeod.
vue | JANUARY/FEBRUARY 2014
25
INDUSTRY N EW S
Marketing Research and Intelligence Association
L’Association de la recherche et de l’intelligence marketing
CMRP – PARM
CERTIFIED MARKETING RESEARCH PROFESSIONAL – CMRP
PROFESSIONNEL AGRÉÉ EN RECHERCHE MARKETING – PARM
SIGNIFIES HIGH LEVEL OF KNOWLEDGE AND CAPABILITY IN MARKETING RESEARCH
THEORY AND PRACTICE, AND ADHERENCE TO RIGOROUS ETHICAL STANDARDS
SET OUT IN MRIA’S CODE OF CONDUCT AND GOOD PRACTICE.
A CMRP has the ability to:
4.1
4.2
4.3
4.4
4.5
4.6
4.7
Apply Core Knowledge*
Identify Issues
Analyze
Integrate Knowledge and Issues
Make a Professional Judgment
Respond to Users’ Needs
Effectively Communicate
*The Core Knowledge of the CMRP includes:
I. Professional Practice
II. Marketing Research Design
III. Statistical Methods
IV. Questionnaire Design
V. Qualitative Marketing Research
VI. Market Intelligence & Competitive Intelligence
VII. Marketing Management
Please see Core Curriculum Competencies document for details. A copy of this document is available online:
http://mria-arim.ca/education/cmrp-certification
GOLD SEAL CERTIFIED CORPORATE RESEARCH AGENCY
INDICATES THAT A CORPORATE RESEARCH AGENCY MEMBER:
• H
as successfully completed an independent, third party Certification Review focused on how effectively
the Agency meets MRIA standards, particularly those related to quality control practices, obligations to
clients and obligations to respondents. The Certification Reviewer has found the Research Agency to be
in compliance with MRIA Standards in all material respects;
• Operates as a business in Canada and derives at least 75% of its revenues from the provision of
marketing and survey research or business intelligence services * to other companies or organizations;
* (Includes consulting services, data collection, data analysis, and adding value to information)
• S ubscribes to the Code of Standards and Ethical Practices of the MRIA;
• Registers its surveys with MRIA’s Research Registration System;
• Has been operating as a business in Canada for at least 24 consecutive months.
Please see the MRIA website for a complete list of Gold Seal Agencies:
http://mria-arim.ca/directories/gold-seal-members
BASIC CORPORATE RESEARCH AGENCY
INDICATES THAT A CORPORATE RESEARCH AGENCY MEMBER:
• O
perates as a business in Canada and derives at least 75% of its revenues from the provision of
marketing and survey research or business intelligence services* to other companies or organizations;
* (Includes consulting services, data collection, data analysis, and adding value to information)
• Subscribes to the Code of Standards and Ethical Practices of the MRIA;
• Registers its surveys with MRIA’s Research Registration System.
For more information on MRIA Corporate Memberships and on the CMRP designation, visit us at www.mria-arim.ca
Pour de plus amples renseignements sur l’adhésion corporative et sur les sociétés de recherche Sceau d’or, visitez
http://mria-arim.ca/fr/adhesion/adhesion-corporative/sous-categories
26 vue | JANUARY/FEBRUARY 2014
IND U STRY NEWS
QUALITATIVE RESEARCH REGISTRY (QRR)
In accordance with federal privacy laws,
MRIA’s Qualitative Research Registry (QRR), or Registre de la
recherche qualitative (RRQ) in French, was created to provide
an ongoing, user-friendly vehicle for tracking those who do
not want to be contacted or should not be contacted for
qualitative research studies.
QRR is a comprehensive do not call list of those who
have recently participated in qualitative research studies,
those who have asked not to be contacted further, and those
felt by recruiters and moderators to be best served by not
being contacted. These respondents are marked as “do not
call” in accordance with established MRIA Standards.
All field and full-service companies are encouraged to submit
a list of their qualitative respondents for entry into the QRR
system each month, including those who do not wish to be
contacted.
Participating firms will receive monthly updates of
respondents to be screened from qualitative recruitment
samples. QRR works effectively to increase the quality and
integrity of the qualitative research process, by serving as
a control to ensure respondents are not contacted more
frequently than is necessary.
However, the ability of the system to function effectively is
directly related to the co-operation received from firms who
provide recruitment services. If you are a full service research
firm or field supplier that is currently participating in the
Qualitative Research Registry program – thank you very much
and keep up the good work!
If you are not currently participating, please get involved! If
you are interested in submitting to QRR, please visit the MRIA
website at http://mria-arim.ca/about-mria/qualitative-researchdivision/qualitative-research-registry for further explanation and
guidance on how to submit qualitative research participants’
names, along with the required electronic forms.
THE FOLLOWING COMPANIES HAVE SUBMITTED NAMES TO QUALITATIVE RESEARCH REGISTRY
JULY
2013
ONTARIO
QUEBEC
Barbara C. Campbell Recruiting
Consumer Vision
CRC Research
Dawn Smith Field Management Services Inc.
I & S Recruiting
Ipsos Reid
Nexus Research
Quality Response
Research House Inc.
Ipsos Reid
WEST
Barbara C. Campbell Recruiting
CRC Research
Ipsos Reid
Trend Research
Qualitative Research Registry submis­sions should be sent to: [email protected]
Submission templates and payment forms can be found at
http://mria-arim.ca/about-mria/qualitative-research-division/qualitative-research-registry-fees
Rules of Conduct and Good Practice for Members of the Marketing Research and Intelligence Association (2007),
Section C Rules Specific to the Conduct of Qualitative Research:
20. R
ecruiters should provide accurate data to the Qualitative Research Registry, where such exists, on a consistent
basis and check all respondents against the Registry.
21. M
oderators buying recruiting services should give primary consideration to recruiting agencies which submit to
the Qualitative Research Registry, where such a service exists, on a regular and ongoing basis.
vue | JANUARY/FEBRUARY 2014
27
INDUSTRY N EW S
RESEARCH REGISTRATION SYSTEM
Since 1994, the RRS has allowed respondents to verify the legitimacy of a research project; helped legislators
and regulators differentiate between legitimate survey researchers and unscrupulous telemarketers,
phishers and scammers; and protected the industry from unnecessary and unwanted regulation.
MRIA’s Research Registration System (RRS) has long been a cornerstone self-regulatory mechanism for the marketing,
survey and public opinion research and market intelligence industry in Canada.
Combined with other self-regulatory initiatives such as our Code of Conduct
and Good Practice and our Charter of Respondent Rights, the RRS has paid
huge dividends in protecting the industry’s positive reputation and good
name with Canadians.
All Gold Seal and Basic Corporate Research Agency members of the
Association are obligated to register all of their research projects with the
RRS, and Client-Side Corporate members are encouraged to require their
agency suppliers to do so.
MRIA’s Research Agency Council provides strategic, policy-level oversight of
the Research Registration System, and receives aggregate data-only on the
System’s performance.
Questions about the Research Registration System should be addressed
to Erica Klie, Member Services & Events Coordinator, at 1-888-602-6742 or
905-602-6854, ext. 8727 or [email protected] or, in her absence, Interim
Executive Director, John Ball, CMRP at ext. 8724 or [email protected].
The following companies have registered research projects with the Research Registration System during JULY 2013:
GOLD SEAL CORPORATE RESEARCH AGENCIES
Academica Group
Advitek Inc.
BBM Analytics
BBM Canada
Blue Ocean Contact Centers
Campaign Research
Canadian Viewpoint Inc.
Cido Research
Consumer Vision Ltd.
EKOS Research Associates Inc.
Greenwich Associates
Harris/Decima Inc.
Head Count
Hotspex Inc.
Ifop North America
Ipsos Reid
Maritz Research Canada
MBA Recherche
MD Analytics Inc.
MQO Research
Nanos Research
NRG Research Group
Opinion Search Inc.
Research Dimensions
Research House Inc.
Tele-Surveys Plus / Télé-Sondages Plus
The Logit Group Inc.
TNS Canada (Canadian Facts)
Trend Research Inc.
BASIC CORPORATE RESEARCH AGENCIES
Barbara C. Campbell Recruiting Inc. (BCCR Inc.)
Goss Gilroy Inc.
Nexus Market Research Inc.
SmartPoint Research Inc.
INDIVIDUAL MEMBER ORGANIZATION
Illumina Research Partners
Rules of Conduct and Good Practice
For Members of the Marketing Research and Intelligence Association (2007):
Section A (5)
Members must uphold the MRIA Charter of Respondent Rights.
Charter of Respondent Rights, Article 2
You can verify that the research you have been invited to participate in is legitimate in one of two ways. You can either obtain a registration
number and the MRIA’s toll-free telephone number for any research registered in the MRIA’s Research Registration System or you can obtain
the contact information of the research director who is conducting the study.
http://mria-arim.ca/about-mria/research-registration/research-registration-overview
28 vue | JANUARY/FEBRUARY 2014
IND U STRY NEWS
n To read more news online, or to submit your “People and
Companies in the News,” s imply fill out our online form at
http://mria-arim.ca/news/people-and-company-news.
n The Vue editorial team reserves the right to select and
edit your submission for appearance in Vue.
n MRIA is neither responsible for the accuracy of this
information nor liable for any false information.
Bruce Anderson Joins Abacus Data as Chairman
Abacus Data is excited to announce that Bruce Anderson has joined
Abacus Data as Chairman, starting December 1, 2013. He will be
responsible for providing strategic advice to Abacus Data clients and
helping Abacus grow its business across Canada and North America.
Bruce has over 30 years’ experience in the market research sector and
is also a partner at I2 Issues and Ideas Advertising.
Website: http://www.abacusdata.ca
Research and Incite Appoints Mary Logan, CMRP,
to President
Research and Incite are pleased to announce the appointment of Mary
Logan to the position of President. Mary has been a guiding force at
Research and Incite since 2000 and most recently held the position of
Executive Vice President. An active member of the research industry for
25 years, Mary has extensive experience of both the North American
and European markets. A professional member of the MRIA with CMRP
status, she holds an honours degree in Social Psychology from the
University of Sussex, U.K. Website: http://www.riconsultants.com
Hank Goertzen, CMRP
Obituary published in The Saskatoon StarPhoenix on Dec. 13, 2013: It
is with great sadness that the family of Hank Goertzen announces his
sudden passing on Tuesday, December 10, 2013 at the age of 59 years.
Hank is survived by his loving wife, Linda; sons, Michael (Heather), Kyle,
and Sean (Allie); his daughters through marriage, Tracy (Todd), Keri,
and Robin (Brian); five grandchildren, Jaimee, Zane, Kai, Lauren, and
Hudson; sisters, Erna (Wilf), Frances, and Ruth (Alan); and many other
extended family.
ESOMAR Applauds Recent Arrests in
Speak Asia Survey Fraud Case
Following an almost three-year investigation, Indian authorities have
arrested the masterminds behind the Speak Asia survey fraud. In 2011,
ESOMAR, along with MRSI had actively advocated strict action against
the company, which had been allegedly running the pyramid fraud
multi-level marketing schemes that encouraged participants to pay
large sums of money to become members of Speak Asia where they
would receive rewards for completing research questionnaires/surveys.
Website: www.esomar.org
MDC Partners Buys Majority Stake in Luntz Global
Marcoms network MDC Partners has acquired a majority partnership
interest in language and communications research specialist Luntz
Global, thereby extending its own Consumer Insights and Analytics
practice. Terms of the deal were not disclosed.The new partnership
will help extend MDC Partners’ Consumer Insights division, which
develops data and analytics capabilities for use by MDC companies
wishing to analyze clients’ integrated digital and multi-channel media
campaigns. Website: www.luntzglobal.com
Leger Metrics Launches Text Analytics Tool
Canadian firm Leger Metrics has launched a solution to automatically
analyze text responses from open-ended survey questions, social
media, and anywhere else customers leave comments about their
brand experiences. Leger Metrics was known as Agility Metrics, prior to
its acquisition by marketing research firm Leger in 2012. The division
specializes in voice-of-the-customer (VoC) and customer experience
management (CEM) solutions, and offers continuous customer
feedback and real-time insights. Website: www.legermetrics.com
Nielsen Names New Marcoms Leader
Nielsen has appointed former Edelman exec Katie Burke as Executive
Vice President of Marketing and Communications, responsible for the
firm’s internal and external communications, content development,
digital marketing and industry relations. Burke joins with nearly 20
years’ experience. Most recently, she served as the Global Chair of
Public Affairs at PR giant Edelman, prior to which she held several
major roles in U.S. politics, including Director of Television News in the
White House. Website: www.nielsen.com
Murdoch Buys Social Media News Agency Storyful
Rupert Murdoch’s News Corp group is paying EUR 18 million (C$24.6
million) for Dublin-based social media news agency Storyful,
People and Companies in the News sponsored by:
vue | JANUARY/FEBRUARY 2014
29
INDUSTRY N EW S
which offers a combination of journalistic expertise and proprietary
technology, including social media dashboards and other real-time
discovery tools and feeds. Storyful was set up in 2010 by former
foreign correspondent Mark Little. The company’s technology allows
clients to integrate video into their news or advertising via online
and mobile platforms, and to monitor social media conversations and
sentiment. So far in 2013, verified user-generated videos managed by
Storyful generated 750 million views for its partners.
Websites: www.newscorp.com and www.storyful.com
achieved online virality, or were identified as memorable by industry
associations and experts. Using its “ComMotion” ad testing approach,
BrainJuicer predicted the in-market efficiency of advertising from
the emotional response it evokes, classifying advertising according
to its “emotional pull” and awarding it a star rating. According to
Chief Juicer and CEO John Kearon, only 5% of the ads tested globally
achieve a 5-star rating, (the U.S. norm is 2-star) – in his view, all brands
should be focused on using advertising for emotional brand building.
Website: www.brainjuicer.com
FTC Approves New Imperium Approach for COPPA
In the U.S., the Federal Trade Commission (FTC) has given its approval
to a new method for obtaining parents’ verifiable consent for online
collection and use of children’s personal information, developed by
technology firm Imperium, whose ChildGuardOnline will offer a series of
“challenge” questions asking for information “not commonly available
or typically found in a person’s wallet” and “difficult for someone
other than the individual to whom the information pertains to answer
correctly’”, in order to verify that the person giving consent is in fact
a parent. This ensures compliance with COPPA, the Children’s Online
Privacy Protection Act. A month ago the FTC rejected a method put
forward by AssertID based on social graph verification. Imperium is
online at www.imperium.com Thanks to www.lexology.com for some of
the above.
IMS Health Plans IPO
Pharma research giant IMS Health Holdings has filed a registration
with the U.S. Securities and Exchange Commission (SEC) for an Initial
Public Offering (IPO) on the New York Stock Exchange. Rumours about
the likelihood of an IPO have been circulating for some time. An initial
maximum offering price of $100 million is said to be a “placeholder”
until the price and number of shares are settled. Some analysts
suggested the amount raised would be up to $1 billion. IMS says it will
use the proceeds of the IPO to repay long-term debt and for “general
corporate purposes.” The company reports revenue up 4 per cent to
$1.87 billion in the nine months to September 30, 2013.
Website: www.imshealth.com
BrainJuicer Debuts FeelMore50 ‘Emotional Ad’ Measure
Online research specialist BrainJuicer has introduced the FeelMore50
– a ranking of the “50 most effective emotional ads” in the U.S. Last
year, the firm tested more than 200 U.S. ads that had won awards,
30 vue | JANUARY/FEBRUARY 2014
EVENTS
MARK YOUR CALENDAR
February 19
Montreal Chapter: DéjeunerConférence
Le service à la clientèle à l’ère
du numérique
Montreal, QC
March 4
Canadian Marketing
Association (CMA) and
MRIA, Client Experience (CX)
Conference
Toronto, ON
February 20
Ottawa Chapter: Driving
Innovation in Mobile Survey
Research and Meet the
Marketing and Business
Intelligence Research Class
(MBIR) of 2013–14
Ottawa, ON
June 8–10
MRIA National Conference
2014 – Dig Deeper & Discover
Saskatoon, SK
February 21
MRIA Qualitative Research
Conference 2014
– Creating Connections
Toronto, ON
June 8
MRIA Annual General Meeting
Saskatoon, SK
vue | JANUARY/FEBRUARY 2014
31
BC CHAPTER
ALBERTA CHAPTER
November’s Design and Deliver Better Presentations workshop,
hosted by Lauren Isaacson, went “swimmingly,” reports the BC
Chapter. There were five engaged and active participants coming
from both the client and agency sides. After discussing best
practices, a real word research example was used to demonstrate
how to best present complex data.
The Alberta Chapter Board of Directors is
sad to say farewell to Fay Poholko, a board
member since 2007 and membership
chair for the past six years. Says Fay, “The
best thing about being a board member
is getting to know a wide range of people
in the industry.” We’d like to thank Fay for
her contributions over the years and wish her well – she will be
missed!
In 2014, the BC Chapter wants to do more skills sharing. If you
have a research technique or professional expertise that you want
to share with your colleagues, please contact chapter president,
Adam DiPaula ([email protected]).
QUEBEC CHAPTER
Qualitative Research Day
On November 29, the MRIA’s Quebec Chapter invited its
members and friends to take part in “Qual Day” to shed some
light on various innovative approaches to qualitative research.
Approximately 30 researchers came together to learn from six
fascinating speakers. Part of the conference was dedicated to new
technologies meant to support traditional qualitative research,
including online research communities and social media.
Journée sur la recherche qualitative Le 29 novembre, le chapitre
québécois de l’ARIM conviait ses membres et amis à une
« Journée Quali » pour faire la lumière sur diverses approches
innovantes en recherche qualitative. Pour l’occasion une trentaine
de chercheurs se sont
réunis pour entendre
nos six passionnants
conférenciers. Il fut
notamment question de
nouvelles technologies
en appui à la recherche
qualitative traditionnelle,
dont les communautés de
recherche en ligne et les
médias sociaux.
Barry Watson, president of Environics Research Group, was one of six speakers
at the Quebec Chapter’s “Qualitative Research Day” on November 29.
About 50 people attended the Quebec Chapter’s Holiday Party held at the Smoking Vallée, on December 5.
Professor André Richelieu from Laval University was the featured speaker at this year’s party.
32 vue | JANUARY/FEBRUARY 2014
CHA P TE R CHAT
OTTAWA CHAPTER
The MRIA Ottawa Chapter ended 2013 with “Public Opinion
Polling: Is there a future for the discipline?” hosted by Steve
Kiar CMRP, president of Phoenix Strategic Perspectives.
Panelists – Joan Bryden (The Canadian Press), Éric Grenier
(ThreeHundredEight.com), John Wright (Ipsos Global Affairs) and
Doug Anderson (Harris/Decima) – discussed the realities our
industry faces, including the role of industry professionals, and
the media, in reporting accurate, dependable and informative
research. There was quite a bit to talk about during the
post-discussion wine and cheese. Approximately 40 people,
including industry professionals, academia, and members of the
media, attended the discussion.
Panelist
John Wright
Ipsos Public Affairs
Panelist
Joan Bryden
The Canadian Press
Key recommendations included:
1. Improve the partnership between the industry and the
media, bordering on full disclosure;
2. Encourage both industry practitioners and media to police
published results;
3. Continue to embrace and promote research on research;
4. Develop working relationships with collaborative interests;
including academia to keep the industry engaged; and,
5. Encourage the media to be careful about printing polling
results focused on hype rather than on the value to the
public.
Panelist
Doug Anderson
Harris/Decima
Panelist
Moderator
Eric Grenier
Steve Kiar
ThreeHundredEight.com Phoenix Strategic Perspectives
TORONTO CHAPTER
Approximately 40 people attended the Ottawa Chapter’s “Public Opinion
Polling: Is there a future for the discipline?” panel discussion on
November 29.
T he Toronto Chapter held its holiday party at Buonanotte on
December 12.
Members and guests are welcome at all MRIA events:
Check our online calendar at http://mria-arim.ca/events-awards/calendar for more information on all events and how to register. Members receive
emails directly with event updates, so please check your inboxes for instructions on how to register for all upcoming events!
vue | JANUARY/FEBRUARY 2014
33
COLUMNISTS
Ask Dr. Ruth
Dylan Cody
Insightrix Research Inc.
Ruth Corbin, CMRP
CorbinPartners Inc.
A column addressing questions
received by the Chair of the
Standards Portfolio.
______________________________
Dear Dr. Ruth:
A major competitor has published
polling results in one of Canada’s
national newspapers, quoting a margin
of error of plus or minus 3%, based on
an Internet survey of its panel members.
Isn’t it contrary to MRIA standards
to attach a margin of error to Internet
panel surveys? Should I be registering a
formal complaint against the company?
(Should I stop dating the CEO?)
Troubled in Toronto
______________________________
Dear Troubled:
The Marketing Research and
Intelligence Association publishes a
Code of Conduct for Members at http://
mria-arim.ca/sites/default/uploads/files/
MRIAConduct-Dec2007REV2010.pdf.
Section 9(iii), the most recent statement
of guidelines concerning the reporting
of margins of error, states as follows:
“Members must not present research
results with greater confidence than the
data warrant. Instead, as responsible
professionals, members must point out
the relevant limitations of the research.
This includes...refraining from making
statements about margins of sampling
error on population estimates when
probability samples are not used.”
34 Bright-Eyed
vue | JANUARY/FEBRUARY 2014
The guideline refers to the characteristic
of the sample (being a probability
sample versus a non-probability sample)
rather than explicitly referring to the
online methodology. In the case of
ambiguity or absence of comment in
the MRIA guidelines for a given topic
of interest, ESOMAR guidelines are to
be treated as applicable to Canadian
practice.
While MRIA has a formal complaint
process, you may first want to check
for other relevant facts, including
any appropriate disclaimers attached
to the results, and whether any
misunderstanding was introduced by
the newspaper writer rather than your
MRIA competitor. Thanks for writing.
p.s. (If you proceed with a formal
complaint against the company, that
may take care of your concern about
dating the CEO.)
______________________________
Please address any questions about
standards or evidentiary issues to
Dr. Ruth at rcorbin@corbinpartners.
com. If your question is published, your
identity will not be revealed unless you
so choose.
I spent the better part of my early 20s
studying at various universities and in
different departments. Although my
years there were engaging and thoughtprovoking, the focus was primarily on
theory while little weight was placed
on practical application. Despite this
emphasis, the years I spent in higher
education are nothing compared to the
challenges I now face upon entering the
world of market research.
With a graduate degree under my belt,
I felt relatively immune to the challenges
I would face in any industry I entered.
Through my education, I learned to
think critically, solve problems, and
communicate fluently. In fact, I thought
that my first “real” job would be a piece
of cake.
When I entered the market research
industry, it felt like a whirlwind. I was
hired by a boutique firm and was quickly
exposed to many aspects of the business.
I was challenged not only to be detail
oriented and complete specific tasks,
but also to connect each element of the
business and envision how all the parts
fit together to make the entire company
function. After almost two years in, I am
still piecing it all together.
Despite the many obstacles that stand
before me, I always have the support of
my colleagues. Academia can leave one
with a lone wolf feeling, always needing
to solve things individually, and the team
environment was something new to me.
I had to learn to trust my colleagues and
their abilities to get job after job done as
a team.
Do I regret not pursuing a career with
fewer challenges that may have been a
better fit with my education? Not at all;
I am learning to think quickly with a
practical, client-oriented focus which is a
far cry from the ivory tower of academia.
Off the Deep End
Julie Fortin
SOM inc. (Québec)
Brian Singh, CMRP
zinc tank
Ah, les répondants… On ne peut vivre
sans eux, mais il faut certainement
apprendre à vivre avec eux!
Bien sûr, il y a le répondant qui
collabore de son plein gré, celui qu’on
espère à chaque appel.
Dans cette catégorie, il y a le
convaincu, celui qui accepte avec joie
(oui, oui, ça existe!) de répondre à une
« étude » (pas un sondage, non, c’est
péjoratif!).
Il y a aussi celui qui se cherche un
tas d’excuses pour ne pas participer
(ex. : je dois sortir les poubelles/le chien/
la belle-mère ou encore « Ma téléréalité
commence à l’instant »), mais qui finit
par accepter, à court d’arguments.
Il y a le « bavard », qui ne peut
s’empêcher de commenter chacune de
ses réponses, voire les questions ellesmêmes, faisant ainsi exploser la durée de
l’entrevue (tout en se plaignant qu’elle
soit trop longue!).
Il y a l’éternel indécis, celui qui n’a
jamais assez d’information pour offrir
une réponse tranchée.
Il y a le « pressé », qui accepte de
faire une entrevue de 12 minutes… à
condition qu’elle ne dure que 5 minutes.
À force d’interrompre l’intervieweur, elle
durera au final 15 minutes, top chrono!
Il y a le « relayeur », soit celui
qui consulte systématiquement son
conjoint(e) avant d’émettre une opinion.
Plus triste, il y a celui pour qui un
sondage téléphonique constitue un
antidote à l’ennui, ou encore la seule
façon d’exprimer son point de vue.
Et il y a les autres, les « refus ».
Certains refusent poliment, fermement,
courtoisement, mais d’autres le font
grossièrement, cavalièrement voire
brutalement. D’aucuns nous accusent
même de harcèlement et, menaçants,
invoquent la sacrosainte « do-notcall-list »… qui ne s’applique pas aux
sondages d’opinion!
Quoi qu’il en soit, je lève mon
chapeau à tous les intervieweurs et les
intervieweuses; c’est un métier beaucoup
plus difficile qu’il n’y paraît.
Can we stop smoking our own crack?
Like many across the world, I have
observed the ongoing saga of Toronto’s
Mayor Rob Ford – from when the news
of the crack video broke, to the present
allegations of drinking and driving,
communicating with criminals, and
using other illicit drugs. But the signs of
a problem were there long in advance –
namely drunken appearances in public.
Amidst allegations and confessions, his
ongoing behaviour indicates denial and
stubbornness. There has clearly been a
problem for a long time, but there’s also
been little will, or drive, to correct poor
and ill-advised behaviour.
Oddly enough Mayor Ford’s
behaviour is analogous to some of our
own in the marketing research industry.
For many years, there were signs of
emerging data sets and new players
entering the industry – players that had
no connection to our business and were
quickly cutting into our territory. We
felt that the realm of insight was firmly
in our grasp and our methodologies
were safe. We too had an addiction to
our past behaviours, and denied the
mounting evidence of the looming
threats delivered via client DIY research
and evolving technology platforms that
provide insight more accurately, cheaply
and timely than was traditionally
available.
Like Rob Ford, we may be in the
same proverbial hole – in a place that
is hard to admit where we are and
without a clear map to safety. This is a
real challenge for an industry that was
supposed to have all the answers. On
the flip side, to steal a thought from
Ford: Are we asking ourselves the right
questions?
One key question is “What is our
appetite for risk?” Risk is something
that contravenes the mindset of our
industry – our existence is predicated
on our ability to mitigate risk and,
hence, we have become risk-averse
ourselves. Illicit drugs aside, risk is a
good thing – it is times like this when
the risk of our survival depends upon
our revival, evolution and ability to
constantly innovate. We need to adjust
our worldview to this reality.
While many in the industry have
begun innovating and incorporating
updated methodologies aligned with
evolving client demand, many of us are
still smoking the same crack. We tend to
seek solutions from within our industry
– and from some of the same folks who
got us here in the first place. Admittedly,
some are masters of re-invention, but
there are many who evolved kicking
and screaming. Change is hard when
you are the master of your domain.
Thus, in trying to address our ongoing
problems I have noted that researchers
are listening to innovation from each
other’s own resurgence experiences, and
rarely stepping outside of our industry
to listen to the level of disintermediation
that is going on in the insight arena.
Bizarrely, this is analogous to Mayor
Ford listening to his brother and his
close circle of advisors. It does appear to
be untimely advice.
So how do we grapple with our own
form of addiction? One that has served
us well from door-to-door, to mail,
telephone through to online?
Go to where the innovation is
happening. Spend time with the
tech community, which is using new
forms of data analysis and collection
and remixing methodologies. Go to a
mobile meet up — they are monthly in
a city where you live (or nearby). Go to
different conferences. Go beyond what
we do within the marketing research
arena, go to the fringes of your interests
and see how people are doing things
differently. Confront the traditional
and expand your horizons. You’d be
surprised how welcome our craft is
among insight innovators who had little
knowledge of our industry.
vue | JANUARY/FEBRUARY 2014
COLUMNISTS
La Belle Vue
35
MRIA INSTITUTE OF PROFESSIONAL DEVELOPMENT
VISIT MRIA-ARIM.CA/EDUCATIONMRIA
FOR A FULL
COURSE
LISTING AND A DETAILED
Institute
for
DESCRIPTION OF EACH COURSE.
Professional Development
Please go online to mria-arim.ca/portal to register.
GET $100 OFF the regular price by registering before the early bird (approximately 4
weeks before the course starts). Registration closes
approximately
1 week prior to course start.
Institut
de développement
professionnel de l’ARIM
Looking BEYOND basic knowledge requirements? MRIA offers the following specialized or advanced
areas of marketing research courses taught in-class.
Categorical Data Analysis
March 18, 2014
MRIA Institute for
Conjoint Analysis: Design, Analysis and Reporting that LeadsInstitute
to BetterforMarketing
Strategy
Professional
Development
développement
Create Winning Research
Presentations and Reports thatInstitut
Deeplyde
Connect
with your professionnel
Audience
Professional
Development
April 22, 2014
May 13, 2014
Measuring Customer Satisfaction, Loyalty and Retention* February 20, 2014
Measuring Customer Satisfaction: Advanced
April 17, 2014
Metrics Madness: Is Your Client’s Digital Marketing Working?
April 17, 2014
Semiotics, Introduction: How Symbols, Packaging & Advertising Communicate April 8, 2014
SPSS: Analyzing Survey Data: An Introductory Workshop
April 23, 2014
SPSS: Analyzing Survey Data: An Advanced Workshop
April 24-25, 2014
http://mria-arim.ca/education/in-class-learning/professional-development-courses
*Ces cours aussi disponibles en français http://mria-arim.ca/fr/formation
Courses covering the core knowledge requirements of the
Certified Marketing Research Professional (CMRP) are as follows:
11 of
these 12 core
courses are
available
anytime
online
Course Next in-class session
Register by
401-Online Research, Best Practices and Innovations
204-Qualitative Marketing Research
403-Advanced Qualitative Marketing Research
302-Market Intelligence
301-Competitive Intelligence, Mystery Shopping, and Benchmarking
201-Marketing Research Design: An Applied Course
101-Introduction to Marketing Research
102-Ethical Issues and Privacy in Marketing Research
202-Questionnaire Design
203-Marketing Research Statistics & Data Analysis*
303-Marketing Management for Researchers
402-Advanced Analysis Techniques
February 26, 2014
March 4, 2014
March 5, 2014
April 3, 2014
April 29, 2014
September 30, 2014
November 4, 2014
Fall 2014
Fall 2014 - Spring 2015
Fall 2014 - Spring 2015
Fall 2014 - Spring 2015
Fall 2014 - Spring 2015
January 29, 2014
February 7, 2014
February 5, 2014
March 6, 2014
April 1, 2014
September 2, 2014
October 2014
online available now
online available now
online available now
online available now
Fall 2014 - Spring 2015
http://mria-arim.ca/education/online-learning/online-learning-faq
Marketing Research and Intelligence Association
l’Association de la recherche et de l’intelligence marketing