Choice-Based Conjoint Workshop October, 2010 With information provided by Sawtooth Software Outline • • • • • Introduction to Conjoint Analysis Formulating Experiments Conjoint Methods -types Use CBC software Introduction to CBC analysis • Introduction to Conjoint Analysis – Identify Your Goal – Design your experiment – 5 stages – Interpreting part-worths and importance – A brief introduction to market simulations Different Perspectives, Different Goals • Public wants all of the most desirable features of environmental assets at lowest possible cost • Providers want to maximize welfare by: 1) minimizing costs of providing features 2) providing products/services that offer greater overall value than other alternatives Demand or Preference Side of Equation • Focus first on demand/preference side of the equation • After figuring out what consumer wants, next assess whether it can be built/provided in a cost- effective manner Products/Services are Composed of Features/Attributes • Olive Oil – Source, Price, Aroma, Size • Forest Harvesting Program Live trees after harvesting, Cost , Dead trees after harvesting, % of forest set aside from harvest • Plastic Bag Management – Cost/tax, % wildlife impact, durability, waste Breaking the Problem Down • If we learn how consumer values the components of a product, we are in a better position to design those that improve profitability • If we learn how the public values the components of environmental goods and services, we are in a better position to design those goods and services to maximize societal welfare How to Learn What Customers/Public Want? • One way: Ask Direct Questions about preference – – – – What Brand do you prefer? How much would you pay for it? What color do you prefer What size of container would you like? • Answers often trivial and unenlightening (e.g. respondents prefer low fees to high fees, medium size than large etc..) How to Learn What Is Important? • One way: Ask Direct Questions about Importances – How important is it that you get the << >> that you want? Stated Importances • Importance Ratings often have low discrimination: Stated Importances • Answers often have low discrimination, with most answers falling in “very important” categories – If they were not important, we probably wouldn’t have included them in the research! • Answers sometimes useful for segmenting market, but still not very actionable – We still don’t exactly what product they want What is Conjoint Analysis? • Research technique developed in early 70s • Measures how buyers value components of a product/service bundle • Dictionary definition-- “Conjoint: Joined together, combined.” • Marketer’s catch-phrase-- “Features CONsidered JOINTly” How Does Conjoint Analysis Work? • We vary the product/service features (independent variables) to build many (usually 12 or more) product concepts • We ask respondents to rate/rank or choose among a subset of those product concepts (dependent variable) • Based on the respondents’ evaluations of the product concepts, we figure out how much unique value (utility) each of the features (attributes) added • (Regress dependent variable on independent variables; estimated betas equal to part worth utilities.) Important Early Articles • • • • • Luce, Duncan and John Tukey (1964), “Simultaneous Conjoint Measurement: A New Type of Fundamental Measurement,” Journal of Mathematical Psychology, 1, 1-27 Green, Paul and Vithala Rao (1971), “Conjoint Measurement for Quantifying Judgmental Data,” Journal of Marketing Research, 8 (Aug), 355-363 Johnson, Richard (1974), “Trade-off Analysis of Consumer Values,” Journal of Marketing Research, 11 (May), 121-127 Green, Paul and V. Srinivasan (1978), “Conjoint Analysis in Marketing: New Development with Implications for Research and Practice,” Journal of Marketing, 54 (Oct), 3-19 Louviere, Jordan and George Woodworth (1983), “Design and Analysis of Simulated Consumer Choice or Allocation Experiments,” Journal of Marketing Research, 20 (Nov), 350-367 Traditional, Less-Effective Questions • How important is horsepower to you in a vehicle? • How important is fuel efficiency to you in a vehicle? • Which is more important to you, horsepower or fuel efficiency? What’s So Good about Conjoint? • More realistic questions: Which product would you prefer . . . - 210 horsepower or - 140 horsepower - 17 MPG - 28 MPG • If choose left, you prefer Power. If you choose right, you prefer Fuel Economy • Rather than ask directly whether you prefer Power over Fuel Economy, we present realistic tradeoff scenarios and infer preferences from your product choices What’s So Good about Conjoint? (cont.) • When respondents are forced to make difficult tradeoffs, we learn what they truly value • These values (utility scores) are associated with specific and actionable attribute levels relevant to the problem at hand Building a Model Building a Model • Inputs: – Attributes – Levels – Respondents – Prior Knowledge – External Data – Experimental Design – Conjoint Method • Outputs: – Utility Scores for each level – Importance Scores for each attribute – Ability to perform Simulations Defining Attributes • Attributes are independent aspects of a product or a service (Brand, Price, Size, Color etc.) • How many attributes? • -Depends on research objectives – One rule of thumb was that no more than 6 0r 7 attributes is too much • May cause respondents to simplify, looking only at 2-3 most important • Attributes should be independent, mutually exclusive – Brand, quality and product life expectancy may all measure the same thing • Each attribute has varying degrees, or “levels” – Cost: $1, $2, $3 Rules for Formulating Attribute Levels • Attributes are assumed to be mutually exclusive – Attribute: Add-on features – Level 1= Sun roof – Level 2= GPS system – Level 3=DVD player – If you define levels in this way, you cannot determine the value of providing 2 or 3 of these features at the same time (or none of them) Solutions • 8 level Attribute: • Features – – – – – – – – None Sunroof GPS system DVD Player Sunroof, GPS Sunroof, DVD GPS, DVD Sunroof, GPS, DVD • 3 Binary Attributes: – Sunroof: • None • Sunroof – GPS System • None • GPS – DVD Player • None • DVD Player Rules for Formulating Attribute Levels • Levels should have concrete/unambiguous meaning • “very expensive” vs “ costs $575” • “weight: 5-7 kilos” vs “ weight 6 kilos” • -One description leaves meaning up to individual interpretation, while the other does not Rules for Formulating Attribute Levels • Don’t include too many levels for any one attribute – The usual number is about 3-5 levels per attribute – Make sure levels from your attributes can combine freely with one another without resulting in utterly impossible combinations (very unlikely combinations OK) Attribute Examples Cost $1 $2 $3 Brand A B C …or graphics as well can be levels. Color Red Black Blue Suggestions for Determining Which Attributes & Levels to Include • Talk to all stakeholders • Focus Groups • Search of competitors websites, sales materials Other Inputs into the Model Conjoint Utilities (Part Worths) • Numeric values that reflect how desirable different features are: Feature Vanilla Chocolate Utility 2.5 1.8 25¢ 35¢ 50¢ 5.3 3.2 1.4 • The higher the utility, the better Interpreting Conjoint Utilities • Interval scaled data (no ratio operations!) • You cannot compare one level from one attribute with one level from another attribute, since conjoint utilities are scaled to an arbitrary constant within each attribute (often zero-centered) • You CAN compare differences between two levels of one attribute versus two levels of another attribute (an addition operation) Conjoint Importances • Ratio scaled data • Measure of how much influence each attribute has on people’s choices • Best minus worst level of each attribute, then percentaged: Vanilla - Chocolate 25¢ - 50¢ (2.5 - 1.8) = (5.3 - 1.4) = 0.7 3.9 ----Totals: 4.6 15.2% 84.8% -------100.0% • Importances are directly affected by the range of levels you choose for each attribute Market Simulations • Make alternative program/services scenarios and predict which program/services respondents would choose • Accumulate (aggregate) respondent predictions to make “Shares of Preference” (some refer to them as “market shares”) Market Simulation Example • Predict market shares for 35¢ Vanilla cone vs. 25¢ Chocolate cone for Respondent #1: Vanilla (2.5) + 35¢ (3.2) Chocolate (1.8) + 25¢ (5.3) = 5.7 = 7.1 • Respondent #1 “chooses” 25¢ Chocolate cone! • Repeat for rest of respondents. . . Market Simulation Results • Predict responses for 500 respondents, and we might see “shares of preference” like: 35% 65% Vanilla @ 35¢ Chocolate @ 25¢ • 65% of respondents prefer the 25¢ Chocolate cone So you want to do a conjoint….. Step 1: Begin With the End in Mind • What is the objective of the research? – How much will public be willing to pay for biological control feature? – Will farmers switch and adopt a different varieties? • The better you define the root problem, the better your research will be! Step 2: Plan Your Analysis • Identify how clients need to use data • Deliver analysis plan to clients as part of research proposal – Makes sure that objectives and deliverables are clear Step 3: Define Attributes and Levels • How many attributes? Depends on research objectives – More than 6 attributes may cause respondents to simplify, looking only at 3-4 most important • Attributes should be independent, mutually exclusive – Brand, quality, product life expectancy may all measure the same thing Rules for Formulating Attribute Levels • Levels are assumed to be mutually exclusive Attribute: Add-on features level 1: Manual level 2: Biological Control level 3: Chemical – If you define levels in this way, you cannot determine the value of providing two or three of these features at the same time (or none of them) Rules for Formulating Attribute Levels • Don’t include too many levels for any one attribute – The usual number is about 3 to 5 levels per attribute • One temptation is to include many levels for price, so we can estimate people’s preferences for each – Better approach usually is to interpolate between fewer more precisely measured levels for “not asked about” prices • Cover the range of probable values Rules for Formulating Attribute Levels • Make sure levels from your attributes can combine freely with one another without resulting in utterly impossible combinations (very unlikely combinations OK) Representing Levels • Text – “High Performance Sports Car” • Pictures / Graphics Medium • Sample boards Low High – Allows respondents to touch or feel samples for tactile attributes (towel softness, greeting card paper quality, etc.) • “Null” Level – Has Stereo vs. __________ Step 3: Choose a conjoint method Step 4: Identify Research Constraints • Sample issues • Sample Population >200 • Length of survey (how long can I keep their attention) • Fielding issues • Budget • Client sophistication Example of a Pair of Soda Product Profile Scenarios Attributes Cost Color Program A: $.50 Red Program B: $.25 Silver Size .33 l .75 l Sugar level Diet/light Regular How to tell if you are in over your head… • You should be okay if… – Small number of attributes – Attributes freely combine with one another – Large sample, even after adjusting for subgroup analysis – Your client can describe the analysis, attributes in one paragraph or less, and you can then explain it to a six year old with little difficulty! Section 2 • Intro to Choice-Based Conjoint (Discrete Choice Modeling) Setting Up CBC Interview (Definitions) Concept 1 Concept 2 Concept 3 How many concepts per task? How many tasks per survey? Concept 4 Task How Many Concepts per Task? • Generally, 2 to 5 concepts are used • Attribute text length, graphical representation affect the decision How Many Tasks per Survey? • Respondents are expensive to recruit. It makes sense to ask respondents multiple choice tasks. • Respondents take about 7 minutes on average to answer 20 tasks (~20 seconds per task) • CBC is very flexible in terms of how many tasks to include. Minimum is just one task! (but you’ll need huge sample size, and will face limitations in analysis) • Typical choice is 12 to 18 choice tasks Test CBC Design • What is a Design? The sum total of information about the attribute levels being shown in the CBC tasks across all respondents. (The independent variable matrix) • If you use ANY prohibitions, or use few questionnaire versions, you MUST test your design • Failure to test the design can invalidate your study • CBC/Web automatically tests your design when it generates the design file--pay attention to the test! The “None” Concept • Pros: – Respondents aren’t forced to choose a product concept that they really don’t like – Lets you capture information about whether respondents (or segments) are more or less interested in buying the product concept • Cons: – Choices of “None” provide much less information for estimating utilities than other choices (reduces the effectiveness of your sample size) – “None” utilities and Shares of Preference for “None” are difficult to interpret Design Stage of CCE Exercise • Decide on topic • Discuss how you are going to decide the attributes and the levels • Begin to think about attitudes • Assign team members tasks for the above • Be efficient