The Usability Test Process: Steps, tips, and more! Dr. Jennifer L. Bowie For Digital Rhetoric The Design Continuum System-Centered • Users dumb • Users same as us • Will use regardless • Bells and whistles • Do what they can, not what they should User-Friendly • Consider the audiences • Users will like this • Often draw on stereotypes • Reasoning not necessarily supported • Based on untested profiles and assumptions User-Centered • Users valuable • Users part of the design process • Early focus on users • Iterative • Involves research of/with users • Includes participatory design, contextual inquiry, ethnography, and usability testing What is Usability? “A function of particular users performing particular tasks in a particular environment” (Smith et al. 68) The “people who use the product can do so quickly and easily to accomplish their own tasks” (Dumas and Redish 4) User-centered design, not “user-friendly” What is Usability Testing? An empirical study of a product’s usability by observing actual users do real tasks with the product Involves: Real users Real tasks Specific usability goals/concerns Observing and recording the testing Data analysis Step 1: User Analysis & Profiles Who are your actual users? You may need to break your users into typical user categories. Consider:  Demographics: age, sex, race, education level, cultural background, socioeconomic status,…  Experience level with the product, with products of the same genre, with required technology,...  Other things:  motivation  learning style  subject matter knowledge  location of use  physical characteristics  people with disabilities or impairments (from color blindness and learning disabilities to more severe disabilities) Step 1: User Analysis & Profiles Con. Create user profiles:  Break users into clear subgroups  Profile/Define the characteristics of each subgroup Choose user profiles to test:  Ideally users from all major profiles will be tested  If limited testing: Choose profiles based on highest number of users in that profile or profiles that you think may have the greatest usability issues Step 2: Decide what to Test 1. Choose an overall purpose  Example: How useable is our new website? 2. Determine objectives or what you are testing for. Examples:  Does our search engine provide usable results in the first 5 links returned?  Are search results clear to the users? 3. Choose type of test: 1. Performance: Can they do it? 2. Understandability: Can they understand it? 3. Read-and-locate: Can they find it? Step 2: Decide what to Test con. 4. Select tasks:    5. Consider tasks with a high chance of user failure (complex tasks, one-of-a-kind tasks, highly abstract or technical tasks) Consider tasks with a high cost of user failure (tasks that require support, like help or support calls, to complete; tasks where data could be damaged or lost) Consider:  First impressions (look and feel)  First tasks  Tasks most performed  Critical tasks  Specific problem areas  New task for the product Select performance objectives (should be individualized for each task)   Time: How long to complete tasks, to find things, to performance procedures Error/Success: user errors, attempts to do/find something, numbers of times section re-read, if the task was completed successfully Step 3: Preparing for the Testing 1. 2. Choose order of tasks: start easy, go sequential, or be random Create written test materials:       3. 4. Recruit participants & determine “payment” Define team member’s roles:      5. 6. 7. Task list for users Written welcome speech/ Intro to be read to user Consent forms Observation forms Pre-task and post task questionnaires & interview questions Other materials Facilitator/Briefer (necessary): Often only team member to interact with users Observation recorder/note taker (necessary) Camera operator (optional) Help desk operator (optional) Test administer (optional) Create written test plan Practice: conduct walkthroughs of the testing and if possible pilot test (the pilot test users could even be a team member) Prepare test environment (day of test) Step 4: Conducting the test 1. Greet & Brief participant: a. b. c. d. e. 2. 3. 4. 5. Read/say welcome Emphasize that you are not testing them, but the product and that they should act as natural as possible Explain think-aloud protocol (if using) Emphasize how user tells you she has completed a task Stress that the testing is “anonymous” Be unbiased (especially the Facilitator/Briefer) Intervene carefully (avoid as much as possible) Observe and record data Debrief user Step 5: Analyzing the Data 1. Collate data into findings: a. Choose an approach:  Top-down approach: predetermine categories of findings (like navigation, design, terminology) and go through data looking for “hits”  Bottom-up approach: put each observation on a sticky note/note card, sort into categories and label categories b. Determine time and errors/success  Examine findings for each user, user profile, and task  Use analysis techniques such as statistics (even averages help) Step 5: Analyzing the Data con. 1. Analyze data: a. Determine cause of problems b. Determine scope/severity of problems c. Make recommendations/changes 2. Report Findings Good Luck & Have Fun! Where to find out more: Barnum Usability Testing and Research Barker Writing Software Documentation, Chapter 6 “Conducting Usability Tests” Hom “General Concepts of Usability Testing” http://jthom.best.vwh.net/usability/general.htm