jmp.com/learn rev 02/2014 MSA Attribute Data Attribute measurement systems produce categorical responses (ex: pass or fail, classification into multiple categories, and ordinal ratings). This page provides information on attribute measurement systems analysis (MSA), including between and within rater (or appraiser) agreement studies, and standard (or expert) agreement studies. For these studies, rater results are stored in separate columns. Attribute Measurement Systems Analysis Example: Attribute Gauge.jmp (Help > Sample Data) 1. Select Analyze > Quality and Process > Variability/Attribute Gauge Chart. 2. Select categorical variables (usually raters) from Select Columns, and click Y, Response (categorical variables have red or green bars). 3. Select a grouping variable (often part), and click X, Grouping. 4. For an expert agreement study, select the standard (expert or reference) and click Standard. 5. Ensure Attribute is displayed under Chart Type (it will change by default) then click OK. JMP displays a Gauge Attribute Chart and several reports to assess agreement: • • • The Gauge Attribute Chart (bottom, left) displays the percent agreement within each part (top) and within each rater. Here we see that parts 6 and 14 were problematic for the raters (low agreement), and that raters A and B had slightly higher overall agreement than rater C. The Agreement Report (not shown) summarizes rater and overall agreement. The Agreement Comparisons report (bottom, middle) provides Kappa statistics for each pair of raters and for each rater against the standard. Kappa is a measure of non-‐chance agreement. Generally, Kappa values of 0.6 or higher indicate acceptable agreement. The Effectiveness report (bottom, right) summarizes rater agreement with the standard. It includes a Misclassification Matrix and a Conformance Report, which provides the probabilities of false alarms (Type I errors) and misses (Type II errors) for each rater. Notes: For more details, search for “Kappa” in the JMP Help, or refer to the “Variability Charts” chapter of the Quality and Process Methods book (under Help > Books).