P E R F O R M A N C E M A N A G E M E N T Balanced Scorecard Report How to Write Performance Analysis That Truly Enhances Decision Making By David McMillan, Consultant; with Barnaby Donlon, Principal, Palladium Group, Inc. “You can’t manage what you can’t measure.” What performance management adherent doesn’t know that mantra? But just presenting measurement data doesn’t give decision makers enough information to manage with. Sadly, most qualitative analysis in performance reporting falls short. Business leaders depend on performance reporting to tell them what’s going on at the front lines. Effective qualitative analysis contextualizes and interprets performance data to give managers a solid basis for sound decision making. In the aftermath of the 2003 space shuttle Columbia disaster, which took the lives of seven astronauts, investigators initially focused on a small piece of foam insulation that broke off the ship during launch. But later, as they peered deeper into the organizational and technical questions, it became clear that NASA’s risk-assessment and decision-making processes were sufficiently flawed to compromise the safety of the mission. Though human life is rarely at stake when leadership teams meet to review their scorecard results, they often suffer from a problem similar to NASA’s: faulty analysis, compounded by a less-thanobjective (often politically driven) decision-making process. The consequences of flawed analysis— the resulting decisions—may not literally be a life-or-death matter, but they very well might spell life or death for an initiative, product, or even an organization. In our work with dozens of organizations, we’ve observed that analysis is often the weak or missing link in the performance reporting chain. As data collection gets easier, many businesses mistakenly believe that more frequently available or betterquality data equals better decision making. Wrong. Without contextu- 10 alizing and interpreting performance data, organizations undermine the potential effectiveness of the reporting process—in effect, making decisions with blinders on. Most organizations generate reports that simply display raw data values for their performance measures, perhaps including the color-coded traffic lights that indicate the level of success in achieving targeted performance. What commentary is provided typically reiterates the data displayed in the graph, with little explanation or context-setting. The whole point of qualitative analysis is to contextualize and interpret performance facts and figures to provide a basis for sound decision making and action. In reporting the quantity of widgets manufactured in Q2, say, you would show the same period’s output last year. Depicting yearover-year performance is a nobrainer in indicating performance against targeted objectives. But suppose one piece of equipment was offline for eight days in May, causing a slight dip in production. Most likely that information was not known by all decision makers. But without it, they could make a false interpretation. Sound qualitative analysis, on the other hand, would qualify the result by citing this circumstance, even suggesting that adding a shift temporarily could recover the lost output. Managers depend on performance reporting to monitor operations and progress toward goals. Senior managers, removed from the front lines—whether the customer interface, the assembly line, or the supplier relationship—depend on performance reporting to show them what they can’t see firsthand. Effective qualitative analysis puts the information into perspective, helping decision makers identify trends, looming problems, improvements, potential risks, or shortfalls—enabling them to assess their options and take action promptly. Luckily, it doesn’t take special expertise to compose insightful, actionable analysis. By following a simple “reporter’s” approach (asking “what?” “why?” and “what next?”), organizations can convert data into information, information into insight, and insight into actionable recommendations— recommendations that will help decision makers keep performance, and strategy, on track. What Happened? To write effective analysis, you need accurate data, and enough of it to get a sense of a trend, and of whether that trend is normal or abnormal. If seasonality matters, you may need to adjust for it by reporting year-over-year results; comparing April this year with April of last year will yield a clearer picture than comparing April to March. In reality, it’s not always possible to have the luxury of several years of historical data. But whether you have one or 10 years’ worth, you can still create insightful analysis. Graph the measure and target data values so they can be easily compared, and it will be obvious whether performance results were favorable or unfavorable. Some November–December 2008 targets may be expressed as a range, by necessity or at the manager’s discretion. A single target, however, is preferable for comparison. With a range, it’s more difficult to discern whether the performance was favorable or unfavorable—and to gain management’s attention to focus on improvements. Effective visual representation takes practice, and amateur graphic designers can easily and unwittingly create misleading graphics. So take the time to create a graph that clearly communicates your message, starting with the title. The x-axis should show the time series, so that a trend will be visible, and the y-axis should display the units of measurement along a clearly marked scale. And be sure to select the right chart type. A stacked bar chart is much more useful than a pie chart since it can show both a breakdown and a trend. Avoid including too many variables. This will make the graph too complicated and confusing—or worse, obscure the pertinent message. The commentary that accompanies your charts should ground the audience in the topic by stating up front, clearly and concisely, exactly what the reader should be seeing when looking at the graphic. This will ensure no misreading or misinterpretation. For the chart shown in Figure 1 you might say “Our revenue from repeat customers has been decreasing steadily over the past year, despite a target of mild revenue growth.” With this single sentence, you communicate the trend in performance compared to a target value over time. From this introduction it will be easy to transition to the more substantive discussion of performance in the context of the internal and external environment, risks, and opportunities that helps generate insights. The past may not always be prologue, but analyzing what happened can provide clues to future performance. How does performance compare to last quarter’s? Last year’s at this time? Is the trendline rising or falling? What’s the outlook? Written commentary should complement and expand on the illustrated data. Aim for high-value, highimpact content—and brevity. Some experts believe that analysis should address only the measure in question. Others (myself included) believe that it should address the entire objective. The scope of a business objective often extends beyond the performance that can be tracked by its associated measures. For example, for the objective “Build Figure 1. Revenue from Repeat Customer Sales (in U.S. millions) $5.5 $5.0 $4.5 $4.0 $3.5 $3.0 Q1 ‘08 Q2 ‘08 Q3 ‘08 Actual Q4 ‘08 Q1 ‘09 Q2 ‘09 Q3 ‘09 Q4 ‘09 Target Note that targets are shown for the upcoming four quarters.We discourage organizations from reporting target data values for the first time when they report actual performance data. It’s important to present a forward-looking view of where the organization expects to take performance. customer loyalty” (the objective encompassing our Figure 1 measure) you may want to know the long-term potential of smaller customers because it has implications for your marketing strategy. No metric underlies it, so it’s harder to analyze, but additional research may support your speculation. Therein lies the value of contextualizing. Why Did It Happen? Identifying the causes of performance is the heart of valuable analysis—it’s the critical information that enables substantive, worthwhile discussion and action. Without getting a plausible explanation about the data, a decision maker is like a carpenter with a hammer but no nails. Yet determining those causes can be challenging. It involves studying both the internal environment (e.g., activities and changes within the organization) as well as the external environment (e.g., events, competitors’ moves, and trends in the industry and broader business environment). To illustrate, let’s look at our objective, “Build customer loyalty,” and its component measure, “revenue from repeat customer sales.” A decline in sales last quarter could be the result of a number of factors, internal or external. What do the measure’s supporting drivers tell you about the reasons for the performance? Figure 2 (next page) shows a simplified driver model for this measure. Two primary drivers, “customer purchase intent” and “product value,” break down further into quantifiable drivers. An uptick in repeat customer sales, for example, might be caused by a new technology need (perhaps triggered by a new regulation). It might be driven by heightened buying power from major customers in key segments, something that an industry survey on 11 Balanced Scorecard Report Figure 2. Driver Model for “Revenue from Repeat Customer Sales” Loyalty Customer purchase intent Buying power Technology need Revenue from repeat customers Functionality Product value Pricing A driver tree offers a logical, systematic approach to analyzing the factors driving performance—information that’s crucial to the “why?” portion of the qualitative analysis. customer spending trends might indicate. Customer loyalty could also be a factor, which customer satisfaction surveys would reveal. Product value breaks down into two component drivers: functionality and pricing, both of which could be examined through thirdparty surveys, product reviews, or customer satisfaction surveys. While driver models are useful in performance analysis, you needn’t develop detailed driver trees for every measure. But you do need to identify the most significant causes of performance because over time they will prove to be useful guideposts for gathering supporting information—and for raising questions that might help those who use the analysis. What internal changes might be affecting performance? Examine the internal factors first; these are the most knowable and most controllable in terms of making improvements. Business drivers related to the measure may reflect specific challenges, such as declining employee morale, budget cuts, the fact that a key account person left, or that the company’s marketing budget 12 was slashed across product lines. Initiatives and other projects are also constantly altering the landscape of performance. When in doubt, consult subject matter experts within your organization (including functional area managers) for their insights. You can also refer to internal communications (such as memos and operational reports) for additional clues that might explain performance. What external factors are at play? A competitor may have gained traction by releasing a new product addressing the same solution. That may explain a dip in sales. Maybe your product just received a great review in an industry magazine, which would augur improved sales next quarter. Product growth is not necessarily a zero-sum game; perhaps the market space is growing, allowing for continued growth for you and your competitors. Having a clearer idea of the most likely factors is essential before any action, remedial or otherwise, can be taken. To understand external forces— competition, overall market trends, and other factors—and their impact on performance, seek resources such as industry publications, customer feedback, general market trend reports, and news reports. Understanding the impact of the external environment will also lead to a better understanding of the risks and opportunities your organization faces; for example, “Despite positive performance this past quarter, we may see a dent in North American sales in the next quarter owing to the cancellation of a major national conference that has served as an important sales and marketing opportunity.” Once again, tap your subject matter experts; they can help prioritize the reference sources to consult. A marketing person could help identify key surveys that might shed light on repeat customer sales trends. An R&D person would know which industry publications or blogs would provide reliable information about new ancillary products. It is not always possible to know unequivocally why a measure performs one way versus another. The point of analysis, however, isn’t to seek the one answer—it’s to provide informed hypotheses that will aid in discussion. Don’t try to represent hypotheses as fact; rather, state them as possible explanations for performance and justify them appropriately. What Are We Going to Do About It? Good performance analysis should inspire appropriate action. So be prepared to offer suggestions about how best to improve performance or perpetuate success. This requires not just an understanding of the current situation and performance gaps, but also of risks and opportunities. Building on this knowledge involves identifying options and weighing them based on relative constraints, costs, and benefits as you develop the most viable recommendation. November–December 2008 Recommendations can take many forms, from a minor follow-up task to a major initiative designed to close a performance gap—or to just maintaining the status quo. until [date] because we wanted to incorporate the added functionality that you, our customers, recently requested.” Here, you’ve converted risk into opportunity. Suppose a survey of your online customers revealed a 10% drop in satisfaction in the shopping experience in the past quarter. Digging deeper, you discover two likely reasons: slow site navigation (noted by 42% of the “unsatisfieds”) and a confusing checkout process (cited by 38% of the “unsatisfieds”). You might recommend that the web design team brainstorm to devise improvements and that marketing and tech conduct a focus group to identify opportunities for improvement in both problem areas. Include a proposed timeline to ensure decisions are followed through: “Both the brainstorming session and the focus group should be held within the next six weeks.” All recommendations should be action-focused. Recommending further analysis is not wise. For one thing, it’s not an action. In addition, it not only delays decision making, but shows that you didn’t do your job the first time. And remember that recommendations are not meant to carry authority or finality. We’ve known performance analysts who felt they had no right to make recommendations—or whose managers expressed discomfort over allowing them to make any. One in particular decided the senior management team should simply come up with recommendations during review meetings. Our response: “How much more productive might your meetings be if the analysts provided a springboard to help stimulate your thinking and decision making—and to use your limited time more efficiently?” The analysis should communicate the risks, both internal and external, to future performance, as well as what the organization is doing to mitigate those risks. It’s equally important to discuss current or future opportunities and what the organization is doing—or could do—to take advantage of them. Example: noting that a competitor with a strong customer base is owned by a man who is looking to sell his company or retire. A common external risk for retailers, especially in a difficult economy, is customer price sensitivity, where customers place value above brand. To mitigate that risk, the analysis might recommend creating special offers for repeat customers. Consider a software producer that must, for manufacturing reasons, delay the release of its latest update (an internal risk). The analysis might propose that the sales division issue a communication to repeat customers along these lines: “We have postponed our release Cultural Resistance and Due Diligence In some corporate cultures, communicating bad news is verboten. Yet although no manager wants bad news, failure without warning is far worse. It’s not easy, but every organization must strive to encourage candor— objective assessment, based on relevant facts, and done not as finger-pointing but in the spirit of common cause. Reality checks, through good analysis, enable proactive decision making that can mitigate or arrest failure— or yield important improvement. Develop a network of information sources, both internal and external, to provide the important information that colors and shapes performance data. Don’t go overboard, but gather a good crosssection from reliable sources to help you establish cause and effect. And since performance measures are tracked frequently, the analysis models you develop can be reused every reporting cycle. An Element of Competitive Advantage Measurements must be more than a “nice to know” commodity if performance reporting is to have value. A combination of clearly represented graphical data and qualitative analysis that provides context, plausible explanations, outlooks, and recommendations will foster the discussion needed to drive strategic performance— helping assess progress and drive healthy decision making. It’s a critical element of your business intelligence—and competitive advantage. I T O L E A R N M O R E For more on performance reporting, see “The How-to’s of BSC Reporting,” Parts I and II, BSR July–August 2003 (Reprint #B0307E) and BSR November– December 2003 (Reprint #0311E), respectively. Many other relevant articles on performance review and reporting have appeared in BSR. Consult the BSR Index, available free via download at www.execution premium.org. Boost your visual design skills with Say It With Charts: The Executive’s Guide to Visual Communication, by Gene Zelazny, Director of Visual Communications at McKinsey & Company. Or consider Envisioning Information or The Visual Display of Quantitative Information, by Edward R. Tufte. Tufte, a statistician and political scientist, taught courses at Yale in statistical evidence, information design, and interface design. Reprint #B0811C 13