What is Business Intelligence Reporting? Business Intelligence reporting is a fast developing IT field. Over the last few years reporting has moved from a passive capability providing users with pre-defined reports to a user driven proactive service that not only helps in analyzing information involved in 1 business oriented decisions but also provides users with the information to make decisions as or before events happen . Just a few years ago, the reporting of information was centered on software such as Crystal Reports where a programmer developed a series of SQL queries that would be used to generate reports. The software would provide the ability to enter search terms or parameters into the queries in order to provide limited customization. Similarly, another popular form of BI involved the linking of third party products, particularly Microsoft’s Excel and Access, through queries to databases (usually with ODBC) or by downloading data to structured text files (e.g. ‘csv’ files) in order to support information analysis. While many of these approaches are still in use, the BI field has evolved significantly in the last few years in such a way that advanced reporting, analysis and visualization tools are available in support of decision making across all enterprise databases. These tools support web-based, user driven analysis, advanced custom reports in a Web 2.0 environment that includes evolving open standards 2 such as XBRL (Extensible Business Reporting Language) for exchanging data. BI is therefore an encompassing term for a rapidly evolving and highly significant field. It draws together a number of technologies concerned with reporting and analysis in support of decision-making. These fields include analytics, business analytics, data mining, OLAP (Online analytical processing) etc. Aside from a rapidly evolving marketplace, BI is also experiencing a number of challenges. The CIO Magazine points out that “although BI holds great promise, implementations can be dogged by technical and cultural 3 challenges. Executives have to ensure that the data feeding BI applications is clean and consistent so that users trust it. ” What is clear is that while BI holds great promise, implementation has to be focused, targeted and well supported by all concerned. Accurate data (whether retrieved directly from various databases or stored in a data warehouse) is essential. In a survey carried out in 2007 for Oracle, the conclusion was that “businesses are striving to get an accurate and up-to-the-minute 4 enterprise-wide picture of current performance but many aren't succeeding”. 5 In its 2010 analysis of the BI market , Garner states that: Gartner's view is that the market for BI platforms will remain one of the fastest growing software markets despite the economic downturn. In tough economic times, when competitiveness depends on the optimization of strategy and execution, organizations continue to turn to BI as a vital tool for smarter, more agile and efficient business. According to Gartner's annual 1 2 3 4 5 More information on what is being called BI 2.0 can be found at http://en.wikipedia.org/wiki/Business_Intelligence_2.0 See http://www.xbrl.org ABC: An Introduction to Business Intelligence, available at http://www.cio.com/article/40296/ABC_An_Introduction_to_Business_Intelligence A summary of the survey can be found at http://www.cio.com/article/487508/Enterprises_Struggle_to_Use_BI_Systems_Effectively See Gartner’s “Magic Quadrant for Business Intelligence Platforms”, 19 January 2009, Rita L. Sallam, Bill Hostmann, James Richardson, Andreas Bitterer survey of CIO technology priorities, BI remained among the top five priorities in 2009 (and it was No. 1 in each of the previous four years). While reporting remained the dominant style of information delivery of BI in 2009, the increased proliferation of interactive visualization tools pushed the power of data analysis and discovery into the hands of a larger number of users than ever before. 6 A Gartner 2010 BI platform user survey provides additional insights into how the various BI/RPT tools are used. Figure 1 – Gartner 2010 survey: Overall Rating of BI Platform Capabilities in Meeting Needs What is clear from this survey is that by far the greatest use of BI/PRT tools is for straight reporting (over 75% of the time), followed by ad-hoc queries (50% of the time). The uses of BI are likely to change as new tools are added and are found to provide benefit to the enterprise. An example of such a relatively new tool is the “Search-Based BI” which at present is little used, but which may hold much greater promise. 6 BI Platforms User Survey, 2010: Customers Rate their BI Platform Functionality, 5 February 2010 Rita L. Sallam, available at http://www.gartner.com/technology/media-products/reprints/microsoft/vol10/article9/article9.html The components of BI Before going through an analysis of what is required of BI reporting for our customers, BILozix present a general overview of some of the features of BI tools. Defining these is somewhat difficult, since BI means different things to different people. Forester Research, a 7 leading market survey company, in a recent paper provided the following diagrammatic representation of the components of BI: Figure 2 – Forrester’s BI architecture stack The report goes on to say that: Contrary to the common misconception, BI enables more than just reporting and analytics. Forrester deﬁnes BI as: A set of methodologies, processes, architectures, and technologies that transform raw data into meaningful and useful information used to enable more eﬀective strategic, tactical, and operational insights and decision-making. In Gartner’s BI review, Gartner define 13 capabilities of a BI package. Because of the neutrality of independent research companies such as Forrester and Gartner, in this report we will use this categorization. These 13 capabilities are then organized into three categories of functionality: integration, information delivery and analysis. Information delivery is the core focus of most BI today. The following categorization is taken from the Gartner report: Integration 7  BI infrastructure — All tools in the platform should use the same security, metadata, administration, portal integration, object model and query engine, and should share the same look and feel.  Metadata management — Not only should all tools leverage the same metadata, but the offering should provide a robust way to search, capture, store, reuse and publish metadata objects such as dimensions, hierarchies, measures, performance metrics and report layout objects. The Forrester Wave: Enterprise Business Intelligence Platforms, Q3 2008, July 31, 2008  Development tools — The BI platform should provide a set of programmatic development tools and a visual development environment, coupled with a software developer's kit for creating BI applications, for integrating them into a business process and/or embedding them in another application. The BI platform should also enable developers to build BI applications without coding by using wizard-like components for a graphical assembly process. The development environment should also support Web services in performing common tasks such as scheduling, delivering, administering and managing. In addition, the BI application should assign and track events or tasks allotted to specific users, based on predefined business rules. Often, this capability is delivered by integrating with a separate portal or workflow tool.  Collaboration — This capability enables BI users to share and discuss information and/or manage hierarchies and metrics via discussion threads, chat and annotations either embedded in the application or through integration with collaboration, analytical master data management (MDM) and social software. Information Delivery  Reporting — Reporting provides the ability to create formatted and interactive reports (parameterized) with highly scalable distribution and scheduling capabilities. In addition, BI platform vendors should handle a wide array of reporting styles (for example, financial, operational and performance dashboards) and should enable users to access and fully interact with BI content delivered to mobile devices.  Dashboards — This subset of reporting includes the ability to publish formal, Web-based reports with intuitive interactive displays of information, including dials, gauges, sliders, check boxes and traffic lights. These displays indicate the state of the performance metric compared with a goal or target value. Increasingly, dashboards are used to disseminate real-time data from operational applications.  Ad hoc query — This capability enables users to ask their own questions of the data, without relying on IT to create a report. In particular, the tools must have a robust semantic layer to allow users to navigate available data sources. These tools should include a disconnected analysis capability that enables users to access BI content and analyze data remotely without being connected to a server-based BI application. In addition, these tools should offer query governance and auditing capabilities to ensure that queries perform well.  Microsoft Office integration — In some cases, BI platforms are used as a middle tier to manage, secure and execute BI tasks, but Microsoft Office (particularly Excel) acts as the BI client. In these cases, it is vital that the BI vendor provides integration with Microsoft Office, including support for document formats, formulas, data "refresh" and pivot tables. Advanced integration includes cell locking and write-back.  Search-based BI — Applies a search index to both structured and unstructured data sources and maps them into a classification structure of dimensions and measures (often leveraging the BI semantic layer) that users can easily navigate and explore using a search (Google-like) interface. Analysis  OLAP — This enables end users to analyze data with extremely fast query and calculation performance, enabling a style of analysis known as "slicing and dicing." This capability could span a variety of storage architectures, such as relational, multidimensional and in-memory.  Interactive visualization — This gives the ability to display numerous aspects of the data more efficiently by using interactive pictures and charts, instead of rows and columns. Over time, advanced visualization will go beyond just slicing and dicing data to include more process-driven BI projects, allowing all stakeholders to better understand the workflow through a visual representation.  Predictive modeling and data mining — This capability enables organizations to classify categorical variables and to estimate continuous variables using advanced mathematical techniques. BI developers are able to integrate models easily into BI reports, dashboards and analysis.  Scorecards — These take the metrics displayed in a dashboard a step further by applying them to a strategy map that aligns key performance indicators with a strategic objective. Scorecard metrics should be linked to related reports and information to perform further analysis. A scorecard implies the use of a performance management methodology such as Six Sigma or a balanced scorecard framework. This categorization does not necessarily cover all of the BI vendor offerings. However, it represents the majority of what is currently termed as BI.