A Look into Rational Reporting Options: How to Meet Your Organization's Reporting Requirements Carlos Ramirez Senior IT Specialist Rational Software IBM Canada Do You Have the Information You Need to Make the Right Decisions? management space −  Reduce escalating cost of information gathering −  Reduce risk by identifying trends before they become issues −  Align SSD activities with strategic and operational objectives   But are still struggling with: −  Accessing heterogeneous data sources −  Managing by exception −  Analyzing progress against objectives −  Implementing and measuring proven and repeatable processes to achieve expected business results − Generation of mandatory and compliant software documentation Activity Models Alerts Plan Monitor Improve Dashboards Value Measures Key Performance Indicators engineering space   Businesses need a mechanism to: In-house Projects Outsource Projects Packaged App Projects How important is measurement? Statistical outcomes for projects with strong and weak measurement practices: Strong Year 1 Year 2 Year 3 Year 4 Year 5 Weak On-time projects 75% 45% Late projects 20% 40% Cancelled projects ROI for Software Measurement: 5% 15% Client satisfaction Optimistic Higher Lower 1. Unsa tbel,changnigrequreimenst 95% 2. Inadequate quality control and poor quality measures Fortune 500 firms with quality measures: 45% Fortune 500 firms with complete measures: 15% Number of software projects measured: 160,000 Number of software projects not measured: 50,000,000 90% 3. Inadequateprogresstrackn ig 85% Staff morale Higher Lower _____________________________________________ Fortune 500 firms with productivity measures: 30% $ 4.5 $ 6.25 $ 8.75 $ 11.50 $ 15.00 Top Reasons for Software Litigations: Defect removal 95% < 85% _____________________________________________ Resource estimates Accurate Cost Return 5% 4% 4% 4% 3% 4. Inadequate cost and schedule estimating 80% 5. False promises by marketing and sales personnel 80% 6. Optimistic schedule estimates or arbitrary dates 75% 7. n Io frma,lunsru tcu treddeveolpment 70% 8. Inexperienced clients who can't articulate requirements 60% Source: Capers Jones, Measurement, Metrics and Industry Leadership, 2009 and Software Engineering Best Practices, McGraw Hill, 2010 Inexperienced project managers 9. 50% 10. Inadequate tools for quality, static analysis, inspections 55% 11. Reusnigmae tarisleld if wh ti bugs 30% But measurement is difficult amongst the chaos… Executive Management Achieve Profit Market Share Headcount Revenue Expense Products Time to Market Performance Enablement Productivity Project Schedule Code Coverage Iteration Burndown Build Health Defect Density Business Alignment Readiness Feedback Satisfaction Deployment Requirements Churn Tests Passed/Failed Code Churn Defect Age Tests for Requirements Quality Renewals Defects Consumability Calls Continuous Integration Tests per Build Work Item Completion Code Growth Defect Priority agile test driven development change management iterative change set bill of materials VELOCITY dashboard WORK ITEM code scan test script requirements project schedule team meeting chat waterfall UNIT TEST Development Artifacts requirement build definition defect TEST CASE deploy image change set bill of materials STREAM custom dashboards design model WORKSPACE Eclipse plug-ins test script SNAPSHOT development assets Demo image Business Objectives Operational Objectives Processes & Practices Development Intelligence Development Intelligence improves predictability by: –  Applying Business Intelligence techniques to software and systems development –  Enabling fact-based decision making (to communicate status, monitor progress, diagnose problems, identify corrective actions) –  Steering projects and programs to deliver on-time 5 How to Meet Your Organization's Reporting Requirements …   Understand the Reporting Requirements, stakeholders, priorities, needs, etc.   Types of reports needed   Reduce escalating cost of information gathering   Reduce risk by identifying trends before they become issues   Raise enterprise visibility and transparency to reduce costs and risk   Measures the effectiveness of processes and practices to improve organizational and business outcomes Make the right decisions at the right time 6 But What Should We Measure? An Example Set of Candidate Lean Metrics…..lots of possibilities! Executive Dashboard Project Health   Defect Backlog   Defect Density                       Defect Repair Latency Build Health Project Velocity Staffing Actuals Process Timeliness Milestone Status Severity Analysis Security Vulnerabilities Static Code Analysis Requirements Met IPD Timeliness   Transactional Survey   PMR / Call Rates   Defect Backlog   Test Escapes Critical Situations Cost of Support Installability Enhancement SLA Useability Consumability Perceived Performance Scalability Integrations with other products   User Experience / Doc Time to Resolution         Functional Test Trends Critical Situations System Test Trends S-Curve Progress             Automation Percentage Customer Testcases Consumability Scorecard Defect Latency Quality Plan Commitments Test Coverage                   Evolutionary Architecture Vulnerability Assessment Concurrent Testing Strategic Health Development Quality Customer Quality Practices   Sales Plays   Partner Enablement                       Support Enablement Technical Enablement Sales Enablement Localization MCIF Index Competition Integrated into Story Green Threads LCM Pipeline / Multiplier Revenue Test Driven Development Whole Team Team Change Management Requirements Management From In Process (Team) To Executive Value Appropriate Metrics for Each Management level Dimensions Time-to-Value (Schedule) Product Value Product Cost Team (In Process) Middle Management Development Executive (Development Mgmt.) (VP Development) User Story Points / Use Case Points Iteration Burndown, Blocking Work Item Release Burndown Iteration Velocity Stakeholder Feedback, # of Enhancement Request, Age of Enhancement Request Tested and Delivered Requirements, Business Value Velocity, Customer Satisfaction Effort (Man-hours) Cost / Unit of work Development / Maintenance Costs Product Quality Predictability Technical Debt (Defect trend, defect density) Test Status, Test Coverage of Requirement, Test Execution Status Quality at Ship User Story Points / Use Case Points Planned/Actual Cost and Velocity Trend Variance. Likelihood of on-time delivery Note: Bold indicates that there is Out-Of-The-Box report supported by Rational tools 8 Measuring Time-to-Value (Schedule) Iteration Burndown Showing how much work is left to do in an iteration. It enables the team to adjust scope or resources to finish the iteration successfully. Release Burndown Shows the estimated functionality remaining to complete the current release. 9 Measuring Product Value Enhancement Request Trend Shows the trend of enhancement requests received, approved, and closed during the project lifecycle. A high number of enhancement requests can indicate that the system is not functioning as stakeholders expected. Age of Enhancement Requests Tracks the length of time stakeholder enhancement requests remain open. Unaddressed requests can impact the stakeholders' perception of value. 10 Measuring Product Cost Cost per unit of work Tracks the cost of delivering a single unit of work (such as a user story point or use case point) across iterations. Monitoring this metric in each iteration helps the team understand if they are burning budget at a sustainable rate. 11 Measuring Product Quality Defect Trends Shows defect arrival and closure rates, determines the remaining defect backlog, projects the future defect arrival/close rate up to and post-ship Test Execution Status Monitors test completion and success 12 Measuring Predictability Velocity Variance Indicates ability of team to deal with uncertainty of the iteration. This helps the team identify the degree to which variance across the lifecycle might impact the cost and schedule of the project. 13 Use dashboards to provide that ONE view of project health 14 IBM Software, Rational Rational Reporting Solution What do we mean by “Reporting” “Reporting” refers to two broad categories of information delivery Business / Development Intelligence Why: - To view, monitor, and answer questions about project and program health and status. - Enables deeper analytics to discover root causes and determine corrective actions What: - Typically information about the development data to show status and progress towards predefined goals and/or thresholds How: - Interactive web based reports, charts, graphs often showing aggregated and/or calculated results - Progressive disclosure via alerts, drill-down, and drillthru Documentation Why: -  To demonstrate compliance, communicate across contractual boundaries, support offline review of development data, and communicate development data with the extended team What: -  Reports typically contain the development data itself mixed with supplementary text (e.g., Requirement/Design Specification) How: -  Typically in highly stylized and formatted documents. Reporting Space use work product Systems and Software Documentation: e.g. RPE prose numbers graphics Business Reports: e.g. Insight/Cognos, BIRT, Crystal Reports management tool content Solution Evolution Past/Present PjC and Dashboard Product Specific Solutions 2010 > Present Rational Insight Powered by Cognos Rational Reporting for Development Intelligence Powered by Cognos BIRT SoDA and DocExpress Rational Publishing Engine (RPE) Work together, with a common architecture Architectural Base   Data Acquisition via REST Services –  Each product provides an HTTP REST service to extract and deliver data in XML –  Insight supplies “Data Services” for products that don’t have a built in REST Service –  Provides data to Rational Data Warehouse, RPE, and live data reports   OSLC: Emerging standard for representing common artifacts –  Defect –  Requirement –  Etc   Common Data Warehouse –  Schema to be used by all products –  Supported on commonly available databases: DB2, Oracle, SQL Server   One set of data presentation technologies used behind the scenes –  Cognos for reports/charts –  RPE for Documents –  BIRT: Legacy and “not customizable” charting Metadata Model Rational Data Architecture Cognos reporting REST RPE BIRT reporting defect Metadata modeling resource url for drill-through customer project XML-relational mapping Rational Data Warehouse ETL Relational View Operational Data Store relationship for drill-through Data Marts Major Reporting Technologies & Solutions  Technologies – Cognos – Rational Publishing Engine (RPE)  Solutions – Rational Reporting For Development Intelligence (powered by Cognos technology) For Document Generation (powered by RPE technology) – Rational Publishing Engine (RPE) – Rational Insight (powered by Cognos technology) Rational Insight Portfolio Management Project Management Pull the big picture together and present to multiple levels in the organization CxO Requirements Management Architecture Management Change Management Process Lead Configuration Management Build Management Quality Management Project Manager Rational Insight High-Level Architecture Requirements Management Architecture Management Change Management Configuration Management Project Management Build Management Portfolio Management Quality Management REST API - native REST API – adapter XML XML – ODBC Mapping DataStream DataStream Open Data Architecture Enterprise Data Warehouse Extract, Transform, Load DB Access Reporting Server RequisitePro TestManager RTC RQM FocalPoint DOORS XML MS Project MS Excel Data Warehouse Reporting Model / Metadata CxO 23 | Supported in Insight V1.0.1 ClearCase Web architecture to retrieve product data in XML via REST Industry Proven BI Platform ClearQuest DB Process Lead Project Manager HP Quality Cntr CAST Self Service Report Authoring Rational Insight Architecture – Report Execution Legacy Products ClearQuest ClearCase RequisitePro … Jazz-based Products … Telelogic Products DOORS FocalPoint … 3rd Party Products MS Project … WebSphere Application Server 6.1 RQM Rational Insight Data Services RTC(ODBC) Rational Insight Reporting Server R E S T S e r v i c e s Rational-branded Cognos Web Portal UI XML/ODBC Cognos 8 BI Reporting Services Cognos 8 BI Data Manager WebSphere Application Server 6.1 ODBC Rational Data Warehouse Rational Insight - Summary and Benefits   Reduce time spent manually collecting, correlating, and analyzing data –  Out-Of-The-Box integrations to Rational and 3rd party tools using an open REST based integration layer optimized for LAN and WAN performance   Enhance visibility into disparate teams and applications –  Pre-build library of SDLC KPIs and measurements provide transparency into a distributed and heterogeneous software development environment   Improve analysis for required for strategic and operational decisions –  Measurement frameworks such as CMMI, Six Sigma, and the OOTB Rational MCIF content provide correlation between development activities and business level objectives   Measure current process efficiency and process improvement effectiveness –  Leverage OOTB KPIs and dashboards to determine the effectiveness of process improvements Rational Publishing Engine: document automation across the development lifecycle   Create documents from data across a wide range of distributed and diverse data sources   Access data from a wide range of Rational tools, including:   Asset Manager, DOORS, ClearCase/ ClearQuest, Focal Point, Quality Manager, RequisitePro, Rhapsody, Requirements Composer, System Architect, Tau, Change   Access data from third party tools via XML and REST interfaces Rational Publishing Engine: automate your document creation process   Document Creation Made Simple Usability   High Quality, Compliant Documents   High Performance Document Generation Rational Publishing Engine Speed Flexible output Rational Publishing Engine: document creation made simple Simple creation of reusable templates Usability   Document Studio interface with intuitive drag and drop template building capability   Integrated preview capability avoids wasted time in validating template changes   Powerful scripting language support (Javascript) with expression editor for ease of use   Create single templates to generate all different types of output format Rational Publishing Engine: document creation made simple Simple generation of complex documents   Launcher interface to quickly and easily control document generation Usability   Also available as a wizard embedded into some data sources   The user generating the document requires no domain knowledge   Use out-of-the-box templates for quick ROI Rational Publishing Engine: high quality, compliant documents   Template driven document generation ensures the output complies with any required formatting standards and contains correct branding and legal information   Support for multiple output formats (MS Word, HTML, PDF, XSL-FO…) Flexible output Rational Publishing Engine: high quality, compliant documents  Support for a huge range of document features  Richly formatted text  Images  Tables  OLEs  Tables of Contents  Figures & Captions  Headers & Footers  Page size & Orientation  Hyperlinks – Internal or external  Full multi-cultural support  Conditional formatting  Etc… Flexible output Rational Publishing Engine Benefits   Increase productivity by allowing engineers to focus on engineering, NOT formatting concerns   Enhance documentation quality and consistency by sharing and reusing templates   Gain quick ROI with out of the box templates and a user community to share templates   Easily implement your own document standards with intuitive Template Editing environment with drag and drop capability   Avoid wasted engineering time waiting for documents to be generated with an easy to use, high performance and low resource use solution 32 www.ibm.com/software/rational 33 © Copyright IBM Corporation 2013. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others. 34