Portraits in Practice
Software Engineering Research & Development
Challenges at Siemens
Juergen Kazmeier
Brian Berenbach
Dan Paulish
Marlon Vieira
Page 1
Copyright 2007 © Siemens Corporate Research
Siemens has a long tradition of technological innovations.
Page 2
Copyright 2007 © Siemens Corporate Research
The rate of innovations is increasing.
Share of sales
with products…
… 5 years
and younger
… 6 to 10
years old
55%
48%
75%
30%
29%
19%
22%
16%
… more than
10 years old
1980
Page 3
1985
6%
2005
Copyright 2007 © Siemens Corporate Research
Siemens is one of the world’s largest software companies.
Page 4
►
Siemens has more than 30,000 software developers.
►
60% of Siemens’ business is based on software.
►
Siemens spends more than 3 billion euros per year on
software development.
Copyright 2007 © Siemens Corporate Research
But, Siemens is not recognized as a software company,
since most of our software is embedded.
Examples:
Automation devices
Industrial control systems
Automotive components
Communication systems
Rail systems
Medical devices
Page 5
Copyright 2007 © Siemens Corporate Research
Siemens at a Glance
Information and
Communications
Automation
and Control
Power
Transportation
Medical
Lighting
Communications
Automation and
Drives
Power Generation
Transportation
Systems
Medical Solutions
Sylvania
Siemens Business
Services
Industrial
Solutions and
Services
Power
Transmission and
Distribution
Siemens VDO
Automotive
Annual Worldwide Sales
Siemens Building
Technologies
Net Income: $2.8B
World’s 21st largest
company
Siemens AG: Worldwide figures for fiscal 2005 1 (U.S. GAAP2)
1Fiscal Year October 1 – September 30
2Average annual exchange rate for FY 2005: €1.00 =$1.273
Page 6
47,000 employees
$98.2 billion
$6.6B dedicated
to global R&D
75% of total sales are from
products and services
developed in the last five years
Copyright 2007 © Siemens Corporate Research
Global presence of Research and Development
More than 47,000 R&D employees at 150 locations
Page 7
Copyright 2007 © Siemens Corporate Research
Corporate Technology:
About 2,300 Researchers and Developers Worldwide
Berlin
St. Petersburg
Moscow
Romsey (RMR)
Princeton
Siemens
Corporate Technology
Beijing
A Global Network
Shanghai
Berkeley
Tokyo
Bangalore
Erlangen
Munich
Page 8
Copyright 2007 © Siemens Corporate Research
Much of our research is done in collaboration
with universities.
Page 9
Copyright 2007 © Siemens Corporate Research
Software Engineering Challenges at Siemens
 Functionality previously realized in electrical or electro-mechanical
systems is now being realized in software => bigger, more complex, &
more software projects (hundreds of developers, millions of lines of
code).
 Meeting functional and non-functional requirements is important to
business success => restricted hardware resources, real-time
performance, safety critical applications.
 Multisite development projects.
 High quality (i.e., thoroughly tested, reliable) software is important to
business success.
Our software engineering methods and technologies
must address the increasing scale and complexity of
emerging software systems.
Page 10
Copyright 2007 © Siemens Corporate Research
There are three major levers for optimizing
software development.
Processes
• Simplification
• Standardization
• Stabilization
Synergies
• Architecture
• Platforms
Structure
• Offshoring/
Outsourcing
• Site strategy
Page 11
Copyright 2007 © Siemens Corporate Research
We address software engineering challenges in R&D programs.
Today’s Portraits in Practice focus: Requirements Engineering,
Architecture Practices, Testing
Processes & Project Management
Requirements Engineering
Assess and improve software
development processes and
project management Process
From business needs to
software design faster, with
better quality and transparency
for project control and
Assessment &
oversight
Improvement
Requirements
Engineering
(Brian)
Project
Management
SCR SE
Quality
Engineering
Ensure high quality
Testing
software based
(Marlon)
systems by Testing,
Code Quality Management and
Performance Engineering
System & Software Quality
Page 12
Architecture
Practices, GSD
(Dan)
SW Platform
Technologies
Building
coherent, interoperable and
future-proof systems
Architecture & Platform Technologies
Copyright 2007 © Siemens Corporate Research
Thank you!
Contact:
Juergen Kazmeier
Department Head, Software & Systems Engineering
[email protected]
Phone +1 (609) 734-3610
Page 13
Copyright 2007 © Siemens Corporate Research
Requirements Engineering
@ Siemens
Brian Berenbach
[email protected]
Page 14
Copyright 2007 © Siemens Corporate Research
The Challenge
To provide effective requirements
development and management processes
for a set of extraordinarily disparate
Siemens companies.
To raise the level of expertise in
requirements engineering throughout
Siemens.
Page 15
Copyright 2007 © Siemens Corporate Research
For Example… Siemens Automotive
Chassis & Car body
Fleet Management
Infotainment
Interior
Marine Solutions
Special OEM Solutions
Power Train
Public Transport Solutions
Replacement Parts
Page 16
Copyright 2007 © Siemens Corporate Research
Customers Provide the Requirements
Stakeholder requests from
customers
in PDF documents
& Excel tables
Manual
Processes
Difficulty Managing Change!
Page 17
Requirements
Database
Copyright 2007 © Siemens Corporate Research
Requirements as State Tables
Again, Difficulty Managing Change!
Page 18
Copyright 2007 © Siemens Corporate Research
Customer
Siemens
Page 19
Copyright 2007 © Siemens Corporate Research
Unique OEM Problems
Customers deliver their requests in varying media
Everything is priority 1
Customers feel under no obligation to limit change requests
Change requests come in the form of new documents
(PDF) and state tables (Excel) with no change matrix. How
do you manage change and traces?
Requests may be vague because customers expect that
“Siemens will figure it out”.
Page 20
Copyright 2007 © Siemens Corporate Research
Providing Integrated Medical
Solutions Leads to Many
Requirement Dependencies
Page 21
Copyright 2007 © Siemens Corporate Research
“V” Model for Traceability
Impact analysis
System
requirements
Subsystem
requirements
Component
requirements
Derivation analysis
Stakeholder
requirements
Are all requirements
covered by the layer
below?
Acceptance
test plan
Derivation analysis
Impact analysis
System
test plan
Integration
test plan
Component
test plan
Are all requirements covered by tests?
Page 22
Copyright 2007 © Siemens Corporate Research
“V” Model leads to tag based tracing
Page 23
Copyright 2007 © Siemens Corporate Research
Tag based training has issues
No rationale for traces
Difficult to cluster traces
Effort rises geometrically with project size
Difficulty managing cascading or crosscutting changes
Page 24
Copyright 2007 © Siemens Corporate Research
Companies such as Siemens Building Technology build
products with distributed (worldwide) stakeholders and
development teams.
Page 25
Copyright 2007 © Siemens Corporate Research
How to capture functional and
non-functional requirements?
Desired Single Customer View but…
 Multiple Regions
 Multiple Business Units
 Everything must plug and play TOGETHER
Page 26
Copyright 2007 © Siemens Corporate Research
Global Challenges for Siemens
Conversion of products to product lines
Capturing requirements across regions and across
business units
Raising the RE skill level of staff world-wide
Page 27
Copyright 2007 © Siemens Corporate Research
 Challenges
One size does NOT fit all!
Effectively Controlling Change
Managing Scale
Managing Customers
Page 28
Copyright 2007 © Siemens Corporate Research
…and Some Possible Solutions
Traceability
Dynamic Tracing
Rich Traces
New Modeling Languages
Complexity, Scale (>10K
Requirements)
Model-Driven RE
RE Skills
Basic RE Courses
Advanced Tools.
Skill Certification
Page 29
Copyright 2007 © Siemens Corporate Research
Thank you!
Contact:
Brian Berenbach
RE Center of Competence Head
[email protected]
Phone +1 (609) 734-3395
Page 30
Copyright 2007 © Siemens Corporate Research
Methods & Processes for
Global Software Development
Dan Paulish
[email protected]
Page 31
Copyright 2007 © Siemens Corporate Research
Agenda
Motivation for Global Software Development
(GSD)
Challenges
An Approach
Lessons Learned
Open Issues and Research Questions
Page 32
Copyright 2007 © Siemens Corporate Research
Global Software Development (GSD):
Motivation
Lower development costs
• Low-wage countries
Capitalizing on a pool of trained workforce
• Quest for talent
Increased output, reduced time
• Improve time-to-market
• Round-the-clock development
Original motivation
was reduced
development cost,
but there are other
reasons for GSD.
Market proximity
• Specific local expertise
• Market acquisition effort
Governmental policies and incentives
Page 33
Copyright 2007 © Siemens Corporate Research
But, Global Software Development is Difficult.
Physical distance
Time Difference: 11.5
hours
Page 34
Copyright 2007 © Siemens Corporate Research
Global Software Development
Siemens has more than 15 years experience developing software products
globally.
Today, most Siemens business groups develop software in low-cost countries.
Organization and process model for global development divides responsibility
between a central headquarters organization & remote development teams.
Requirements
Domain
expertise
Management
&
Process
expertise
Software
Technology
engineering
expertise
expertise
Architecture
& design
Supplier Managers
Software
engineering
expertise
Development
experience
Development
plan
Acceptance
tests
Central team
Verified by
Remote development
teams
Component requirements
Component design
Integrated by
Code
Verified by
Tests
Page 35
Copyright 2007 © Siemens Corporate Research
Global Studio Project (GSP)
Experimental global software development project using university student teams and
researchers.
Shadows real Siemens global development project, process, & organization.
Document processes, best practices, and understanding of how to successfully execute
global projects.
University of Limerick,
Harvard Business School,
Ireland
USA
Carnegie Mellon University,
Technical University of
USA
Munich, Germany
Siemens Corporate
Research, USA Monmouth University,
USA
Pontifical Catholic
University of Rio de Sul,
Brazil
Page 36
International Institute of
Information Technology,
Bangalore, India
Copyright 2007 © Siemens Corporate Research
GSP Research Goals
Identify and document best (and worst) practices for
global software development.
Identify the prerequisites for successful global software
development.
Test a global software development reference process.
Determine artifacts for commissioning a remote
development team.
Identify communications necessary for effective global
development.
Page 37
Copyright 2007 © Siemens Corporate Research
Social Network Analysis (SNA)
Affiliations (Sites)
SCR Central Team
Tech. Univ. Munich
CMU Sapphire
IIITB
Monmouth #5
Monmouth Codicons
Monmouth TCT
Limerick ArdnaCroise
Roles
Developer
Team Leader
Function Manager
Remote Supp. Manager
Central Supp. Manager
Executive
Page 38
Copyright 2007 © Siemens Corporate Research
Some Lessons Learned
Amount of Required Communications
Cultural Differences among Teams
Start-up Time for Remote Teams
Supplier Managers as the interface to Remote Teams
Team Size
Agility
Page 39
Copyright 2007 © Siemens Corporate Research
Open Issues and Research Questions
1. Given that the technical artifacts that are delivered to the
remote component development teams are not adequate
in specifying the precise work to be done, in what ways
are they deficient?
2. What strategies do the remote teams employ to
compensate for the deficiencies found in the received
technical artifacts?
3. What are the early warning signs that an issue is
imminent? Can communication patterns, for example,
between the central and remote teams be used to predict
future component integration problems?
Page 40
Copyright 2007 © Siemens Corporate Research
Thank you!
Contact:
Dan Paulish
Distinguished Member of Technical Staff
[email protected]
Phone +1 (609) 734-6579
Page 41
Copyright 2007 © Siemens Corporate Research
Issues and Best Practices in Testing Automation
- An Industrial View -
Marlon E. Vieira
[email protected]
Page 42
Copyright
2007. AllResearch
rights reserved.
Copyright
2007©©Siemens
SiemensAG
Corporate
Agenda
 Motivation and Some Challenges
 Observations on Automation of Testing for Diverse
Siemens Projects
 Some Observable Best Practices
 Our Research Focus
 Suggestions on Research Topics
 Final Comments
Page 43
Copyright 2007 © Siemens Corporate Research
Automation of Testing: Motivation
Motivation
• Pressure to release software to the
market (with quality and on time)
• Some tests cannot be run without
automation
• Test teams have time to address
bigger test issues
• Improve tester job satisfaction
Page 44
Copyright 2007 © Siemens Corporate Research
Automation of Testing:
Some Challenges to Adopt It in Practice
Some Challenges
• Global Development
• Investment in the automation itself (ROI calculation)
• Testers with development skills
• Industrial scale solutions
Page 45
Copyright 2007 © Siemens Corporate Research
Automation of Testing:
Observations (Diverse Siemens Projects)
The test process and its automation in practice (Tools)
Plan
plan tests
test
Design tests
Test Management Tools
MBT Tools
Unit
High *
Page 46
Component
Medium *
* Automation level (usage)
Execute tests
Analyze results
Capture-Replay
Test Analysis Tools
Simulators/emulators
Defect Classification
Custom Frameworks
Integration
Medium *
Metrics Analysis
Validation
Low *
Copyright 2007 © Siemens Corporate Research
Other Observations:
Best Practices
 Observable Best Practices
• Dealing with testing automation as a
software development process
• Designing tests first before deciding to
automate
– Test creation (or generation)
separated from execution
• Applying an incremental approach
– Start small and grow - Provide
frameworks (libraries) and exercise
tests with different data
– Not everything is automated (allow
exploratory tests)
Page 47
Copyright 2007 © Siemens Corporate Research
Other Observations:
Emerging Strategies
 Emerging Industrial Strategies
• Test-driven development (Unit Testing Frameworks)
• Model-based testing
• Keyword-driven automation
Page 48
Copyright 2007 © Siemens Corporate Research
Our Research Focus on Automation of Testing
Model Based Testing
• Test case generation based on UML Analysis Models
• UML 2.0 Models – Use Case, Activity, Sequence and
Statechart diagrams
• U2TP (UML 2 Testing Profile)
Data driven test case generation
• Category-Partition Method
Architectural issues on flexible/robust testing frameworks
• Customizable framework to execute tests in a specific domain
Page 49
Copyright 2007 © Siemens Corporate Research
Suggestions for Research:
Improvements for Industrial Scale Problems
Testability
• Design changes (dynamic)
• Different environments
• Failure treatment (restart and
remote test execution)
• Visibility and Control (stimulus
and result verification)
Event-Triggered Real-time
Systems
• Non-deterministic execution
orders
• Impact of the execution
environment
Page 50
Copyright 2007 © Siemens Corporate Research
Final Comments
Automation of Testing must fit within
project and organizational context.
The industrial automation approach
needs to be affordable.
1. Usable and Scalable
2. Allow early results
3. Easy Evolution
A broad range of academic research is
currently being used in the software
industry.
Examples:
– Assertions
– Model-Based Testing
– Model Checking
Page 51
Copyright 2007 © Siemens Corporate Research
Thank you!
Contact:
Marlon Vieira
Program Manager, Software Testing
[email protected]
Phone +1 (609) 734-3313
Page 52
Copyright 2007 © Siemens Corporate Research