1. technology and computing
2. data centers

IBM SAP International Competence Center
Ciba demonstrates
how to implement a
complex, highly
available SAP
landscape without
wasting processing
capacity, using IBM
virtualization and
database
technologies
“The solution we have implemented is
highly flexible and makes optimal use of
our IT resources. The high availability
design has turned out to be extremely
efficient and easy to manage.”
Hermann Medam
Chief Architect
Ciba
Cover photograph: Ciba® Antioxidants extend the life of lubricants and allow longer drain intervals in industrial oils and engine oils.
This photograph and all other photographs in this brochure are © Ciba 2000-2007
2
Ciba demonstrates how to implement a
complex, highly available SAP landscape
without wasting processing capacity, using IBM
virtualization and database technologies.
About this paper
This technical solution brief describes the implemented SAP landscape for CIBA’s mySAP ERP 2005 based solution. It will show how
IBM virtualization technologies in POWER5 and AIX 5L, complemented by the IBM DB2 High Availability Disaster Recovery (HADR)
feature, combine to form a highly available and very resource-efficient infrastructure for CIBA’s complex SAP application landscape.
Customer Objectives
Customer Benefits
l Introduction of a New Global Single Instance “mySAP”
l IBM POWER5 and IBM System Storage technologies
implementation
help the customer
l Standardized approach for all SAP software
environments across the entire landscape
o
maintain a flexible yet easy to manage
infrastructure
l High availability and performance
share resources optimally between different
l Implementation of a disaster recovery site for legal
compliance
o
landscape components
l IBM DB2 allows the customer to minimize planned and
unplanned downtime due to
o
automatic and seamless failover to hot standby
IBM Solution
systems
l IBM POWER5 virtualization technologies for managing
manual failover capabilities for system
o
the different SAP environments efficiently and
maintenance
exploiting idle resources for peak load compensation
on-line administration capabilities
l IBM DB2 High Availability Disaster Recovery for hot
o
l End user satisfaction has increased due to improved
standby of production systems and automatic failover
response times and greater system availability
l IBM Storage DS8100 for high performance data
l Overall operational cost has been reduced due to
access and flexible storage allocation
lower licensing costs, better resource utilization and
l Libelle BusinessShadow for SAP complements the
streamlined operations.
disaster recovery concept with long distance
replication capabilities.
3
4
Background, starting point and objectives
About Ciba
The need for high availability
Ciba Specialty Chemicals is a leading global company
Consolidating worldwide systems into a single SAP software
dedicated to producing high-value effects for its customers’
solution also increases the need for 24x7 availability. Any
products. Ciba creates effects that improve the quality of life –
downtime – whether planned or unplanned – can have a
adding performance, protection, color and strength to plastics,
significant impact on operations, so a dedicated high availability
paper, automobiles, buildings, home and personal care
architecture is required to ensure that systems remain online at
products and much more. Ciba brings new and creative thought
all times.
to the processes and products of customers in more than 120
countries.
Extending the high availability requirements, Ciba wanted to
create a disaster recovery (DR) site in a different tectonic zone,
Ciba Specialty Chemicals is headquartered in Basel,
more than 100km away from the regular Ciba IT centers. Ciba
Switzerland. Switzerland also hosts key research, development
needed to be able to replicate the entire solution to the off-site
and production centers. More than 2,700 of its 15,000
location, while still keeping the same operational guidelines for
employees work in one of the four Swiss sites – Basel, Kaisten,
both the main site and the DR site.
Monthey and Schweizerhalle. These sites play an important role
in management, research and development, production,
marketing and sales for the whole company.
Moving to mySAP ERP
With operations in many countries, Ciba decided in 2005 to
replace its legacy systems with a single globally consolidated
solution based around mySAP ERP 2005. Exploiting most of the
capabilities that SAP NetWeaver and the mySAP ERP
application suite offer, one can imagine the complexity of the
underlying server infrastructure and high availability
architecture.
Considering that an SAP software landscape always requires
additional development, test, pre-production and education
systems, operational excellence will depend on designing a
system architecture that allows consistent management of the
various environments and their resource consumption.
5
6
Guiding Principles for the landscape architecture
The joint Ciba and IBM architecture team established a set of
IBM DB2 High Availability Disaster Recovery (HADR)
guiding principles to drive the design decisions during the
The integrated solution for hot-standby databases within DB2,
architectural phase.
HADR eliminates the most important single point of failure in the
high availability concept. IBM DB2 HADR allows the
l Highly centralized global infrastructure with the same
synchronous replication of any changes from the production
configuration for all countries
system to the backup system. In case of an automatic failover
l Highest possible application performance
due to an incident on the production server, the standby
l Flexibility to changing demand
database is already “up and running”, so application
l Emphasis on simple operation and management
components can simply connect to the failover server and
continue to work.
The innovative solution design implemented at Ciba combines
IBM Tivoli Storage Manager (TSM)
different IBM technologies to serve these established principles.
IBM TSM complements the Backup/Recovery and Archiving
IBM POWER5 Virtualization
architecture by providing a consistent, integrated interface for
Dynamic Logical Partitioning (DLPAR) allows Ciba to
DB2, enabling the centralized management of all backup and
consolidate all the different SAP software components onto a
archiving processes for the entire SAP software landscape.
single server, while still being able to maintain each component
as a single entity. In addition, DLPAR manages the different load
balancing requirements across the SAP software components
and still allows dedicated resource allocation when required. By
virtualizing CPU and I/O capacities, the systems have the
flexibility to accommodate changes in workload characteristics,
and are able to scale to meet growing demand. In addition, with
all SAP software components residing on a single server,
network traffic is minimized – resulting in better overall
performance.
7
Architectural Details: High Availability
In this section we will look at various design decisions that Ciba
every year, it provides an opportunity to practise the procedures
made, and how these have affected the system landscape. In
for failover to the secondary system. In case of an unplanned
some areas the approaches taken are new, and it took several
system failure, these tested and verified procedures are
iterations before the current implementation was achieved.
executed automatically.
High availability design
Eliminating single points of failure
High availability has been a key requirement, as the entire
Ciba decided to implement a two data center strategy to
worldwide operation depends on the centralized systems. The
eliminate single system failure (e.g. power outage, fire etc), and
design needed to cover two distinct cases: first, the unpredicted
in addition to implement a disaster data center for major outages
failure of system components (either hardware or software), and
(earthquake, chemical disaster in the primary location etc). Each
second, the planned downtime required for system and
data center is equipped with a single high-capacity System p
component maintenance. It was a clear goal to have a single
server that can run the entire production landscape. For
common procedure to handle both cases. As system
workload balancing, the production workload is split across the
maintenance is a regular task that is performed multiple times
two data centers (see Diagram 1 and Diagram 2)
High Availability in DC1 & DC2
DC1 in Basel
DC2 in Basel
SAP-License
Database Server /
Enqueue Server /
Application Server
Database Server (back-up) /
Enqueue Server (back-up) /
Application Server
Enterprise LAN
GB-Ethernet
Network
HACMP
PC
Director
Fiber Channel
Storage Area Network
PC
Director
PC
Director
PC
Director
HADR Synchronization
IBM DS8000
IBM DS8000
Diagram 1: two data center approach with server and storage clustering
8
Architectural Details: High Availability
As one can see in Diagram 2, the SAP software environment
Technical implementation of the fail-over procedures
consists of 10 independent SAP instances (one for each major
As shown in Diagram 1, CIBA is using HACMP to monitor each
component). To simplify its fail-over scenarios, Ciba has decided
single resource in the production environment. The complete
not to split each instance into two or three tiers, as is usually the
LPAR environment (operating system, SAP software etc) is
case with SAP software environments. As a result, the database
located on a Fiber Channel attached storage environment (IBM
server, the central instance and all application servers required
DS8100), and mirrored through the Logical Volume Manager
for a single SAP component (e.g. ERP) run in a single AIX LPAR.
(LVM) to the secondary site. If HACMP detects one of the
resources required for the whole operation is failing, it will
This also means that all single points of failure in the SAP
automatically start the fail-over procedure.
component stack (e.g. database, SAP enqueue server etc.) can
be eliminated with a single fail-over procedure. The only impact
The only exception to the mirroring with LVM is the database
this simplification has is that in the event of a fail-over, all users
component. In order to achieve the required SLA for component
will need to reconnect to their SAP applications.
fail-over, CIBA decided to implement a hot-standby database
approach. With hot-standby the fail-over for the database
components is reduced from minutes to seconds.
System 1
System 2
ERP
SRE
GTS
SCA
EP
ERP
SRE
GTS
SCA
EP
BI
CPR
XI
ISA
EMS
BI
CPR
XI
ISA
EMS
Diagram 2: workload balancing (red) and failover (yellow) for SAP components
9
Architectural Details: High Availability
IBM DB2 High Availability Disaster Recovery (HADR)
As mentioned above, this high availability concept is also used
DB2 HADR helps to automate and maintain a hot-standby
to minimize downtime for software maintenance. In case of
database, and the solution is fully supported by SAP. For each
software upgrades (AIX, DB2 or SAP) the existing LVM mirror will
primary database (production) one can specify a secondary
be split, the secondary site will be upgraded and tested, and the
database (hot-standby) and the mode of synchronization
fail-over procedure will be activated to force the user onto the
between the two. Ciba selected the synchronous mode, which
new systems. Then the back-level primary site will be upgraded
will only commit changes in the primary database when these
and the LVM and HADR setup will be resynchronized.
changes have also been applied to the secondary database.
VIP
CI Instance (A)
CI Instance (J)
Idling
(A) – ABAP Stack
VIP
DB Instance
SAP Component
Idling LPAR
Active
(J) – JAVA Stack
VIP – Virtual IP
DB Instance
CI Instance (A)
VIP
VIP
SAP Component
Active LPAR
CI Instance (J)
SCS Inst. (A)
VIP
VIP
SCS Inst. (A)
SCS Inst. (J)
AI Instance (A)
AI Instance (A)
VIP
VIP
SCS Inst. (J)
AI Instance (J)
AI Instance (J)
CI
LVM Mirror
CI
DB
HADR
DB
Diagram 3: high availability concept for a single LPAR (SAP component stack)
10
Architectural Details: Disaster Recovery
Disaster Recovery
In addition, the computing capacity at this site is used to provide
In addition to the second data center for high availability, Ciba
pre-production and quality assurance for the SAP software
also needed to implement a disaster recovery (DR) site around
landscape. In case of a disaster, these computing resources will
100km away, in a different tectonic zone. In order to replicate any
be transferred to the production LPARs. As a disaster usually
changes in the production environment to the DR site, CIBA
requires a lot of critical decisions to be made, the whole fail-over
decided to use Libelle BusinessShadow for SAP.
to the disaster recovery site, including allocation of hardware, is
based on manual procedures.
This replication technology allows for database log-file shipment
(cold-standby) and file system synchronisation at the same time.
In addition Libelle BusinessShadow for SAP compresses the
data before transferring it to the DR site, which enables the use of
lower bandwidth connections.
11
Architectural Details: Virtualization
Virtualization using IBM System p5
The consolidation of the entire SAP software environment onto a
Each of the boxes in Diagram 2 above represents a logical
single machine also enables the different LPARs to share I/O and
partition (LPAR) within the large p5-590 production systems. All
network adapters. This again makes configuration and change
LPARs in Ciba’s environment share all the available CPUs. This
management considerably easier. If, for example, Ciba needs to
mode, called “uncapped LPAR”, will virtualize all physical CPUs
add a partition for a new component, it only has to define the
and simulate a defined number of CPUs for each partition. In
resources needed – there is no need to interrupt the running
consequence, the software in the LPAR will only “see” the
system.
number of CPUs that are defined.
In addition, the LPARs within the machine can communicate
Resources are shifted between LPARs based on their actual
through dedicated memory areas, which reduces network traffic
consumption. For example, the ERP LPAR has a defined CPU
and increases speed, delivering better performance across the
number of 4 CPUs, but if it is only using the capacity of 2 CPUs,
whole system.
the additional resources can be utilized by the LPAR running the
BI solution. The total average utilization of the machine can
therefore be increased, as it is possible to define more “virtual”
CPUs for each physical CPU (see Diagram 4).
RRDTOOL / TOBI OETIKER
aggregate CPU Usage information
6
5
CPUs
4
3
2
1
0
Mon
Tue
Wed
Thu
Diagram 4: CPU usage information aggregated across the different LPARs
12
Fri
Sat
Sun
Architectural Details: Backup
Backup Concept
with the TSM server. This integration will minimize the effort of
The final building block for Ciba’s high availability and disaster
administration and reduce the risk of losing critical files for the
recovery architecture is the backup concept, which is currently
restore operation.
being implemented. IBM Tivoli Storage Manager (TSM) will be
used to set up a fiber-channel based (LAN-Free) backup
Ciba is also currently investigating the use of the IBM FlashCopy
solution for the major SAP components. Data will transferred
functionality built into the IBM DS8100. Using FlashCopy to back
from the storage systems directly to a tape library managed by
up the production server will further minimize the impact on the
TSM, minimizing the impact on CPU and I/O within the
production landscape. FlashCopy will also be used at the
production environment.
disaster recovery site to refresh the SAP pre-production and QA
systems with the latest image of the production environment.
For on-line database backup, CIBA will rely on a combination of
different backup procedures available in DB2 to deliver
increased flexibility and optimized restore times. During a restore
procedure, the database will communicate directly with the IBM
TSM server to acquire all necessary files, including the database
log files. Log file management in DB2 will also be synchronized
Storage system for
Active DB
2nd Storage
System
DB2 database
DB2 database
HADR mirror
DB2 Logs
DB2 Logs
DS8100 - 1
DS8100 - 2
Datacenter 4
Tape Library
Log shipping
(asynchron)
Tape Library
Datacenter 3
(Zurich)
Storage system
Diagram 5: database backup architecture
13
“IBM System p virtualization and IBM
DB2 High Availability Disaster Recovery
are two major building blocks in our
centralized global SAP architecture. These
features have allowed us to implement a
highly available, cost-effective and
manageable infrastructure for our SAP
software landscape.”
Summary
Ciba has taken an innovative approach to the implementation of
a global SAP software environment. The company’s strong focus
on manageability and high availability has led to an architecture
that exploits the leading-edge capabilities of IBM’s hardware
and software. Consolidating all SAP components onto a single
IBM System p server, and using virtualization to manage each
SAP component individually has also led to greatle increased
Hans Hamburger
utilization of CPU and memory resources.
Lead Architect
Ciba
By focusing less on single points of failure and more on total
system availability, the new high availability architecture greatly
simplifies Ciba’s failover scenarios, enabling the company to use
the same fail-over procedures for both un-planned system
outages and planned system maintenance downtime.
In addition, by exploiting the IBM DB2 High Availability Disaster
Recovery feature and the integrated IBM Tivoli Storage Manager
backup interfaces, Ciba will be able to minimize fail-over time
and operational effort.
Overall, the architecture implemented at Ciba is the perfect
exemplar of a modern, cost-effective SAP software roll-out.
15
For more information:
To learn more about the solutions from IBM
and SAP visit: ibm-sap.com
For more information about SAP products and
services, contact an SAP representative or
visit: sap.com
For more information about IBM products and
services, contact an IBM representative or
visit: ibm.com
For further questions please contact the IBM
SAP International Competency Center via
[email protected]
© Copyright IBM Corp. 2007 All Rights Reserved.
IBM Deutschland GmbH
D-70548 Stuttgart
ibm.com
Produced in Germany
IBM, the IBM logo, IBM System z, IBM System p,
IBM System i, IBM System x, BladeCenter, z/OS,
z/VM, i5/OS, AIX, DB2, Domino, FlashCopy, Lotus,
POWER, POWER5, QuickPlace, SameTime, Tivoli
and WebSphere are trademarks of International
Business Machines Corporation in the United
States, other countries, or both.
UNIX is a registered trademark of The Open
Group in the United States and other countries.
Linux is a trademark of Linus Torvalds in the
United States, other countries, or both. Microsoft,
Windows, Windows NT, and the Windows logo are
trademarks of Microsoft Corporation in the United
States, other countries, or both. Other company,
product or service names may be trademarks, or
service marks of others.
This brochure illustrates how IBM customers may
be using IBM and/or IBM Business Partner
technologies/services. Many factors have
contributed to the results and benefits described.
IBM does not guarantee comparable results. All
information contained herein was provided by the
featured customer/s and/or IBM Business
Partner/s. IBM does not attest to its accuracy. All
customer examples cited represent how some
customers have used IBM products and the
results they may have achieved. Actual
environmental costs and performance
characteristics will vary depending on individual
customer configurations and conditions.
This publication is for general guidance only.
Photographs may show design models.
GK12-4318-00 (11/07)