Business Continuity Planning & Disaster Recovery Solutions Continuous
Business Continuity Planning & Disaster Recovery Solutions Achieving Continuous Availability Introduction 3 Today’s business continuity problem 5 Planning for disaster recovery and business continuity 8 Regulatory squeeze and ubiquitous deployment 10 The ideal enterprise continuity solution 13 Achieving continuous availability with Sybase 18 Conclusion 24 Appendix: Business Continuity Planning and Disaster Recovery in practice 25 First Published 2002 www.datamonitor.com Datamonitor USA 1 Park Avenue 14th Floor New York, NY 10016-5802 USA Datamonitor Europe Charles House 108-110 Finchley Road London NW3 5JJ United Kingdom Datamonitor Germany Messe Turm Box 23 60308 Frankfurt Deutschland Datamonitor Asia Pacific Room 2413-18, 24/F Shui On Centre 6-8 Harbour Road Hong Kong t: +1 212 686 7400 f: +1 212 686 2626 e: [email protected] t: +44 20 7675 7000 f: +44 20 7675 7500 e: [email protected] t: +49 69 9754 4517 f: +49 69 9754 4900 e: [email protected] t: +852 2520 1177 f: +852 2520 1165 e: [email protected] ABOUT DATAMONITOR Datamonitor plc is a premium business information company specializing in industry analysis. We help our clients, 5000 of the world’s leading companies, to address complex strategic issues. Through our proprietary databases and wealth of expertise, we provide clients with unbiased expert analysis and in-depth forecasts for six industry sectors: Automotive, Consumer Markets, Energy, Financial Services, Healthcare, Technology. Datamonitor maintains its headquarters in London and has regional offices in New York, Frankfurt and Hong Kong. All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior permission of the publisher, Datamonitor plc. The facts of this report are believed to be correct at the time of publication but cannot be guaranteed. Please note that the findings, conclusions and recommendations that Datamonitor delivers will be based on information gathered in good faith from both primary and secondary sources, whose accuracy we are not always in a position to guarantee. As such Datamonitor can accept no liability whatever for actions taken based on any information that may subsequently prove to be incorrect. Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 2 BCP & DR – Achieving Continuous Availability Introduction As a solution, Disaster Recovery has been around for some time - the need to back up IT assets in the event of incidents such as flooding, fire, power failure or other physical and electronic attack is clear. Business continuity is an extension of this to ensure processes and transactions can continue under all circumstances. Recent events – notably those of the September 11th 2001– have made it clear that BCP and DR (Business Continuity Planning and Disaster Recovery) is increasingly vital for all large enterprises. The solutions themselves have also moved on; it has become apparent that classic back-up and restore where tapes containing data assets would be shipped to a remote location is no longer feasible. The complexity of the modern IT environment exposes it both to an increased internal technical risk of failure as well as external risks, in addition to the need for continuous data access in a ‘real-time’ IT world. There has been massive investment over the past few years in enterprise applications, such as CRM, ERP and SCM, to name but a few. This investment has greatly affected business continuity and disaster recovery, as many organisations are integrating their business processes with those of their customers, suppliers and business partners. Recover times have shrunk to minutes and hours, and in some cases moved to zero – this means 24x7x52 continuous business process availability. Furthermore, scenario plans have broadened to take on the new risks of eBusiness, for example, downtime due to: • operational risk (such as the Microsoft.com three-day outage); • security risk (denial-of-service attacks bringing down Yahoo); • lack of capacity (the launch of the UK Governments 1901 census site resulted in a 1.2 million hits per hour causing the site to crash and be withdrawn for several months); • application failure (such as the full day outage last year by the London Stock Exchange); • partner / outsourcer unavailability (such as ISP network failure or links from a web site to those of partners that are unavailable); • natural disasters (such as floods on the River Oder and the Rhine, earthquakes in Bulgaria and Turkey); Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 3 BCP & DR – Achieving Continuous Availability • terrorist attacks (such as those common to the United Kingdom and Spain). Any downtime risk must be a concern to every business, as any downtime today results in a press event which could impact the image and reputation of the enterprise. Therefore, the aim of this paper is to increase the awareness and understanding of the benefits surrounding Business Continuity and Disaster Recovery technologies, by: • discussing the increasing need for BCP and DR solutions by organisations; • highlighting the business benefits of business continuity solutions; • outlining the various technology solutions that can be used to achieve BCP and DR; • explaining the advantages of using a 3-tiered, continuously available softwarebased solution. Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 4 BCP & DR – Achieving Continuous Availability Today’s business continuity problem The issue surrounding business continuity planning (BCP) is essentially the same for a traditional bricks and mortar company or an innovative ‘dot com’; business processes and information systems must be continually available, 24x7x52. However, what has changed over time is the speed in which recovery is required and the increased willingness of competitors to capitalise on a company’s downtime. With increasing reliance on electronic markets and the impact of natural disasters and criminal activity on the company’s technology base, its supply chain and the customer base, companies are becoming more and more concerned about business continuity planning. The main drivers and factors that have created the need for comprehensive business continuity solutions are: • the global nature of modern business practices; • corporate, internal and external supply chain interdependency • speed and timeliness • IT-dependant business processes; • the increasing value of data; • legislation / regulation. Business continuity planning means formalising a company’s strategy for dealing with the unexpected and unknown by planning, training and testing for the recovery of critical business processes and IT systems in a timely fashion to minimise the impact of any disruption on the business and the customer. Typical natural disaster scenarios addressed by many businesses include fire, floods, tornadoes, hurricanes, earthquakes, snow / ice and extreme heat. However, many businesses are willing to accept the risk of the above natural disasters due to their perceived unlikelihood and hence they feel business continuity planning is not necessary. Many threats that have nothing to do with geographic location or threat of natural disaster are often overlooked, such as: • work-place violence; • terrorism; • workforce unavailability; Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 5 BCP & DR – Achieving Continuous Availability • computer / Internet based crime (including denial of service attacks); • geographical restrictions caused by events, such as chemical contamination; • power outage; • computer viruses; • telecommunications failure; • asset malfunction (including hardware and software failures); • human error. Business continuity planning is not simply ‘ticking a box’ by purchasing a hot-site contract or a business continuity planning software package. Similarly, having a documented plan is not enough. In the beginning As illustrated in Figure 1, the modern concepts of disaster recovery and business continuity were hatched around the beginning of the nineties mainly in the form of legal requirements for banks and financial institutions to set up contingency plans in the event of a disaster. Figure 1: The evolution of disaster recovery and business continuity R e c o v e ry tim e s c a le seconds m in u te s A p p lic a tio n re c o v e ry & c o n tin u ity h o u rs D is a s te r re c o v e ry 2 4 -h o u r s days B u s in e s s C o n tin u ity D is a s te r m itig a tio n 1990 1995 Source: Datamonitor 2000 2000 + DATAMONITOR Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 6 BCP & DR – Achieving Continuous Availability These solutions mainly mitigated the impact of a major disaster. Consequently, these contingency arrangements were designed to enable the restoration of business and data over a period of a few days protecting the business from a permanent or longterm loss. As such the contingencies developed in this period did not allow for any immediate continuity in the running of the business. Over the following half-decade the concept of being able to recover the operations of the business in the case of a disaster gained further ground with enterprises establishing remote locations where workers could be relocated for the restoration of a business in the case of a disaster. These facilities are to this day the ultimate protection against major disasters, but are as such only available to enterprises whose wealth and data value makes a facility of this nature possible. The dawn of the millennium At the end of the last millennium, the spread of the Internet and the advent of eCommerce saw the timeframes of business transactions shrink to levels never seen before. For example the average patience of people surfing the web is less than 10 seconds until they give up and move to another site, unlikely to return to the site that did not work. At the same time IT has been integrated into business processes, leading to the increased importance of maintaining the availability of applications and IT infrastructure at all times. These solutions still mainly involved back-up and restore implementations, typically to tape, with the aim of minimising any data loss in case of a systems failure. Furthermore a backup and restore facility facilitated the process of restoring the last known working configuration of a system thereby enabling higher levels of availability. This process was perfected with more frequent back-up windows and redundant application servers enabling the fail-over of application to occur in a matter of minutes. The fall in the price of back-up and restore solutions over this period to levels that were affordable for most enterprises meant that the majority of companies with an IT department could afford a simple tape library for the backup of critical data. The birth of Business Continuity In the scare relating to the Y2K bug, many enterprises performed a close audit of their business involving contingency plans in the event of downtime or disaster. The resulting solutions for the first time saw the widespread acceptance of the concept of business continuity whereby the business operations were to be maintained at all times in some form. Business continuity is meant to encompass the whole business, not just the web-site or key applications. Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 7 BCP & DR – Achieving Continuous Availability Today the main issue in business continuity and disaster recovery is no longer merely related to the attainment of higher rates of availability as the economics of moving from the highly expensive availability level of ‘four or five 9s’ of reliability (i.e. 99.99% or 99.999% reliable) to even higher levels rarely makes sense to even the most downtime sensitive-organization. Instead, the main focus in business continuity and disaster recovery should be to concentrate of the level of deployment within the enterprise. Furthermore, in today’s world of interdependence and collaboration it is also important to think about partner companies, most notably suppliers, to ensure that agreed levels of performance can be maintained. Most organizations at the moment still only have quite rudimentary solutions in use for disaster recovery and business continuity. Planning for disaster recovery and business continuity Datamonitor research on issues relating to business continuity and disaster recovery indicated that enterprises had a very poor idea of what these solutions entail and that levels of deployment seem to be quite low. It is this low level of deployment that needs to be addressed, as with so many IT issues, through education. Despite the higher priority that business continuity and disaster recovery have been able to gain in overall IT budgets, there is still a clear shortfall in the level of spending in enterprises resulting in inadequate cover against disruptive events. Furthermore, many BCP mistakes or assumptions are made by companies dependant on traditional technology infrastructures, as well as those relying more heavily on the Internet, and include: • over resilience – relying on a business continuity plan can lead to a false sense of security and potential business failure if the plan is not updated regularly and fully tested. Formal mechanisms should be in place to force a plan update on a regular basis or when significant systems or business process change occurs. A comprehensive BCP plan should include mechanisms to ensure periodic updates; • segregated planning processes – companies often limit the scope of their efforts to systems recovery or consider IT assets and business processes separately. BCP requires consideration of both business process and systems recovery together, given that technology often play’s a critical role in the business processes. The plan must address those processes that coincide with corporate strategy and objectives; • lack of planning prioritisation – prioritising key business processes is a critical step that often does not get the appropriate attention. Without prioritisation, a plan may recover less-than-critical business processes rather than the ones crucial for Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 8 BCP & DR – Achieving Continuous Availability survival. Furthermore, vulnerability should also be considered and taken into account when prioritising which processes will be planned first. The fact that more and more business processes are interdependent, the order in which processes and systems are recovered is important; • safety deficiency – all stages of the BCP processes, employee, business partner and customer safety must be taken into account. The plan should include safety controls to minimise casualties in the event of a disaster and ways to contain the situation to minimise or eliminate risk; • inadequate communications – communication issues are often overlooked. Often, businesses lack formal communications plans to contact employees, business partners and clients, in the event of problems. Strategies to address how these groups obtain recovery status updates are usually inadequate; • poor security – physical and information systems security controls are often disregarded during plan development and implementation, resulting in greater risk exposure during recovery operations. In order to recover equipment quickly and without interference, as well as to process insurance claims in a timely manner, physical security over a disaster site is an important consideration. Also, similar logical security controls should be in place for the back-up information systems as the primary processing environment; • ineffective response to insurance requirements – many business continuity plans fail to adequately plan to support the filing of insurance claims, resulting in delayed or reduced settlements; • poor recovery services evaluation – many companies poorly evaluate recovery products (hot sites, cold sites, off site storage and planning software), relying on vendor information. Furthermore, some companies eliminate the product or service simply due to cost without understanding how it could significantly affect the timely recovery of critical business processes. Lack of foresight may lead to a solution that does not adequately address a company’s needs; • regulatory / legislative compliance – adherence to legislative issues is also often overlooked, with many companies being unaware of their legal requirement to plan for the continuity of business processes and must understand local statutes and industry regulations governing business resumption and disaster recovery; Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 9 BCP & DR – Achieving Continuous Availability Regulatory squeeze and ubiquitous deployment Until now the regulation relating to business continuity and disaster recovery has been quite limited, relating mostly to financial companies and financial records within an enterprise. Very little legislation is actually in place that forces businesses in normal circumstances to implement measures for business continuity. However, driven by the political will created following the events of September 11 regulatory bodies across the Western world will most likely step up their requirements on business continuity features of businesses, especially publicly listed businesses. Primarily, this would be to ensure the going concern of the business but also would set down standards for business availability- in essence setting down a legally enforceable SLA. However, there are two regulatory bodies that are providing guidance with the adoption of risk-based approaches to business operations. They include the Basel Committee for the financial services sector and the Turnbull Committee for companies of any industry. A discussion of both regulatory bodies is given below. Basel Committee In 1988, the Basel committee of the Bank for International Settlements (BIS) classified the risk levels of different types of credit and the minimum amount of capital that should be held as a cushion against the risks of each. The revised Accord (Basel II) conceived in June 1999, attempts to tackle the shortfalls of its predecessor but more importantly, it encourages banks to develop internal models rather than have capital reserves imposed by regulators. Proposals focus on incentivizing banks to develop more sophisticated approaches to credit and operational risk based on the banks’ own internal ratings and measurement systems. In order to benefit from capital incentives, banks must be able to demonstrate high levels of data quality, accuracy and integrity in order to obtain approval for the internal rating based approach that will give the institutions the ability to self regulate risk management. Basel II also recognises that risks other than credit and the market can be substantial, and therefore also incorporates directives on operational risk, i.e. the risk of losses from inadequate or failed internal processes, people, systems or external events. These losses can be catastrophic, especially with the increased reliance on electronic payment processing, electronic trading and the automation of other middle and back office functions. The increased complexity and granularity of Basel II over its predecessor forces banks to revise existing credit and operational risk management systems and data Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 10 BCP & DR – Achieving Continuous Availability standards. Critical to this is the management of large volumes of high quality loss data and detailed documentation to enable audit trails and third party verification. Figure 2: The Basel II data challenge DATA MANAGEMENT Data collection Assimilation of data from across the enterprise at a business unit and geographical level Data standardisation Imposing a uniform discipline on the captured data Data cleansing Quality control Data consolidation Developing and maintaining a dynamic database Source: Datamonitor DATAMONITOR A significant factor in a Basel II project will be the process for collecting, standardizing and consolidating data through the business units to model risk. Banks need to establish a workable definition of risk and for this definition to be applied across all geographies and business units. While presenting a significant technological and business process challenge, possibly the single greatest factor will be cultural. Many banks are not culturally accustomed to amassing large volumes of historical operational loss data given that operational risk by definition has traditionally been regarded as a risk category immune to standard linear risk management methodologies. The links between Basel II and Business Continuity are clear. Firstly, it integrates business risk and the need to collect and store information. Business Continuity as a concept is tightly integrated into the requirements for Basel II as a method of mitigating risk and ensuring data integrity against low frequency high consequence and high frequency low risk events. Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 11 BCP & DR – Achieving Continuous Availability Turnbull Committee Report The Turnbull Report is the popular name given to the ICAEW's Guidance on Internal Control which was produced in clarification of The Combined Code on Corporate Governance. The Turnbull guidance is linked via the Combined Code on Corporate Governance to the Listing Rule disclosure requirements of the London Stock exchange. The guidance is concerned with the adoption of a risk-based approach to establishing a system of internal control and reviewing its effectiveness and can be instituted by any company in any industry. The guidance is intended to reflect sound business practice whereby internal control is embedded within the business processes themselves. This ensures that rather than being merely an exercise to meet regulatory hurdles it is incorporated into the company’s normal management and governance processes. Crucial to this is the system of internal control; sound internal control systems should ensure the enterprise will not be hindered in achieving its objectives. Building on the internal control aspect, the Turnbull guidance mandates the need for an ongoing internal audit stating how the enterprise identifies, evaluates and manages risks. As with Basel II, there is a requirement for transparency of the results of this and the enterprise is required to state what actions are being taken to rectify or improve any major failings or weaknesses with the control system or risk management system. Although this requirement involves non-IT functions it is clear that in any enterprise where information handling or processing is paramount, the institution of a business continuity capability is key. An effective system should not only mitigate risk, but also satisfies the IT internal control requirements for Turnbull. Business Continuity for the masses Traditionally, Business Continuity solutions have been associated with the financial services sector. However, with the evolution of eBusiness and the wide-spread use of mission critical applications, organisations from all verticals are investing in continuity and recovery solutions. In 2002, financial services institutions will account for 35% of a €2.9 billion market in Europe. Datamonitor expects the dominance of financial services end-users to remain, due to various initiatives such as GSTPA, T+1 and multi-channel banking. The uptake of BC and DR solutions in other verticals is relatively similar to the security market – with the public sector, utilities and telecommunications organisations showing an increasing spend pattern. By 2005, Datamonitor expects European businesses to spend a total of €7.7 billion in achieving continuous availability of their operations. Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 12 BCP & DR – Achieving Continuous Availability The ideal enterprise continuity solution The need to ensure the availability and continuous operation of business systems, in spite of potential failures ranging from disk crashes and CPU failures to catastrophic losses of computing facilities or communications networks, and planned downtime and maintenance is of critical importance to today’s businesses. Typically, many companies also have geographically dispersed data centres. Information must be available across the data centres and steps should be taken to ensure data continuity, particularly when a data centre has an outage, planned or unplanned. Although solutions exist to provide tolerance to component failure, the issue of site loss is often overlooked, with potentially disastrous consequences – business interruption and loss of information. Traditionally, off-site tape dumps have satisfied the requirements for disaster recovery for batch systems but they are typically inadequate for protecting the information stored in real-time eBusinesses and online transaction processing systems. Asynchronous replication facilities can provide continuous duplication of critical eBusiness applications to off-site backup facilities, without the high latency inherent in tape backup strategies. Once established, such an environment can be automated to ensure that information is replicated in a timely manner and the switch to backup systems is accomplished with minimal or no business interruption. Companies must develop a solution to provide continuous availability of their systems, or risk losing business to their competitors. It is essential that high availability solutions address the availability of data in three areas: • managed planned downtime (e.g. routine maintenance downtime, software upgrades, etc.); • protect during unplanned downtime (machine / network outages); • provide disaster recovery. Any system providing high availability (HA) should provide continuous availability of data in all three scenarios. The remainder of this section will provide an understanding of the various components needed to achieve continuously available systems. In combination, these components can provide a near-continuous service. Hardware redundancy and physical replication Hardware redundancy is often though as the first line of defence in continuous systems and it protects against computer and disk failure. Although this can be an Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 13 BCP & DR – Achieving Continuous Availability expensive option, it is the only way to protect against the failure of a machine or disk drive. Multiple processors in a single machine can protect against processor failure, while multiple machines in a cluster formation protects against machine failure, and these form the basis of hardware redundancy. Drive redundancy can be achieved by using RAID (redundant array of inexpensive disk) and disk mirroring solutions: • RAID – provides redundancy within a single RAID cabinet that contains extra storage / disk drives; • Disk mirroring – a subset of RAID, two separate disk storage devices are used to provide redundancy. Both of these solutions protect against disk media failure as long as the redundant storage contains a valid copy of the data and is also known as ‘cold standby’. However, these solutions only secure data in its raw low-level format and do not address the actual meaning and usage of the data, or the interaction with applications. Today’s most critical eBusiness systems depend on data which has a complicated internal structure and is used in ways that are dependent on the state of key applications and business processes, than a conventional set of data files. The delay and perception of unreliability involved in such incidents can represent millions of Euros in direct and indirect costs or loss of opportunity. Reputations are also at risk due to the external impression that the organisation is not in full control of its data, or that strict audit trails may have been compromised. Logical replication Logical replication is a process whereby logical operations / processes are replicated at the level of the database instead of at the disk level. Typically, database operations are replicated between two or more databases. Therefore, it replicates meaningful operations such as ‘Customer Y with account number 00192 has placed an order that needs delivered in 36 hours’. Using logical replication allows the customer information to be validated before the redundant copy is written to disk. Redundant hardware is used in conjunction with logical replication, with two or more databases running on separate machines, writing to separate disk devices. This solution can be combined with physical replication, providing a line of defence against loss of availability and logical replication providing coverage in the event of physical Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 14 BCP & DR – Achieving Continuous Availability replication failure. This is also known as a ‘warm standby’ system and can be located anywhere in the world, technology and reliable bandwidth permitting. Surviving planned downtime Generally, high availability is thought of in the context of server failures or disasters that are actually quite rare. Today’s administrators have to consider another measure of availability – keeping systems / applications available during normal planned maintenance. Although this downtime is planned, it accounts for the majority of time that systems are unavailable. This becomes critical for businesses that are operating a true 24x7 system that serves customers globally, and taking any data offline can have severe consequences. Therefore, systems and software must be designed so that maintenance activities do not disrupt normal usage. This process is usually known as switchover. Previous solutions involved minimising the level of data impacted by maintenance operations. Although this reduces the number of end-users affected, it still leaves some data unavailable, making it far from an ideal solution. Application and business process continuity Combining hardware redundancy, physical replication and logical replication reduces the amount of time it takes to recover in the event of an outage. However, it takes more than just fast recovery to provide continuous availability. Even if there is an immediately available copy of data when there is a disruption, users and applications must be switched over to the copy. This process of switching over to a good copy of data or another copy of an application is known as failover. The modes of failover include: • active-passive – the fastest failover mechanism using a cold standby copy of the data, either a genuine copy or switchable disks attached to both machines in a pair. However, this is not the most rapid and seamless approach, as a second server has to be started which involves its own overheads and wastes resources because the second server is not available for other work during the time when there is no emergency. Therefore, although failover is rapid, achieving true business continuity can be slow procedure; • active-active – this method solves the active-passive problem by running a second server alongside the main server. This server is available to perform other duties and means that resources are not wasted during normal operation. Also, the time to take over from the main server during failure is significantly shorter; Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 15 BCP & DR – Achieving Continuous Availability • connection transparency – allows the various protection and failover solutions to work without any of the users or applications noticing that there has been a failure in the system. Supporting continuous availability requires that failover is automatic and immediate so that it is transparent to users and applications accessing the failed component. Also, once this component has been repaired or replaced and brought back online, users and applications will need to be switched back to the original component. Again, this process must be immediate and transparent. Achieving automatic, immediate and transparent failover requires appropriate failover support in both database management systems and applications servers. What about the network? A common mistake to make in BCP and DR deployments is to ignore the vulnerability of the actual network, whether private or public. Typically organisations will use a secure VPN between the onsite and offsite facilities to mirror / RAID their information assets. However, what happens if this link goes down during planned maintenance, or when network traffic becomes excessive for example? Any loss of data, transactions or applications could prove very costly to the organisation. There is a method of bypassing this problem however and businesses must accept that networks are unreliable and plan for it through their BCP and DR solution. It is essential that storing and queuing techniques are used, where the most recent version of the replicated data is queued at the failed network node. Once the network is restored, this information can then be passed securely to the offsite facility. Restoration – the weakest link? Backup of information has to be fast enough to keep up with the size of the database, must scale with the database as it grows and must have minimal performance impact on active database operations. One method of minimising this impact is to offload as much of the backup processing from the database itself to a backup server. Restoring the database is the last step in recovering from an outage. However, this can be the limiting step in the whole recovery process. Typically in day-to-day operations the key issue in a smoothly running system is backup performance, not restoration performance and it is often the case that not enough attention is paid to restoration time. For example, a database management system that backs up at a rate of 500Mb / hour and restores at a rate of 40Mb / hour does little more than provide a false sense Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 16 BCP & DR – Achieving Continuous Availability of security. Under these conditions, a one terabyte database can be backed up in two hours, but restoration would take 25 hours. These restoration time frames are now unacceptable for most businesses who wish to retain their customers, partners and suppliers, without losing competitive edge. Therefore, it is essential that fast restoration systems are used. Common mis-conceptions about BCP Many businesses have spent millions on technology infrastructure and the resilience solutions to support them, however many areas still exist where they are left exposed. This is usually a result of the mis-perceptions and lack of education and understanding among the end-user base, and includes: • throwing hardware at the problem will not solve it – disk mirroring, clustered machine pairs and offsite disk or tape backups may secure critical data, but are not enough to allow the continuous availability of applications that customers demand; • successful failover is only the beginning – mirroring or taking snapshots preserves critical data, but unless there is a systematic, rapid and reliable way of going back to normality, organisations will be exposed to further failures; • clustering is not sufficient – clustered pairs will not protect against all emergency situations, especially those that would be categorised as disasters. A pair of machines will not guarantee continuous availability when fires, floods or terrorist attacks occur; • dual level resilience is not perfect – this becomes significant during planned maintenance. The applications and users are dependent on the second environment while the first is undergoing maintenance. What happens if the second system fails during the maintenance session? The answer is to have three layers (or more) of resilience, preferably combined with offsite secure copies of the data; • messaging has its limitations – the use of messaging to some extent insulates one application from a failure in other applications feeding it. However, this does not guarantee that an entire chain of related systems will continue to function smoothly whatever the combination of emergencies. Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 17 BCP & DR – Achieving Continuous Availability Achieving continuous availability with Sybase This paper has, so far, discussed numerous continuity planning solutions, such as warm standby, avoidance of low level database corruption, triple layer resilience and connection transparency. Combinations of the above solutions are essential to make an enterprise truly resilient. Depending on the business need, Sybase can provide a variety of levels of availability that address the various degrees of severity in disasters and emergencies, in various combinations. Their solution offering is illustrated in Figure 3. It can be observed that Sybase offers a complete solutions architecture – the products, design and implementation templates to make them work. Each element of the architecture raises the bar in terms of continuously availability to higher levels on the 4x9’s, 5x9’s scale. Their three key product offerings are: • Adaptive Server Enterprise- High Availability (ASE HA); • Replication Server; • OpenSwitch. Figure 3: Sybase Triple layer resilience solution D a ta b a s e o n d ua l p o rte d d is k a rra y M irro r in g W a rm S ta n d b y R e p lic a tio n S e rv e r AS E H A AS E H A O p e n S w itch S o ftw a re F a ilure s C O N T IN U O U S L Y A V A IL A B L E A P P L IC A T IO N S H a rd w a re F a ilure s 2 4 x7 x5 2 B us ine s s o p e ra tio n Da ta b e as o r ro m ri s e bl ou F lo Te d, O ffs ite S ta nd b y A S E O ffs ite R e p lic a te d D a ta b a s e D Source: Datamonitor s re H a rd w a re C lu s te rin g ilu C lu s te r C o n tro l S o ftw a re Fa e, F ir Co on p ti rru DATAMONITOR Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 18 BCP & DR – Achieving Continuous Availability Adaptive Server Enterprise (High Availiability) Sybase ASE (HA) 12.5 is a data management platform for mission-critical, transaction-intensive enterprise applications. It can be used to provide the continuous availability of database systems to customers, partners and suppliers, without creating any headaches, confusion or requiring them to reconnect. It has the ability to manage each level of a BCP and DR solution, including hardware redundancy, cold standby (backup and restore), warm standby (replication) and active / active hot standby. In the event of a failure, ASE (HA) enables the movement of end-users from a primary system to a back-up system at any point in time, without causing any disruption to the applications or information that they are using. Essentially, it insulates the user from the complexities of back-end systems. Sybase ASE (HA) leverages the cluster architecture discussed previously and provides failover to a backup server without losing any non-committed data or severing user connections. It also works in concert with existing hardware and software high availability solutions from third party vendors, to deliver maximum systems availability. Furthermore, Sybase ASE (HA) incorporates a Companion Server that allows the configuration of two ASE (HA) servers as companions in either asymmetric (masterslave) or symmetric companion (active / active hot standby) to create a hot standby capability that can further reduce unplanned downtime. This approach involves the deployment of a two-node hardware cluster with two ASE (HA) databases running as companion servers to each other. In this configuration, both servers run applications, so that if Server 1 fails, Server 2 opens up the devices on which the primary’s databases are built and perform a fast recovery to bring them online, while continuing to handle its own clients. The failover process is simplified through the use of ASE (HA), as it sends the enduser an error message indicating that failover has occurred and that the current transaction must be resubmitted. Replication Server Sybase’s replication server provides automatic server failover solutions that reach across LAN and WAN, providing simple replication of data over extensive geographic regions. The replication server provides high availability and disaster recovery services, giving greater protection against failures through asynchronous, wide-area delivery of database transactions. It is also possible to share information across an Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 19 BCP & DR – Achieving Continuous Availability organization by replicating data to and from heterogeneous hardware and data sources without losing transactional integrity of the data. The replication server operates in a warm standby mode where databases are kept in close synchronisation using a database replication system. This provides simple configuration, database mirroring with geographical reach of up to 1000s of km, the ability for rapid switching of the direction of replication and automatic switchover in the event of failure (when used in combination with OpenSwitch). This warm standby feature enables customers to more easily configure and manage a high-availability, distributed recovery environment at lower administrative cost than traditional replication products. Also, this feature simplifies the configuration and operation of a warm standby environment by eliminating the traditional requirement to define individual objects eligible for replication and explicit subscriptions to the objects. With the replication server operating in the warm standby mode, it is possible to switch the direction of replication when the primary system is unavailable, with the applications being routed to the standby system. Activation of the switch results in the reverse flow of transaction traffic from the standby system to the active system (which maybe currently unavailable). This enables the replication server to queue up any transactions applied to the standby system by the clients, until the primary system is brought online. Once online, the primary system can be re-synchronised from the queued data. This functionality is critical in situations where failback is as important as failover, for example when managing planned downtime, or unplanned downtime where the primary systems are recovered relatively quickly and need to be resynchronised rather than built from scratch. Sybase Replication Server is fully interoperable with ASE (HA) and supports the high availability features in a cluster configuration. For example, in the event of a node failure (or process failure), the replication server process is migrated over to the companion node without administrator intervention, thereby presenting the organisation with cost and time benefits. OpenSwitch – facilitating continuous availability Adaptive Server Enterprise’s Companion Server option provides the fastest failover functionality and can also guarantee that users will not be disconnected in the event of a system failure. However, organisations that wish to provide continuous Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 20 BCP & DR – Achieving Continuous Availability availability to a variety of clients over extended geographic regions will benefit from the enhanced “connection transparency” facilities of Sybase OpenSwitch Sybase OpenSwitch is a solution that provides transparent client connection failover as well as transparent client connection routing, load balancing and central management. OpenSwitch sits between client applications and the servers to which they connect and works either manually, in response to an administrative request, or automatically when it detects a server failure. It transparently transfers incoming connections to any Sybase server product, or another instance of OpenSwitch. The true beauty of OpenSwitch is that it monitors server availability on an ongoing basis and when a server becomes unavailable, it transfers the client connection to another server without disturbing the connection. By monitoring and restoring the transaction state, communications state and connection state of each connection, OpenSwitch is able to transfer client connections without the need to disconnect and reconnect. From an end-users perspective, nothing has changed, while behind the scenes a great deal may be happening. The user may notice a pause or short delay, but the connection and the customer is not lost. This functionality is available to existing applications without requiring any programming changes and highlights the continuous availability available from Sybase solutions. Advantages of OpenSwitch include: • load balancing and routing – allows organisations to transfer connections based on the type of users coming into pools of servers. For example, different users may have different performance requirements for their applications, ranging from the need for sub-second query responses to the need to run less time sensitive batch reports. This allocation process is performed on the fly without having to bring any servers down; • central management – provides tools designed to make the end-user experience as simple as possible and to simplify the challenges IT faces behind the scenes. For example, consider an organization in which the administrator needs to perform a data load involving hundreds of gigabytes. Such loads can take hours and are normally done in the middle of the night. Using OpenSwitch, the load can be performed during the day and once the load is completed, users can be moved over to the updated server with a simple flick of a switch. Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 21 BCP & DR – Achieving Continuous Availability Why Sybase? • builds on existing investments – in mirroring, clustered hardware, cluster control software, remote DR recovery, etc. but adds the database and application level resilience that enhances basic data protection and hardware duplications; • active–active – configuration for the quickest failover, and a secondary server that is always running; • secure, physical and logical warm standby – of database copies, providing insurance against lurking corruption and provision of an offsite copy that is secure against sever emergencies like fires, floods and terrorist attacks; • triple level resilience – architected with active-active and warm standby combined. Provides integrated, automated protection against service interruptions, database corruption and severe emergencies; • connection transparency – provides protection and failover without applications or users noticing. Real assurance provided for each part of an STP, reporting or customer service chain will continue smoothly. The whole value chain will continue correct operation both in parts and as a whole; • multiple benefits from each investment – remote DR sites also support failover operations if hardware failure occurs during planned maintenance and OpenSwitch ensures this is invisible to end-users, while also ensuring minimum disruption to applications during any normal high availability failover; • ease of switching back to normal – well defined procedures are inherent in the Sybase architecture and allow for ‘returning to normal’ configuration. Also, the Sybase solution set works at the database and the application level, providing logical integrity of the system at all times. Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 22 BCP & DR – Achieving Continuous Availability The true value of the Sybase offering arises when each of the products work as one, to provide continuously available BCP and DR solutions. The benefits of such an integrated solution include: • fast failover – locally provides resilience against any hardware breakdowns; • secure logical copy – provides a highly up to date logical copy of all data at the remote site, providing resilience against any site disasters, or inaccessibility of the main site to the key staff; • invisibility – any maintenance activities to hardware or applications are invisible to the end-users; • triple layer resilience – possible breakdowns during planned maintenance periods go unnoticed by users through the secure triple layer resilience; • highly automated – procedures providing rapid failover and failback are provided through the automation of the solution; • immune to corruptions – the solution provides a remote copy which is immune to any database corruptions originating in the original copy; • intelligent – the data replication functionality is closely related to the individual databases supporting individual applications, meaning that the failback procedures are less error prone than if all the data is mirrored indiscriminately. Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 23 BCP & DR – Achieving Continuous Availability Conclusion Today’s world of eBusiness means that it is essential for businesses to have some form of business continuity planning and disaster recovery solution in place. This is not only limited to the financial service sector, which is dependent on high volume and value transactions to run its business, but to businesses in all industry sectors. Now it is not only the internal employees that are dependent on system availability, but also your business partners, customers and suppliers. Enterprise applications and collaborative initiatives have further highlighted the need BCP and DR. The cost of not having a robust continuity solution in place could be catastrophic – lost revenues, bad press coverage, loss of customers and competitive mindshare to name but a few. The level of continuity needed is obviously dependent on the business need, however BCP and DR demands a certain level of skill, expertise and experience that may be lacking in an enterprise. Using an outside vendor for the writing, testing and implementation stages of any plan may be an expensive option. On the other hand, a ‘build it yourself’ approach could prove complicated and time consuming, especially when multiple organizations and sites are involved. Therefore, it is essential that businesses select a continuity vendor with a proven track record and a comprehensive solution offering that meets the business need. This is where Sybase comes in with its various continuity offerings. Together, the Adaptive Server Enterprise, Replication Server and OpenSwitch solutions provide a truly available and continuous continuity solution for use with all angles of the business, on a 24x7x52 basis. Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 24 BCP & DR – Achieving Continuous Availability Appendix: Business Continuity Planning and Disaster Recovery in practice Case study – large European clearing bank The business challenge: A large European clearing bank is offering one of its largest customers an extranet web-based system allowing frequent and high volume transactions to be performed. These transactions would be aggregated and calculations performed based on external data feeds and the aggregated transactional feeds sent to existing bank foreign exchange systems. As this business offering was to support the customers’ own web sites, 24x7x52 availability of the transactional receipt process was essential for customer satisfaction. In order to avoid backlogs, minimal downtime in any part of the system could be tolerated. Also, to meet regulatory and auditor requirement, the ability to switch processing to an alternate site was essential. The business solution: This was a greenfield deployment and the bank opted for a three tier solution, using a well known application server for the middle tier and web clients at the top. The banks appreciation of Sybase’s high availability solution in ASE 12.5 led them to insist on this architecture, in preference to other database vendor’s offerings. Also, the bank’s long track record with Replication Server gave them faith in this as the most reliable way to transport data rapidly and safely offsite, while filtering out any low level physical database corruptions. The Sybase High Availability Solution: This customer used all elements of the Sybase continuity offering bar one, Sybase OpenSwitch. The customer felt that connection transparency could be built into the J2EE based middle tier. No other parts of the system would require direct connection to the database. For a site with many existing 2 tier applications, the decision would have been made differently. This example indicates that, while comprehensive, the Sybase architecture is flexible and capable of working with 3rd party products where necessary. Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 25 BCP & DR – Achieving Continuous Availability To fulfil the business and technology requirements, the bank opted for the ASE 12.5 solution, providing dynamic memory management features for administration without frequent restarts, in addition to the High Availability, fast failover support. Replication servers were used onsite and offsite to avoid physical low level data corruptions. The benefits: The bank is able to provide the ‘always available’ solution demanded by the end customers using the web sites of the bank’s own customers, and comfortably meet the regulatory requirements and service level agreements (SLAs) in place. Use of such a scalable solution will allow the bank to expand these operations easily to include the growing number of customers who are interested in this business offering. Case study – European branch of a major Asian bank The business challenge: A European branch of a major Asian bank was running a significant number of business critical applications and Sybase based systems that were supported by a variety of technical architectures. The day-to-day operational management and administrative workload generated by these various architectures / hosts was becoming and unacceptable cost to the business. Furthermore, the resilience strategy for a number of these systems and applications (notably Summit and Fidessa) was not going to meet the latest demands of the business user communities, both in terms of loss of data and time to recover in the event of a significant primary and / or host failure. The bank was facing a situation where the operational risks that its business systems infrastructure presented were now considered unacceptable and needed to be addressed as a matter of urgency. The business solution: To address these problems, the bank chose a Sybase systems consolidation and resilience solution that addressed two key criteria: • migration / consolidation of these disparate Sybase based systems to a single technical architecture based on two ASE servers; • incorporation of the migrated solution into a single standardised high availability solution, based on proven / complementary Sybase technologies. Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 26 BCP & DR – Achieving Continuous Availability The result of the project was that it produced a new high resilience architecture for the bank’s operations. The Sybase High Availability Solution: For each of the systems / applications that were migrated to the new architecture, exactly the same high availability solution was deployed. Even if the SLA requirements for a given system did not require the level of resilience the solution provided, it was considered that taking an uniform approach would be most cost effective in terms of operational support, control and management. The high availability solution deployed utilised two Sybase ASE(HA)’s, one on the primary site and the other on the DR site. Warm standby functionality was provided by the replication server and allowed near real time replication of all database data between the two ASE(HA)s. This server was deployed at the DR site host. This bank made use of the Sybase OpenSwitch technology, providing user and application transparency in the event of a systems failure. Management of the OpenSwitch technology is achieved through the OpenSwitch Coordination Module and it monitors all other Sybase components in the architecture and runs alongside the OpenSwitch server on the DR host. Its primary role is to detect significant failure in the primary database server environment and then coordinate the switching of user connections within OpenSwitch, with management activities being performed by the replication server. The benefits: In addition to solving the critical business problem, the project aimed to maximise the return on investment, by ensuring that best use was made of the new host architecture. The migration of a given system / application to the new architecture / high availability solution would be transparent to business users and in house application development teams. Disruption to 3rd party systems was nonexistent, thereby allowing the data movement and interfaces to remain the same. The resulting high availability solution is platform independent and therefore allows for flexibility in client choices, particularly when it comes to platform upgrades such as operating systems or disk sub-systems. This also allows for a scalable architecture, both in terms of hardware and software performance. Using OpenSwitch has allowed the bank to perform essential maintenance tasks by seamlessly switching business users between the primary and disaster recovery database servers. Business Continuity Planning and Disaster Recovery – Achieving Continuous Availability Page 27
© Copyright 2024