EMC
Object Storage Solutions
Helmut Gotsche
Advisory Solutions Engineer
Object Storage Platforms, EMEA
Backup and Recovery Systems
[email protected]
© Copyright 2011 EMC Corporation. All rights reserved.
1
Agenda
• Object Storage versus File system
• Centera Overview
– Tech Refresh
• Atmos Overview
– GeoDrive
– ACDP
• CTA
© Copyright 2011 EMC Corporation. All rights reserved.
2
The Portfolio
Centera
Centera
Atmos
Policy driven
Purpose built,
Object
Storage
WORM based,
with
long term Archive
Data Retention/
with
Expiration
Enhanced Retention
© Copyright 2011 EMC Corporation. All rights reserved.
VNX(e)
Primary
storage
with
Data Retention
Isilon
Scalable Primary
storage
with
Data Retention
Data Domain
Deduplicating
Backup storage
with
Data Retention
3
EMC Centera
Purpose built archive platform
• Centera and XAM API interface
– End-to-End content authenticity and data integrity
– Application driven retention policies
•
•
•
•
•
Governance and Compliance WORM capable
Self-Configuring, Managing and Healing
Deduplication at file level
High availability through RAIN
Up to Multi Petabyte addressable capacity
© Copyright 2011 EMC Corporation. All rights reserved.
4
Redundant Array of Independent Nodes (RAIN)
Centera node







Four
node
16-node 2 cubes/
cube cabinet
Members
Host
Storage nodes/access nodes
1.66 GHz dual core processor
1024 MB DDR RAM
Four - 1 TB , 2TB or 3 TB SATA-II
Two 1 Gbit network-interfaces (internal/Backend)
One 1 Gbit to outside LAN (copper/optical)
A node can have 1-4 roles
–
–
–
–
Application Access
Storage
Replication
Management
Centera network
 Dual 24-port cube switches
 Gigabit Ethernet connections to
facilitate additional cubes
 Redundant connection to each node
Extreme scalability
Multiple cubes form
a single cluster
Multiple Cluster form
a Virtual Archive
© Copyright 2011 EMC Corporation. All rights reserved.
 Massive parallel processing
 Add storage: processing power, memory, bandwidth
5
Centera Requires No Backups
Centera
 Dramatically reduces opportunities for errors to affect data access
or authenticity
 If an error occurs, it can be discovered and healed
How?
 Fixed content prevents data overwrites by applications
 Content authenticity, independent copies, self-monitoring, selfhealing
–
–
–
–
Detection and healing of bad disk blocks
Content regeneration from loss of entire disk
Detection and healing of FS errors
Content regeneration from total loss of FS
Limited configuration
 Human error cannot affect the archive filesystems or disks
– No active management of these resources
© Copyright 2011 EMC Corporation. All rights reserved.
6
How EMC Centera Works:
EMC Centera
performs Content
Address calculation
and sends address
back to application
Simple
Object is created
and sent to
application server
CA
LAN
CA
Application server sends object to
EMC Centera over IP network
Database stores
Content Address
for future reference
Content Address
10001010
10111011
© Copyright 2011 EMC Corporation. All rights reserved.
Content
Address
algorithm
Content
Address
algorithm
 Digital
fingerprint
 Globally
unique
 Locationindependent
7
Centera Stores and Retrieves Objects
Object: File(s) and metadata
date
name
<My_Archiving_Application>
<MagazineCover name=“Time” photo=“Annan” date=“Sep 4, 2000”/>
<Reviewer name=“Jones, Ted”/>
</ My_Archiving_Application >
photo
• Applications create metadata associated with one or more files
• Objects are stored independent of volume/directory information
© Copyright 2011 EMC Corporation. All rights reserved.
8
How Application writes to Centera
Clip ID:
The Content Address of the metadata file
(CDF) that Centera returns to the application
3C08JM40C8AMMe0N8ATEJHC2DQN
<User: Jane Doe, VP Sales>
<Date: April 2, 2003>
<Report from: T. Smith>
<Retention.period; 3years
<Comment: Great Division Sales numbers>
Clip ID:
The Content Address of the metadata file
Brown,
EVP>
(CDF) that <User:
CenteraBill
returns
to the
application
<Date: April 10, 2003>
4AE7B39A2CEFe6J2PTDRWE4YYZ
<Report from: T. Smith>
<Retention.period: 1year>
<Comment:
Inconsistent Revenue production
revised Calculation as XLS>
<CA>: 38498REM37A32x3445FHR4239585
<CA>:Q734F23412MNEx2810QR6T548E5
Q734F23412MNEx2810QR6T548E5
<CA>:
© Copyright 2011 EMC Corporation. All rights reserved.
9
Content Protection: Mirror
• Object is written to Centera, Centera finds
least busy node
• A duplicate is immediately made
on another node
• Once there are 2 copies,
the write is acknowledged
• Content Address is stored by application
• If a node or disk fails,
the contents are re-mirrored automatically
Mirrored = 50% of raw capacity
© Copyright 2011 EMC Corporation. All rights reserved.
10
Content Protection: Parity
• Object is written to Centera,
Splits up object into 6 pieces
• A Parity Piece is calculated (XOR)
• Each piece is distributed to individual nodes
(based on which is least busy)
• Content Address is stored by the application
• If a node or a disk fails,
automatically the contents are
mathematically regenerated (XOR)
Parity ~ 83% of raw capacity
© Copyright 2011 EMC Corporation. All rights reserved.
11
Multiple “Virtual Pools” in one Physical Cluster
Pool 1
Application 1(profile1)
Pool 2
Application 2(profile2)
Pool 3
Application 3 (profile3)
Default Pool
Legacy/Anonymous
Blob
CDF
© Copyright 2011 EMC Corporation. All rights reserved.
12
Distributed Content for
Business Continuity and Disaster Recovery
 Replicate
 Replicate
selected
all Virtual
Virtual
Pools
Pools
IP-Address
Target’
Source’
Pool 1
Pool 2
Pool 3
Source
Target
IP-Address
© Copyright 2011 EMC Corporation. All rights reserved.
13
Advanced Replication: Star and Chain
 Incoming
 Replication
Replicate
Star
…and
andin
restore
incoming
Chain
Chain…
combined
Star…
1
A
B
C
2
3
© Copyright 2011 EMC Corporation. All rights reserved.
14
Centera and Compliance
Centera Basic
 Provides all functionality without enforcement of retention periods
Centera Governance Edition
 Process-centric on the lifecycle of electronic records and enabling policies
and technologies
 Restricts the retention and deletion of data but
does not conform to SEC regulations
 Suitable for most regulations
Centera Compliance Edition Plus (CE+)
 Designed for the strictest of regulation requirements,
specifically SEC 17a-4
 Restricts the retention and deletion of data according to SEC regulations
© Copyright 2011 EMC Corporation. All rights reserved.
15
Retention and Disposition Functionality
Optional Software Configurations
Governance Edition
Software Configuration
Compliance Edition Plus
Software Configuration
Yes
Yes
modifiable
Only prolongable
E-shredding
Yes
Yes
Remote management
Yes
No
Privileged delete
Yes
No
Can be set by SYSOP; factoryset to zero
Automatic default to infinity
Event-based retention
Yes
Yes
Litigation Hold
Yes
Yes
Min/Max retention
Yes
Yes
Features
Enforced retention period
Retention Classes
Configurable default
retention period
© Copyright 2011 EMC Corporation. All rights reserved.
16
Advanced Retention Management
Optional Software Configurations
Basic
Advanced Retention Management
Centera Governance Edition
Centera Compliance Edition Plus
CentraStar V3.1
• Event-based Retention
–
–
“Undetermined” retention period set when content is stored
specific retention period starts counting at the time of an “event”
• Litigation Hold
–
–
Enables an application to put a “hold” on a content item beyond any retention setting
Prevents deletion until the hold is removed
• Minimum Maximum Retention period
–
Writes are only accepted if retention period setting meets the limits
© Copyright 2011 EMC Corporation. All rights reserved.
17
Advanced Features
• Network Segregation
• Access
• Management
• Replication
• Multiple Access NWs
Access
• Time Synchronization
Management
NTP
Replication
© Copyright 2011 EMC Corporation. All rights reserved.
18
Universal Access Makes Archiving Easy
FTP
HTTP
NFS/CIFS
Emulation
Direct Call
from Application
XAM
Centera
nywhere, any time, any application from virtually any platform
© Copyright 2011 EMC Corporation. All rights reserved.
19
325+ Commercial Applications using Centera
• E-mail
• Voice recordings
• File systems
• Video
• Documents and images (enterprise
content management)
• Check/document imaging
• Microsoft SharePoint
• Life Sciences
• Medical images
• Security Event and Information
Management (SEIM)
• SAP information
• Other archiving applications
• Databases
© Copyright 2011 EMC Corporation. All rights reserved.
20
Centera—Meeting the Needs of
Today’s Production Long Term Archives
• Centera delivers:
– A multibillion object,
long-term archive
– Sub-second time to first byte
– Assured lifetime
content authenticity
– Bulletproof content
protection
– Five-nines availability
– Low TCO
• Defacto standard:
– Healthcare and
e-mail archiving
© Copyright 2011 EMC Corporation. All rights reserved.
21
Centera Tech Refresh
© Copyright 2011 EMC Corporation. All rights reserved.
22
Centera Tech Refresh Program Overview
• Tech Refresh with new Centera or Atmos
• Centera hardware generations to refresh with Gen4LP 4TB, 8TB or 12TB
–
–
–
–
–
Gen1 -> EOSL
Gen2 -> EOSL
Gen3 -> EOSL
Gen4 (1.2TB and 2TB nodes) -> EOSL 2013-02-28
Gen4LP (3TB nodes)
-> EOSL 2014-02-09
• Includes professional services migrations – not customer executable
– Centera intracluster migration service (CICM)
• PS-BAS-CICM3G / PS-BAS-CICM
– Centera to Atmos migration service
• PS-CUS-EMC
• Migration process will be designed with no or minimal downtime
© Copyright 2011 EMC Corporation. All rights reserved.
23
Common TechRefresh
<= Gen3
>= Gen4
NEW
Warrenty
Four
node
© Copyright 2011 EMC Corporation. All rights reserved.
16-node
cube
24
EMC Object Storage
Fixed Content
Compliance
Authentication
Longevity
Object Storage
Global name space
Low cost
Plug&Play expansion
Unattend operation
Scale-Out benefits
Centera
© Copyright 2011 EMC Corporation. All rights reserved.
Policies
Multi-tenancy
BLOB store
RESTful
GeoScale
GeoProtect
ATMOS
25
Optimize Applications Across Your Business
WEB APPLICATIONS
Atmos SDK
(.NET, Java, FX, C,
PHP, Ruby, Python,
Javascript, Drupal, CAS*)
MOBILITY
Atmos
GeoDrive for
Windows or
Linux
CONTENT MANAGEMENT
FILE SERVER TIERING
MEDICAL IMAGING
EMC VNX
EMC Celerra
NetApp
CTA
Intelligently manages a massive amount of
unstructured content over WAN or LAN
HTTP/S (REST, SOAP), CAS, S3, IFS, NFS, CIFS
Global. Intelligent. Web. Self-Service.
FRONT END
FRONT END
FRONT END
FRONT END
Atmos Virtual Edition
DISK ARRAY
DISK ARRAY
DISK ARRAY
VIRTUAL DISK
Site #1
LOS ANGELES
Site #2
CHICAGO
Site #3
NEW YORK
Site #4
LONDON
ATMOS CLOUD
© Copyright 2011 EMC Corporation. All rights reserved.
26
Deploy a Cloud Storage Infrastructure
EMC Atmos stores, manages and protects globally distributed unstructured content
Single system across distributed storage
Custom
App
Multi-Site Active/Active, Single pane of glass
Automated storage placement, protection,
services
Metadata drives policies to deliver business value of data
Fast, easy access – from anywhere
LAN, WAN, WEB, REST, API, other access methods
Windows, Linux, Mobile Devices, and seamless user access
Multi-tenancy
Separate, restricted access; common storage
Metered, self-service access simplifies Storage-as-aNEW YORK
U.K.
service
Flexible, elastic, extensible infrastructure
Access, store and manage any content rich web application,
unstructured content via Atmos private, hybrid or public cloud
The 4-M’s easily qualify Atmos opportunities
© Copyright 2011 EMC Corporation. All rights reserved.
27
Atmos Solution Options today
ATMOS CE
ATMOS LE
ATMOS VE
Complete Atmos functionality:
• Multiple Active/Active
• Unlimited Multi-tenancy
• Full policy controls
Subset of Atmos functionality:
• 2 sites
• 1 Tenant, 1 Subtenant
• Limited Policy Configuration
• Upgrade path to Atmos CE
Complete Atmos functionality
certified to run on EMC VNX,
VMware-Supported Servers, & ThirdParty Storage
Policy Engine, Access Methods, Operating Environment
Atmos Virtual Edition
Policy Engine, Access Methods,
Operating Environment
VMware vSphere
Flexibility
VMware vSphere
NFS / FC / iSCSI Storage
WS2-120
WS2-240
WS2-360
 Starts at 10 TB
 Up to 960 TB per site
ATMOS PURPOSE-BUILT HARDWARE
© Copyright 2011 EMC Corporation. All rights reserved.
28
Atmos Solution Options today
ATMOS CE
ATMOS LE
ATMOS VE
Complete Atmos functionality:
• Multiple Active/Active
• Unlimited Multi-tenancy
• Full policy controls
Subset of Atmos functionality:
• 2 sites
• 1 Tenant, 1 Subtenant
• Limited Policy Configuration
• Upgrade path to Atmos CE
Complete Atmos functionality
certified to run on EMC VNX,
VMware-Supported Servers, & ThirdParty Storage
Policy Engine, Access Methods, Operating Environment
Atmos Virtual Edition
Policy Engine, Access Methods,
Operating Environment
VMware vSphere
Flexibility
VMware vSphere
NFS / FC / iSCSI Storage
WS2-120
WS2-240
G3-DENSE-480
 Starts at 10 TB
 Up to 960 TB per site
ATMOS PURPOSE-BUILT HARDWARE
© Copyright 2011 EMC Corporation. All rights reserved.
29
Atmos Solution Options today
ATMOS CE
ATMOS LE
ATMOS VE
Complete Atmos functionality:
• Multiple Active/Active
• Unlimited Multi-tenancy
• Full policy controls
Subset of Atmos functionality:
• 2 sites
• 1 Tenant, 1 Subtenant
• Limited Policy Configuration
• Upgrade path to Atmos CE
Complete Atmos functionality
certified to run on EMC VNX,
VMware-Supported Servers, & ThirdParty Storage
Policy Engine, Access Methods, Operating Environment
Atmos Virtual Edition
Policy Engine, Access Methods,
Operating Environment
VMware vSphere
Replaces:
WS2-120
WS2-240
Flexibility
VMware vSphere
NFS / FC / iSCSI Storage
G3-FLEX-360
G3-DENSE-480
 Starts at 10 TB
 Up to 960 TB per site
ATMOS PURPOSE-BUILT HARDWARE
© Copyright 2011 EMC Corporation. All rights reserved.
30
Atmos/Virtual Edition
• Smaller
– No dedicated ESX servers required
– Shared VMs
• Cheaper through shared infrastructure
• Easier to install
–
–
–
–
Max nodes supported per RMG is 32
Up-to 960TB per RMG
Max capacity per node is 30TB
Enhanced VMWare feature support
• BUT uses primary resources
© Copyright 2011 EMC Corporation. All rights reserved.
31
Protect your assets
• GeoProtect
– Dial the right performance and protection levels to your
content
– Two implementation methods to choose from
GeoMirror
GeoParity
• Multiple copies
across locations
• Active content
• Highly Available
• Longtail content
retention
• Decrease storage
overhead
• Survive multiple
component failures
Implement both to maximize content distribution efficiency and cost
© Copyright 2011 EMC Corporation. All rights reserved.
32
Atmos GeoProtect
Protect your content investment from multiple site failures
Object written into M data segments
and N parity% segments
Determine fault domains or
unavailable components
Object can still be retrieved!
Can recreate the object from
M segments increasing data security
© Copyright 2011 EMC Corporation. All rights reserved.
Flexible options:
9/12 config.  33% overhead  tolerate 3 failures
• 10/16 config.  60% overhead  tolerate 6 failures
33
Additional Data Services
• Compression
– Reduction of physical space required for object
storage
• Deduplication
– Reduction of replicated identical objects
Object-level
data services
Node-level
data services
System-level
auto-configuration
services
• Spin Down
– Power-efficient method for long-term storage
• Striping
– Data striping within nodes or across nodes for
higher throughput
• Auto-configuration and auto-healing
– Install once—new capacity is automatically added
© Copyright 2011 EMC Corporation. All rights reserved.
34
Atmos Policy
Automated data transformation helps manage cost and service
Business determines business objectives and value of data
Applications trigger policy with metadata: “Status = UserPaid”, “UserNotPaid”
Policies drive actions at the object level:
Where to store information (location)
What data services to apply (Drive spin-down, compression, deduplication, striping)
What protection techniques to use (Atmos GeoProtect)
Premium User
Free User
Status = UserPaid
Status = UserNotPaid
Two synch copies,
striped
One asynch
copy
One asynch
copy
LOS ANGELES
NEW YORK
LONDON
© Copyright 2011 EMC Corporation. All rights reserved.
One GeoParity copy, spun down, not replicated to
save on bandwidth, delete in 90 days
NEW YORK
35
Atmos Policies User Interface
Define Conditions
based on Metadata
Select location
of Metadata
Replicated by
policy across
your Atmos Cloud
New GeoParity
options
Retained and
archived to the
cloud
© Copyright 2011 EMC Corporation. All rights reserved.
36
Atmos Integrated Solutions
Archive to the Cloud
Cloud Backup and Recovery
SourceOne
NetWorker
Content Management and Collaboration
File Tiering
FMA
DCTM
DiskXtender
Medical Imaging Archive
Mobile and File Synch
Server Gateway
WAN Optimization and Fast File Transfer
© Copyright 2011 EMC Corporation. All rights reserved.
39
Atmos Proven Customer Success
40
© Copyright 2011 EMC Corporation. All rights reserved.
40
Atmos 2.0: CAS
•
•
•
•
•
Centera apps leverage cloud storage
Centera API and XAM support
325+ ISV partner applications
Guaranteed Authenticity
Centera “Basic” functionality
LOS ANGELES
NEW YORK
LONDON
ATMOS CLOUD
© Copyright 2011 EMC Corporation. All rights reserved.
41
Interoperability for Amazon S3 Applications
• S3 Applications can leverage
– Atmos
– Atmos powered service provider
• No application changes required
• Reduces integration
• Expands service provider
opportunity
© Copyright 2011 EMC Corporation. All rights reserved.
42
Atmos GeoDrive: Windows(server) Access
Windows
Windows
Linux
Windows
Windows
Windows
7
LOS ANGELES
•
•
•
•
•
Content in the Cloud in one minute
Supports end users and applications
WAN or LAN friendly
Caching, compression, encryption
Easy to roll out, install, configure
NEW YORK
LONDON
ATMOS CLOUD
© Copyright 2011 EMC Corporation. All rights reserved.
43
Atmos GeoDrive
Simple to Install ...
Transparent movement of files …
Ready for Use …
© Copyright 2011 EMC Corporation. All rights reserved.
44
ATMOS Clouds Delivery Platform (ACDP)
Interactive
Portal
Internet
Metering
Modules
LOS ANGELES
NEW YORK
ATMOS CLOUD
© Copyright 2011 EMC Corporation. All rights reserved.
LONDON
Atmos Admin
45
With Custom Branding or Without
ADCP Default Portal
Safaricom (Kenya)
© Copyright 2011 EMC Corporation. All rights reserved.
AtmosOnline (Beta)
Telecom Italia
KDDI(Japan)
46
Cloud Tiering Appliance
© Copyright 2011 EMC Corporation. All rights reserved.
61
Cloud Tiering Appliance
• Automated file movement
• Stub-based
VNX
• Seamless retrieval
Celerra NetApp
• File retention (WORM)
requirements
VNX
VNXe
Isilon
DD
© Copyright 2011 EMC Corporation. All rights reserved.
Celerra
Atmos
Centera
62
EMC Cloud Tiering Appliance Makes it Easy
Get the right data to the right place …at the right time
 Create a policy
 Define rules
 Save and enact policy
© Copyright 2011 EMC Corporation. All rights reserved.
63
End User and Application Transparency
“The ability to archive
and recall as quick
as you can through
[Cloud Tiering
Appliance] is
unbelievable. The
user doesn’t know
the difference…
The user just sees
that it is online.”
SYSTEMS ADMINISTRATOR,
BOSTON RED SOX
Icon now
shows clock
Original file
size displayed
Offline bit
set
• Files are still visible and can be quickly recalled
• No need to re-train users or modify applications
• Backup and anti-virus processes need not be
changed
© Copyright 2011 EMC Corporation. All rights reserved.
64
The Portfolio
Centera
Centera
Atmos
Policy driven
Purpose built,
Object
Storage
WORM based,
with
long term Archive
Data Retention/
with
Expiration
Enhanced Retention
© Copyright 2011 EMC Corporation. All rights reserved.
VNX(e)
Primary
storage
with
Data Retention
Isilon
Scalable Primary
storage
with
Data Retention
Data Domain
Deduplicating
Backup storage
with
Data Retention
71
THANK YOU
© Copyright 2011 EMC Corporation. All rights reserved.
72
© Copyright 2011 EMC Corporation. All rights reserved.
73