1. technology and computing
2. operating systems
3. linux

ORACLE VM 3.3 STORAGE CONNECT
FOR VMAX WITH EMC STORAGE
INTEGRATOR V2.0
EMC® VMAX® Engineering White Paper
ABSTRACT
Oracle VM and Storage Connect framework of API, together with EMC Storage
Integrator for Oracle VM x86, enhance IT operations for managing virtualized
infrastructure by allowing EMC storage discovery, provisioning, and replication
operations to be performed with ease directly from Oracle VM Manager.
December, 2014 Version 1.1
EMC WHITE PAPER
To learn more about how EMC products, services, and solutions can help solve your business and IT challenges, contact your local
representative or authorized reseller, visit www.emc.com, or explore and compare products in the EMC Store.
Copyright © 2014 EMC Corporation. All Rights Reserved.
EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without
notice.
The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with
respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a
particular purpose.
Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.
For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.
Part Number H13603 Version 1.1
2
TABLE OF CONTENTS
EXECUTIVE SUMMARY .............................................................................. 4
AUDIENCE ................................................................................................ 5
PRODUCT OVERVIEW ................................................................................ 5
Oracle VM overview ............................................................................................ 5
Oracle VM Server ............................................................................................... 5
Oracle VM Manager ............................................................................................ 5
Oracle VM Storage Connect ................................................................................. 6
ESI for Oracle VM overview ................................................................................. 6
VMAX3 overview ................................................................................................. 6
INSTALLATION AND CONFIGURATION ..................................................... 8
Test environment and Installation Steps ............................................................... 8
Installation of Oracle VM Server for x86 ................................................................ 9
Installation of Oracle VM Server ......................................................................... 10
Installation of EMC VMAX SMI-S Provider ............................................................ 12
Installation of Oracle VM Manager ...................................................................... 14
Installation of ESI for Oracle VM for x86 ............................................................. 16
View VMAX Storage Information ........................................................................ 20
USE CASES .............................................................................................. 23
Use Case 1 – Basic Storage Management Operations ............................................ 23
Use Case 2 – Provision Storage to Oracle VM Servers ........................................... 27
Use Case 3 – Create a Server Pool ..................................................................... 29
Use Case 4 – Create a Repositry for Server Pool .................................................. 31
Use Case 5 – Create Virtual Machines from a Server Pool ...................................... 33
Use Case 6 – Clone Virtual Machines .................................................................. 36
CONCLUSION .......................................................................................... 38
REFERENCES ........................................................................................... 38
3
EXECUTIVE SUMMARY
Many IT organizations embrace virtualization and ‘as-a-Service’ Cloud deployments as a better way of achieving business agility and
efficiency. While traditional IT operations require elongated processes between different groups and departments (such as Storage,
Network, Systems, Virtualization, Database, and Applications) it is much more desirable to create an infrastructure that is suited to
support such processes in an automated, secure, and optimized way via a set of standard processes and APIs. Such an infrastructure
is provided by Oracle VM and Storage Connect APIs.
Oracle VM is the server virtualization offering from Oracle Corporation. Oracle VM Server for x86 is application-driven built for rapid
deployment of enterprise applications. Oracle VM is a Xen-based hypervisor that can support Linux, Oracle Solaris and Windows
guests. Oracle VM includes an integrated Web based management console, Oracle VM Manager.
EMC Storage Integrator (ESI) for Oracle VM version 3.3 is a plug-in that enables Oracle VM to discover and provision EMC storage
arrays such as VMAX and VNX. The ESI plug-in is built to work in conjunction with Oracle Storage Connect framework. The
framework provides a set of storage discovery and provisioning APIs that enhances the ability to manage and provision storage
devices in an Oracle VM environment.
ESI for Oracle VM uses EMC SMI-S Provider to do the actual storage management work. SMI-S Provider supports the Storage
Networking Industry Association (SNIA) Storage Management Initiative (SMI), an American National Standards Institute (ANSI)
standard for storage management.
Oracle VM and Storage Connect APIs, together with EMC Storage Integrator, enhance IT operations for managing virtualized
infrastructure. As can be seen in Figure 1, the Oracle VM administrator can add storage devices from a pre-assigned storage pools,
create snapshots of these devices, or clone complete VMs with all their attached storage. All this is done with ease from Oracle VM
Manager, utilizing EMC ESI and SMI-S APIs, and take place without need to call a storage admin.
Figure 1 Oracle VM storage tasks
4
In summary, EMC Storage Integrator for Oracle VM for x86 enables the following:
Agile – Discover and provision EMC storage for virtual machines without the need to involve a storage administrator.
Simple – EMC Storage operations are a simple to execute with a few clicks in the Oracle VM Manager.
Integrated – enables flexibility, scalability, and protection of Oracle VM environments with EMC storage.
AUDIENCE
This white paper is intended for database and system administrators, storage administrators, and system architects who are
responsible for implementing, managing, and supporting Oracle VM virtual infrastructures, and Oracle databases and storage
systems. It is assumed that readers have some familiarity with Oracle VM and EMC VMAX and VMAX3 family storage arrays, and are
interested in achieving server and database virtualization, as well as higher database availability and protection.
PRODUCT OVERVIEW
ORACLE VM OVERVIEW
Oracle VM is an integrated application-driven server virtualization solution designed for rapid deployment and simplified management
of enterprise applications. Oracle VM supports a variety of guests such as Linux, Solaris and Windows. Oracle VM is designed to
support mission-critical workloads such as Oracle Database, business applications as well as third-party enterprise applications.
ORACLE VM SERVER
Oracle VM Server provides the host on which virtual machines can run. It is lightweight and secure and while a single Oracle VM
server is sufficient.
Oracle VM Server includes a light-weight Linux kernel (known as dom0), and can manage many virtual machines (known as domU),
where each virtual machine can run Linux, Oracle Solaris, or Microsoft Windows TM. Each Oracle VM Server also runs an Oracle VM
Agent.
Oracle VM Servers can be grouped together into a cluster called server pools, where multiple server pools can be configured and
managed within an Oracle VM environment. At least one server pool is required to be configured, even if it includes only one Oracle
VM Sever. Each server pool allows for high-availability and load balancing of virtual machines across the Oracle VM servers in the
sever pool.
ORACLE VM MANAGER
Oracle VM Manager provides the Oracle VM environment with a framework that includes a web-based GUI, a command line interface,
and an integrated database as a repository. Oracle VM Manager doesn’t necessarily require high-availability as an Oracle VM
environment can continue operations even if Oracle VM Manager is down.
Oracle VM Manager consists of Oracle VM Manager Server, and Oracle VM Manager Database. Other components are interfaced via
APIs to Oracle VM Manager.
5
ORACLE VM STORAGE CONNECT
Oracle VM Storage Connect is a program and a framework that contains a set of API for storage discovery and provisioning. Using
this framework and API, users can perform such storage discovery and provisioning operations directly from Oracle VM Manager.
This enhances business agility, simplifies storage management, and allows for more robust virtualized infrastructure.
The plugins for Oracle VM Storage Connect are written by each storage vendor, utilizing their specific and advanced storage
capabilities. By using these plugins the virtualization manager doesn’t need to know how to operate and manage each storage
system, yet benefit from correct operations based on the storage vendor’s implementation of their best practices via API. In this way
storage provisioning is performed as part of the overall management of a virtualized infrastructure.
Note: For more information about the Oracle Storage Connect framework and Oracle VM, visit:
Oracle Storage Connect (http://www.oracle.com/us/technologies/virtualization/storage-connect/overview/index.html)
Oracle VM for x86 (http://www.oracle.com/us/technologies/virtualization/oraclevm/overview/index.html)
ESI FOR ORACLE VM OVERVIEW
The ESI for Oracle VM is a plug-in that enables Oracle VM to discover and provision EMC storage arrays. The plug-in is built upon the
Oracle Storage Connect framework. The framework provides a set of storage discovery and provisioning Application Programming
Interfaces (APIs) that enhance the ability to manage and provision storage devices in an Oracle VM environment.
ESI for Oracle VM supports the following storage management and provisioning operations for EMC storage arrays:
LUN discovery and inquiry
Thin LUN creation and removal
Auto-provisioning (LUN masking)
Register and refresh arrays
Create and delete access groups
Clone device creation and removal
Snap device creation and removal
EMC storage information (total and pools capacities, free space, and more)
ESI for Oracle VM uses EMC SMI-S Provider to do the actual storage management work. SMI-S Provider supports the Storage
Networking Industry Association (SNIA) Storage Management Initiative (SMI), an American National Standards Institute (ANSI)
standard for storage management.
VMAX3 OVERVIEW1
The EMC VMAX3 family storage arrays are built on the strategy of simple, intelligent, modular storage, and incorporates a Virtual
Matrix interface that connects and shares resources across all VMAX3 engines, allowing the storage array to seamlessly grow from an
entry-level configuration into the world’s largest storage array. It provides the highest levels of performance and availability
featuring new hardware and software capabilities.
The newest additions to the EMC VMAX3 family, VMAX 100K, 200K and 400K, deliver the latest in Tier-1 scale-out multi-controller
architecture with consolidation and efficiency for the enterprise. With completely redesigned hardware and software, the new VMAX3
array provides unprecedented performance and scale. It offers dramatic increases in floor tile density with engines and high capacity
disk enclosures for both 2.5" and 3.5" drives consolidated in the same system bay.
1
At the time this paper is written, VMAX with Enginuity 5876 and SMI-S 5.7.6 is supported. VMAX3 with HYPERMAX OS 5977 and SMI-S 8.0 support is
planned to be added shortly.
6
In addition, VMAX3 arrays can be configured as either hybrid or all- flash arrays. All new VMAX3 models come pre-configured from
the factory to significantly shorten the time from installation to first I/O.
1 – 8 redundant VMAX3 Engines
Up to 4 PB usable capacity
Up to 256 FC host ports
Up to 16 TB global memory (mirrored)
Up to 384 Cores, 2.7 GHz Intel Xeon E5-2697-v2
Up to 5,760 drives
SSD Flash drives 200/400/800 GB 2.5”/3.5”
300 GB – 1.2 TB 10K RPM SAS drives 2.5”/3.5”
300 GB 15K RPM SAS drives 2.5”/3.5”
2 TB/4 TB SAS 7.2K RPM 3.5”
Figure 2 VMAX3 storage array
VMAX3 engines provide the complete foundation for high-availability storage arrays. Each engine contains two VMAX3 directors and
redundant interfaces to the new Dynamic Virtual Matrix dual InfiniBand® fabric interconnect. Each director consolidates front-end,
global memory, and back-end functions, enabling direct memory access to data for optimized I/O operations. Depending on the
array chosen, up to eight VMAX3 engines can be interconnected via a set of active fabrics that provide scalable performance and high
availability. Systems ports are no longer mapped directly to CPU resources. CPU resources are allocated as needed using pools
(front-end, back-end, or data services pool) of CPU cores which can service all activity in the VMAX3 array.
All storage in the VMAX3 array is virtually provisioned. All data pools are created in containers called Storage Resource Pools. The
array uses the dynamic and intelligent capabilities of Fully Automated Storage Tiering (FAST) to meet system or user-defined Service
Level Objectives throughout the lifecycle of each application. VMAX3 Service Level Objectives are tightly integrated with EMC FAST
software to optimize agility and array performance across all drive types in the system. EMC FAST technology improves application
performance while reducing cost by using Flash drives combined with cost-effective high-capacity drives.
7
For local replications, VMAX3 adds a new feature to TimeFinder software called SnapVX, which provides “hybrid cloud scale”
snapshots. Unlike previous VMAX snapshots, SnapVX snapshots do not require the use of dedicated target devices. SnapVX allows
for up to 256 snapshots per individual source. These snapshots can be linked to copy their data to new target devices, or re-linked to
update just the incremental data changes.
VMAX3 new features and capabilities provide Oracle VM and Oracle databases more flexibility, scalability, protection, and
performance, together with a dramatic increase in management simplification and ease of use.
INSTALLATION AND CONFIGURATION
TEST ENVIRONMENT AND INSTALLATION STEPS
Overview of Test Environment
The test environment was kept small for demonstration purposes. A larger scale environment can be easily set in a similar way with
larger server pools, more networks, and Oracle VM Servers.
Figure 3 Test environment
The Oracle VM servers were configured using two Dell PowerEdge R710. Oracle VM Manager was configured using a Dell R900. The
SMI-S Provider was installed on the same server as Oracle VM Manager for simplicity. A management public network was set
between all three servers and a private network was set just between the Oracle VM servers as they were clustered and belonged to
the same server pool. SAN connectivity was set between the VMAX 10K storage array and the Oracle VM Servers leveraging
redundant FC switches, host multipathing, and multiple ports on the array. The Storage Management host had connectivity to only a
few small devices for communication with the storage array.
8
List of hardware components can be seen in Table 1, and software component in Table 2.
Table 1 Hardware components
Item
Type
Oracle VM Servers
2 x Dell R710
Oracle VM Manager and SMI-S Provider server
1 x Dell R900
Storage array
1 x VMAX 10K
SAN Switch
2 x EMC Connectrix DS-6510B
Host HBAs (per host)
2 x Emulex LPe12002
Table 2 Software components
Item
Version
Oracle VM Server
3.3
Oracle VM Manager / Host operating system
3.3 / Oracle Linux 6.5
ESI for Oracle VM 3.3
V2.0
EMC SMI-S Provider for VMAX and VNX
7.6.2.9
Summary of Installation Steps
The following are the high-level steps for installing and configuring ESI for Oracle VM:
1.
Install all Oracle VM Servers for x86
2.
Install EMC VMAX SMI-S Provider
3.
Install Oracle VM Manager
4.
Install ESI for Oracle VM
The following sections explain each step in more details.
INSTALLATION OF ORACLE VM SERVER FOR X86
Oracle VM Servers are the physical compute nodes hosting the virtual machines. Oracle installation for Oracle VM Server is simple,
light weight, and contains only packages necessary for its task. For this reason it is not recommended to use the Oracle VM Server
for other purposes, such as Solutions Enabler array management software, or Linux package repositories for upgrades.
Note: Do not create a VMAX Masking View2 for the Oracle VM Servers. All storage provisioning operations between VMAX and the
Oracle VM Servers should be performed in Oracle VM Manager utilizing Storage Connect APIs. Device masking operations outside of
Oracle Storage Connect (such as by using Unisphere for VMAX or Solutions Enabler CLI) won’t be recognized by Oracle VM Manager.
An exception is device creation as Oracle VM Manager can discover and use existing VMAX devices.
2
VMAX Masking View is a component of VMAX Auto-provisioning Group. The masking view task is to make storage devices visible to hosts by creating a
relationship between a group of devices, a group of host HBA ports (initiators), and a group of storage array ports the devices are visible to.
9
INSTALLATION OF ORACLE VM SERVER
Oracle VM Server installation tasks
To install and configure Oracle VM Server, perform the following actions:
1. Navigate to http://www.oracle.com and sign in with your user account.
2. Navigate to Downloads > Linux and Oracle VM and download Oracle VM 3.3 Server for x86 installation media.
3. Follow the installation steps described in Oracle Installation and Upgrade Guide for Release 3.3. This will allow you to:
Install Oracle VM Server software
Configure the hostname
Configure network information
Define Oracle VM Agent and root user passwords
Note: During the installation the first password asked for is for the Oracle VM Agent. This password will be used later when Oracle
VM Manager discovers and connects with the Oracle VM Server.
Oracle VM Server post installation tasks
Post installation tasks may include the configuration of additional network interfaces and other tasks3. Oracle VM Servers are
discovered via the public network, however, Oracle recommends using dedicated networks or VLAN for communication between
Oracle VM Servers within a Server Pool. Additional networks can be configured for purposes such as Virtual Machine private network,
Live Migration, and Cluster Heart-bit. Refer to Oracle VM Concept Guide for release 3.3 for more details about Oracle VM networking.
Note: All hosts in the cluster should have the same subnet
For simplicity, this document describes the configuration of two Oracle VM Servers, with a dedicated private network, as shown in
Table 3.
Table 3 Oracle VM Server Networks
ORACLE VM SERVER 1
ORACLE VM SERVER 2
ORACLE VM MANAGER
SERVER NAME
DSIB0062
DSIB0063
DSIB0064
PUBLIC IP/NIC
10.108.244.62/EM1
10.108.244.63/EM1
10.108.244.64
PRIVATE IP/NIC
192.168.1.62/EM4
192.168.1.63/EM4
During Oracle VM Server installation, the hostname and public network information are configured. To set additional network
interfaces perform the following actions on each of the Oracle VM Servers, making changes to the network information as
applicable.
3
Oracle VM 3.3 Administration Guide documents the steps to increase dom0 memory size to avoid performance issues, and to add SNMP monitoring
tools.
10
The example in this paper demonstrates the following activities:
Manual configuration of an additional network interface for private network
Update of /etc/hosts to include the private network
Test the new interface
Edit the appropriate network interface connected to the private network switch:
[root@dsib0062 ~]# cd /etc/sysconfig/network-scripts/
[root@dsib0062 network-scripts]# cp ifcfg-em4 sav_ifcfg-em4.sav
[root@dsib0062 network-scripts]# vi ifcfg-em4
Update the file to include the private network information:
DEVICE="em4"
BOOTPROTO="none"
DHCP_HOSTNAME="dsib0062"
HWADDR="A4:BA:DB:41:7B:6B"
NM_CONTROLLED="yes"
#orig: ONBOOT="no"
TYPE="Ethernet"
UUID="439b72dc-75d9-485c-9468-1118576e9eff"
# Added
ONBOOT="yes"
IPADDR="192.168.1.62"
NETMASK="255.255.255.0"
Restart network service for the changes to take effect:
[root@dsib0062 network-scripts]# service network restart
Update hosts file with the new host information (unless DNS is used):
[root@dsib0062 ~]# vi /etc/hosts
Include the relevant network information:
127.0.0.1
10.108.244.62
10.108.244.63
192.168.1.62
192.168.1.63
localhost
dsib0062
dsib0063
dsib0062-priv
dsib0063-priv
Ping the private network between the servers to test the connection:
[root@dsib0062 ~]# ping dsib0063-priv
PING dsib0063-priv (192.168.1.63) 56(84) bytes of data.
64 bytes from dsib0063-priv (192.168.1.63): icmp_seq=1 ttl=64
64 bytes from dsib0063-priv (192.168.1.63): icmp_seq=2 ttl=64
64 bytes from dsib0063-priv (192.168.1.63): icmp_seq=3 ttl=64
64 bytes from dsib0063-priv (192.168.1.63): icmp_seq=4 ttl=64
time=0.190
time=0.075
time=0.072
time=0.072
ms
ms
ms
ms
11
INSTALLATION OF EMC VMAX SMI-S PROVIDER
Storage management host
EMC SMI-S Provider is the software that complies with SNIA SMI-S standards for storage management and works in conjunction with
Solutions Enabler4 software. Oracle VM Storage Connect for VMAX uses SMI-S for all storage monitoring and control operations.
Note: EMC SMI-S Provider and Solutions Enabler software should not be installed on the Oracle VM Servers. Instead, they should be
installed on a host referred to as the Storage Management host. In some deployments, the user may prefer to combine the Storage
Management host with Oracle VM Manager host though it is not required.
Note: Solutions Enabler requires a number of very small devices called Gatekeepers to be presented to the Storage Management
host for communication with the VMAX. There should be a minimum of six Gatekeepers visible to the Storage Management host
before Solutions Enabler and SMI-S are installed.
SMI-S Provider installation
In order to install the SMI-S Provider, perform the following actions on the Storage Management host.
Host Preparation
1. Engage the storage administrator and have 6 or more Gatekeepers provisioned and visible to the Storage Management host.
2. Update the Storage Management host path to include the path to Solutions Enabler and SMI-S binaries.
[root@dsib0064 ~]# vi ~.bashrc
...
export PATH=$PATH:/usr/symcli/bin:/opt/emc/ECIM/ECOM/bin:.
3. Source the file to take effect.
[root@dsib0064 ~]# . ~/.bashrc
SMI-S Installation Steps
1. Navigate to http://www.emc.com.
2. Navigate to Support > Downloads and search for SMI-S Provider.
3. Download SMI-S Provider 4.6.2.9 for SMI-S for Linux (or a later release)5.
4. Copy the file to the Storage Management server and extract it.
4
Solutions Enabler software allows management of VMAX storage array using APIs, CLIs, and REST. It works in conjunction with other VMAX
management solutions such as Unisphere for VMAX GUI, EMC SMI-S Provider, EMC ViPR, and others.
5
SMI-S 4.6.2.9 release is considered a ‘customer hot-fix’ for SMI-S. If SMI-S 4.6.2.9 (or a later release) is not available on EMC SMI-S download page
request it from your EMC account representative as a minimum software requirement.
12
5. Follow the installation steps described in EMC SMI-S Provider Version 4.6.2 Release Notes. The main steps are described below.
Extract the installation software and run it:
[root@dsib0064 SE]# tar xvfz se7629-Linux-x86_64-SMI.tar.gz
[root@dsib0064 SE]# ./se7629_install.sh –install
...
#----------------------------------------------------------------# The following HAS BEEN INSTALLED in /opt/emc via the rpm utility.
#----------------------------------------------------------------ITEM PRODUCT
VERSION
01
EMC Solutions Enabler
V7.6.2.9
02
SMI KIT
V4.6.2
#----------------------------------------------------------------Run the storage discovery command6:
[root@dsib0064 SE]# symcfg discover
Test the SMI-S provider installation and communication with the storage array. To do so run7 TestSmiProvider command, accept
all the defaults until the command screen shows, then use Display Version Info (dv) command to test connectivity
[root@dsib0064 ~]# TestSmiProvider
...
Built with EMC SMI-S Provider: V4.6.2
Namespace: root/emc
repeat count: 1
(localhost:5988) ? dv
++++ Display version information ++++
CIM ObjectManager Name: EMC:10.108.244.63
CIMOM Version: EMC CIM Server Version 2.7.3.3.0.8
SMI-S qualified version: 1.5.0
SMI-S Provider version: V4.6.2.3
SMI-S Provider Location: Proxy
SMI-S Provider Server:
Linux dsib0063 3.8.13-35.1.1.el6uek.x86_64 #2 SMP Fri Jun 6 10:52:33 PDT 2014 x86_64
(64bit Libraries)
Solutions Enabler version: V7.6-1808 2.9
Firmware version information:
(Local) Symmetrix Array 000198700067 (VMAX10K) : 5876.268.174
Retrieve and Display data - 1 Iteration(s) In 0.327698 Seconds
Please press enter key to continue...
6
It is assumed that the root user PATH was updated with Solutions Enabler binaries location, which has a default location of: /usr/symcli/bin
It is assumed that the root user PATH was updated with the SMI-S Provider binaries location, which has a default location of:
/opt/emc/ECIM/ECOM/bin
7
13
Restarting SMI-S
If SMI-S Provider is not running, or if Oracle VM shows messages such as “Connection to Admin host failed” restart SMI-S Provider
by using the following command:
[root@dsib0064 ~]# ECOM -d
INSTALLATION OF ORACLE VM MANAGER
Oracle VM Prerequisites
Oracle VM Manager is required to be installed on a separate host than Oracle VM Servers.
Note: The Storage Management host and Oracle VM Manager may reside on the same host, or separate hosts. In this paper the
same host has been used for both.
For more information on Oracle VM Manager installation requirements refer to Oracle VM Installation and Upgrade Guide for Release
3.3. The main tasks are described below.
Install Oracle VM Manager
Install NTP Server on the Oracle VM Manager host to provide time coordination for the VMs.
1. Install the NTP server.
[root@dsib0064 ~]# yum install ntp
2. Configure the NTP Daemon to start at boot.
[root@dsib0064 ~]# chkconfig ntpd on
3. Start the NTP service.
[root@dsib0064 ~]# service ntpd start
4. Mount the Oracle VM Manager media and run the installer.
[root@dsib0064 ~]# mount -o loop /tmp/ovmm-3.3.1-installer-OracleLinux-b1065.iso /mnt
[root@dsib0064 ~]# cd /mnt
[root@dsib0064 mnt]# ./runInstaller.sh
During the installation MySQL database is installed for Oracle VM Manager repository.
Oracle VM Manager Post Installation Tasks
Oracle offers Oracle VM Utilities to help with the management of Oracle VM tasks from a command line. Refer to Oracle VM
Administrator’s Guide for Release 3.3 for a description of Oracle VM Utilities and their installation procedures.
Oracle also provides a package that should be installed on Oracle VM Manager host in order to provide VNC access to the Oracle VMs.
Download the ovmcore-console package from Oracle VM Download page and install it. If Linux Firewall is configured you’ll need to
open the appropriate ports or disable it. Refer to Oracle VM Installation and Upgrade Guide for details on the installation tasks of VNC
and Serial Console for Oracle VM.
14
In this example the ovmcore package is installed on the Oracle VM Manager host.
[root@dsib0064 OVM]# rpm -ivh ovmcore-console-1.0-36.el6.noarch.rpm
warning: ovmcore-console-1.0-36.el6.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Preparing...
########################################### [100%]
1:ovmcore-console
########################################### [100%]
If you wish to disable Linux firewall:
[root@dsib0064 ~]# service iptables stop
[root@dsib0064 ~]# chkconfig iptables off
If you wish to disable SELinux:
[root@dsib0064 ~]# vi /etc/selinux/config
...
SELINUX=disabled
Networking
Networking in a virtualized environment is an important topic that requires adequate thought and planning. Refer to Oracle VM
Concept Guide for Release 3.3 for details on such considerations. In this paper a single private network is added, as shown in Figure
4. It will be used for VM’s communication and Live Migration.
1. To create an additional network, choose the Networking tab.
2. Click
to create a new network.
Figure 4 Configure Oracle VM networks
15
INSTALLATION OF ESI FOR ORACLE VM FOR X86
ESI for Oracle VM version 3.3 is the software that utilizes Oracle Storage Connect plugins to monitor and manage the VMAX storage
array from Oracle VM. Refer to Table 4 for installation prerequisites.
Prerequisites checklist
Table 4 ESI for Oracle VM prerequisites
ESI FOR ORACLE VM PREREQUISITES
1.
ORACLE VM SERVERS INSTALLED
2.
ORACLE VM MANAGER INSTALLED
3.
SMI-S PROVIDER INSTALLED
COMPLETE
Installation steps
1. Navigate to http://www.emc.com.
2. Navigate to Support > Downloads and search for ESI for Oracle VM.
3. Download the ESI for Oracle VM plugin.
4. Copy the plugin to each of the Oracle VM Servers.
5. Install the rpm on each of the Oracle VM Servers.
[root@dsib0062 ESI]# tar xvf esi_plugin.2.0.0.5_OVM-3.3.tar
emc-osc-isa-2.0.0.5-1.el6.x86_64.rpm
[root@dsib0062 ESI]# rpm -ivh emc-osc-isa-2.0.0.5-1.el6.x86_64.rpm
Preparing...
########################################### [100%]
1:emc-osc-isa
########################################### [100%]
16
ESI Configuration File
ESI for Oracle VM uses a configuration file to pass user parameters that affect its behavior with Storage Connect.
Note: It is highly recommended to read the following section and be familiar with ESI configuration file options.
ESI for Oracle VM Configuration File Parameters:
AccessGroupPrefix: this parameter allows modifying the default prefix for VMAX Auto-provisioning Group names8. While
this prefix is not visible in Oracle VM Manager screens, it will be visible to the Storage Admin monitoring the VMAX storage
array using tools such as Unisphere for VMAX or Solutions Enabler CLI.
AutoMetaEnabled9: True means that the VMAX Auto-meta feature is configured and enabled. False means that it isn’t (you
can check by running Solutions Enabler command: symcfg list -v | grep “Auto Meta”). If left unset, the plugin will
inquire with the storage via SMI-S to check whether AutoMeta is enabled and will take the appropriate value.
Note: It is recommended to not set this parameter as its correct value will be found automatically by ESI plugin.
VolumeGroupPrefix: Provides a way of limiting which VMAX Pools are visible to Oracle VM. In this way Oracle VM can view
and access only certain storage Pools. The parameter should be consistent across all the Oracle VM Servers accessing the
same VMAX storage. Modifying this parameter after the storage was discovered will require removal of the array in Oracle
VM Manager and rediscovery. Prior to ESI for Oracle VM version 2.0.0.5, the Prefix can only contain part of the Pool name,
but not the full name of the pool. Starting with version 2.0.0.5 the prefix can include partial or full Pool name.
LogLevel: provides a way to control the level of detail in the Oracle VM Server log file located at /var/log/ovs-agent.log.
Set to “debug” for detailed information. Other values allowed: “info”, and “error”.
The log referenced in isa.conf is located at: /var/log/ovs-agent.log on the Oracle VM Server.
Create a new configuration file, or copy an existing one from a previously installed Oracle VM Server with ESI:
[root@dsib0062 ESI]# vi /opt/storage-connect/plugins/emc/isa/isa.conf
AccessGroupPrefix=osc_
AutoMetaEnabled=false
VolumeGroupPrefix=FC,SATA
LogLevel=debug
Note: If the ESI plugin is upgraded, perform rediscovery of the Oracle VM Servers by right clicking on each and choosing
Rediscover Server from the menu.
8
The default prefix is osc_ for Oracle Storage Connect. A different one, such as ovm_ can be easily used.
Meta volumes is a feature available in VMAX with Enginuity code level 5876 and below. It allows for storage devices aggregation as a striped or
concatenated group of devices, and at the same time allows the group (meta-volume) to grow beyond the 240GB limit of a single device. VMAX3 with
HYPERMAX OS 5977 and above no longer supports or requires meta-volumes as all devices are virtual and natively striped, and each device capacity
can reach multi-terabytes.
9
17
Registration of Oracle VM Servers containing ESI Plugin with Oracle VM Manager
Once the ESI for Oracle VM plugin has been installed on the Oracle VM Servers, the Oracle VM Servers and VMAX storage array will
both need to be registered (even if the servers were previously registered with Oracle VM Manager).
1. Open a browser and navigate to https://<hostname>:7002/ovm/console
Example: https://dsib0064:7002/ovm/console
2. Log in with username admin using the password that was set during the installation of Oracle VM Manager.
3. Navigate to the Servers and VMs tab.
4. Click on the Discover Servers icon
.
5. Enter the Oracle VM Agent password and Oracle VM servers IP addresses/hostnames. Click OK, as shown in Figure 5.
Figure 5 Discover Oracle VM Servers
18
6. Once discovered, the servers will be displayed under Unassigned Servers, as shown in Figure 6.
Figure 6 Discovered Oracle VM Servers
Note: In case the discovery encounters a failure during storage refresh action validate that the Oracle VM Servers can each ping the
management host back (/etc/hosts or DNS are set correctly in each Oracle VM Server), and that SMI-S is running.
19
VIEW VMAX STORAGE INFORMATION
Discovery of VMAX Storage Array
In order to discover the VMAX storage array and perform cloning and provisioning operations, the first task is to register the SMI-S
provider that has access to the VMAX storage.
1. Navigate to the Storage tab and click on the Discover SAN Server icon
, as shown in Figure 7.
Figure 7 Discover VMAX Storage Array
2. Update the following fields appropriately, as shown in Figure 8.
Name – User defined name for EMC storage array
Description – User defined description for EMC storage array
Storage Type – Select FibreChannel
Storage Plug-in – Select EMC FC SCSI Plugin
Plug-in Private Data – Provide the VMAX array ID (either full or partial ID number)
Note: This field is mandatory for ESI plugin. Partial ID is allowed only for the end of the ID (e.g. 198700067, or 067 are valid)
Admin Host – Provide the host name or IP address of the host where SMI-S is installed
Admin Username – Provide the SMI-S admin username. The default is admin
Admin Password – Provide the SMI-S admin user password. The default is #1Password
20
Figure 8 Enter Storage Discovery Information
3. Click Next to continue. The Oracle VM Servers appear under Available Server(s). Move them to the right side under Selected
Server(s) then click Finish.
Identifying VMAX storage information from Oracle VM Manager
After the discovery job completes, the storage will appear with basic information, as shown in Figure 9.
Figure 9 Storage Summary
21
More information about the storage array such as the Pools and the devices associated with each pool can be viewed by selecting the
storage from the list of SAN Servers. Additional VMAX device information, such as device ID (appears as “Name”, or “Extra
Information”) can be viewed by expanding any of the storage devices listed, as shown in Figure 10. Storage devices also show which
of the Oracle VM Servers they are visible to, Access Groups they belong to, etc.
Figure 10 Storage Devices Information
Identifying VMAX storage devices directly from Oracle VM Server
Oracle VM Administrator can view basic VMAX device information from the Oracle VM Servers by using Linux commands such as
multipath or scsi_id.
With EMC PowerPath, powermt display dev=all command can be used. If Linux dm-multipathing is used, it can list multipath
devices with their name, serial, and capacity.
[root@dsib0062 ~]# multipath -l
360000970000198700067533030313435 dm-1 EMC,SYMMETRIX
size=100G features='1 queue_if_no_path' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=0 status=active
`- 3:0:0:1 sdc 8:32 active undef running
360000970000198700067533030323235 dm-10 EMC,SYMMETRIX
size=100G features='1 queue_if_no_path' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=0 status=active
`- 3:0:0:10 sdl 8:176 active undef running
360000970000198700067533030323234 dm-9 EMC,SYMMETRIX
size=100G features='1 queue_if_no_path' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=0 status=active
`- 3:0:0:9 sdk 8:160 active undef running
...
22
Linux scsi_id command can also list storage device information with the addition of the VMAX Device ID. The field ID_SCSI_SERIAL
contains the last 6 digits of the VMAX Storage ID followed by the 3 digits of the VMAX Device ID.
[root@dsib0062 ~]# for i in `ls /dev/dm-*`; do echo $i; scsi_id -g -x $i | grep ID | grep SERIAL= ; done
/dev/dm-0
ID_SERIAL=360000970000198700067533030303134
ID_SCSI_SERIAL=700067014000
/dev/dm-1
ID_SERIAL=360000970000198700067533030313435
ID_SCSI_SERIAL=700067145000
/dev/dm-10
ID_SERIAL=360000970000198700067533030323235
ID_SCSI_SERIAL=700067225000
...
USE CASES
The following section assumes that all the configuration steps have been completed and provide an introduction to some of the basic
Oracle VM operations that Storage Connect helps achieve.
Oracle VM Storage Connect with ESI provides the Oracle VM Administrator with a way of performing tasks such as storage creation,
provisioning, and cloning with ease and simplicity.
The first use case shows basic storage provisioning tasks. Following use cases demonstrate how to provision storage devices to
Oracle VM Servers, Virtual Machines, create storage, repositories, and how to perform cloning operations at the VM level.
USE CASE 1 – BASIC STORAGE MANAGEMENT OPERATIONS
Review available storage pools capacities
Before adding new storage devices, let’s inspect the available capacities of the storage pools.
1. Choose the Storage tab at the top of the Oracle VM Manager screen.
2. On the left, under SAN Servers, choose the appropriate storage array, as shown in Figure 11. Click on the small triangle to the
left of the Storage name to expand its details and the available storage pools will be listed. By using the Info perspective it is
easy to view how much capacity is available in each storage pool. In Figure 11 we can see that the FC_Pool has 3,611.39 GB of
available capacity.
Figure 11 View Storage Pool Available Capacity
23
Create New Storage Devices
To create a new storage device change to Physical Disks perspective (refer also to Figure 14).
1. Click on the Create Physical Disk icon
, as shown in Figure 12.
Figure 12 Create Physical Disk
Volume Group: If a specific array Storage Pool was already selected it will be used as Volume Group. If the Array Name was
selected instead, a drop-down list of the available storage pools will be listed. Select the appropriate storage pool.
Name: Provide a descriptive name for the new device.
Extra Information: This box can be ignored. It can be used during creation of storage meta volumes when Auto-meta is not
configured in the array. For example, it could say: meta_member_size=200G. However, it is optional even when creating meta
volumes.
Description: Provide an appropriate description for the new device.
Shareable: Specifies for Oracle VM that the device can be shared between multiple Virtual Machines with simultaneous read/write
access.
Thin Provision: Only Thin devices are supported for Oracle VM Storage Connect with ESI 2.0
24
2. Click OK to create the storage device.
Once the device is created, it will show in the Physical Disk list for the storage array. By clicking on the small triangle to the left of
the device name more information can be viewed, as shown in Figure 13. The VMAX device ID is shown to the right of Extra
Information field. For example, Oracle ERP Disk 1 is using device ID 625.
Figure 13 Show Device Information
The Storage Admin can identify the Oracle VM devices either by their ID (for example: device ID 625), or by their name. To list
device names they can use Solutions Enabler command: symdev list -ident device_name.
[root@dsib0064 ~]# symdev list -ident device_name
Symmetrix ID: 000198700067
Device
------------------------------------------------------------------------------Sym Config
Attr Device Name
---- --------------- ---- ----------------------------------------------------01F3 TDEV
ORA_DATA1
0622 TDEV
Template1
0623 TDEV
Volume 00147_clone
0624 TDEV
Volume 00148_clone
0625 TDEV
Oracle ERP Disk 1
0626 TDEV
Volume 00147_clone
...
25
Manage Storage Devices
By selecting a VMAX storage device, a number of operations can be executed on the device, as shown in Figure 14.
Figure 14 Manage Storage Device
Device operations:
Edit Device: Attributes such as Device Name, Description, and Shareable can be modified.
Note: It is not allowed to change device capacity.
Delete Device: allows the user to delete the device.
Clone Device: allows the user to clone a device.
In the Clone Target Type field select: Storage Array.
In the Name field provide an optional name for the clone.
o
If a name wasn’t provided the cloned device will show as the original device name with the addition of a
bracket and a clone number. For example, ‘Oracle ERP Disk 1 (2)’ is a default name for a clone to a device
named ‘Oracle ERP Disk 1’.
o
If the name provided ends with ‘_EMC_SNAP’, then rather than a full-sized clone, a space-efficient snapshot
will be created with copy-on-write functionality (only changes to either source or target devices consume
additional storage capacity)
Note: As there is no other way in Storage Connect to pass information to the array, whether to create a full
copy or space-efficient copy of the source device, use the clone name to do so. The suffix ‘_EMC_SNAP’ will
result in a space-efficient clone, aka snap.
26
Only Thin Clone value is allowed in the Clone Type field.
After the snap or clone was created, as before, by clicking on the small triangle to the left of the clone device name its
VMAX device ID can be viewed in the Extra Information field.
USE CASE 2 – PROVISION STORAGE TO ORACLE VM SERVERS
Provisioning storage devices to the Oracle VM Servers is done by creating Access Groups. Storage devices in Access Groups can later
be used as storage for the Oracle VM Servers (for example: Server Pool storage, or Repositories), or provisioned to the virtual
machines as physical disks.
Note: This section assumes that storage devices were already created for Oracle VM and can be simply added to the Access Group.
To learn about creating new storage devices refer to Use Case 1.
In order to create an Access Group:
1. Select the Storage tab.
2. Choose Access Group perspective from the drop-down menu.
3. Click on the
icon that creates a new Access Group, as shown in Figure 15.
Figure 15 Create Access Group
4. In the window that opens provide a name and description for the Access Group. Click Next.
Note: Access Group name must not contain any spaces. It will be used to create a VMAX Storage Group name (together with a predefined prefix that can be modified in the ESI configuration file).
27
5. In the Initiators (HBA ports) selection screen, on the left side you can see a list of the Oracle VM Servers initiators prefixed by
the server name and followed by the port WWN. You may notice some WWN’s showing only zeros. These can be ignored as they
are not showing storage connectivity. Only initiators that have been zoned correctly to the storage array will show their WWN as
a non-zero address and can be selected for the Access Group HBA ports connectivity.
6. Move the appropriate initiators to the right side of the screen as shown in Figure 16. An Access Group that includes initiators of
multiple Oracle VM Servers can provide shared storage to these servers. Click Next.
Figure 16 Access Group initiators selection
7. In the next screen select the storage devices that will be provisioned to the Oracle VM Servers, as shown in Figure 17. Click
Finish to complete the task.
Figure 17 Access Group storage selection
28
After the Access Group is created, by switching to the Physical Disk perspective, the Server column will reflect the Oracle VM
Servers in the Access Group for the appropriate devices, as shown in Figure 18.
Figure 18 Access Group servers listed
USE CASE 3 – CREATE A SERVER POOL
The Oracle VM Servers can be placed in a Server Pool to allow shared storage for repositories, high availability and easy migration of
virtual machines.
1. To create a Server Pool select the Servers and VMs tab and click on the icon that creates a Server Pool, as shown in Figure 19.
Figure 19 Create server pool
29
2. Provide a Server Pool Name and a Virtual IP to manage the pool. Note that Clustered Server Pool will be checked by default.
3. In the Storage for Server Pool field choose Physical Disk and click on the search icon to select storage, as shown in Figure
20.
Figure 20 Storage selection for server pool
Storage devices visible to the Oracle VM Servers will be listed. Select one of them for the Server Pool shared storage. An OCFS2
filesystem will be created on this device and mounted on the Oracle VM Servers selected for the Server Pool. Click OK and you’ll be
returned to the Create Server Pool screen where you can add a description. Click Next. At this time the Oracle VM Servers for the
Server Pool should be selected. Move the appropriate servers to the right side of the screen and click Finish to complete the task, or
Next to add Tags. The Sever Pool will be listed upon completion, as shown in Figure 21.
Figure 21 Server pool created
30
USE CASE 4 – CREATE A REPOSITRY FOR SERVER POOL
For Server Pool clustering, a shared storage is required. For SAN storage, a disk image is created and formatted using OCFS2 (Oracle
Clustered Filesystem).
1. Change to the Repositories tab and click on the icon to add a repository, as shown in Figure 22.
Figure 22 Create a storage repository
2. In the Create a Repository window enter the repository name.
3. Choose the Physical Disk radio button and the Server Pool the Repository will belong to.
4. Next, click on the icon to discover a Physical Disk for the repository storage allocation, as shown in Figure 23.
Figure 23 New Repository - Storage Selection
5. In the Select Physical Disk screen select the VMAX storage array and Pool. A list of shared storage devices will be available.
Note in the example that the first device (volume 00145) is no longer visible as it was selected previously for the Server Pool
storage. The next volume, 00146, is therefore selected in this example.
6. After the storage selection for the repository you’ll be returned to the Create Repository screen where you can add a
description. Click Next. Select the Oracle VM Servers for the repository by moving them to the right side of the screen and click
Finish.
31
The Repository, as shown in Figure 24, is created and divided into 5 categories as explained in the Oracle VM Concept Guide:
Assemblies, which are a preconfigured collection of virtual machines, typically created with Oracle Assembly Builder.
ISO files that can be mounted as a virtual CD/DVD.
VM Templates for virtual machine templates (templates can be downloaded from Oracle website, or created).
VM Files containing the configuration files of the virtual machines.
Virtual Disks that can be dedicated or shared between VMs.
Figure 24 Newly Created Repository
32
USE CASE 5 – CREATE VIRTUAL MACHINES FROM A SERVER POOL
Import an Oracle VM Assembly
An easy way to create virtual machines is from Assemblies or Templates. Assemblies and Templates can be downloaded from Oracle
Software Delivery Cloud under Oracle VM Templates. They are not installed directly into the Oracle VM Servers and rather imported
by Oracle VM Manager using http, https, or ftp.
In this example, vsftpd (secure ftp), is installed on the Oracle VM Manager host and the vsftpd service started. An OVF set is selected
from Oracle website: Oracle VM 3 Templates (OVF) for Oracle Linux 6 Media Pack v7 for x86_64 (64 bit). It opens a new window in
the browser with multiple OVFs. Download and copy to the Oracle VM Manager host the appropriate template. Once copied, unzip the
packages.
[root@dsib0064 OVM]# cd /var/ftp/OVM
[root@dsib0064 OVM]# unzip OVF_OEL65_V45138-01.zip
Import the OVA as an Assembly first, and then turn it into a Template. To do so:
1. Go to the Repository tab and choose Assemblies under the previously created Repository (Repository 1).
2. Click on the Import icon, as seen in Figure 25 . In this example ftp protocol is used with the ftp user on the Oracle VM Manager
host. The OVM is a sub-directory under the ftp user home directory (/var/ftp/OVM) where the .ova file is stored. The following
link is inserted in the Assembly download location: ftp://ftp:ftp@dsib0064/OVM/OVM_OL6U5_X86_64_PVM.ova.
Figure 25 Import an Assembly
33
Create an Oracle VM Template from an Assembly
1. Once the Assembly is imported select it, and click the icon to create a VM Template, as shown in Figure 26.
2. In the window that opens enter the template name and click OK.
Figure 26 Create a VM Template
Create an Oracle VM from a Template
Under Servers and VM tab click on the icon to create a new virtual machine. For example, the previously imported template can be
used to create 3 VMs by selecting Clone from an existing VM Template, and providing the appropriate information, as shown in
Figure 27.
Figure 27 Create New Virtual Machines
34
3. Select a VM and click on the Edit icon, as shown in Figure 28.
4. Under the Network tab add the private network created earlier for VMs. Under Storage tab add Physical or Virtual disks as
necessary. Virtual Disks capacity is taken from the storage device assigned to the Server Pool Repository. Physical Disks can be
chosen from the Access Group storage associated with the Server Pool.
5. After the update is completed the VM can be started.
Figure 28 Update VM Information
35
USE CASE 6 – CLONE VIRTUAL MACHINES
A Virtual Machine can easily be cloned together with its assigned storage devices. To do so:
1. Select the VM from the list, right click and choose Clone or Move, or click on the icon: Clone or Move Virtual Machine.
2. Optionally, provide a Clone Name, Count, and Index, as shown in Figure 29. It is also possible to set an advanced cloning with
Clone Customizer (for example, to use storage from another Repository).
3. To finish the operation click OK.
Figure 29 Clone a Virtual Machine
36
In this following example the VM OL6U5_X86_64_PVM.1 is cloned. When the clone operation completes, a new VM appears with a .0
suffix to the originator VM name (as the Name Index in the Clone Creation screen was set to “0”). If the Cloned VM storage details
are reviewed, as shown in Figure 30, it is easy to see that the original Physical and Virtual Disks were cloned. The new Virtual Disk
capacity is again taken from the Repository storage, while the Physical disk capacity is created as a new VMAX storage device and
automatically added to the Server Pool (by automatically joining the Access Group).
Figure 30 Review Cloned VM Storage Details
Figure 31 Review Cloned Storage Device
To see the new device information move to the Storage Tab, and review the new device details. Under Extra Information it is easy to
see that it has the VMAX device ID 00626, as shown in Figure 31.
37
CONCLUSION
Oracle VM and Storage Connect APIs, together with ESI and VMAX provides a robust set of storage capabilities, offer the
virtualization administrator a complete set of tools to manage their Oracle VM environment while performing tasks such as storage
provisioning and VM cloning with ease. By utilizing this integration, many tasks that previously required intervention by multiple IT
groups, such as Storage, Network, and Virtualization, can now be executed with just a few clicks directly from Oracle VM Manager.
The overall result is increased business agility, simplicity of operations, and better ability to manage virtualization and cloud
operations under Oracle VM.
REFERENCES
EMS Storage Integrator for Oracle VM Version 3.3 Product Guide
EMS Storage Integrator for Oracle VM Version 3.3 Release Notes
Oracle VM for x86 (http://www.oracle.com/us/technologies/virtualization/oraclevm/overview/index.html)
Oracle product documentation library: http://www.oracle.com/technetwork/indexes/documentation/index.html#virtualization
EMC SMI-S Provider version 4.6.2 Release Notes
38