White Paper
ORACLE VM 3.3 STORAGE CONNECT FOR VMAX WITH EMC STORAGE INTEGRATOR V2.0 EMC® VMAX® Engineering White Paper ABSTRACT Oracle VM and Storage Connect framework of API, together with EMC Storage Integrator for Oracle VM x86, enhance IT operations for managing virtualized infrastructure by allowing EMC storage discovery, provisioning, and replication operations to be performed with ease directly from Oracle VM Manager. December, 2014 Version 1.1 EMC WHITE PAPER To learn more about how EMC products, services, and solutions can help solve your business and IT challenges, contact your local representative or authorized reseller, visit www.emc.com, or explore and compare products in the EMC Store. Copyright © 2014 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. Part Number H13603 Version 1.1 2 TABLE OF CONTENTS EXECUTIVE SUMMARY .............................................................................. 4 AUDIENCE ................................................................................................ 5 PRODUCT OVERVIEW ................................................................................ 5 Oracle VM overview ............................................................................................ 5 Oracle VM Server ............................................................................................... 5 Oracle VM Manager ............................................................................................ 5 Oracle VM Storage Connect ................................................................................. 6 ESI for Oracle VM overview ................................................................................. 6 VMAX3 overview ................................................................................................. 6 INSTALLATION AND CONFIGURATION ..................................................... 8 Test environment and Installation Steps ............................................................... 8 Installation of Oracle VM Server for x86 ................................................................ 9 Installation of Oracle VM Server ......................................................................... 10 Installation of EMC VMAX SMI-S Provider ............................................................ 12 Installation of Oracle VM Manager ...................................................................... 14 Installation of ESI for Oracle VM for x86 ............................................................. 16 View VMAX Storage Information ........................................................................ 20 USE CASES .............................................................................................. 23 Use Case 1 – Basic Storage Management Operations ............................................ 23 Use Case 2 – Provision Storage to Oracle VM Servers ........................................... 27 Use Case 3 – Create a Server Pool ..................................................................... 29 Use Case 4 – Create a Repositry for Server Pool .................................................. 31 Use Case 5 – Create Virtual Machines from a Server Pool ...................................... 33 Use Case 6 – Clone Virtual Machines .................................................................. 36 CONCLUSION .......................................................................................... 38 REFERENCES ........................................................................................... 38 3 EXECUTIVE SUMMARY Many IT organizations embrace virtualization and ‘as-a-Service’ Cloud deployments as a better way of achieving business agility and efficiency. While traditional IT operations require elongated processes between different groups and departments (such as Storage, Network, Systems, Virtualization, Database, and Applications) it is much more desirable to create an infrastructure that is suited to support such processes in an automated, secure, and optimized way via a set of standard processes and APIs. Such an infrastructure is provided by Oracle VM and Storage Connect APIs. Oracle VM is the server virtualization offering from Oracle Corporation. Oracle VM Server for x86 is application-driven built for rapid deployment of enterprise applications. Oracle VM is a Xen-based hypervisor that can support Linux, Oracle Solaris and Windows guests. Oracle VM includes an integrated Web based management console, Oracle VM Manager. EMC Storage Integrator (ESI) for Oracle VM version 3.3 is a plug-in that enables Oracle VM to discover and provision EMC storage arrays such as VMAX and VNX. The ESI plug-in is built to work in conjunction with Oracle Storage Connect framework. The framework provides a set of storage discovery and provisioning APIs that enhances the ability to manage and provision storage devices in an Oracle VM environment. ESI for Oracle VM uses EMC SMI-S Provider to do the actual storage management work. SMI-S Provider supports the Storage Networking Industry Association (SNIA) Storage Management Initiative (SMI), an American National Standards Institute (ANSI) standard for storage management. Oracle VM and Storage Connect APIs, together with EMC Storage Integrator, enhance IT operations for managing virtualized infrastructure. As can be seen in Figure 1, the Oracle VM administrator can add storage devices from a pre-assigned storage pools, create snapshots of these devices, or clone complete VMs with all their attached storage. All this is done with ease from Oracle VM Manager, utilizing EMC ESI and SMI-S APIs, and take place without need to call a storage admin. Figure 1 Oracle VM storage tasks 4 In summary, EMC Storage Integrator for Oracle VM for x86 enables the following: Agile – Discover and provision EMC storage for virtual machines without the need to involve a storage administrator. Simple – EMC Storage operations are a simple to execute with a few clicks in the Oracle VM Manager. Integrated – enables flexibility, scalability, and protection of Oracle VM environments with EMC storage. AUDIENCE This white paper is intended for database and system administrators, storage administrators, and system architects who are responsible for implementing, managing, and supporting Oracle VM virtual infrastructures, and Oracle databases and storage systems. It is assumed that readers have some familiarity with Oracle VM and EMC VMAX and VMAX3 family storage arrays, and are interested in achieving server and database virtualization, as well as higher database availability and protection. PRODUCT OVERVIEW ORACLE VM OVERVIEW Oracle VM is an integrated application-driven server virtualization solution designed for rapid deployment and simplified management of enterprise applications. Oracle VM supports a variety of guests such as Linux, Solaris and Windows. Oracle VM is designed to support mission-critical workloads such as Oracle Database, business applications as well as third-party enterprise applications. ORACLE VM SERVER Oracle VM Server provides the host on which virtual machines can run. It is lightweight and secure and while a single Oracle VM server is sufficient. Oracle VM Server includes a light-weight Linux kernel (known as dom0), and can manage many virtual machines (known as domU), where each virtual machine can run Linux, Oracle Solaris, or Microsoft Windows TM. Each Oracle VM Server also runs an Oracle VM Agent. Oracle VM Servers can be grouped together into a cluster called server pools, where multiple server pools can be configured and managed within an Oracle VM environment. At least one server pool is required to be configured, even if it includes only one Oracle VM Sever. Each server pool allows for high-availability and load balancing of virtual machines across the Oracle VM servers in the sever pool. ORACLE VM MANAGER Oracle VM Manager provides the Oracle VM environment with a framework that includes a web-based GUI, a command line interface, and an integrated database as a repository. Oracle VM Manager doesn’t necessarily require high-availability as an Oracle VM environment can continue operations even if Oracle VM Manager is down. Oracle VM Manager consists of Oracle VM Manager Server, and Oracle VM Manager Database. Other components are interfaced via APIs to Oracle VM Manager. 5 ORACLE VM STORAGE CONNECT Oracle VM Storage Connect is a program and a framework that contains a set of API for storage discovery and provisioning. Using this framework and API, users can perform such storage discovery and provisioning operations directly from Oracle VM Manager. This enhances business agility, simplifies storage management, and allows for more robust virtualized infrastructure. The plugins for Oracle VM Storage Connect are written by each storage vendor, utilizing their specific and advanced storage capabilities. By using these plugins the virtualization manager doesn’t need to know how to operate and manage each storage system, yet benefit from correct operations based on the storage vendor’s implementation of their best practices via API. In this way storage provisioning is performed as part of the overall management of a virtualized infrastructure. Note: For more information about the Oracle Storage Connect framework and Oracle VM, visit: Oracle Storage Connect (http://www.oracle.com/us/technologies/virtualization/storage-connect/overview/index.html) Oracle VM for x86 (http://www.oracle.com/us/technologies/virtualization/oraclevm/overview/index.html) ESI FOR ORACLE VM OVERVIEW The ESI for Oracle VM is a plug-in that enables Oracle VM to discover and provision EMC storage arrays. The plug-in is built upon the Oracle Storage Connect framework. The framework provides a set of storage discovery and provisioning Application Programming Interfaces (APIs) that enhance the ability to manage and provision storage devices in an Oracle VM environment. ESI for Oracle VM supports the following storage management and provisioning operations for EMC storage arrays: LUN discovery and inquiry Thin LUN creation and removal Auto-provisioning (LUN masking) Register and refresh arrays Create and delete access groups Clone device creation and removal Snap device creation and removal EMC storage information (total and pools capacities, free space, and more) ESI for Oracle VM uses EMC SMI-S Provider to do the actual storage management work. SMI-S Provider supports the Storage Networking Industry Association (SNIA) Storage Management Initiative (SMI), an American National Standards Institute (ANSI) standard for storage management. VMAX3 OVERVIEW1 The EMC VMAX3 family storage arrays are built on the strategy of simple, intelligent, modular storage, and incorporates a Virtual Matrix interface that connects and shares resources across all VMAX3 engines, allowing the storage array to seamlessly grow from an entry-level configuration into the world’s largest storage array. It provides the highest levels of performance and availability featuring new hardware and software capabilities. The newest additions to the EMC VMAX3 family, VMAX 100K, 200K and 400K, deliver the latest in Tier-1 scale-out multi-controller architecture with consolidation and efficiency for the enterprise. With completely redesigned hardware and software, the new VMAX3 array provides unprecedented performance and scale. It offers dramatic increases in floor tile density with engines and high capacity disk enclosures for both 2.5" and 3.5" drives consolidated in the same system bay. 1 At the time this paper is written, VMAX with Enginuity 5876 and SMI-S 5.7.6 is supported. VMAX3 with HYPERMAX OS 5977 and SMI-S 8.0 support is planned to be added shortly. 6 In addition, VMAX3 arrays can be configured as either hybrid or all- flash arrays. All new VMAX3 models come pre-configured from the factory to significantly shorten the time from installation to first I/O. 1 – 8 redundant VMAX3 Engines Up to 4 PB usable capacity Up to 256 FC host ports Up to 16 TB global memory (mirrored) Up to 384 Cores, 2.7 GHz Intel Xeon E5-2697-v2 Up to 5,760 drives SSD Flash drives 200/400/800 GB 2.5”/3.5” 300 GB – 1.2 TB 10K RPM SAS drives 2.5”/3.5” 300 GB 15K RPM SAS drives 2.5”/3.5” 2 TB/4 TB SAS 7.2K RPM 3.5” Figure 2 VMAX3 storage array VMAX3 engines provide the complete foundation for high-availability storage arrays. Each engine contains two VMAX3 directors and redundant interfaces to the new Dynamic Virtual Matrix dual InfiniBand® fabric interconnect. Each director consolidates front-end, global memory, and back-end functions, enabling direct memory access to data for optimized I/O operations. Depending on the array chosen, up to eight VMAX3 engines can be interconnected via a set of active fabrics that provide scalable performance and high availability. Systems ports are no longer mapped directly to CPU resources. CPU resources are allocated as needed using pools (front-end, back-end, or data services pool) of CPU cores which can service all activity in the VMAX3 array. All storage in the VMAX3 array is virtually provisioned. All data pools are created in containers called Storage Resource Pools. The array uses the dynamic and intelligent capabilities of Fully Automated Storage Tiering (FAST) to meet system or user-defined Service Level Objectives throughout the lifecycle of each application. VMAX3 Service Level Objectives are tightly integrated with EMC FAST software to optimize agility and array performance across all drive types in the system. EMC FAST technology improves application performance while reducing cost by using Flash drives combined with cost-effective high-capacity drives. 7 For local replications, VMAX3 adds a new feature to TimeFinder software called SnapVX, which provides “hybrid cloud scale” snapshots. Unlike previous VMAX snapshots, SnapVX snapshots do not require the use of dedicated target devices. SnapVX allows for up to 256 snapshots per individual source. These snapshots can be linked to copy their data to new target devices, or re-linked to update just the incremental data changes. VMAX3 new features and capabilities provide Oracle VM and Oracle databases more flexibility, scalability, protection, and performance, together with a dramatic increase in management simplification and ease of use. INSTALLATION AND CONFIGURATION TEST ENVIRONMENT AND INSTALLATION STEPS Overview of Test Environment The test environment was kept small for demonstration purposes. A larger scale environment can be easily set in a similar way with larger server pools, more networks, and Oracle VM Servers. Figure 3 Test environment The Oracle VM servers were configured using two Dell PowerEdge R710. Oracle VM Manager was configured using a Dell R900. The SMI-S Provider was installed on the same server as Oracle VM Manager for simplicity. A management public network was set between all three servers and a private network was set just between the Oracle VM servers as they were clustered and belonged to the same server pool. SAN connectivity was set between the VMAX 10K storage array and the Oracle VM Servers leveraging redundant FC switches, host multipathing, and multiple ports on the array. The Storage Management host had connectivity to only a few small devices for communication with the storage array. 8 List of hardware components can be seen in Table 1, and software component in Table 2. Table 1 Hardware components Item Type Oracle VM Servers 2 x Dell R710 Oracle VM Manager and SMI-S Provider server 1 x Dell R900 Storage array 1 x VMAX 10K SAN Switch 2 x EMC Connectrix DS-6510B Host HBAs (per host) 2 x Emulex LPe12002 Table 2 Software components Item Version Oracle VM Server 3.3 Oracle VM Manager / Host operating system 3.3 / Oracle Linux 6.5 ESI for Oracle VM 3.3 V2.0 EMC SMI-S Provider for VMAX and VNX 7.6.2.9 Summary of Installation Steps The following are the high-level steps for installing and configuring ESI for Oracle VM: 1. Install all Oracle VM Servers for x86 2. Install EMC VMAX SMI-S Provider 3. Install Oracle VM Manager 4. Install ESI for Oracle VM The following sections explain each step in more details. INSTALLATION OF ORACLE VM SERVER FOR X86 Oracle VM Servers are the physical compute nodes hosting the virtual machines. Oracle installation for Oracle VM Server is simple, light weight, and contains only packages necessary for its task. For this reason it is not recommended to use the Oracle VM Server for other purposes, such as Solutions Enabler array management software, or Linux package repositories for upgrades. Note: Do not create a VMAX Masking View2 for the Oracle VM Servers. All storage provisioning operations between VMAX and the Oracle VM Servers should be performed in Oracle VM Manager utilizing Storage Connect APIs. Device masking operations outside of Oracle Storage Connect (such as by using Unisphere for VMAX or Solutions Enabler CLI) won’t be recognized by Oracle VM Manager. An exception is device creation as Oracle VM Manager can discover and use existing VMAX devices. 2 VMAX Masking View is a component of VMAX Auto-provisioning Group. The masking view task is to make storage devices visible to hosts by creating a relationship between a group of devices, a group of host HBA ports (initiators), and a group of storage array ports the devices are visible to. 9 INSTALLATION OF ORACLE VM SERVER Oracle VM Server installation tasks To install and configure Oracle VM Server, perform the following actions: 1. Navigate to http://www.oracle.com and sign in with your user account. 2. Navigate to Downloads > Linux and Oracle VM and download Oracle VM 3.3 Server for x86 installation media. 3. Follow the installation steps described in Oracle Installation and Upgrade Guide for Release 3.3. This will allow you to: Install Oracle VM Server software Configure the hostname Configure network information Define Oracle VM Agent and root user passwords Note: During the installation the first password asked for is for the Oracle VM Agent. This password will be used later when Oracle VM Manager discovers and connects with the Oracle VM Server. Oracle VM Server post installation tasks Post installation tasks may include the configuration of additional network interfaces and other tasks3. Oracle VM Servers are discovered via the public network, however, Oracle recommends using dedicated networks or VLAN for communication between Oracle VM Servers within a Server Pool. Additional networks can be configured for purposes such as Virtual Machine private network, Live Migration, and Cluster Heart-bit. Refer to Oracle VM Concept Guide for release 3.3 for more details about Oracle VM networking. Note: All hosts in the cluster should have the same subnet For simplicity, this document describes the configuration of two Oracle VM Servers, with a dedicated private network, as shown in Table 3. Table 3 Oracle VM Server Networks ORACLE VM SERVER 1 ORACLE VM SERVER 2 ORACLE VM MANAGER SERVER NAME DSIB0062 DSIB0063 DSIB0064 PUBLIC IP/NIC 10.108.244.62/EM1 10.108.244.63/EM1 10.108.244.64 PRIVATE IP/NIC 192.168.1.62/EM4 192.168.1.63/EM4 During Oracle VM Server installation, the hostname and public network information are configured. To set additional network interfaces perform the following actions on each of the Oracle VM Servers, making changes to the network information as applicable. 3 Oracle VM 3.3 Administration Guide documents the steps to increase dom0 memory size to avoid performance issues, and to add SNMP monitoring tools. 10 The example in this paper demonstrates the following activities: Manual configuration of an additional network interface for private network Update of /etc/hosts to include the private network Test the new interface Edit the appropriate network interface connected to the private network switch: [root@dsib0062 ~]# cd /etc/sysconfig/network-scripts/ [root@dsib0062 network-scripts]# cp ifcfg-em4 sav_ifcfg-em4.sav [root@dsib0062 network-scripts]# vi ifcfg-em4 Update the file to include the private network information: DEVICE="em4" BOOTPROTO="none" DHCP_HOSTNAME="dsib0062" HWADDR="A4:BA:DB:41:7B:6B" NM_CONTROLLED="yes" #orig: ONBOOT="no" TYPE="Ethernet" UUID="439b72dc-75d9-485c-9468-1118576e9eff" # Added ONBOOT="yes" IPADDR="192.168.1.62" NETMASK="255.255.255.0" Restart network service for the changes to take effect: [root@dsib0062 network-scripts]# service network restart Update hosts file with the new host information (unless DNS is used): [root@dsib0062 ~]# vi /etc/hosts Include the relevant network information: 127.0.0.1 10.108.244.62 10.108.244.63 192.168.1.62 192.168.1.63 localhost dsib0062 dsib0063 dsib0062-priv dsib0063-priv Ping the private network between the servers to test the connection: [root@dsib0062 ~]# ping dsib0063-priv PING dsib0063-priv (192.168.1.63) 56(84) bytes of data. 64 bytes from dsib0063-priv (192.168.1.63): icmp_seq=1 ttl=64 64 bytes from dsib0063-priv (192.168.1.63): icmp_seq=2 ttl=64 64 bytes from dsib0063-priv (192.168.1.63): icmp_seq=3 ttl=64 64 bytes from dsib0063-priv (192.168.1.63): icmp_seq=4 ttl=64 time=0.190 time=0.075 time=0.072 time=0.072 ms ms ms ms 11 INSTALLATION OF EMC VMAX SMI-S PROVIDER Storage management host EMC SMI-S Provider is the software that complies with SNIA SMI-S standards for storage management and works in conjunction with Solutions Enabler4 software. Oracle VM Storage Connect for VMAX uses SMI-S for all storage monitoring and control operations. Note: EMC SMI-S Provider and Solutions Enabler software should not be installed on the Oracle VM Servers. Instead, they should be installed on a host referred to as the Storage Management host. In some deployments, the user may prefer to combine the Storage Management host with Oracle VM Manager host though it is not required. Note: Solutions Enabler requires a number of very small devices called Gatekeepers to be presented to the Storage Management host for communication with the VMAX. There should be a minimum of six Gatekeepers visible to the Storage Management host before Solutions Enabler and SMI-S are installed. SMI-S Provider installation In order to install the SMI-S Provider, perform the following actions on the Storage Management host. Host Preparation 1. Engage the storage administrator and have 6 or more Gatekeepers provisioned and visible to the Storage Management host. 2. Update the Storage Management host path to include the path to Solutions Enabler and SMI-S binaries. [root@dsib0064 ~]# vi ~.bashrc ... export PATH=$PATH:/usr/symcli/bin:/opt/emc/ECIM/ECOM/bin:. 3. Source the file to take effect. [root@dsib0064 ~]# . ~/.bashrc SMI-S Installation Steps 1. Navigate to http://www.emc.com. 2. Navigate to Support > Downloads and search for SMI-S Provider. 3. Download SMI-S Provider 4.6.2.9 for SMI-S for Linux (or a later release)5. 4. Copy the file to the Storage Management server and extract it. 4 Solutions Enabler software allows management of VMAX storage array using APIs, CLIs, and REST. It works in conjunction with other VMAX management solutions such as Unisphere for VMAX GUI, EMC SMI-S Provider, EMC ViPR, and others. 5 SMI-S 4.6.2.9 release is considered a ‘customer hot-fix’ for SMI-S. If SMI-S 4.6.2.9 (or a later release) is not available on EMC SMI-S download page request it from your EMC account representative as a minimum software requirement. 12 5. Follow the installation steps described in EMC SMI-S Provider Version 4.6.2 Release Notes. The main steps are described below. Extract the installation software and run it: [root@dsib0064 SE]# tar xvfz se7629-Linux-x86_64-SMI.tar.gz [root@dsib0064 SE]# ./se7629_install.sh –install ... #----------------------------------------------------------------# The following HAS BEEN INSTALLED in /opt/emc via the rpm utility. #----------------------------------------------------------------ITEM PRODUCT VERSION 01 EMC Solutions Enabler V7.6.2.9 02 SMI KIT V4.6.2 #----------------------------------------------------------------Run the storage discovery command6: [root@dsib0064 SE]# symcfg discover Test the SMI-S provider installation and communication with the storage array. To do so run7 TestSmiProvider command, accept all the defaults until the command screen shows, then use Display Version Info (dv) command to test connectivity [root@dsib0064 ~]# TestSmiProvider ... Built with EMC SMI-S Provider: V4.6.2 Namespace: root/emc repeat count: 1 (localhost:5988) ? dv ++++ Display version information ++++ CIM ObjectManager Name: EMC:10.108.244.63 CIMOM Version: EMC CIM Server Version 2.7.3.3.0.8 SMI-S qualified version: 1.5.0 SMI-S Provider version: V4.6.2.3 SMI-S Provider Location: Proxy SMI-S Provider Server: Linux dsib0063 3.8.13-35.1.1.el6uek.x86_64 #2 SMP Fri Jun 6 10:52:33 PDT 2014 x86_64 (64bit Libraries) Solutions Enabler version: V7.6-1808 2.9 Firmware version information: (Local) Symmetrix Array 000198700067 (VMAX10K) : 5876.268.174 Retrieve and Display data - 1 Iteration(s) In 0.327698 Seconds Please press enter key to continue... 6 It is assumed that the root user PATH was updated with Solutions Enabler binaries location, which has a default location of: /usr/symcli/bin It is assumed that the root user PATH was updated with the SMI-S Provider binaries location, which has a default location of: /opt/emc/ECIM/ECOM/bin 7 13 Restarting SMI-S If SMI-S Provider is not running, or if Oracle VM shows messages such as “Connection to Admin host failed” restart SMI-S Provider by using the following command: [root@dsib0064 ~]# ECOM -d INSTALLATION OF ORACLE VM MANAGER Oracle VM Prerequisites Oracle VM Manager is required to be installed on a separate host than Oracle VM Servers. Note: The Storage Management host and Oracle VM Manager may reside on the same host, or separate hosts. In this paper the same host has been used for both. For more information on Oracle VM Manager installation requirements refer to Oracle VM Installation and Upgrade Guide for Release 3.3. The main tasks are described below. Install Oracle VM Manager Install NTP Server on the Oracle VM Manager host to provide time coordination for the VMs. 1. Install the NTP server. [root@dsib0064 ~]# yum install ntp 2. Configure the NTP Daemon to start at boot. [root@dsib0064 ~]# chkconfig ntpd on 3. Start the NTP service. [root@dsib0064 ~]# service ntpd start 4. Mount the Oracle VM Manager media and run the installer. [root@dsib0064 ~]# mount -o loop /tmp/ovmm-3.3.1-installer-OracleLinux-b1065.iso /mnt [root@dsib0064 ~]# cd /mnt [root@dsib0064 mnt]# ./runInstaller.sh During the installation MySQL database is installed for Oracle VM Manager repository. Oracle VM Manager Post Installation Tasks Oracle offers Oracle VM Utilities to help with the management of Oracle VM tasks from a command line. Refer to Oracle VM Administrator’s Guide for Release 3.3 for a description of Oracle VM Utilities and their installation procedures. Oracle also provides a package that should be installed on Oracle VM Manager host in order to provide VNC access to the Oracle VMs. Download the ovmcore-console package from Oracle VM Download page and install it. If Linux Firewall is configured you’ll need to open the appropriate ports or disable it. Refer to Oracle VM Installation and Upgrade Guide for details on the installation tasks of VNC and Serial Console for Oracle VM. 14 In this example the ovmcore package is installed on the Oracle VM Manager host. [root@dsib0064 OVM]# rpm -ivh ovmcore-console-1.0-36.el6.noarch.rpm warning: ovmcore-console-1.0-36.el6.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY Preparing... ########################################### [100%] 1:ovmcore-console ########################################### [100%] If you wish to disable Linux firewall: [root@dsib0064 ~]# service iptables stop [root@dsib0064 ~]# chkconfig iptables off If you wish to disable SELinux: [root@dsib0064 ~]# vi /etc/selinux/config ... SELINUX=disabled Networking Networking in a virtualized environment is an important topic that requires adequate thought and planning. Refer to Oracle VM Concept Guide for Release 3.3 for details on such considerations. In this paper a single private network is added, as shown in Figure 4. It will be used for VM’s communication and Live Migration. 1. To create an additional network, choose the Networking tab. 2. Click to create a new network. Figure 4 Configure Oracle VM networks 15 INSTALLATION OF ESI FOR ORACLE VM FOR X86 ESI for Oracle VM version 3.3 is the software that utilizes Oracle Storage Connect plugins to monitor and manage the VMAX storage array from Oracle VM. Refer to Table 4 for installation prerequisites. Prerequisites checklist Table 4 ESI for Oracle VM prerequisites ESI FOR ORACLE VM PREREQUISITES 1. ORACLE VM SERVERS INSTALLED 2. ORACLE VM MANAGER INSTALLED 3. SMI-S PROVIDER INSTALLED COMPLETE Installation steps 1. Navigate to http://www.emc.com. 2. Navigate to Support > Downloads and search for ESI for Oracle VM. 3. Download the ESI for Oracle VM plugin. 4. Copy the plugin to each of the Oracle VM Servers. 5. Install the rpm on each of the Oracle VM Servers. [root@dsib0062 ESI]# tar xvf esi_plugin.2.0.0.5_OVM-3.3.tar emc-osc-isa-2.0.0.5-1.el6.x86_64.rpm [root@dsib0062 ESI]# rpm -ivh emc-osc-isa-2.0.0.5-1.el6.x86_64.rpm Preparing... ########################################### [100%] 1:emc-osc-isa ########################################### [100%] 16 ESI Configuration File ESI for Oracle VM uses a configuration file to pass user parameters that affect its behavior with Storage Connect. Note: It is highly recommended to read the following section and be familiar with ESI configuration file options. ESI for Oracle VM Configuration File Parameters: AccessGroupPrefix: this parameter allows modifying the default prefix for VMAX Auto-provisioning Group names8. While this prefix is not visible in Oracle VM Manager screens, it will be visible to the Storage Admin monitoring the VMAX storage array using tools such as Unisphere for VMAX or Solutions Enabler CLI. AutoMetaEnabled9: True means that the VMAX Auto-meta feature is configured and enabled. False means that it isn’t (you can check by running Solutions Enabler command: symcfg list -v | grep “Auto Meta”). If left unset, the plugin will inquire with the storage via SMI-S to check whether AutoMeta is enabled and will take the appropriate value. Note: It is recommended to not set this parameter as its correct value will be found automatically by ESI plugin. VolumeGroupPrefix: Provides a way of limiting which VMAX Pools are visible to Oracle VM. In this way Oracle VM can view and access only certain storage Pools. The parameter should be consistent across all the Oracle VM Servers accessing the same VMAX storage. Modifying this parameter after the storage was discovered will require removal of the array in Oracle VM Manager and rediscovery. Prior to ESI for Oracle VM version 2.0.0.5, the Prefix can only contain part of the Pool name, but not the full name of the pool. Starting with version 2.0.0.5 the prefix can include partial or full Pool name. LogLevel: provides a way to control the level of detail in the Oracle VM Server log file located at /var/log/ovs-agent.log. Set to “debug” for detailed information. Other values allowed: “info”, and “error”. The log referenced in isa.conf is located at: /var/log/ovs-agent.log on the Oracle VM Server. Create a new configuration file, or copy an existing one from a previously installed Oracle VM Server with ESI: [root@dsib0062 ESI]# vi /opt/storage-connect/plugins/emc/isa/isa.conf AccessGroupPrefix=osc_ AutoMetaEnabled=false VolumeGroupPrefix=FC,SATA LogLevel=debug Note: If the ESI plugin is upgraded, perform rediscovery of the Oracle VM Servers by right clicking on each and choosing Rediscover Server from the menu. 8 The default prefix is osc_ for Oracle Storage Connect. A different one, such as ovm_ can be easily used. Meta volumes is a feature available in VMAX with Enginuity code level 5876 and below. It allows for storage devices aggregation as a striped or concatenated group of devices, and at the same time allows the group (meta-volume) to grow beyond the 240GB limit of a single device. VMAX3 with HYPERMAX OS 5977 and above no longer supports or requires meta-volumes as all devices are virtual and natively striped, and each device capacity can reach multi-terabytes. 9 17 Registration of Oracle VM Servers containing ESI Plugin with Oracle VM Manager Once the ESI for Oracle VM plugin has been installed on the Oracle VM Servers, the Oracle VM Servers and VMAX storage array will both need to be registered (even if the servers were previously registered with Oracle VM Manager). 1. Open a browser and navigate to https://<hostname>:7002/ovm/console Example: https://dsib0064:7002/ovm/console 2. Log in with username admin using the password that was set during the installation of Oracle VM Manager. 3. Navigate to the Servers and VMs tab. 4. Click on the Discover Servers icon . 5. Enter the Oracle VM Agent password and Oracle VM servers IP addresses/hostnames. Click OK, as shown in Figure 5. Figure 5 Discover Oracle VM Servers 18 6. Once discovered, the servers will be displayed under Unassigned Servers, as shown in Figure 6. Figure 6 Discovered Oracle VM Servers Note: In case the discovery encounters a failure during storage refresh action validate that the Oracle VM Servers can each ping the management host back (/etc/hosts or DNS are set correctly in each Oracle VM Server), and that SMI-S is running. 19 VIEW VMAX STORAGE INFORMATION Discovery of VMAX Storage Array In order to discover the VMAX storage array and perform cloning and provisioning operations, the first task is to register the SMI-S provider that has access to the VMAX storage. 1. Navigate to the Storage tab and click on the Discover SAN Server icon , as shown in Figure 7. Figure 7 Discover VMAX Storage Array 2. Update the following fields appropriately, as shown in Figure 8. Name – User defined name for EMC storage array Description – User defined description for EMC storage array Storage Type – Select FibreChannel Storage Plug-in – Select EMC FC SCSI Plugin Plug-in Private Data – Provide the VMAX array ID (either full or partial ID number) Note: This field is mandatory for ESI plugin. Partial ID is allowed only for the end of the ID (e.g. 198700067, or 067 are valid) Admin Host – Provide the host name or IP address of the host where SMI-S is installed Admin Username – Provide the SMI-S admin username. The default is admin Admin Password – Provide the SMI-S admin user password. The default is #1Password 20 Figure 8 Enter Storage Discovery Information 3. Click Next to continue. The Oracle VM Servers appear under Available Server(s). Move them to the right side under Selected Server(s) then click Finish. Identifying VMAX storage information from Oracle VM Manager After the discovery job completes, the storage will appear with basic information, as shown in Figure 9. Figure 9 Storage Summary 21 More information about the storage array such as the Pools and the devices associated with each pool can be viewed by selecting the storage from the list of SAN Servers. Additional VMAX device information, such as device ID (appears as “Name”, or “Extra Information”) can be viewed by expanding any of the storage devices listed, as shown in Figure 10. Storage devices also show which of the Oracle VM Servers they are visible to, Access Groups they belong to, etc. Figure 10 Storage Devices Information Identifying VMAX storage devices directly from Oracle VM Server Oracle VM Administrator can view basic VMAX device information from the Oracle VM Servers by using Linux commands such as multipath or scsi_id. With EMC PowerPath, powermt display dev=all command can be used. If Linux dm-multipathing is used, it can list multipath devices with their name, serial, and capacity. [root@dsib0062 ~]# multipath -l 360000970000198700067533030313435 dm-1 EMC,SYMMETRIX size=100G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=0 status=active `- 3:0:0:1 sdc 8:32 active undef running 360000970000198700067533030323235 dm-10 EMC,SYMMETRIX size=100G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=0 status=active `- 3:0:0:10 sdl 8:176 active undef running 360000970000198700067533030323234 dm-9 EMC,SYMMETRIX size=100G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=0 status=active `- 3:0:0:9 sdk 8:160 active undef running ... 22 Linux scsi_id command can also list storage device information with the addition of the VMAX Device ID. The field ID_SCSI_SERIAL contains the last 6 digits of the VMAX Storage ID followed by the 3 digits of the VMAX Device ID. [root@dsib0062 ~]# for i in `ls /dev/dm-*`; do echo $i; scsi_id -g -x $i | grep ID | grep SERIAL= ; done /dev/dm-0 ID_SERIAL=360000970000198700067533030303134 ID_SCSI_SERIAL=700067014000 /dev/dm-1 ID_SERIAL=360000970000198700067533030313435 ID_SCSI_SERIAL=700067145000 /dev/dm-10 ID_SERIAL=360000970000198700067533030323235 ID_SCSI_SERIAL=700067225000 ... USE CASES The following section assumes that all the configuration steps have been completed and provide an introduction to some of the basic Oracle VM operations that Storage Connect helps achieve. Oracle VM Storage Connect with ESI provides the Oracle VM Administrator with a way of performing tasks such as storage creation, provisioning, and cloning with ease and simplicity. The first use case shows basic storage provisioning tasks. Following use cases demonstrate how to provision storage devices to Oracle VM Servers, Virtual Machines, create storage, repositories, and how to perform cloning operations at the VM level. USE CASE 1 – BASIC STORAGE MANAGEMENT OPERATIONS Review available storage pools capacities Before adding new storage devices, let’s inspect the available capacities of the storage pools. 1. Choose the Storage tab at the top of the Oracle VM Manager screen. 2. On the left, under SAN Servers, choose the appropriate storage array, as shown in Figure 11. Click on the small triangle to the left of the Storage name to expand its details and the available storage pools will be listed. By using the Info perspective it is easy to view how much capacity is available in each storage pool. In Figure 11 we can see that the FC_Pool has 3,611.39 GB of available capacity. Figure 11 View Storage Pool Available Capacity 23 Create New Storage Devices To create a new storage device change to Physical Disks perspective (refer also to Figure 14). 1. Click on the Create Physical Disk icon , as shown in Figure 12. Figure 12 Create Physical Disk Volume Group: If a specific array Storage Pool was already selected it will be used as Volume Group. If the Array Name was selected instead, a drop-down list of the available storage pools will be listed. Select the appropriate storage pool. Name: Provide a descriptive name for the new device. Extra Information: This box can be ignored. It can be used during creation of storage meta volumes when Auto-meta is not configured in the array. For example, it could say: meta_member_size=200G. However, it is optional even when creating meta volumes. Description: Provide an appropriate description for the new device. Shareable: Specifies for Oracle VM that the device can be shared between multiple Virtual Machines with simultaneous read/write access. Thin Provision: Only Thin devices are supported for Oracle VM Storage Connect with ESI 2.0 24 2. Click OK to create the storage device. Once the device is created, it will show in the Physical Disk list for the storage array. By clicking on the small triangle to the left of the device name more information can be viewed, as shown in Figure 13. The VMAX device ID is shown to the right of Extra Information field. For example, Oracle ERP Disk 1 is using device ID 625. Figure 13 Show Device Information The Storage Admin can identify the Oracle VM devices either by their ID (for example: device ID 625), or by their name. To list device names they can use Solutions Enabler command: symdev list -ident device_name. [root@dsib0064 ~]# symdev list -ident device_name Symmetrix ID: 000198700067 Device ------------------------------------------------------------------------------Sym Config Attr Device Name ---- --------------- ---- ----------------------------------------------------01F3 TDEV ORA_DATA1 0622 TDEV Template1 0623 TDEV Volume 00147_clone 0624 TDEV Volume 00148_clone 0625 TDEV Oracle ERP Disk 1 0626 TDEV Volume 00147_clone ... 25 Manage Storage Devices By selecting a VMAX storage device, a number of operations can be executed on the device, as shown in Figure 14. Figure 14 Manage Storage Device Device operations: Edit Device: Attributes such as Device Name, Description, and Shareable can be modified. Note: It is not allowed to change device capacity. Delete Device: allows the user to delete the device. Clone Device: allows the user to clone a device. In the Clone Target Type field select: Storage Array. In the Name field provide an optional name for the clone. o If a name wasn’t provided the cloned device will show as the original device name with the addition of a bracket and a clone number. For example, ‘Oracle ERP Disk 1 (2)’ is a default name for a clone to a device named ‘Oracle ERP Disk 1’. o If the name provided ends with ‘_EMC_SNAP’, then rather than a full-sized clone, a space-efficient snapshot will be created with copy-on-write functionality (only changes to either source or target devices consume additional storage capacity) Note: As there is no other way in Storage Connect to pass information to the array, whether to create a full copy or space-efficient copy of the source device, use the clone name to do so. The suffix ‘_EMC_SNAP’ will result in a space-efficient clone, aka snap. 26 Only Thin Clone value is allowed in the Clone Type field. After the snap or clone was created, as before, by clicking on the small triangle to the left of the clone device name its VMAX device ID can be viewed in the Extra Information field. USE CASE 2 – PROVISION STORAGE TO ORACLE VM SERVERS Provisioning storage devices to the Oracle VM Servers is done by creating Access Groups. Storage devices in Access Groups can later be used as storage for the Oracle VM Servers (for example: Server Pool storage, or Repositories), or provisioned to the virtual machines as physical disks. Note: This section assumes that storage devices were already created for Oracle VM and can be simply added to the Access Group. To learn about creating new storage devices refer to Use Case 1. In order to create an Access Group: 1. Select the Storage tab. 2. Choose Access Group perspective from the drop-down menu. 3. Click on the icon that creates a new Access Group, as shown in Figure 15. Figure 15 Create Access Group 4. In the window that opens provide a name and description for the Access Group. Click Next. Note: Access Group name must not contain any spaces. It will be used to create a VMAX Storage Group name (together with a predefined prefix that can be modified in the ESI configuration file). 27 5. In the Initiators (HBA ports) selection screen, on the left side you can see a list of the Oracle VM Servers initiators prefixed by the server name and followed by the port WWN. You may notice some WWN’s showing only zeros. These can be ignored as they are not showing storage connectivity. Only initiators that have been zoned correctly to the storage array will show their WWN as a non-zero address and can be selected for the Access Group HBA ports connectivity. 6. Move the appropriate initiators to the right side of the screen as shown in Figure 16. An Access Group that includes initiators of multiple Oracle VM Servers can provide shared storage to these servers. Click Next. Figure 16 Access Group initiators selection 7. In the next screen select the storage devices that will be provisioned to the Oracle VM Servers, as shown in Figure 17. Click Finish to complete the task. Figure 17 Access Group storage selection 28 After the Access Group is created, by switching to the Physical Disk perspective, the Server column will reflect the Oracle VM Servers in the Access Group for the appropriate devices, as shown in Figure 18. Figure 18 Access Group servers listed USE CASE 3 – CREATE A SERVER POOL The Oracle VM Servers can be placed in a Server Pool to allow shared storage for repositories, high availability and easy migration of virtual machines. 1. To create a Server Pool select the Servers and VMs tab and click on the icon that creates a Server Pool, as shown in Figure 19. Figure 19 Create server pool 29 2. Provide a Server Pool Name and a Virtual IP to manage the pool. Note that Clustered Server Pool will be checked by default. 3. In the Storage for Server Pool field choose Physical Disk and click on the search icon to select storage, as shown in Figure 20. Figure 20 Storage selection for server pool Storage devices visible to the Oracle VM Servers will be listed. Select one of them for the Server Pool shared storage. An OCFS2 filesystem will be created on this device and mounted on the Oracle VM Servers selected for the Server Pool. Click OK and you’ll be returned to the Create Server Pool screen where you can add a description. Click Next. At this time the Oracle VM Servers for the Server Pool should be selected. Move the appropriate servers to the right side of the screen and click Finish to complete the task, or Next to add Tags. The Sever Pool will be listed upon completion, as shown in Figure 21. Figure 21 Server pool created 30 USE CASE 4 – CREATE A REPOSITRY FOR SERVER POOL For Server Pool clustering, a shared storage is required. For SAN storage, a disk image is created and formatted using OCFS2 (Oracle Clustered Filesystem). 1. Change to the Repositories tab and click on the icon to add a repository, as shown in Figure 22. Figure 22 Create a storage repository 2. In the Create a Repository window enter the repository name. 3. Choose the Physical Disk radio button and the Server Pool the Repository will belong to. 4. Next, click on the icon to discover a Physical Disk for the repository storage allocation, as shown in Figure 23. Figure 23 New Repository - Storage Selection 5. In the Select Physical Disk screen select the VMAX storage array and Pool. A list of shared storage devices will be available. Note in the example that the first device (volume 00145) is no longer visible as it was selected previously for the Server Pool storage. The next volume, 00146, is therefore selected in this example. 6. After the storage selection for the repository you’ll be returned to the Create Repository screen where you can add a description. Click Next. Select the Oracle VM Servers for the repository by moving them to the right side of the screen and click Finish. 31 The Repository, as shown in Figure 24, is created and divided into 5 categories as explained in the Oracle VM Concept Guide: Assemblies, which are a preconfigured collection of virtual machines, typically created with Oracle Assembly Builder. ISO files that can be mounted as a virtual CD/DVD. VM Templates for virtual machine templates (templates can be downloaded from Oracle website, or created). VM Files containing the configuration files of the virtual machines. Virtual Disks that can be dedicated or shared between VMs. Figure 24 Newly Created Repository 32 USE CASE 5 – CREATE VIRTUAL MACHINES FROM A SERVER POOL Import an Oracle VM Assembly An easy way to create virtual machines is from Assemblies or Templates. Assemblies and Templates can be downloaded from Oracle Software Delivery Cloud under Oracle VM Templates. They are not installed directly into the Oracle VM Servers and rather imported by Oracle VM Manager using http, https, or ftp. In this example, vsftpd (secure ftp), is installed on the Oracle VM Manager host and the vsftpd service started. An OVF set is selected from Oracle website: Oracle VM 3 Templates (OVF) for Oracle Linux 6 Media Pack v7 for x86_64 (64 bit). It opens a new window in the browser with multiple OVFs. Download and copy to the Oracle VM Manager host the appropriate template. Once copied, unzip the packages. [root@dsib0064 OVM]# cd /var/ftp/OVM [root@dsib0064 OVM]# unzip OVF_OEL65_V45138-01.zip Import the OVA as an Assembly first, and then turn it into a Template. To do so: 1. Go to the Repository tab and choose Assemblies under the previously created Repository (Repository 1). 2. Click on the Import icon, as seen in Figure 25 . In this example ftp protocol is used with the ftp user on the Oracle VM Manager host. The OVM is a sub-directory under the ftp user home directory (/var/ftp/OVM) where the .ova file is stored. The following link is inserted in the Assembly download location: ftp://ftp:ftp@dsib0064/OVM/OVM_OL6U5_X86_64_PVM.ova. Figure 25 Import an Assembly 33 Create an Oracle VM Template from an Assembly 1. Once the Assembly is imported select it, and click the icon to create a VM Template, as shown in Figure 26. 2. In the window that opens enter the template name and click OK. Figure 26 Create a VM Template Create an Oracle VM from a Template Under Servers and VM tab click on the icon to create a new virtual machine. For example, the previously imported template can be used to create 3 VMs by selecting Clone from an existing VM Template, and providing the appropriate information, as shown in Figure 27. Figure 27 Create New Virtual Machines 34 3. Select a VM and click on the Edit icon, as shown in Figure 28. 4. Under the Network tab add the private network created earlier for VMs. Under Storage tab add Physical or Virtual disks as necessary. Virtual Disks capacity is taken from the storage device assigned to the Server Pool Repository. Physical Disks can be chosen from the Access Group storage associated with the Server Pool. 5. After the update is completed the VM can be started. Figure 28 Update VM Information 35 USE CASE 6 – CLONE VIRTUAL MACHINES A Virtual Machine can easily be cloned together with its assigned storage devices. To do so: 1. Select the VM from the list, right click and choose Clone or Move, or click on the icon: Clone or Move Virtual Machine. 2. Optionally, provide a Clone Name, Count, and Index, as shown in Figure 29. It is also possible to set an advanced cloning with Clone Customizer (for example, to use storage from another Repository). 3. To finish the operation click OK. Figure 29 Clone a Virtual Machine 36 In this following example the VM OL6U5_X86_64_PVM.1 is cloned. When the clone operation completes, a new VM appears with a .0 suffix to the originator VM name (as the Name Index in the Clone Creation screen was set to “0”). If the Cloned VM storage details are reviewed, as shown in Figure 30, it is easy to see that the original Physical and Virtual Disks were cloned. The new Virtual Disk capacity is again taken from the Repository storage, while the Physical disk capacity is created as a new VMAX storage device and automatically added to the Server Pool (by automatically joining the Access Group). Figure 30 Review Cloned VM Storage Details Figure 31 Review Cloned Storage Device To see the new device information move to the Storage Tab, and review the new device details. Under Extra Information it is easy to see that it has the VMAX device ID 00626, as shown in Figure 31. 37 CONCLUSION Oracle VM and Storage Connect APIs, together with ESI and VMAX provides a robust set of storage capabilities, offer the virtualization administrator a complete set of tools to manage their Oracle VM environment while performing tasks such as storage provisioning and VM cloning with ease. By utilizing this integration, many tasks that previously required intervention by multiple IT groups, such as Storage, Network, and Virtualization, can now be executed with just a few clicks directly from Oracle VM Manager. The overall result is increased business agility, simplicity of operations, and better ability to manage virtualization and cloud operations under Oracle VM. REFERENCES EMS Storage Integrator for Oracle VM Version 3.3 Product Guide EMS Storage Integrator for Oracle VM Version 3.3 Release Notes Oracle VM for x86 (http://www.oracle.com/us/technologies/virtualization/oraclevm/overview/index.html) Oracle product documentation library: http://www.oracle.com/technetwork/indexes/documentation/index.html#virtualization EMC SMI-S Provider version 4.6.2 Release Notes 38
© Copyright 2024