Oracle EXADATA and ZFS NAS Appliance for Faster Platform as Service 11 Pranilesh Chand Lead Engineer - Database Oracle Certified Professional 9i, 10g,11g Speciality: Performance Tuning Oracle RAC/HA Oracle Exadata Disaster Recovery 22 Agenda • • • • • Technology Overview Why we chose this solution How we did it Issues Q&A 33 SUN Oracle Database Machine • 2 Sun Fire™X4170 Oracle Database servers • 3 Exadata Storage Servers (All SAS or all SATA) • 2 Sun Datacenter InfiniBand Switch • Preconfigured out of the box high performance machine • Balanced performance configuration • Oracle database 11.2 preinstalled • No ASM Instance • Smart Scan 44 Sun ZFS Storage 7420 •Only NAS appliance that supports Hybrid Columnar Compression •Cluster Ready System for HA •Can be easily integrated with Oracle database •ZFS Filesystem •Inline de-duplication and compression to save space •Filesystem snapshot and cloning 55 Why We Chose This Solution 66 Problem Presented •Performance Slowness •Batch jobs taking 9hrs to run •User experience affected as some had to work in weekends •Database backups •OLTP reports not able to run and times out •Some important reports had to run in weekend as it took 14hrs •No High Availability •Database Refresh •Takes almost 16-20hrs to refresh 77 Problem Analysis Performance •Database IO Bound •Server Low RAM Environment Refresh Cost 88 Solution Accepted •Platform as a service to customer •Oracle Exadata for production DB hosting •Consolidation of all production databases •Sun ZFS and Sun Fire servers for test/dev environment •RAC environment to be created •Consolidation of all test/dev databases 99 Exadata Benefits Data Warehouse Jobs •From 9hrs to 90mins •From 8hrs to 40mins •ETL now running in minutes from hours before OLTP Performance •Reports running in minutes from hours Business Improvement •Staff engagement •Cost Faster RMAN Backup Consolidation 10 10 Oracle ZFS Appliance Benefits Hybrid Columnar Compression Quick Refresh of Test/Dev from standby database Consolidated platform for test databases Attached to Exadata via IB Clone DB requires very small space 11 11 How We Did It 12 12 Backup: High Level Design OEM Server Exadata Server OEM Repository/ RMAN Catalog Tape Backup OEM agent 10G Ethernet Backup Server /backup (DNFS mount) Compute node Operating System Oracle RDBMS/RAC binaries OCR and voting disk backup Database RMAN backups Regular IB configuration export SAN 13 13 Sample: OEL Backup Script # Create snapshot backup of filesystems /, /u01, /boot lvcreate -L1G -s -n root_snap /dev/VGExaDb/LVDbSys1 e2label /dev/VGExaDb/root_snap DBSYS_SNAP mkdir -p /root/mnt mount /dev/VGExaDb/root_snap /root/mnt lvcreate -L5G -s -n u01_snap /dev/VGExaDb/LVDbOra1 e2label /dev/VGExaDb/u01_snap DBORA_SNAP mkdir -p /root/mnt/u01 mount /dev/VGExaDb/u01_snap /root/mnt/u01 # Backup the snapshots to the NFS mount /OS_backup cd /root/mnt # Check for any significant errors reported in the /tmp/backup_tar.stderr file tar -pjcvf /OS_backup/node1_backup.tar.bz2 * /boot --exclude node1_backup.tar.bz2 >/tmp/backup_tar.stdout 2>/tmp/backup_tar.stderr # Remove snapshots cd / umount /root/mnt/u01 umount /root/mnt /bin/rm -rf /root/mnt lvremove -f /dev/VGExaDb/u01_snap lvremove -f /dev/VGExaDb/root_snap 14 14 Backup: IB Switch 15 15 Monitoring: High Level Design OEM Server ASR Manager Triggered Alert SMS Sent OEM Agent Breached Hardware Alert Breached SoftwareAlert SR Details Exadata Machine Gen-I Engineer Oracle Support Exadata Storage cells registered in ASR Exadata compute nodes registered in ASR Compute node and Storage Cell ILOM registered in ASR 16 16 DR – High Level Design Primary Site DR Site OEM Server OEM Server OEM repository/ RMAN Catalog OEM repository/ RMAN Catalog Redo Shipping OEM Agent 10G Ethernet Load Balancer Dark Fiber OEM Agent 10G Ethernet RAC Database Exadata Exadata IB RAC Database Redo Shipping Rac1 Test/DEV DB ZFS 7420 Rac2 17 17 NFS Shares Key Notes (762374.1) •ZFS Shares mounted as NFS on Exadata DB nodes •DNFS configured for each share used by DB •To configure DNFS, run the following for each Oracle Home •make -f ins_rdbms.mk dnfs_on •Restart each instance from the Oracle Home •Add NFS shares to /etc/oranfstab for global access server: heada path: 192.168.20.10 export: /export/db_logs mount: /u01/app/oracle/db/reco server: headb path: 192.168.20.11 export: /export/db_data mount: /u01/app/oracle/db/data •Once DNFS is enabled, the following message will be displayed in alert log file •Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 2.0 •Following query can be used to check if DNFS is used •select * from v$dnfs_servers; 18 18 NFS Mount Options (RAC) Binaries Mount options for Oracle Datafiles Mount options for CRS Voting Disk and OCR Operating System Mount options for Sun Solaris rw,bg,hard,nointr,rsize=32768, rw,bg,hard,nointr,rsize=32768, rw,bg,hard,nointr,rsize=32768, wsize=32768,proto=tcp,noac, wsize=32768,proto=tcp,noac, wsize=32768,proto=tcp,vers=3, forcedirectio, vers=3,suid noac,forcedirectio vers=3,suid AIX (5L) rw,bg,hard,nointr,rsize=32768, cio,rw,bg,hard,nointr,rsize=3276 cio,rw,bg,hard,intr,rsize=32768, 8, wsize=32768,proto=tcp, wsize=32768,tcp,noac, wsize=32768,proto=tcp,noac, vers=3,timeo=600 vers=3,timeo=600 vers=3,timeo=600 HPUX 11.23 rw,bg,vers=3,proto=tcp,noac, hard,nointr,timeo=600, rsize=32768,wsize=32768,suid rw,bg,vers=3,proto=tcp,noac, forcedirectio,hard,nointr,timeo= 600, rsize=32768,wsize=32768,suid Windows Not Supported Not Supported rw,bg,vers=3,proto=tcp,noac, forcedirectio,hard,nointr,timeo= 600 ,rsize=32768,wsize=32768,suid Linux x86 Not Supported rw,bg,hard,nointr,rsize=32768, rw,bg,hard,nointr,rsize=32768, rw,bg,hard,nointr,rsize=32768, wsize=32768,tcp,noac,actimeo= wsize=32768,tcp, vers=3, wsize=32768,tcp,actimeo=0, 0, timeo=600, actimeo=0 vers=3,timeo=600 vers=3,timeo=600 Linux x86-64 rw,bg,hard,nointr,rsize=32768, rw,bg,hard,nointr,rsize=32768, rw,bg,hard,nointr,rsize=32768, wsize=32768,tcp,vers=3, wsize=32768,tcp,actimeo=0, wsize=32768,tcp,noac,vers=3, timeo=600, actimeo=0 vers=3,timeo=600 timeo=600,actimeo=0 Linux - Itanium rw,bg,hard,nointr,rsize=32768, rw,bg,hard,nointr,rsize=32768, rw,bg,hard,nointr,rsize=32768, wsize=32768,tcp,vers=3, wsize=32768,tcp,actimeo=0, wsize=32768,tcp,noac,vers=3, timeo=600, actimeo=0 vers=3,timeo=600 timeo=600,actimeo=0 MOSS: 359515.1 19 19 DB Refresh 1. Suspend managed recovery of standby DB Primary DB SQL> alter system set log_archive_dest_state_2=DEFER sid=’*’; DR DB SQL> alter database recover managed standby database cancel; 2. Take snapshot of the of the filesystem 3. Resume managed recovery of standby DB Primary DB SQL> alter system set log_archive_dest_state_2=ENABLE sid=’*’; DR DB SQL > alter database recover managed standby database using current logfile disconnect; 4. Create clone of snapshot 20 20 DB Refresh 5. Mount the filesystems on both the test RAC servers 6. Create a softlink to refer old location of standby redo logs Update db_file_name_convert and log_file_name_convert 7. Startup DB in mount state 8. Drop the standby redo logs 9. Activate the database SQL> alter database activate standby database ; SQL> shutdown immediate ; SQL> startup 10. Use NID to rename the database. 21 21 DB Refresh 22 22 Technical Issues 23 23 Issues: Exadata Exadata IPoIB protocol died •The cause is still not known fully •This happened when ZFS was hooked to Exadata via IB cable Workaround •Oracle support has confirmed that ZFS can be hooked to Exadata via IB •Transmit queue length to be increased to 1000 for the IB interfaces #ifconfig ib0 txqueuelen 1000 #ifconfig ib1 txqueuelen 1000 24 24 Issues: ZFS NAS Appliance Memory leak at the server heads •Cause known to be with SNMP service •Firmware to be patched until Oracle releases it •Workaround is to disable SNMP service HCC on ZFS needs SNMP service to be enabled 25 25 Q&A 26 26
© Copyright 2024