How to Update a Red Hat Enterprise Linux Cluster while Minimizing Downtime Authors: Adam Drew and Eva Schaller Editor: Allison Pranger 11/25/2010 OVERVIEW A rolling upgrade is the process of taking a running cluster and updating packages either from one minor version of Red Hat Enterprise Linux to another (for example, updating a Red Hat Enterprise Linux 5.4 cluster to Red Hat Enterprise Linux 5.5) or updating to the latest errata packages in the same minor release (for example, updating a Red Hat Enterprise Linux 5.5 cluster with the latest Red Hat Enterprise Linux 5.5.z release). NOTE: Rolling upgrades between minor releases of Red Hat Enterprise Linux are fully supported, but rolling upgrades between major releases of Red Hat Enterprise Linux are not supported at this time. This document describes how to perform a rolling upgrade to update software on every node of a cluster while minimizing downtime. Environment • Red Hat Cluster Suite • Red Hat Enterprise Linux 5+ Advanced Platform (Clustering and GFS/GFS2) RECOMMENDATIONS During the upgrade process, the cluster will be running some nodes at one software revision level and other nodes at a higher software revision. It is best practice to minimize the amount of time that the cluster is running with a heterogeneous software configuration. There is no set time limit for how long a cluster may run in this mode, but returning all nodes to the same software versions as quickly as possible is recommended. The specific process you should follow depends on how many nodes are in your cluster and how many versions you are moving between. NOTE: Red Hat only tests upgrading from one point release to the next (for example, 5.3 to 5.4 or 5.2 to 5.3). Skipping point releases (for example, 5.2 to 5.5) has not been tested. Cluster with Three Nodes or More The following processes apply to clusters with three or more nodes. Moving from One Point Release to the Next 1. 2. 3. Select one node in the cluster. Move its clustered service(s) to another node. Gracefully remove the node from the cluster. How to Update a Red Hat Enterprise Linux Cluster while Minimizing Downtime | Adam Drew and Eva Schaller 1 # service rgmanager stop ; service gfs stop ; service gfs2 stop ; service clvmd stop ; service qdiskd stop ; service cman stop 4. Update the node in question. # yum groupupdate clustering cluster-storage 5. Bring the node back into the cluster. # service cman start ; service qdiskd start ; service clvmd start ; service gfs start ; service gfs2 start ; service rgmanager start 6. 7. Migrate the services back to the updated node. Choose another node and repeat the process until the entire cluster is updated. Skipping Point Releases 1. Stop the cluster services on all nodes. # service rgmanager stop ; service gfs stop ; service gfs2 stop ; service clvmd stop ; service qdiskd stop ; service cman stop 2. 3. Update all nodes. Start the cluster services on all nodes. # service cman start ; service qdiskd start ; service clvmd start ; service gfs start ; service gfs2 start ; service rgmanager start 4. Verify the cluster is operational. Clusters with Two Nodes The following processes apply to two-node clusters. Moving from One Point Release to the Next 1. 2. 3. Select one node in the cluster. Move its clustered service(s) to the other node in the cluster. Gracefully remove the node from the cluster. # service rgmanager stop ; service gfs stop ; service gfs2 stop ; service clvmd stop ; service qdiskd stop ; service cman stop 4. Update the node in question. # yum groupupdate clustering cluster-storage 5. 6. 7. 8. Bring the node back into the cluster. Migrate the services back to the updated node. Migrate the services from the other node to the updated node. Repeat the process for the other node in the cluster. How to Update a Red Hat Enterprise Linux Cluster while Minimizing Downtime | Adam Drew and Eva Schaller 2 Skipping Point Releases 1. 2. 3. Select one node in the cluster. Move its clustered service(s) to the other node in the cluster. Gracefully remove the node from the cluster. # service rgmanager stop ; service gfs stop ; service gfs2 stop ; service clvmd stop ; service qdiskd stop ; service cman stop 4. Update the node in question. # yum groupupdate clustering cluster-storage 5. Bring the other node in the cluster down. # service rgmanager stop ; service gfs stop ; service gfs2 stop ; service clvmd stop ; service qdiskd stop ; service cman stop 6. Bring up the updated node. # service cman start ; service qdiskd start ; service clvmd start ; service gfs start ; service gfs2 start ; service rgmanager start 7. Update the downed node. # yum groupupdate clustering cluster-storage 8. Bring the most recently updated node up and back into the cluster. # service cman start ; service qdiskd start ; service clvmd start ; service gfs start ; service gfs2 start ; service rgmanager start How to Update a Red Hat Enterprise Linux Cluster while Minimizing Downtime | Adam Drew and Eva Schaller 3 Copyright © 2012 Red Hat, Inc. “Red Hat,” Red Hat Linux, the Red Hat “Shadowman” logo, and the products listed are trademarks of Red Hat, Inc., registered in the U.S. and other countries. Linux® is the registered trademark of Linus Torvalds in the U.S. and other countries. www.redhat.com