Percona Replication Manager – Renaming Cluster Hostnames
Recently I’ve had to rename a node on a production cluster – however, I did not have an experience on this yet so I decided to try on a sandbox and establish a known procedure. Without further ado, assuming we have 3 nodes and we want to rename ha3’s hostname to ha03.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
[root@ha01 ~]# crm status Last updated: Wed Jun 4 21:55:01 2014 Last change: Wed Jun 4 21:53:35 2014 via crm_attribute on ha01.localdomain Stack: classic openais (with plugin) Current DC: ha01.localdomain - partition with quorum Version: 1.1.10-14.el6_5.2-368c726 3 Nodes configured, 3 expected votes 6 Resources configured Online: [ ha01.localdomain ha02.localdomain ha3.localdomain ] Master/Slave Set: ms_MySQL [p_mysql] Masters: [ ha01.localdomain ] Slaves: [ ha02.localdomain ha3.localdomain ] Clone Set: ping_gw_clone [ping_gw] Started: [ ha01.localdomain ha02.localdomain ha3.localdomain ] |
Here we go:
- Put ha3 on standby:
1[root@ha01 ~]# crm node standby ha3.localdomain - Stop the cluster services on the node and delete it from the cluster, because I was using Pacemaker and Corosync, I followed the steps here, then rename the hostname as you would on any Linux system.
123456789101112[root@ha3 mysql]# crm_node -i1111009472[root@ha3 mysql]# service pacemaker stopSignaling Pacemaker Cluster Manager to terminate [ OK ]Waiting for cluster services to unload. [ OK ][root@ha3 mysql]# service corosync stopSignaling Corosync Cluster Engine (corosync) to terminate: [ OK ]Waiting for corosync services to unload:. [ OK ][root@ha01 ~]# crm_node --force -R 1111009472[root@ha01 ~]# cibadmin --delete --obj_type nodes --crm_xml '<node uname="ha3.localdomain"/>'[root@ha01 ~]# cibadmin --delete --obj_type status --crm_xml '<node_state uname="ha3.localdomain"/>' - Put the whole cluster in maintenance mode and restart the Pacemaker and Corosync services. I could not find a way to rename cleanly without doing this, if I don’t I always end up with ghost nodes on the cluster. Because ha01 is the current Master, I would cycle all cluster services – stopping at ha01 last and starting with it first.
12345678910111213141516171819202122232425[root@ha01 ~]# crm configure property maintenance-mode=true[root@ha02 ~]# service pacemaker stopSignaling Pacemaker Cluster Manager to terminate [ OK ]Waiting for cluster services to unload [ OK ][root@ha02 ~]# service corosync stopSignaling Corosync Cluster Engine (corosync) to terminate: [ OK ]Waiting for corosync services to unload: [ OK ][root@ha01 ~]# service pacemaker stopSignaling Pacemaker Cluster Manager to terminate [ OK ]Waiting for cluster services to unload. [ OK ][root@ha01 ~]# service corosync stopSignaling Corosync Cluster Engine (corosync) to terminate: [ OK ]Waiting for corosync services to unload:. [ OK ][root@ha01 ~]# service corosync startStarting Corosync Cluster Engine (corosync): [ OK ][root@ha01 ~]# service pacemaker startStarting Pacemaker Cluster Manager [ OK ][root@ha02 ~]# service corosync startStarting Corosync Cluster Engine (corosync): [ OK ][root@ha02 ~]# service pacemaker startStarting Pacemaker Cluster Manager [ OK ] - Disable maintenance-mode and verify the 2 actives nodes are up and running:
123456789101112131415161718[root@ha01 ~]# crm configure property maintenance-mode=false[root@ha01 ~]# crm statusLast updated: Wed Jun 4 22:56:24 2014Last change: Wed Jun 4 22:56:23 2014 via crm_attribute on ha02.localdomainStack: classic openais (with plugin)Current DC: ha02.localdomain - partition with quorumVersion: 1.1.10-14.el6_5.2-368c7262 Nodes configured, 3 expected votes5 Resources configuredOnline: [ ha01.localdomain ha02.localdomain ]Master/Slave Set: ms_MySQL [p_mysql]Masters: [ ha01.localdomain ]Slaves: [ ha02.localdomain ]Clone Set: ping_gw_clone [ping_gw]Started: [ ha01.localdomain ha02.localdomain ] - Then start the cluster services on ha03 which I renamed:
123456789101112131415161718192021[root@ha03 mysql]# service corosync startStarting Corosync Cluster Engine (corosync): [ OK ][root@ha03 mysql]# service pacemaker startStarting Pacemaker Cluster Manager [ OK ][root@ha03 mysql]# crm statusLast updated: Wed Jun 4 23:14:30 2014Last change: Wed Jun 4 23:14:25 2014 via crm_attribute on ha02.localdomainStack: classic openais (with plugin)Current DC: ha02.localdomain - partition with quorumVersion: 1.1.10-14.el6_5.2-368c7263 Nodes configured, 3 expected votes6 Resources configuredOnline: [ ha01.localdomain ha02.localdomain ha03.localdomain ]Master/Slave Set: ms_MySQL [p_mysql]Masters: [ ha01.localdomain ]Slaves: [ ha02.localdomain ha03.localdomain ]Clone Set: ping_gw_clone [ping_gw]Started: [ ha01.localdomain ha02.localdomain ha03.localdomain ]
Do you have any more convenient way of doing this?