Percona Replication Manager – Renaming Cluster Hostnames
Recently I’ve had to rename a node on a production cluster – however, I did not have an experience on this yet so I decided to try on a sandbox and establish a known procedure. Without further ado, assuming we have 3 nodes and we want to rename ha3’s hostname to ha03.
[root@ha01 ~]# crm status Last updated: Wed Jun 4 21:55:01 2014 Last change: Wed Jun 4 21:53:35 2014 via crm_attribute on ha01.localdomain Stack: classic openais (with plugin) Current DC: ha01.localdomain - partition with quorum Version: 1.1.10-14.el6_5.2-368c726 3 Nodes configured, 3 expected votes 6 Resources configured Online: [ ha01.localdomain ha02.localdomain ha3.localdomain ] Master/Slave Set: ms_MySQL [p_mysql] Masters: [ ha01.localdomain ] Slaves: [ ha02.localdomain ha3.localdomain ] Clone Set: ping_gw_clone [ping_gw] Started: [ ha01.localdomain ha02.localdomain ha3.localdomain ]
Here we go:
- Put ha3 on standby:
[root@ha01 ~]# crm node standby ha3.localdomain
- Stop the cluster services on the node and delete it from the cluster, because I was using Pacemaker and Corosync, I followed the steps here, then rename the hostname as you would on any Linux system.
[root@ha3 mysql]# crm_node -i 1111009472 [root@ha3 mysql]# service pacemaker stop Signaling Pacemaker Cluster Manager to terminate [ OK ] Waiting for cluster services to unload. [ OK ] [root@ha3 mysql]# service corosync stop Signaling Corosync Cluster Engine (corosync) to terminate: [ OK ] Waiting for corosync services to unload:. [ OK ] [root@ha01 ~]# crm_node --force -R 1111009472 [root@ha01 ~]# cibadmin --delete --obj_type nodes --crm_xml '<node uname="ha3.localdomain"/>' [root@ha01 ~]# cibadmin --delete --obj_type status --crm_xml '<node_state uname="ha3.localdomain"/>'
- Put the whole cluster in maintenance mode and restart the Pacemaker and Corosync services. I could not find a way to rename cleanly without doing this, if I don’t I always end up with ghost nodes on the cluster. Because ha01 is the current Master, I would cycle all cluster services – stopping at ha01 last and starting with it first.
[root@ha01 ~]# crm configure property maintenance-mode=true [root@ha02 ~]# service pacemaker stop Signaling Pacemaker Cluster Manager to terminate [ OK ] Waiting for cluster services to unload [ OK ] [root@ha02 ~]# service corosync stop Signaling Corosync Cluster Engine (corosync) to terminate: [ OK ] Waiting for corosync services to unload: [ OK ] [root@ha01 ~]# service pacemaker stop Signaling Pacemaker Cluster Manager to terminate [ OK ] Waiting for cluster services to unload. [ OK ] [root@ha01 ~]# service corosync stop Signaling Corosync Cluster Engine (corosync) to terminate: [ OK ] Waiting for corosync services to unload:. [ OK ] [root@ha01 ~]# service corosync start Starting Corosync Cluster Engine (corosync): [ OK ] [root@ha01 ~]# service pacemaker start Starting Pacemaker Cluster Manager [ OK ] [root@ha02 ~]# service corosync start Starting Corosync Cluster Engine (corosync): [ OK ] [root@ha02 ~]# service pacemaker start Starting Pacemaker Cluster Manager [ OK ]
- Disable maintenance-mode and verify the 2 actives nodes are up and running:
[root@ha01 ~]# crm configure property maintenance-mode=false [root@ha01 ~]# crm status Last updated: Wed Jun 4 22:56:24 2014 Last change: Wed Jun 4 22:56:23 2014 via crm_attribute on ha02.localdomain Stack: classic openais (with plugin) Current DC: ha02.localdomain - partition with quorum Version: 1.1.10-14.el6_5.2-368c726 2 Nodes configured, 3 expected votes 5 Resources configured Online: [ ha01.localdomain ha02.localdomain ] Master/Slave Set: ms_MySQL [p_mysql] Masters: [ ha01.localdomain ] Slaves: [ ha02.localdomain ] Clone Set: ping_gw_clone [ping_gw] Started: [ ha01.localdomain ha02.localdomain ]
- Then start the cluster services on ha03 which I renamed:
[root@ha03 mysql]# service corosync start Starting Corosync Cluster Engine (corosync): [ OK ] [root@ha03 mysql]# service pacemaker start Starting Pacemaker Cluster Manager [ OK ] [root@ha03 mysql]# crm status Last updated: Wed Jun 4 23:14:30 2014 Last change: Wed Jun 4 23:14:25 2014 via crm_attribute on ha02.localdomain Stack: classic openais (with plugin) Current DC: ha02.localdomain - partition with quorum Version: 1.1.10-14.el6_5.2-368c726 3 Nodes configured, 3 expected votes 6 Resources configured Online: [ ha01.localdomain ha02.localdomain ha03.localdomain ] Master/Slave Set: ms_MySQL [p_mysql] Masters: [ ha01.localdomain ] Slaves: [ ha02.localdomain ha03.localdomain ] Clone Set: ping_gw_clone [ping_gw] Started: [ ha01.localdomain ha02.localdomain ha03.localdomain ]
Do you have any more convenient way of doing this?
Comments
Leave a Comment