Percona Replication Manager – Renaming Cluster Hostnames

Recently I’ve had to rename a node on a production cluster – however, I did not have an experience on this yet so I decided to try on a sandbox and establish a known procedure. Without further ado, assuming we have 3 nodes and we want to rename ha3′s hostname to ha03.

[root@ha01 ~]# crm status
Last updated: Wed Jun  4 21:55:01 2014
Last change: Wed Jun  4 21:53:35 2014 via crm_attribute on ha01.localdomain
Stack: classic openais (with plugin)
Current DC: ha01.localdomain - partition with quorum
Version: 1.1.10-14.el6_5.2-368c726
3 Nodes configured, 3 expected votes
6 Resources configured


Online: [ ha01.localdomain ha02.localdomain ha3.localdomain ]

 Master/Slave Set: ms_MySQL [p_mysql]
     Masters: [ ha01.localdomain ]
     Slaves: [ ha02.localdomain ha3.localdomain ]
 Clone Set: ping_gw_clone [ping_gw]
     Started: [ ha01.localdomain ha02.localdomain ha3.localdomain ]

Here we go:

  1. Put ha3 on standby:
    [root@ha01 ~]# crm node standby ha3.localdomain
  2. Stop the cluster services on the node and delete it from the cluster, because I was using Pacemaker and Corosync, I followed the steps here, then rename the hostname as you would on any Linux system.
    [root@ha3 mysql]# crm_node -i
    1111009472
    [root@ha3 mysql]# service pacemaker stop
    Signaling Pacemaker Cluster Manager to terminate           [  OK  ]
    Waiting for cluster services to unload.                    [  OK  ]
    [root@ha3 mysql]# service corosync stop
    Signaling Corosync Cluster Engine (corosync) to terminate: [  OK  ]
    Waiting for corosync services to unload:.                  [  OK  ]
    
    [root@ha01 ~]# crm_node --force -R 1111009472
    [root@ha01 ~]# cibadmin --delete --obj_type nodes --crm_xml '<node uname="ha3.localdomain"/>'
    [root@ha01 ~]# cibadmin --delete --obj_type status --crm_xml '<node_state uname="ha3.localdomain"/>'
  3. Put the whole cluster in maintenance mode and restart the Pacemaker and Corosync services. I could not find a way to rename cleanly without doing this, if I don’t I always end up with ghost nodes on the cluster. Because ha01 is the current Master, I would cycle all cluster services – stopping at ha01 last and starting with it first.
    [root@ha01 ~]# crm configure property maintenance-mode=true
    
    [root@ha02 ~]# service pacemaker stop
    Signaling Pacemaker Cluster Manager to terminate           [  OK  ]
    Waiting for cluster services to unload                     [  OK  ]
    [root@ha02 ~]# service corosync stop
    Signaling Corosync Cluster Engine (corosync) to terminate: [  OK  ]
    Waiting for corosync services to unload:                   [  OK  ]
    
    [root@ha01 ~]# service pacemaker stop
    Signaling Pacemaker Cluster Manager to terminate           [  OK  ]
    Waiting for cluster services to unload.                    [  OK  ]
    [root@ha01 ~]# service corosync stop
    Signaling Corosync Cluster Engine (corosync) to terminate: [  OK  ]
    Waiting for corosync services to unload:.                  [  OK  ]
    
    [root@ha01 ~]# service corosync start
    Starting Corosync Cluster Engine (corosync):               [  OK  ]
    [root@ha01 ~]# service pacemaker start
    Starting Pacemaker Cluster Manager                         [  OK  ]
    
    [root@ha02 ~]# service corosync start
    Starting Corosync Cluster Engine (corosync):               [  OK  ]
    [root@ha02 ~]# service pacemaker start
    Starting Pacemaker Cluster Manager                         [  OK  ]
  4. Disable maintenance-mode and verify the 2 actives nodes are up and running:
    [root@ha01 ~]# crm configure property maintenance-mode=false
    [root@ha01 ~]# crm status
    Last updated: Wed Jun  4 22:56:24 2014
    Last change: Wed Jun  4 22:56:23 2014 via crm_attribute on ha02.localdomain
    Stack: classic openais (with plugin)
    Current DC: ha02.localdomain - partition with quorum
    Version: 1.1.10-14.el6_5.2-368c726
    2 Nodes configured, 3 expected votes
    5 Resources configured
    
    
    Online: [ ha01.localdomain ha02.localdomain ]
    
     Master/Slave Set: ms_MySQL [p_mysql]
         Masters: [ ha01.localdomain ]
         Slaves: [ ha02.localdomain ]
     Clone Set: ping_gw_clone [ping_gw]
         Started: [ ha01.localdomain ha02.localdomain ]
  5. Then start the cluster services on ha03 which I renamed:
    [root@ha03 mysql]# service corosync start
    Starting Corosync Cluster Engine (corosync):               [  OK  ]
    [root@ha03 mysql]# service pacemaker start
    Starting Pacemaker Cluster Manager                         [  OK  ]
    [root@ha03 mysql]# crm status
    Last updated: Wed Jun  4 23:14:30 2014
    Last change: Wed Jun  4 23:14:25 2014 via crm_attribute on ha02.localdomain
    Stack: classic openais (with plugin)
    Current DC: ha02.localdomain - partition with quorum
    Version: 1.1.10-14.el6_5.2-368c726
    3 Nodes configured, 3 expected votes
    6 Resources configured
    
    
    Online: [ ha01.localdomain ha02.localdomain ha03.localdomain ]
    
     Master/Slave Set: ms_MySQL [p_mysql]
         Masters: [ ha01.localdomain ]
         Slaves: [ ha02.localdomain ha03.localdomain ]
     Clone Set: ping_gw_clone [ping_gw]
         Started: [ ha01.localdomain ha02.localdomain ha03.localdomain ]

Do you have any more convenient way of doing this?