Playing around with Percona Replication Manager, and being new to Pacemaker, some errors were somewhat cryptic.

May 26 09:53:39 [2181] ha01.localdomain    pengine:   notice: unpack_rsc_op:    Preventing p_mysql from re-starting on ha02.localdomain: operation monitor failed 'not installed' (rc=5)
May 26 09:53:39 [2181] ha01.localdomain    pengine:   notice: unpack_rsc_op:    Preventing op from re-starting on ha02.localdomain: operation monitor failed 'not installed' (rc=5)

First, it says here that I have ‘operation monitor failed’ on node ha02.localdomain. This error simply means ‘I cannot run the operation monitor because it is not installed’. Where does this operation supposed to come from, from the resource agent. Turns out, I did not have the mysql resource agent in place under /usr/lib/ocf/resource.d/percona.

May 27 23:24:29 [2013] ha01.localdomain crm_resource:    error: unpack_rsc_op:  Preventing reader_vip_2 from re-starting anywhere in the cluster : operation start failed 'not configured' (rc=6)
May 27 23:24:29 [2013] ha01.localdomain crm_resource:    error: unpack_rsc_op:  Preventing reader_vip_1 from re-starting anywhere in the cluster : operation start failed 'not configured' (rc=6)

Second, here it says it failed to start the the reader_vip resources, lookint only from the cluster status outputs the cause is not obvious. If you dig closely from the cluster logs, you may find the reason why. In my case, the reader VIP cannot be bound to an interface which was non-existent.

May 27 23:24:29 [1581] ha01.localdomain       lrmd:   notice: operation_finished:       reader_vip_1_start_0:1971 [ eth1: unknown interface: No such device ]
May 27 23:24:29 [1581] ha01.localdomain       lrmd:   notice: operation_finished:       reader_vip_1_start_0:1971 [  ]
May 27 23:24:29 [1581] ha01.localdomain       lrmd:   notice: operation_finished:       reader_vip_1_start_0:1971 [ /usr/lib64/heartbeat/findif version 2.99.1 Copyright Alan Robertson ]

I plan to write more as I explore the exciting world of Pacemaker!