[Linux-HA] crm_mon showing different status on cluster nodes
Alex Strachan
astrachan at inter-systems.com.au
Sun Nov 18 23:53:35 MST 2007
In a production cluster (four nodes) the crm_mon command is showing
different statuses.
How to resync the status - without affecting services! Not sure how the
sync option in cibadmin works.
crm_mon output from the four nodes, basically
3 resources running on sinfids3a1 are being reported as stopped although
they are running OKAY.
1 node is reported as being offline - although it is running the resources
as reported by other nodes
Very puzzling.
All nodes report sinfids3b2 as the DC
[root at sinfids3a1 init.d]#
============
Last updated: Mon Nov 19 14:15:35 2007
Current DC: sinfids3b2 (b641a173-b7a1-4cf9-a8d3-a90cea2d7beb)
4 Nodes configured.
18 Resources configured.
============
Node: sinfids3a1 (3e940b7a-0e41-4df4-9279-71205d5038e8): online
Node: sinfids3a2 (8cc209ea-9be8-48a7-a1a1-3fdc7b7a64d4): online
Node: sinfids3b2 (b641a173-b7a1-4cf9-a8d3-a90cea2d7beb): online
Node: sinfids3b1 (ac809f97-d6b7-4c14-ab50-a8aa6246e132): online
Resource Group: group_sinfids3a
resource_sinfids3a_drbd (heartbeat:sinfids3a_drbddisk): Started
sinfids3a1
resource_sinfids3a_fs (heartbeat::ocf:sinfids3a_Filesystem):
Started sinfids3a1
resource_sinfids3a_vip (heartbeat::ocf:IPaddr): Started
sinfids3a1
resource_sinfids3a_oracle (heartbeat::ocf:sinfids3a_oracle):
Stopped
resource_sinfids3a_oralsnr (heartbeat::ocf:sinfids3a_oralsnr):
Stopped
resource_sinfids3a_aims (lsb:sinfids3a_aims): Started sinfids3a1
resource_sinfids3a_apache (heartbeat::ocf:sinfids3a_apache):
Stopped
resource_sinfids3a_smb (lsb:sinfids3a_smb): Started sinfids3a1
Resource Group: group_sinfids3b
resource_sinfids3b_drbd (heartbeat:sinfids3b_drbddisk): Started
sinfids3b1
resource_sinfids3b_fs (heartbeat::ocf:sinfids3b_Filesystem):
Started sinfids3b1
resource_sinfids3b_vip (heartbeat::ocf:IPaddr): Started
sinfids3b1
resource_sinfids3b_oracle (heartbeat::ocf:sinfids3b_oracle):
Started sinfids3b1
resource_sinfids3b_oralsnr (heartbeat::ocf:sinfids3b_oralsnr):
Started sinfids3b1
resource_sinfids3b_aims (lsb:sinfids3b_aims): Started sinfids3b1
resource_sinfids3b_apache (heartbeat::ocf:sinfids3b_apache):
Started sinfids3b1
resource_sinfids3b_smb (lsb:sinfids3b_smb): Started sinfids3b1
resource_sinfids3_vip (heartbeat::ocf:IPaddr): Started sinfids3a1
resource_sinfids3_skel1 (heartbeat::ocf:sinfids3_ISskel): Started
sinfids3a1
resource_sinfids3_gatemid1 (heartbeat::ocf:sinfids3_ISgatemid):
Started sinfids3a1
resource_sinfids3_itsixcats01 (heartbeat::ocf:sinfids3_ITSIXcats):
Started sinfids3a1
resource_sinfids3_dummy (heartbeat::ocf:Dummy): Started sinfids3a1
resource_sinfids3a_itsixatob (heartbeat::ocf:sinfids3a_ITSIXatob):
Started sinfids3a1
resource_sinfids3a_apache_nagios
(heartbeat::ocf:sinfids3a_apache_nagios): Started sinfids3a1
resource_sinfids3a_nagios (lsb:sinfids3a_nagios): Started sinfids3a1
resource_sinfids3b_apache_nagios
(heartbeat::ocf:sinfids3b_apache_nagios): Started sinfids3b1
resource_sinfids3b_nagios (lsb:sinfids3b_nagios): Started sinfids3b1
resource_sinfids3a_fids_flight (lsb:sinfids3a_fids_flight): Started
sinfids3a1
resource_sinfids3a_fids_weather (lsb:sinfids3a_fids_weather): Started
sinfids3a1
resource_sinfids3a_fids_proxy (lsb:sinfids3a_fids_proxy): Started
sinfids3a2
resource_sinfids3b_fids_flight (lsb:sinfids3b_fids_flight): Started
sinfids3b1
resource_sinfids3b_fids_weather (lsb:sinfids3b_fids_weather): Started
sinfids3b1
resource_sinfids3b_fids_proxy (lsb:sinfids3b_fids_proxy): Started
sinfids3b2
[root at sinfids3a2 ~]#
============
Last updated: Mon Nov 19 14:15:51 2007
Current DC: sinfids3b2 (b641a173-b7a1-4cf9-a8d3-a90cea2d7beb)
4 Nodes configured.
18 Resources configured.
============
Node: sinfids3a1 (3e940b7a-0e41-4df4-9279-71205d5038e8): online
Node: sinfids3a2 (8cc209ea-9be8-48a7-a1a1-3fdc7b7a64d4): online
Node: sinfids3b2 (b641a173-b7a1-4cf9-a8d3-a90cea2d7beb): online
Node: sinfids3b1 (ac809f97-d6b7-4c14-ab50-a8aa6246e132): online
Resource Group: group_sinfids3a
resource_sinfids3a_drbd (heartbeat:sinfids3a_drbddisk): Started
sinfids3a1
resource_sinfids3a_fs (heartbeat::ocf:sinfids3a_Filesystem):
Started sinfids3a1
resource_sinfids3a_vip (heartbeat::ocf:IPaddr): Started
sinfids3a1
resource_sinfids3a_oracle (heartbeat::ocf:sinfids3a_oracle):
Stopped
resource_sinfids3a_oralsnr (heartbeat::ocf:sinfids3a_oralsnr):
Stopped
resource_sinfids3a_aims (lsb:sinfids3a_aims): Started sinfids3a1
resource_sinfids3a_apache (heartbeat::ocf:sinfids3a_apache):
Stopped
resource_sinfids3a_smb (lsb:sinfids3a_smb): Started sinfids3a1
Resource Group: group_sinfids3b
resource_sinfids3b_drbd (heartbeat:sinfids3b_drbddisk): Started
sinfids3b1
resource_sinfids3b_fs (heartbeat::ocf:sinfids3b_Filesystem):
Started sinfids3b1
resource_sinfids3b_vip (heartbeat::ocf:IPaddr): Started
sinfids3b1
resource_sinfids3b_oracle (heartbeat::ocf:sinfids3b_oracle):
Started sinfids3b1
resource_sinfids3b_oralsnr (heartbeat::ocf:sinfids3b_oralsnr):
Started sinfids3b1
resource_sinfids3b_aims (lsb:sinfids3b_aims): Started sinfids3b1
resource_sinfids3b_apache (heartbeat::ocf:sinfids3b_apache):
Started sinfids3b1
resource_sinfids3b_smb (lsb:sinfids3b_smb): Started sinfids3b1
resource_sinfids3_vip (heartbeat::ocf:IPaddr): Started sinfids3a1
resource_sinfids3_skel1 (heartbeat::ocf:sinfids3_ISskel): Started
sinfids3a1
resource_sinfids3_gatemid1 (heartbeat::ocf:sinfids3_ISgatemid):
Started sinfids3a1
resource_sinfids3_itsixcats01 (heartbeat::ocf:sinfids3_ITSIXcats):
Started sinfids3a1
resource_sinfids3_dummy (heartbeat::ocf:Dummy): Started sinfids3a1
resource_sinfids3a_itsixatob (heartbeat::ocf:sinfids3a_ITSIXatob):
Started sinfids3a1
resource_sinfids3a_apache_nagios
(heartbeat::ocf:sinfids3a_apache_nagios): Started sinfids3a1
resource_sinfids3a_nagios (lsb:sinfids3a_nagios): Started sinfids3a1
resource_sinfids3b_apache_nagios
(heartbeat::ocf:sinfids3b_apache_nagios): Started sinfids3b1
resource_sinfids3b_nagios (lsb:sinfids3b_nagios): Started sinfids3b1
resource_sinfids3a_fids_flight (lsb:sinfids3a_fids_flight): Started
sinfids3a1
resource_sinfids3a_fids_weather (lsb:sinfids3a_fids_weather): Started
sinfids3a1
resource_sinfids3a_fids_proxy (lsb:sinfids3a_fids_proxy): Started
sinfids3a2
resource_sinfids3b_fids_flight (lsb:sinfids3b_fids_flight): Started
sinfids3b1
resource_sinfids3b_fids_weather (lsb:sinfids3b_fids_weather): Started
sinfids3b1
resource_sinfids3b_fids_proxy (lsb:sinfids3b_fids_proxy): Started
sinfids3b2
[root at sinfids3b1 ~]#
============
Last updated: Mon Nov 19 14:16:15 2007
Current DC: sinfids3b2 (b641a173-b7a1-4cf9-a8d3-a90cea2d7beb)
4 Nodes configured.
18 Resources configured.
============
Node: sinfids3a1 (3e940b7a-0e41-4df4-9279-71205d5038e8): OFFLINE
Node: sinfids3a2 (8cc209ea-9be8-48a7-a1a1-3fdc7b7a64d4): online
Node: sinfids3b2 (b641a173-b7a1-4cf9-a8d3-a90cea2d7beb): online
Node: sinfids3b1 (ac809f97-d6b7-4c14-ab50-a8aa6246e132): online
Resource Group: group_sinfids3b
resource_sinfids3b_drbd (heartbeat:sinfids3b_drbddisk): Started
sinfids3b1
resource_sinfids3b_fs (heartbeat::ocf:sinfids3b_Filesystem):
Started sinfids3b1
resource_sinfids3b_vip (heartbeat::ocf:IPaddr): Started
sinfids3b1
resource_sinfids3b_oracle (heartbeat::ocf:sinfids3b_oracle):
Started sinfids3b1
resource_sinfids3b_oralsnr (heartbeat::ocf:sinfids3b_oralsnr):
Started sinfids3b1
resource_sinfids3b_aims (lsb:sinfids3b_aims): Started sinfids3b1
resource_sinfids3b_apache (heartbeat::ocf:sinfids3b_apache):
Started sinfids3b1
resource_sinfids3b_smb (lsb:sinfids3b_smb): Started sinfids3b1
resource_sinfids3b_apache_nagios
(heartbeat::ocf:sinfids3b_apache_nagios): Started sinfids3b1
resource_sinfids3b_nagios (lsb:sinfids3b_nagios): Started sinfids3b1
resource_sinfids3a_fids_proxy (lsb:sinfids3a_fids_proxy): Started
sinfids3a2
resource_sinfids3b_fids_flight (lsb:sinfids3b_fids_flight): Started
sinfids3b1
resource_sinfids3b_fids_weather (lsb:sinfids3b_fids_weather): Started
sinfids3b1
resource_sinfids3b_fids_proxy (lsb:sinfids3b_fids_proxy): Started
sinfids3b2
[root at sinfids3b2 ~]#
============
Last updated: Mon Nov 19 14:16:38 2007
Current DC: sinfids3b2 (b641a173-b7a1-4cf9-a8d3-a90cea2d7beb)
4 Nodes configured.
18 Resources configured.
============
Node: sinfids3a1 (3e940b7a-0e41-4df4-9279-71205d5038e8): online
Node: sinfids3a2 (8cc209ea-9be8-48a7-a1a1-3fdc7b7a64d4): online
Node: sinfids3b2 (b641a173-b7a1-4cf9-a8d3-a90cea2d7beb): online
Node: sinfids3b1 (ac809f97-d6b7-4c14-ab50-a8aa6246e132): online
Resource Group: group_sinfids3a
resource_sinfids3a_drbd (heartbeat:sinfids3a_drbddisk): Started
sinfids3a1
resource_sinfids3a_fs (heartbeat::ocf:sinfids3a_Filesystem):
Started sinfids3a1
resource_sinfids3a_vip (heartbeat::ocf:IPaddr): Started
sinfids3a1
resource_sinfids3a_oracle (heartbeat::ocf:sinfids3a_oracle):
Stopped
resource_sinfids3a_oralsnr (heartbeat::ocf:sinfids3a_oralsnr):
Stopped
resource_sinfids3a_aims (lsb:sinfids3a_aims): Started sinfids3a1
resource_sinfids3a_apache (heartbeat::ocf:sinfids3a_apache):
Stopped
resource_sinfids3a_smb (lsb:sinfids3a_smb): Started sinfids3a1
Resource Group: group_sinfids3b
resource_sinfids3b_drbd (heartbeat:sinfids3b_drbddisk): Started
sinfids3b1
resource_sinfids3b_fs (heartbeat::ocf:sinfids3b_Filesystem):
Started sinfids3b1
resource_sinfids3b_vip (heartbeat::ocf:IPaddr): Started
sinfids3b1
resource_sinfids3b_oracle (heartbeat::ocf:sinfids3b_oracle):
Started sinfids3b1
resource_sinfids3b_oralsnr (heartbeat::ocf:sinfids3b_oralsnr):
Started sinfids3b1
resource_sinfids3b_aims (lsb:sinfids3b_aims): Started sinfids3b1
resource_sinfids3b_apache (heartbeat::ocf:sinfids3b_apache):
Started sinfids3b1
resource_sinfids3b_smb (lsb:sinfids3b_smb): Started sinfids3b1
resource_sinfids3_vip (heartbeat::ocf:IPaddr): Started sinfids3a1
resource_sinfids3_skel1 (heartbeat::ocf:sinfids3_ISskel): Started
sinfids3a1
resource_sinfids3_gatemid1 (heartbeat::ocf:sinfids3_ISgatemid):
Started sinfids3a1
resource_sinfids3_itsixcats01 (heartbeat::ocf:sinfids3_ITSIXcats):
Started sinfids3a1
resource_sinfids3_dummy (heartbeat::ocf:Dummy): Started sinfids3a1
resource_sinfids3a_itsixatob (heartbeat::ocf:sinfids3a_ITSIXatob):
Started sinfids3a1
resource_sinfids3a_apache_nagios
(heartbeat::ocf:sinfids3a_apache_nagios): Started sinfids3a1
resource_sinfids3a_nagios (lsb:sinfids3a_nagios): Started sinfids3a1
resource_sinfids3b_apache_nagios
(heartbeat::ocf:sinfids3b_apache_nagios): Started sinfids3b1
resource_sinfids3b_nagios (lsb:sinfids3b_nagios): Started sinfids3b1
resource_sinfids3a_fids_flight (lsb:sinfids3a_fids_flight): Started
sinfids3a1
resource_sinfids3a_fids_weather (lsb:sinfids3a_fids_weather): Started
sinfids3a1
resource_sinfids3a_fids_proxy (lsb:sinfids3a_fids_proxy): Started
sinfids3a2
resource_sinfids3b_fids_flight (lsb:sinfids3b_fids_flight): Started
sinfids3b1
resource_sinfids3b_fids_weather (lsb:sinfids3b_fids_weather): Started
sinfids3b1
resource_sinfids3b_fids_proxy (lsb:sinfids3b_fids_proxy): Started
sinfids3b2
More information about the Linux-HA
mailing list