[Linux-HA] ldirectord problem, monitor fails
Stephan Berlet
SBerlet at sysdesign-edv.de
Tue Feb 5 03:57:12 MST 2008
Hi guys,
thank you for answering my annoying questions ;-)
> -----Original Message-----
> From: Todd Lyons [mailto:tlyons at ivenue.com]
> Sent: Friday, February 01, 2008 6:44 PM
> To: linux-ha at lists.linux-ha.org
> Subject: Re: [Linux-HA] ldirectord problem, monitor fails
>
> On Fri, Feb 01, 2008 at 12:04:44PM +0100, Stephan Berlet wrote:
>
> >Hi Todd,
> >thank you for your suggestions!
> >
> >I had simplified my /etc/sysctl.conf and removed the VIP
> from lo device.
> >I will let heartbeat manage the VIPs on both nodes with corresponding
> >constraints, so that the lo VIP never can run where the eth0
> VIP runs.
>
> There should not *BE* an lo VIP on the director. Until I get
> corrected
> by someone, I have never seen a configuration where you
> needed or wanted that.
I'm agree with that, that the director node should not have an
lo VIP, but I always thought that the other node, who is acting
as real server, need to have the VIP on lo:0 with ARP suppressed
to be able to accept packets to the VIP. Isn't that correct?
> >I tried a lot of configurations, and I arrive at the
> >conclusion that the ldirectord is the cause of my
> >problem. Even if I configure ldirectord as
> >the only resource, I get always this error:
> >
> >Feb 1 10:50:18 localhost crmd: [8164]: info: do_lrm_rsc_op:
> >Performing op=ldirectord_1_monitor_0
> >key=3:0:315504da-cde4-4824-9e7e-082eae17b1b8)
> >Feb 1 10:50:18 localhost lrmd: [8168]: ERROR:
> >(raexecocf.c:execra:173) execl failed for
> >/usr/lib/ocf/resource.d//hearbeat/ldirectord: No such file
> >or directory
>
> Does that file exist? You appear to be running
> RedHat/Fedora/CentOS of some sort, so just do:
> yum install heartbeat-ldirectord
> The packagers have split it into several packages so that you do not
> have to have every single component and its dependency installed. In
> your case, your system will probably be needing:
> heartbeat
> heartbeat-pils
> heartbeat-stonith
> heartbeat-ldirectord
> and maybe heartbeat-gui
>
> >There is a slash too much in the path:
> >"/usr/lib/ocf/resource.d//hearbeat/ldirectord"
>
> That doesn't matter. Linux treats multiple slashes as /:
> CentOS5[root at ivlb01 ~]# ls
> /usr/lib/ocf/resource.d/heartbeat/ldirectord
> /usr/lib/ocf/resource.d/heartbeat/ldirectord
> CentOS5[root at ivlb01 ~]# ls
> /usr/lib/ocf/resource.d////heartbeat//ldirectord
> /usr/lib/ocf/resource.d////heartbeat//ldirectord
>
> Your problem simply appears to be that you don't actually have the
> ldirectord component of heartbeat installed. Get that installed and
> then restart heartbeat and see what you see.
>
Cool, I didn't know about this yet! That file actually exists,
but anyway I found an error in this path. There is a missing "t"
in heartbeat: "/usr/lib/ocf/resource.d//hearbeat/ldirectord:
No such file or directory"
However, I forgot that I tried a version of ldirectord RA
(OCF compatible) I found on this mailing list. So I restored
the backuped file, and the error message disapeared.
> -----Original Message-----
> From: Andreas Mock [mailto:Andreas.Mock at web.de]
> Sent: Sunday, February 03, 2008 6:45 PM
> To: 'General Linux-HA mailing list'
> Subject: Re: [Linux-HA] ldirectord problem, monitor fails
>
>
> > Feb 1 10:50:18 localhost crmd: [8164]: ERROR:
> process_lrm_event: LRM
> > operation ldirectord_1_monitor_0 (call=2, rc=1) Error unknown error
> >
> > There is a slash too much in the path:
> > "/usr/lib/ocf/resource.d//hearbeat/ldirectord"
> > Has anybody a hint where this is stored, or how I can remove this?
> >
>
> Hi Stephan,
>
> the LRM is stating the resources and it seems that it isn't able to
> "exec"ute it. I'm pretty sure that it has nothing to do with
> this extra
> slash (/) as unixoide OSes ignore this.
>
> a) Is the file adressed by
> '/usr/lib/ocf/resource.d//hearbeat/ldirectord'
> there? I ask this for completeness. I guess you checked this.
The file was there, but I forgot that it wasn't the original file.
Now it is the original file again and this error is away.
(And new problems appear...)
> b) Which rights are set for this file. Is the executable bit set for
> the right user/gorup. Is user and group set correctly?
> c) Look at the sheebang path at the beginning of the script. Does it
> refer to the right script interpreter?
>
> Best regards
> Andreas Mock
>
>
Since the LRM can execute the ldirectord resource (I must admit that
this was my fault) I get the error that appears before I played with
that file. Here the crm_mon output:
============
Last updated: Tue Feb 5 10:54:13 2008
Current DC: defas2 (8d083aa4-14cc-4cca-9a8c-e7308fb2743c)
2 Nodes configured.
1 Resources configured.
============
Node: defas2 (8d083aa4-14cc-4cca-9a8c-e7308fb2743c): online
Node: defas1 (6ca8e370-6b00-4b8c-85fe-a67da4c562ba): online
Resource Group: group_1
ldirectord_1 (heartbeat:ldirectord): Started defas1 FAILED
LVSSyncDaemonSwap_2 (heartbeat:LVSSyncDaemonSwap): Started defas1
(unmanaged) FAILED
IPaddr_eth (heartbeat::ocf:IPaddr): Stopped
Failed actions:
LVSSyncDaemonSwap_2_start_0 (node=defas1, call=8, rc=255): Error
LVSSyncDaemonSwap_2_stop_0 (node=defas1, call=9, rc=255): Error
ldirectord_1_monitor_120000 (node=defas1, call=7, rc=7): complete
And in /var/log/messages I have found this: (on defas1, not DC)
--------------
crmd: info: do_lrm_rsc_op: Performing op=ldirectord_1_monitor_0
key=7:0:acb11936-6402-42e7-8834-d94a3c9a86fc)
lrmd: info: RA output: (ldirectord_1:monitor:stderr) ldirectord for
/etc/ha.d/ldirectord.cf is running with pid: 16879
lrmd: WARN: There is something wrong: the first line isn't read in. Maybe
the heartbeat does not ouput string correctly for status operation. Or the
code (myself) is wrong.
crmd: info: process_lrm_event: LRM operation ldirectord_1_monitor_0 (call=2,
rc=7) complete
crmd: info: do_lrm_rsc_op: Performing op=LVSSyncDaemonSwap_2_monitor_0
key=8:0:acb11936-6402-42e7-8834-d94a3c9a86fc)
crmd: info: process_lrm_event: LRM operation LVSSyncDaemonSwap_2_monitor_0
(call=3, rc=7) complete
crmd: info: do_lrm_rsc_op: Performing op=IPaddr_eth_monitor_0
key=9:0:acb11936-6402-42e7-8834-d94a3c9a86fc)
crmd: info: process_lrm_event: LRM operation IPaddr_eth_monitor_0 (call=4,
rc=7) complete
crmd: info: do_lrm_rsc_op: Performing op=ldirectord_1_start_0
key=4:1:acb11936-6402-42e7-8834-d94a3c9a86fc)
crmd: info: process_lrm_event: LRM operation ldirectord_1_start_0 (call=5,
rc=0) complete
crmd: info: log_data_element: build_operation_update: digest:source
<parameters 1="ldirectord.cf"/>
crmd: info: do_lrm_rsc_op: Performing op=ldirectord_1_start_0
key=5:2:acb11936-6402-42e7-8834-d94a3c9a86fc)
crmd: info: process_lrm_event: LRM operation ldirectord_1_start_0 (call=6,
rc=0) complete
crmd: info: log_data_element: build_operation_update: digest:source
<parameters 1="ldirectord.cf"/>
crmd: info: do_lrm_rsc_op: Performing op=ldirectord_1_monitor_120000
key=6:2:acb11936-6402-42e7-8834-d94a3c9a86fc)
lrmd: info: RA output: (ldirectord_1:monitor:stderr) ldirectord for
/etc/ha.d/ldirectord.cf is running with pid: 16879
lrmd: WARN: There is something wrong: the first line isn't read in. Maybe
the heartbeat does not ouput string correctly for status operation. Or the
code (myself) is wrong.
crmd: info: process_lrm_event: LRM operation ldirectord_1_monitor_120000
(call=7, rc=7) complete
crmd: info: do_lrm_rsc_op: Performing op=LVSSyncDaemonSwap_2_start_0
key=7:2:acb11936-6402-42e7-8834-d94a3c9a86fc)
kernel: IPVS: set_ctl: len 92 != 24
lrmd: info: RA output: (LVSSyncDaemonSwap_2:start:stderr) Module is wrong
version
lrmd: info: RA output: (LVSSyncDaemonSwap_2:start:stdout) ERROR:
ipvsadm --start-daemon master --mcast-interface=eth0 failed.
crmd: ERROR: process_lrm_event: LRM operation LVSSyncDaemonSwap_2_start_0
(call=8, rc=255) Error <unknown>
crmd: info: log_data_element: build_operation_update: digest:source
<parameters 1="master"/>
crmd: info: do_lrm_rsc_op: Performing op=LVSSyncDaemonSwap_2_stop_0
key=1:3:acb11936-6402-42e7-8834-d94a3c9a86fc)
kernel: IPVS: set_ctl: len 92 != 24
lrmd: info: RA output: (LVSSyncDaemonSwap_2:stop:stderr) Module is wrong
version
lrmd: info: RA output: (LVSSyncDaemonSwap_2:stop:stdout) ERROR:
ipvsadm --start-daemon backup --mcast-interface=eth0 failed.
crmd: ERROR: process_lrm_event: LRM operation LVSSyncDaemonSwap_2_stop_0
(call=9, rc=255) Error <unknown>
lrmd: info: RA output: (ldirectord_1:monitor:stderr) ldirectord for
/etc/ha.d/ldirectord.cf is running with pid: 16879
lrmd: WARN: There is something wrong: the first line isn't read in. Maybe
the heartbeat does not ouput string correctly for status operation. Or the
code (myself) is wrong.
--------------
The complete logs of both nodes are attached.
The output from cibadmin -Ql is attached , too.
IMHO there are two issues:
1. WARN: There is something wrong: the first line isn't read
in. Maybe the heartbeat does not ouput string correctly
for status operation. Or the code (myself) is wrong.
Can I correct this with using class=ocf instead class=heartbeat?
2. RA output: (LVSSyncDaemonSwap_2:stop:stderr) Module is wrong version
Do I have the wrong version of ipvsadm? My version is:
# ipvsadm
IP Virtual Server version 1.2.0 (size=4096)
My version of ldirectord is:
# ldirectord --version
Linux Director v1.186-ha-2.1.2
heartbeat: 2.1.2-3 (CentOS 4 RPMs)
kernel: 2.6.9-55.EL i686
linux: CentOS release 4.5 (Final)
Thank you for helping me in advance,
I've been working on this since several days.
Best regards, Stephan Berlet
HELPING HEADS for Hard- and Software
-------------------------------------------------------------------------
Für Ihre Projekte entwickeln wir maßgeschneiderte Lösungen - schnell,
flexibel und direkt vor Ort. Unser eingespieltes Team an erfahrenen Hard-
und Software-Spezialisten unterstützt Sie dort, wo Sie uns brauchen.
--------------------------------------------------------------------------
SysDesign GmbH
Säntisstrasse 25
D-88079 Kressbronn am Bodensee
Geschäftsführer: Franz Kleiner, Achim Solle
Handelsregister: Ulm 632138
--------------------------------------------------------------------------
-------------- next part --------------
Feb 5 10:48:21 localhost heartbeat: [8263]: info: Enabling logging daemon
Feb 5 10:48:21 localhost heartbeat: [8263]: info: logfile and debug file are those specified in logd config file (default /etc/logd.cf)
Feb 5 10:48:21 localhost heartbeat: [8263]: info: Version 2 support: on
Feb 5 10:48:22 localhost heartbeat: [8263]: info: **************************
Feb 5 10:48:22 localhost heartbeat: [8263]: info: Configuration validated. Starting heartbeat 2.1.2
Feb 5 10:48:22 localhost heartbeat: [8264]: info: heartbeat: version 2.1.2
Feb 5 10:48:22 localhost heartbeat: [8264]: info: Heartbeat generation: 1194446617
Feb 5 10:48:22 localhost heartbeat: [8264]: info: G_main_add_TriggerHandler: Added signal manual handler
Feb 5 10:48:22 localhost heartbeat: [8264]: info: G_main_add_TriggerHandler: Added signal manual handler
Feb 5 10:48:22 localhost heartbeat: [8264]: info: Removing /var/run/heartbeat/rsctmp failed, recreating.
Feb 5 10:48:22 localhost heartbeat: [8264]: info: glib: UDP Broadcast heartbeat started on port 694 (694) interface eth1
Feb 5 10:48:22 localhost heartbeat: [8264]: info: glib: UDP Broadcast heartbeat closed on port 694 interface eth1 - Status: 1
Feb 5 10:48:22 localhost heartbeat: [8264]: info: glib: Starting serial heartbeat on tty /dev/ttyS0 (19200 baud)
Feb 5 10:48:22 localhost heartbeat: [8264]: info: G_main_add_SignalHandler: Added signal handler for signal 17
Feb 5 10:48:22 localhost heartbeat: [8264]: info: Local status now set to: 'up'
Feb 5 10:48:23 localhost heartbeat: [8264]: info: Link defas2:/dev/ttyS0 up.
Feb 5 10:48:23 localhost heartbeat: [8264]: info: Link defas1:eth1 up.
Feb 5 10:48:23 localhost heartbeat: [8264]: info: Status update for node defas2: status up
Feb 5 10:48:24 localhost heartbeat: [8264]: info: Comm_now_up(): updating status to active
Feb 5 10:48:24 localhost heartbeat: [8264]: info: Local status now set to: 'active'
Feb 5 10:48:24 localhost heartbeat: [8264]: info: Starting child client "/usr/lib/heartbeat/ccm" (90,90)
Feb 5 10:48:24 localhost heartbeat: [8264]: info: Starting child client "/usr/lib/heartbeat/cib" (90,90)
Feb 5 10:48:24 localhost heartbeat: [8264]: info: Starting child client "/usr/lib/heartbeat/lrmd -r" (0,0)
Feb 5 10:48:24 localhost heartbeat: [8264]: info: Starting child client "/usr/lib/heartbeat/stonithd" (0,0)
Feb 5 10:48:24 localhost heartbeat: [8264]: info: Starting child client "/usr/lib/heartbeat/attrd" (90,90)
Feb 5 10:48:24 localhost heartbeat: [8264]: info: Starting child client "/usr/lib/heartbeat/crmd" (90,90)
Feb 5 10:48:24 localhost heartbeat: [8264]: info: Starting child client "/usr/lib/heartbeat/mgmtd -v" (0,0)
Feb 5 10:48:24 localhost heartbeat: [8274]: info: Starting "/usr/lib/heartbeat/ccm" as uid 90 gid 90 (pid 8274)
Feb 5 10:48:24 localhost heartbeat: [8275]: info: Starting "/usr/lib/heartbeat/cib" as uid 90 gid 90 (pid 8275)
Feb 5 10:48:24 localhost cib: [8275]: info: G_main_add_SignalHandler: Added signal handler for signal 15
Feb 5 10:48:24 localhost cib: [8275]: info: G_main_add_TriggerHandler: Added signal manual handler
Feb 5 10:48:24 localhost cib: [8275]: info: G_main_add_SignalHandler: Added signal handler for signal 17
Feb 5 10:48:24 localhost cib: [8275]: info: main: Retrieval of a per-action CIB: disabled
Feb 5 10:48:24 localhost cib: [8275]: info: readCibXmlFile: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <cib admin_epoch="0" have_quorum="false" ignore_dtd="false" num_peers="2" ccm_transition="2" cib_feature_revision="1.3" generated="false" epoch="1" num_updates="1" cib-last-written="Tue Feb 5 10:26:49 2008">
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <configuration>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <crm_config>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <cluster_property_set id="cib-bootstrap-options">
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <attributes>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-symmetric-cluster" name="symmetric-cluster" value="true"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-no-quorum-policy" name="no-quorum-policy" value="stop"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-default-resource-stickiness" name="default-resource-stickiness" value="0"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-default-resource-failure-stickiness" name="default-resource-failure-stickiness" value="0"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" value="false"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-stonith-action" name="stonith-action" value="reboot"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-stop-orphan-resources" name="stop-orphan-resources" value="true"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-stop-orphan-actions" name="stop-orphan-actions" value="true"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-remove-after-stop" name="remove-after-stop" value="false"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-short-resource-names" name="short-resource-names" value="true"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-transition-idle-timeout" name="transition-idle-timeout" value="5min"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-default-action-timeout" name="default-action-timeout" value="15s"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-is-managed-default" name="is-managed-default" value="true"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </attributes>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </cluster_property_set>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </crm_config>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <nodes>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <node id="8d083aa4-14cc-4cca-9a8c-e7308fb2743c" uname="defas2" type="normal"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <node id="6ca8e370-6b00-4b8c-85fe-a67da4c562ba" uname="defas1" type="normal"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </nodes>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <resources>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <group id="group_1">
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <primitive class="heartbeat" id="ldirectord_1" provider="heartbeat" type="ldirectord">
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <operations>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <op id="ldirectord_1_mon" interval="120s" name="monitor" timeout="60s"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </operations>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <instance_attributes id="ldirectord_1_inst_attr">
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <attributes>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="ldirectord_1_attr_1" name="1" value="ldirectord.cf"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </attributes>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </instance_attributes>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </primitive>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <primitive class="heartbeat" id="LVSSyncDaemonSwap_2" provider="heartbeat" type="LVSSyncDaemonSwap">
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <operations>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <op id="LVSSyncDaemonSwap_2_mon" interval="120s" name="monitor" timeout="60s"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </operations>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <instance_attributes id="LVSSyncDaemonSwap_2_inst_attr">
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <attributes>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="LVSSyncDaemonSwap_2_attr_1" name="1" value="master"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </attributes>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </instance_attributes>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </primitive>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <primitive class="ocf" id="IPaddr_eth" provider="heartbeat" type="IPaddr">
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <operations>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <op id="IPaddr_eth_mon" interval="5s" name="monitor" timeout="5s"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </operations>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <instance_attributes id="IPaddr_eth_inst_attr">
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <attributes>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="IPaddr_eth_attr_0" name="ip" value="172.30.4.170"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="IPaddr_eth_attr_1" name="netmask" value="16"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="IPaddr_eth_attr_2" name="nic" value="eth0"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </attributes>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </instance_attributes>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </primitive>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </group>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </resources>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <constraints>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <rsc_location id="rsc_location_group_1" rsc="group_1">
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <rule id="prefered_location_group_1" score="100">
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <expression attribute="#uname" id="prefered_location_group_1_expr" operation="eq" value="defas1"/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </rule>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </rsc_location>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </constraints>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </configuration>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] <status/>
Feb 5 10:48:24 localhost cib: [8275]: info: log_data_element: readCibXmlFile: [on-disk] </cib>
Feb 5 10:48:24 localhost cib: [8275]: info: startCib: CIB Initialization completed successfully
Feb 5 10:48:24 localhost cib: [8275]: info: cib_register_ha: Signing in with Heartbeat
Feb 5 10:48:24 localhost ccm: [8274]: info: Hostname: defas1
Feb 5 10:48:24 localhost cib: [8275]: info: cib_register_ha: FSA Hostname: defas1
Feb 5 10:48:24 localhost cib: [8275]: WARN: cib_init: CCM Activation failed
Feb 5 10:48:24 localhost cib: [8275]: WARN: cib_init: CCM Connection failed 1 times (30 max)
Feb 5 10:48:24 localhost heartbeat: [8276]: info: Starting "/usr/lib/heartbeat/lrmd -r" as uid 0 gid 0 (pid 8276)
Feb 5 10:48:25 localhost heartbeat: [8277]: info: Starting "/usr/lib/heartbeat/stonithd" as uid 0 gid 0 (pid 8277)
Feb 5 10:48:25 localhost heartbeat: [8278]: info: Starting "/usr/lib/heartbeat/attrd" as uid 90 gid 90 (pid 8278)
Feb 5 10:48:25 localhost heartbeat: [8279]: info: Starting "/usr/lib/heartbeat/crmd" as uid 90 gid 90 (pid 8279)
Feb 5 10:48:25 localhost crmd: [8279]: info: main: CRM Hg Version: 9a995a8a7052 tip
Feb 5 10:48:25 localhost crmd: [8279]: info: crmd_init: Starting crmd
Feb 5 10:48:25 localhost crmd: [8279]: info: G_main_add_SignalHandler: Added signal handler for signal 15
Feb 5 10:48:25 localhost crmd: [8279]: info: G_main_add_TriggerHandler: Added signal manual handler
Feb 5 10:48:25 localhost crmd: [8279]: info: G_main_add_SignalHandler: Added signal handler for signal 17
Feb 5 10:48:25 localhost heartbeat: [8280]: info: Starting "/usr/lib/heartbeat/mgmtd -v" as uid 0 gid 0 (pid 8280)
Feb 5 10:48:25 localhost mgmtd: [8280]: info: G_main_add_SignalHandler: Added signal handler for signal 15
Feb 5 10:48:25 localhost mgmtd: [8280]: info: G_main_add_SignalHandler: Added signal handler for signal 10
Feb 5 10:48:25 localhost mgmtd: [8280]: info: G_main_add_SignalHandler: Added signal handler for signal 12
Feb 5 10:48:25 localhost mgmtd: [8280]: WARN: lrm_signon: can not initiate connection
Feb 5 10:48:25 localhost mgmtd: [8280]: info: login to lrm: 0, ret:0
Feb 5 10:48:25 localhost lrmd: [8276]: info: G_main_add_SignalHandler: Added signal handler for signal 15
Feb 5 10:48:25 localhost lrmd: [8276]: info: G_main_add_SignalHandler: Added signal handler for signal 17
Feb 5 10:48:25 localhost lrmd: [8276]: info: G_main_add_SignalHandler: Added signal handler for signal 10
Feb 5 10:48:25 localhost lrmd: [8276]: info: G_main_add_SignalHandler: Added signal handler for signal 12
Feb 5 10:48:25 localhost lrmd: [8276]: info: Started.
Feb 5 10:48:25 localhost stonithd: [8277]: info: G_main_add_SignalHandler: Added signal handler for signal 10
Feb 5 10:48:25 localhost stonithd: [8277]: info: G_main_add_SignalHandler: Added signal handler for signal 12
Feb 5 10:48:25 localhost stonithd: [8277]: info: Signing in with heartbeat.
Feb 5 10:48:25 localhost attrd: [8278]: info: G_main_add_SignalHandler: Added signal handler for signal 15
Feb 5 10:48:25 localhost stonithd: [8277]: notice: /usr/lib/heartbeat/stonithd start up successfully.
Feb 5 10:48:25 localhost stonithd: [8277]: info: G_main_add_SignalHandler: Added signal handler for signal 17
Feb 5 10:48:25 localhost attrd: [8278]: info: register_with_ha: Hostname: defas1
Feb 5 10:48:25 localhost heartbeat: [8264]: info: Status update for node defas2: status active
Feb 5 10:48:25 localhost attrd: [8278]: info: register_with_ha: UUID: 6ca8e370-6b00-4b8c-85fe-a67da4c562ba
Feb 5 10:48:25 localhost cib: [8275]: WARN: cib_init: CCM Activation failed
Feb 5 10:48:25 localhost cib: [8275]: WARN: cib_init: CCM Connection failed 2 times (30 max)
Feb 5 10:48:25 localhost mgmtd: [8280]: info: init_crm
Feb 5 10:48:26 localhost cib: [8275]: WARN: cib_init: CCM Activation failed
Feb 5 10:48:26 localhost cib: [8275]: WARN: cib_init: CCM Connection failed 3 times (30 max)
Feb 5 10:48:26 localhost ccm: [8274]: info: G_main_add_SignalHandler: Added signal handler for signal 15
Feb 5 10:48:27 localhost cib: [8275]: info: cib_init: Starting cib mainloop
Feb 5 10:48:27 localhost crmd: [8279]: info: do_cib_control: CIB connection established
Feb 5 10:48:27 localhost cib: [8275]: info: cib_client_status_callback: Status update: Client defas1/cib now has status [join]
Feb 5 10:48:28 localhost crmd: [8279]: info: register_with_ha: Hostname: defas1
Feb 5 10:48:28 localhost cib: [8275]: info: cib_null_callback: Setting cib_refresh_notify callbacks for crmd: on
Feb 5 10:48:28 localhost cib: [8275]: info: cib_null_callback: Setting cib_diff_notify callbacks for mgmtd: on
Feb 5 10:48:28 localhost cib: [8275]: info: cib_client_status_callback: Status update: Client defas1/cib now has status [online]
Feb 5 10:48:28 localhost cib: [8281]: info: write_cib_contents: Wrote version 0.1.1 of the CIB to disk (digest: 5add9b53f9d640959765169f2836fa77)
Feb 5 10:48:28 localhost crmd: [8279]: info: register_with_ha: UUID: 6ca8e370-6b00-4b8c-85fe-a67da4c562ba
Feb 5 10:48:28 localhost cib: [8275]: info: cib_client_status_callback: Status update: Client defas2/cib now has status [offline]
Feb 5 10:48:28 localhost cib: [8275]: info: cib_client_status_callback: Status update: Client defas2/cib now has status [join]
Feb 5 10:48:28 localhost crmd: [8279]: info: populate_cib_nodes: Requesting the list of configured nodes
Feb 5 10:48:29 localhost heartbeat: [8264]: WARN: 1 lost packet(s) for [defas2] [14:16]
Feb 5 10:48:29 localhost heartbeat: [8264]: info: No pkts missing from defas2!
Feb 5 10:48:30 localhost heartbeat: [8264]: WARN: 1 lost packet(s) for [defas2] [18:20]
Feb 5 10:48:30 localhost crmd: [8279]: notice: populate_cib_nodes: Node: defas2 (uuid: 8d083aa4-14cc-4cca-9a8c-e7308fb2743c)
Feb 5 10:48:30 localhost heartbeat: [8264]: info: No pkts missing from defas2!
Feb 5 10:48:30 localhost crmd: [8279]: notice: populate_cib_nodes: Node: defas1 (uuid: 6ca8e370-6b00-4b8c-85fe-a67da4c562ba)
Feb 5 10:48:30 localhost crmd: [8279]: info: do_ha_control: Connected to Heartbeat
Feb 5 10:48:30 localhost crmd: [8279]: info: do_ccm_control: CCM connection established... waiting for first callback
Feb 5 10:48:30 localhost crmd: [8279]: info: do_started: Delaying start, CCM (0000000000100000) not connected
Feb 5 10:48:30 localhost crmd: [8279]: info: crmd_init: Starting crmd's mainloop
Feb 5 10:48:30 localhost crmd: [8279]: notice: crmd_client_status_callback: Status update: Client defas1/crmd now has status [online]
Feb 5 10:48:31 localhost crmd: [8279]: notice: crmd_client_status_callback: Status update: Client defas1/crmd now has status [online]
Feb 5 10:48:31 localhost crmd: [8279]: notice: crmd_client_status_callback: Status update: Client defas2/crmd now has status [online]
Feb 5 10:48:31 localhost crmd: [8279]: notice: crmd_client_status_callback: Status update: Client defas2/crmd now has status [online]
Feb 5 10:48:32 localhost crmd: [8279]: info: do_started: Delaying start, CCM (0000000000100000) not connected
Feb 5 10:48:32 localhost crmd: [8279]: notice: cluster_option: Using default value '10s' for cluster option 'dc_deadtime'
Feb 5 10:48:32 localhost crmd: [8279]: notice: cluster_option: Using default value '0' for cluster option 'cluster_recheck_interval'
Feb 5 10:48:32 localhost crmd: [8279]: notice: cluster_option: Using default value '2min' for cluster option 'election_timeout'
Feb 5 10:48:32 localhost crmd: [8279]: notice: cluster_option: Using default value '20min' for cluster option 'shutdown_escalation'
Feb 5 10:48:32 localhost crmd: [8279]: notice: cluster_option: Using default value '3min' for cluster option 'crmd-integration-timeout'
Feb 5 10:48:32 localhost crmd: [8279]: notice: cluster_option: Using default value '10min' for cluster option 'crmd-finalization-timeout'
Feb 5 10:48:32 localhost crmd: [8279]: info: do_started: Delaying start, CCM (0000000000100000) not connected
Feb 5 10:48:32 localhost attrd: [8278]: info: main: Starting mainloop...
Feb 5 10:48:34 localhost crmd: [8279]: info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm
Feb 5 10:48:34 localhost crmd: [8279]: info: mem_handle_event: instance=2, nodes=2, new=2, lost=0, n_idx=0, new_idx=0, old_idx=4
Feb 5 10:48:34 localhost crmd: [8279]: info: crmd_ccm_msg_callback: Quorum (re)attained after event=NEW MEMBERSHIP (id=2)
Feb 5 10:48:34 localhost crmd: [8279]: info: ccm_event_detail: NEW MEMBERSHIP: trans=2, nodes=2, new=2, lost=0 n_idx=0, new_idx=0, old_idx=4
Feb 5 10:48:34 localhost crmd: [8279]: info: ccm_event_detail: CURRENT: defas2 [nodeid=1, born=1]
Feb 5 10:48:34 localhost crmd: [8279]: info: ccm_event_detail: CURRENT: defas1 [nodeid=0, born=2]
Feb 5 10:48:34 localhost crmd: [8279]: info: ccm_event_detail: NEW: defas2 [nodeid=1, born=1]
Feb 5 10:48:34 localhost crmd: [8279]: info: ccm_event_detail: NEW: defas1 [nodeid=0, born=2]
Feb 5 10:48:34 localhost crmd: [8279]: info: do_started: The local CRM is operational
Feb 5 10:48:34 localhost crmd: [8279]: info: do_state_transition: State transition S_STARTING -> S_PENDING [ input=I_PENDING cause=C_CCM_CALLBACK origin=do_started ]
Feb 5 10:48:34 localhost crmd: [8279]: info: update_dc: Set DC to <null> (<null>)
Feb 5 10:48:34 localhost cib: [8275]: info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm
Feb 5 10:48:34 localhost cib: [8275]: info: mem_handle_event: instance=2, nodes=2, new=2, lost=0, n_idx=0, new_idx=0, old_idx=4
Feb 5 10:48:34 localhost cib: [8275]: info: cib_ccm_msg_callback: PEER: defas2
Feb 5 10:48:34 localhost cib: [8275]: info: cib_ccm_msg_callback: PEER: defas1
Feb 5 10:48:35 localhost mgmtd: [8280]: info: Started.
Feb 5 10:49:18 localhost crmd: [8279]: info: do_election_count_vote: Election check: vote from defas2
Feb 5 10:49:18 localhost crmd: [8279]: info: update_dc: Set DC to <null> (<null>)
Feb 5 10:49:20 localhost crmd: [8279]: info: crm_timer_popped: Election Trigger (I_DC_TIMEOUT) just popped!
Feb 5 10:49:20 localhost crmd: [8279]: WARN: do_log: [[FSA]] Input I_DC_TIMEOUT from crm_timer_popped() received in state (S_PENDING)
Feb 5 10:49:20 localhost crmd: [8279]: info: do_state_transition: State transition S_PENDING -> S_ELECTION [ input=I_DC_TIMEOUT cause=C_TIMER_POPPED origin=crm_timer_popped ]
Feb 5 10:49:20 localhost crmd: [8279]: info: update_dc: Set DC to <null> (<null>)
Feb 5 10:49:20 localhost crmd: [8279]: info: do_election_count_vote: Updated voted hash for defas1 to vote
Feb 5 10:49:20 localhost crmd: [8279]: info: do_election_count_vote: Election ignore: our vote (defas1)
Feb 5 10:49:20 localhost crmd: [8279]: info: do_election_check: Still waiting on 1 non-votes (2 total)
Feb 5 10:49:21 localhost crmd: [8279]: WARN: do_log: [[FSA]] Input I_JOIN_OFFER from route_message() received in state (S_ELECTION)
Feb 5 10:49:21 localhost cib: [8275]: info: cib_process_diff: Diff 0.1.2 -> 0.1.3 not applied to 0.1.1: current "num_updates" is less than required
Feb 5 10:49:21 localhost cib: [8275]: info: cib_process_diff: Requesting re-sync from peer: current "num_updates" is less than required
Feb 5 10:49:21 localhost cib: [8275]: WARN: do_cib_notify: cib_apply_diff of <diff > FAILED: Application of an update diff failed, requesting a full refresh
Feb 5 10:49:21 localhost cib: [8275]: WARN: cib_process_request: cib_apply_diff operation failed: Application of an update diff failed, requesting a full refresh
Feb 5 10:49:22 localhost crmd: [8279]: info: do_election_count_vote: Election check: vote from defas2
Feb 5 10:49:22 localhost crmd: [8279]: info: update_dc: Set DC to <null> (<null>)
Feb 5 10:49:22 localhost crmd: [8279]: info: do_election_check: Still waiting on 1 non-votes (2 total)
Feb 5 10:49:22 localhost crmd: [8279]: info: do_state_transition: State transition S_ELECTION -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Feb 5 10:49:22 localhost crmd: [8279]: info: update_dc: Set DC to <null> (<null>)
Feb 5 10:49:22 localhost crmd: [8279]: info: do_dc_release: DC role released
Feb 5 10:49:25 localhost crmd: [8279]: info: update_dc: Set DC to defas2 (1.0.9)
Feb 5 10:49:25 localhost cib: [8275]: info: cib_replace_notify: Replaced: 0.1.1 -> 0.1.3 from <null>
Feb 5 10:49:25 localhost crmd: [8279]: info: populate_cib_nodes: Requesting the list of configured nodes
Feb 5 10:49:25 localhost crmd: [8279]: notice: populate_cib_nodes: Node: defas2 (uuid: 8d083aa4-14cc-4cca-9a8c-e7308fb2743c)
Feb 5 10:49:26 localhost crmd: [8279]: notice: populate_cib_nodes: Node: defas1 (uuid: 6ca8e370-6b00-4b8c-85fe-a67da4c562ba)
Feb 5 10:49:29 localhost crmd: [8279]: info: update_dc: Set DC to defas2 (1.0.9)
Feb 5 10:49:29 localhost crmd: [8279]: info: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Feb 5 10:49:33 localhost crmd: [8279]: info: do_lrm_rsc_op: Performing op=ldirectord_1_monitor_0 key=7:0:acb11936-6402-42e7-8834-d94a3c9a86fc)
Feb 5 10:49:33 localhost crmd: [8279]: info: do_lrm_rsc_op: Performing op=LVSSyncDaemonSwap_2_monitor_0 key=8:0:acb11936-6402-42e7-8834-d94a3c9a86fc)
Feb 5 10:49:33 localhost crmd: [8279]: info: process_lrm_event: LRM operation LVSSyncDaemonSwap_2_monitor_0 (call=3, rc=7) complete
Feb 5 10:49:34 localhost crmd: [8279]: info: do_lrm_rsc_op: Performing op=IPaddr_eth_monitor_0 key=9:0:acb11936-6402-42e7-8834-d94a3c9a86fc)
Feb 5 10:49:34 localhost lrmd: [8276]: info: RA output: (ldirectord_1:monitor:stderr) ldirectord for /etc/ha.d/ldirectord.cf is running with pid: 16879
Feb 5 10:49:34 localhost lrmd: [8276]: WARN: There is something wrong: the first line isn't read in. Maybe the heartbeat does not ouput string correctly for status operation. Or the code (myself) is wrong.
Feb 5 10:49:34 localhost crmd: [8279]: info: process_lrm_event: LRM operation ldirectord_1_monitor_0 (call=2, rc=7) complete
Feb 5 10:49:34 localhost crmd: [8279]: info: process_lrm_event: LRM operation IPaddr_eth_monitor_0 (call=4, rc=7) complete
Feb 5 10:49:41 localhost crmd: [8279]: info: do_lrm_rsc_op: Performing op=ldirectord_1_start_0 key=4:1:acb11936-6402-42e7-8834-d94a3c9a86fc)
Feb 5 10:49:42 localhost crmd: [8279]: info: process_lrm_event: LRM operation ldirectord_1_start_0 (call=5, rc=0) complete
Feb 5 10:49:42 localhost crmd: [8279]: info: build_operation_update: Digest for 0:0;4:1:acb11936-6402-42e7-8834-d94a3c9a86fc (ldirectord_1_start_0) was 9369160ba28ec8f79eb02de50b45f82d
Feb 5 10:49:42 localhost crmd: [8279]: info: log_data_element: build_operation_update: digest:source <parameters 1="ldirectord.cf"/>
Feb 5 10:49:46 localhost crmd: [8279]: info: do_lrm_rsc_op: Performing op=ldirectord_1_start_0 key=5:2:acb11936-6402-42e7-8834-d94a3c9a86fc)
Feb 5 10:49:47 localhost crmd: [8279]: info: process_lrm_event: LRM operation ldirectord_1_start_0 (call=6, rc=0) complete
Feb 5 10:49:47 localhost crmd: [8279]: info: build_operation_update: Digest for 0:0;5:2:acb11936-6402-42e7-8834-d94a3c9a86fc (ldirectord_1_start_0) was 9369160ba28ec8f79eb02de50b45f82d
Feb 5 10:49:47 localhost crmd: [8279]: info: log_data_element: build_operation_update: digest:source <parameters 1="ldirectord.cf"/>
Feb 5 10:49:51 localhost crmd: [8279]: info: do_lrm_rsc_op: Performing op=ldirectord_1_monitor_120000 key=6:2:acb11936-6402-42e7-8834-d94a3c9a86fc)
Feb 5 10:49:51 localhost crmd: [8279]: info: do_lrm_rsc_op: Performing op=LVSSyncDaemonSwap_2_start_0 key=7:2:acb11936-6402-42e7-8834-d94a3c9a86fc)
Feb 5 10:49:51 localhost kernel: IPVS: set_ctl: len 92 != 24
Feb 5 10:49:51 localhost lrmd: [8276]: info: RA output: (LVSSyncDaemonSwap_2:start:stderr) Module is wrong version
Feb 5 10:49:51 localhost lrmd: [8276]: info: RA output: (LVSSyncDaemonSwap_2:start:stdout) ERROR: ipvsadm --start-daemon master --mcast-interface=eth0 failed.
Feb 5 10:49:51 localhost crmd: [8279]: ERROR: process_lrm_event: LRM operation LVSSyncDaemonSwap_2_start_0 (call=8, rc=255) Error <unknown>
Feb 5 10:49:51 localhost crmd: [8279]: info: build_operation_update: Digest for 4:255;7:2:acb11936-6402-42e7-8834-d94a3c9a86fc (LVSSyncDaemonSwap_2_start_0) was 71cb1bc7f37b5a41e950dbe46794a64f
Feb 5 10:49:51 localhost crmd: [8279]: info: log_data_element: build_operation_update: digest:source <parameters 1="master"/>
Feb 5 10:49:52 localhost lrmd: [8276]: info: RA output: (ldirectord_1:monitor:stderr) ldirectord for /etc/ha.d/ldirectord.cf is running with pid: 16879
Feb 5 10:49:52 localhost lrmd: [8276]: WARN: There is something wrong: the first line isn't read in. Maybe the heartbeat does not ouput string correctly for status operation. Or the code (myself) is wrong.
Feb 5 10:49:52 localhost crmd: [8279]: info: process_lrm_event: LRM operation ldirectord_1_monitor_120000 (call=7, rc=7) complete
Feb 5 10:49:57 localhost crmd: [8279]: info: do_lrm_rsc_op: Performing op=LVSSyncDaemonSwap_2_stop_0 key=1:3:acb11936-6402-42e7-8834-d94a3c9a86fc)
Feb 5 10:49:57 localhost kernel: IPVS: set_ctl: len 92 != 24
Feb 5 10:49:57 localhost lrmd: [8276]: info: RA output: (LVSSyncDaemonSwap_2:stop:stderr) Module is wrong version
Feb 5 10:49:57 localhost lrmd: [8276]: info: RA output: (LVSSyncDaemonSwap_2:stop:stdout) ERROR: ipvsadm --start-daemon backup --mcast-interface=eth0 failed.
Feb 5 10:49:57 localhost crmd: [8279]: ERROR: process_lrm_event: LRM operation LVSSyncDaemonSwap_2_stop_0 (call=9, rc=255) Error <unknown>
Feb 5 10:51:53 localhost lrmd: [8276]: info: RA output: (ldirectord_1:monitor:stderr) ldirectord for /etc/ha.d/ldirectord.cf is running with pid: 16879
Feb 5 10:51:53 localhost lrmd: [8276]: WARN: There is something wrong: the first line isn't read in. Maybe the heartbeat does not ouput string correctly for status operation. Or the code (myself) is wrong.
-------------- next part --------------
Feb 5 10:48:22 localhost ccm: [14696]: WARN: Initializing connection to logging daemon failed. Logging daemon may not be running
Feb 5 10:48:22 localhost stonithd: [14699]: WARN: Initializing connection to logging daemon failed. Logging daemon may not be running
Feb 5 10:48:22 localhost stonithd: [14699]: info: G_main_add_SignalHandler: Added signal handler for signal 10
Feb 5 10:48:22 localhost stonithd: [14699]: info: G_main_add_SignalHandler: Added signal handler for signal 12
Feb 5 10:48:22 localhost attrd: [14700]: WARN: Initializing connection to logging daemon failed. Logging daemon may not be running
Feb 5 10:48:22 localhost attrd: [14700]: info: G_main_add_SignalHandler: Added signal handler for signal 15
Feb 5 10:48:22 localhost ccm: [14696]: info: Hostname: defas2
Feb 5 10:48:22 localhost attrd: [14700]: info: register_with_ha: Hostname: defas2
Feb 5 10:48:22 localhost stonithd: [14699]: info: Signing in with heartbeat.
Feb 5 10:48:22 localhost attrd: [14700]: info: register_with_ha: UUID: 8d083aa4-14cc-4cca-9a8c-e7308fb2743c
Feb 5 10:48:22 localhost cib: [14697]: WARN: Initializing connection to logging daemon failed. Logging daemon may not be running
Feb 5 10:48:22 localhost cib: [14697]: info: G_main_add_SignalHandler: Added signal handler for signal 15
Feb 5 10:48:22 localhost cib: [14697]: info: G_main_add_TriggerHandler: Added signal manual handler
Feb 5 10:48:22 localhost cib: [14697]: info: G_main_add_SignalHandler: Added signal handler for signal 17
Feb 5 10:48:22 localhost cib: [14697]: info: main: Retrieval of a per-action CIB: disabled
Feb 5 10:48:22 localhost cib: [14697]: info: readCibXmlFile: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml
Feb 5 10:48:22 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <cib admin_epoch="0" epoch="1" have_quorum="false" ignore_dtd="false" num_peers="0" cib_feature_revision="1.3" generated="false" num_updates="2" cib-last-written="Tue Feb 5 10:36:15 2008">
Feb 5 10:48:22 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <configuration>
Feb 5 10:48:22 localhost lrmd: [14698]: WARN: Initializing connection to logging daemon failed. Logging daemon may not be running
Feb 5 10:48:22 localhost crmd: [14701]: WARN: Initializing connection to logging daemon failed. Logging daemon may not be running
Feb 5 10:48:22 localhost mgmtd: [14702]: WARN: Initializing connection to logging daemon failed. Logging daemon may not be running
Feb 5 10:48:22 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <crm_config>
Feb 5 10:48:22 localhost lrmd: [14698]: info: G_main_add_SignalHandler: Added signal handler for signal 15
Feb 5 10:48:22 localhost crmd: [14701]: info: main: CRM Hg Version: 9a995a8a7052 tip
Feb 5 10:48:22 localhost mgmtd: [14702]: info: G_main_add_SignalHandler: Added signal handler for signal 15
Feb 5 10:48:22 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <cluster_property_set id="cib-bootstrap-options">
Feb 5 10:48:22 localhost lrmd: [14698]: info: G_main_add_SignalHandler: Added signal handler for signal 17
Feb 5 10:48:22 localhost crmd: [14701]: info: crmd_init: Starting crmd
Feb 5 10:48:22 localhost stonithd: [14699]: notice: /usr/lib/heartbeat/stonithd start up successfully.
Feb 5 10:48:23 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <attributes>
Feb 5 10:48:23 localhost lrmd: [14698]: info: G_main_add_SignalHandler: Added signal handler for signal 10
Feb 5 10:48:23 localhost crmd: [14701]: info: G_main_add_SignalHandler: Added signal handler for signal 15
Feb 5 10:48:23 localhost stonithd: [14699]: info: G_main_add_SignalHandler: Added signal handler for signal 17
Feb 5 10:48:23 localhost mgmtd: [14702]: info: G_main_add_SignalHandler: Added signal handler for signal 10
Feb 5 10:48:23 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-symmetric-cluster" name="symmetric-cluster" value="true"/>
Feb 5 10:48:23 localhost lrmd: [14698]: info: G_main_add_SignalHandler: Added signal handler for signal 12
Feb 5 10:48:23 localhost crmd: [14701]: info: G_main_add_TriggerHandler: Added signal manual handler
Feb 5 10:48:23 localhost mgmtd: [14702]: info: G_main_add_SignalHandler: Added signal handler for signal 12
Feb 5 10:48:23 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-no-quorum-policy" name="no-quorum-policy" value="stop"/>
Feb 5 10:48:23 localhost lrmd: [14698]: info: Started.
Feb 5 10:48:23 localhost crmd: [14701]: info: G_main_add_SignalHandler: Added signal handler for signal 17
Feb 5 10:48:23 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-default-resource-stickiness" name="default-resource-stickiness" value="0"/>
Feb 5 10:48:23 localhost mgmtd: [14702]: info: init_crm
Feb 5 10:48:23 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-default-resource-failure-stickiness" name="default-resource-failure-stickiness" value="0"/>
Feb 5 10:48:23 localhost mgmtd: [14702]: info: login to cib: 0, ret:-10
Feb 5 10:48:23 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" value="false"/>
Feb 5 10:48:23 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-stonith-action" name="stonith-action" value="reboot"/>
Feb 5 10:48:23 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-stop-orphan-resources" name="stop-orphan-resources" value="true"/>
Feb 5 10:48:23 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-stop-orphan-actions" name="stop-orphan-actions" value="true"/>
Feb 5 10:48:23 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-remove-after-stop" name="remove-after-stop" value="false"/>
Feb 5 10:48:23 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-short-resource-names" name="short-resource-names" value="true"/>
Feb 5 10:48:23 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-transition-idle-timeout" name="transition-idle-timeout" value="5min"/>
Feb 5 10:48:23 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-default-action-timeout" name="default-action-timeout" value="15s"/>
Feb 5 10:48:23 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="cib-bootstrap-options-is-managed-default" name="is-managed-default" value="true"/>
Feb 5 10:48:23 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </attributes>
Feb 5 10:48:23 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </cluster_property_set>
Feb 5 10:48:23 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </crm_config>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <nodes>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <node id="8d083aa4-14cc-4cca-9a8c-e7308fb2743c" uname="defas2" type="normal"/>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <node id="6ca8e370-6b00-4b8c-85fe-a67da4c562ba" uname="defas1" type="normal"/>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </nodes>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <resources>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <group id="group_1">
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <primitive class="heartbeat" id="ldirectord_1" provider="heartbeat" type="ldirectord">
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <operations>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <op id="ldirectord_1_mon" interval="120s" name="monitor" timeout="60s"/>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </operations>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <instance_attributes id="ldirectord_1_inst_attr">
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <attributes>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="ldirectord_1_attr_1" name="1" value="ldirectord.cf"/>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </attributes>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </instance_attributes>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </primitive>
Feb 5 10:48:24 localhost crmd: [14701]: WARN: do_cib_control: Couldn't complete CIB registration 1 times... pause and retry
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <primitive class="heartbeat" id="LVSSyncDaemonSwap_2" provider="heartbeat" type="LVSSyncDaemonSwap">
Feb 5 10:48:24 localhost crmd: [14701]: info: crmd_init: Starting crmd's mainloop
Feb 5 10:48:24 localhost mgmtd: [14702]: info: login to cib: 1, ret:-10
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <operations>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <op id="LVSSyncDaemonSwap_2_mon" interval="120s" name="monitor" timeout="60s"/>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </operations>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <instance_attributes id="LVSSyncDaemonSwap_2_inst_attr">
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <attributes>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="LVSSyncDaemonSwap_2_attr_1" name="1" value="master"/>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </attributes>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </instance_attributes>
Feb 5 10:48:24 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </primitive>
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <primitive class="ocf" id="IPaddr_eth" provider="heartbeat" type="IPaddr">
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <operations>
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <op id="IPaddr_eth_mon" interval="5s" name="monitor" timeout="5s"/>
Feb 5 10:48:25 localhost crmd: [14701]: info: crm_timer_popped: Wait Timer (I_NULL) just popped!
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </operations>
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <instance_attributes id="IPaddr_eth_inst_attr">
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <attributes>
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="IPaddr_eth_attr_0" name="ip" value="172.30.4.170"/>
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="IPaddr_eth_attr_1" name="netmask" value="16"/>
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <nvpair id="IPaddr_eth_attr_2" name="nic" value="eth0"/>
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </attributes>
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </instance_attributes>
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </primitive>
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </group>
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </resources>
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <constraints>
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <rsc_location id="rsc_location_group_1" rsc="group_1">
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <rule id="prefered_location_group_1" score="100">
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <expression attribute="#uname" id="prefered_location_group_1_expr" operation="eq" value="defas1"/>
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </rule>
Feb 5 10:48:25 localhost mgmtd: [14702]: info: login to cib: 2, ret:-10
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </rsc_location>
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </constraints>
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </configuration>
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] <status/>
Feb 5 10:48:25 localhost cib: [14697]: info: log_data_element: readCibXmlFile: [on-disk] </cib>
Feb 5 10:48:25 localhost cib: [14697]: info: startCib: CIB Initialization completed successfully
Feb 5 10:48:25 localhost cib: [14697]: info: cib_register_ha: Signing in with Heartbeat
Feb 5 10:48:26 localhost ccm: [14696]: info: G_main_add_SignalHandler: Added signal handler for signal 15
Feb 5 10:48:26 localhost cib: [14697]: info: cib_register_ha: FSA Hostname: defas2
Feb 5 10:48:26 localhost cib: [14697]: info: cib_init: Starting cib mainloop
Feb 5 10:48:26 localhost crmd: [14701]: info: do_cib_control: CIB connection established
Feb 5 10:48:26 localhost crmd: [14701]: info: register_with_ha: Hostname: defas2
Feb 5 10:48:26 localhost cib: [14697]: info: cib_client_status_callback: Status update: Client defas2/cib now has status [join]
Feb 5 10:48:26 localhost cib: [14697]: info: cib_client_status_callback: Status update: Client defas2/cib now has status [online]
Feb 5 10:48:26 localhost cib: [14697]: info: cib_null_callback: Setting cib_refresh_notify callbacks for crmd: on
Feb 5 10:48:26 localhost cib: [14703]: info: write_cib_contents: Wrote version 0.1.2 of the CIB to disk (digest: b7c728c1433a9aae799a908e9e81c54e)
Feb 5 10:48:26 localhost cib: [14697]: info: cib_null_callback: Setting cib_diff_notify callbacks for mgmtd: on
Feb 5 10:48:27 localhost crmd: [14701]: info: register_with_ha: UUID: 8d083aa4-14cc-4cca-9a8c-e7308fb2743c
Feb 5 10:48:27 localhost cib: [14697]: info: cib_client_status_callback: Status update: Client defas1/cib now has status [online]
Feb 5 10:48:27 localhost crmd: [14701]: info: populate_cib_nodes: Requesting the list of configured nodes
Feb 5 10:48:28 localhost crmd: [14701]: notice: populate_cib_nodes: Node: defas2 (uuid: 8d083aa4-14cc-4cca-9a8c-e7308fb2743c)
Feb 5 10:48:28 localhost crmd: [14701]: notice: populate_cib_nodes: Node: defas1 (uuid: 6ca8e370-6b00-4b8c-85fe-a67da4c562ba)
Feb 5 10:48:28 localhost crmd: [14701]: info: do_ha_control: Connected to Heartbeat
Feb 5 10:48:28 localhost crmd: [14701]: info: do_ccm_control: CCM connection established... waiting for first callback
Feb 5 10:48:28 localhost crmd: [14701]: info: do_started: Delaying start, CCM (0000000000100000) not connected
Feb 5 10:48:28 localhost crmd: [14701]: notice: crmd_client_status_callback: Status update: Client defas2/crmd now has status [online]
Feb 5 10:48:29 localhost crmd: [14701]: notice: crmd_client_status_callback: Status update: Client defas2/crmd now has status [online]
Feb 5 10:48:29 localhost ccm: [14696]: info: Break tie for 2 nodes cluster
Feb 5 10:48:29 localhost cib: [14697]: info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm
Feb 5 10:48:29 localhost cib: [14697]: info: mem_handle_event: instance=1, nodes=1, new=1, lost=0, n_idx=0, new_idx=0, old_idx=3
Feb 5 10:48:29 localhost cib: [14697]: info: cib_ccm_msg_callback: PEER: defas2
Feb 5 10:48:29 localhost crmd: [14701]: notice: crmd_client_status_callback: Status update: Client defas1/crmd now has status [online]
Feb 5 10:48:29 localhost crmd: [14701]: info: do_started: Delaying start, CCM (0000000000100000) not connected
Feb 5 10:48:29 localhost crmd: [14701]: notice: cluster_option: Using default value '10s' for cluster option 'dc_deadtime'
Feb 5 10:48:29 localhost crmd: [14701]: notice: cluster_option: Using default value '0' for cluster option 'cluster_recheck_interval'
Feb 5 10:48:29 localhost crmd: [14701]: notice: cluster_option: Using default value '2min' for cluster option 'election_timeout'
Feb 5 10:48:29 localhost crmd: [14701]: notice: cluster_option: Using default value '20min' for cluster option 'shutdown_escalation'
Feb 5 10:48:29 localhost crmd: [14701]: notice: cluster_option: Using default value '3min' for cluster option 'crmd-integration-timeout'
Feb 5 10:48:29 localhost crmd: [14701]: notice: cluster_option: Using default value '10min' for cluster option 'crmd-finalization-timeout'
Feb 5 10:48:29 localhost crmd: [14701]: info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm
Feb 5 10:48:29 localhost crmd: [14701]: info: mem_handle_event: instance=1, nodes=1, new=1, lost=0, n_idx=0, new_idx=0, old_idx=3
Feb 5 10:48:29 localhost crmd: [14701]: info: crmd_ccm_msg_callback: Quorum (re)attained after event=NEW MEMBERSHIP (id=1)
Feb 5 10:48:29 localhost crmd: [14701]: info: ccm_event_detail: NEW MEMBERSHIP: trans=1, nodes=1, new=1, lost=0 n_idx=0, new_idx=0, old_idx=3
Feb 5 10:48:29 localhost crmd: [14701]: info: ccm_event_detail: CURRENT: defas2 [nodeid=1, born=1]
Feb 5 10:48:29 localhost crmd: [14701]: info: ccm_event_detail: NEW: defas2 [nodeid=1, born=1]
Feb 5 10:48:29 localhost crmd: [14701]: info: do_started: The local CRM is operational
Feb 5 10:48:29 localhost crmd: [14701]: info: do_state_transition: State transition S_STARTING -> S_PENDING [ input=I_PENDING cause=C_CCM_CALLBACK origin=do_started ]
Feb 5 10:48:29 localhost crmd: [14701]: info: update_dc: Set DC to <null> (<null>)
Feb 5 10:48:31 localhost cib: [14697]: info: mem_handle_event: Got an event OC_EV_MS_INVALID from ccm
Feb 5 10:48:31 localhost cib: [14697]: info: mem_handle_event: no mbr_track info
Feb 5 10:48:31 localhost crmd: [14701]: info: mem_handle_event: Got an event OC_EV_MS_INVALID from ccm
Feb 5 10:48:31 localhost crmd: [14701]: info: mem_handle_event: no mbr_track info
Feb 5 10:48:31 localhost cib: [14697]: info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm
Feb 5 10:48:31 localhost cib: [14697]: info: mem_handle_event: instance=2, nodes=2, new=1, lost=0, n_idx=0, new_idx=2, old_idx=4
Feb 5 10:48:31 localhost cib: [14697]: info: cib_ccm_msg_callback: PEER: defas2
Feb 5 10:48:31 localhost cib: [14697]: info: cib_ccm_msg_callback: PEER: defas1
Feb 5 10:48:31 localhost crmd: [14701]: info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm
Feb 5 10:48:31 localhost crmd: [14701]: info: mem_handle_event: instance=2, nodes=2, new=1, lost=0, n_idx=0, new_idx=2, old_idx=4
Feb 5 10:48:31 localhost crmd: [14701]: info: crmd_ccm_msg_callback: Quorum (re)attained after event=NEW MEMBERSHIP (id=2)
Feb 5 10:48:31 localhost crmd: [14701]: info: ccm_event_detail: NEW MEMBERSHIP: trans=2, nodes=2, new=1, lost=0 n_idx=0, new_idx=2, old_idx=4
Feb 5 10:48:31 localhost crmd: [14701]: info: ccm_event_detail: CURRENT: defas2 [nodeid=1, born=1]
Feb 5 10:48:31 localhost crmd: [14701]: info: ccm_event_detail: CURRENT: defas1 [nodeid=0, born=2]
Feb 5 10:48:31 localhost crmd: [14701]: info: ccm_event_detail: NEW: defas1 [nodeid=0, born=2]
Feb 5 10:48:32 localhost attrd: [14700]: info: main: Starting mainloop...
Feb 5 10:48:35 localhost mgmtd: [14702]: info: Started.
Feb 5 10:49:15 localhost crmd: [14701]: info: crm_timer_popped: Election Trigger (I_DC_TIMEOUT) just popped!
Feb 5 10:49:15 localhost crmd: [14701]: WARN: do_log: [[FSA]] Input I_DC_TIMEOUT from crm_timer_popped() received in state (S_PENDING)
Feb 5 10:49:15 localhost crmd: [14701]: info: do_state_transition: State transition S_PENDING -> S_ELECTION [ input=I_DC_TIMEOUT cause=C_TIMER_POPPED origin=crm_timer_popped ]
Feb 5 10:49:15 localhost crmd: [14701]: info: update_dc: Set DC to <null> (<null>)
Feb 5 10:49:16 localhost crmd: [14701]: info: do_election_count_vote: Updated voted hash for defas2 to vote
Feb 5 10:49:16 localhost crmd: [14701]: info: do_election_count_vote: Election ignore: our vote (defas2)
Feb 5 10:49:16 localhost crmd: [14701]: info: do_election_check: Still waiting on 1 non-votes (2 total)
Feb 5 10:49:17 localhost crmd: [14701]: info: do_election_count_vote: Updated voted hash for defas1 to no-vote
Feb 5 10:49:17 localhost crmd: [14701]: info: do_election_count_vote: Election ignore: no-vote from defas1
Feb 5 10:49:17 localhost crmd: [14701]: info: do_state_transition: State transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_election_check ]
Feb 5 10:49:17 localhost crmd: [14701]: info: start_subsystem: Starting sub-system "tengine"
Feb 5 10:49:17 localhost crmd: [14701]: info: start_subsystem: Starting sub-system "pengine"
Feb 5 10:49:17 localhost crmd: [14701]: info: do_dc_takeover: Taking over DC status for this partition
Feb 5 10:49:17 localhost crmd: [14701]: info: update_dc: Set DC to <null> (<null>)
Feb 5 10:49:17 localhost crmd: [14701]: info: join_make_offer: Making join offers based on membership 2
Feb 5 10:49:17 localhost crmd: [14701]: info: do_dc_join_offer_all: join-1: Waiting on 2 outstanding join acks
Feb 5 10:49:17 localhost cib: [14697]: info: cib_process_readwrite: We are now in R/W mode
Feb 5 10:49:17 localhost tengine: [14704]: WARN: Initializing connection to logging daemon failed. Logging daemon may not be running
Feb 5 10:49:17 localhost tengine: [14704]: info: G_main_add_SignalHandler: Added signal handler for signal 15
Feb 5 10:49:17 localhost tengine: [14704]: info: G_main_add_TriggerHandler: Added signal manual handler
Feb 5 10:49:17 localhost cib: [14697]: info: cib_null_callback: Setting cib_diff_notify callbacks for tengine: on
Feb 5 10:49:17 localhost tengine: [14704]: info: te_init: Registering TE UUID: acb11936-6402-42e7-8834-d94a3c9a86fc
Feb 5 10:49:17 localhost pengine: [14705]: WARN: Initializing connection to logging daemon failed. Logging daemon may not be running
Feb 5 10:49:17 localhost tengine: [14704]: info: set_graph_functions: Setting custom graph functions
Feb 5 10:49:17 localhost pengine: [14705]: info: G_main_add_SignalHandler: Added signal handler for signal 15
Feb 5 10:49:17 localhost tengine: [14704]: info: unpack_graph: Unpacked transition -1: 0 actions in 0 synapses
Feb 5 10:49:17 localhost pengine: [14705]: info: pe_init: Starting pengine
Feb 5 10:49:17 localhost tengine: [14704]: info: te_init: Starting tengine
Feb 5 10:49:18 localhost crmd: [14701]: info: update_dc: Set DC to defas2 (1.0.9)
Feb 5 10:49:18 localhost crmd: [14701]: info: do_election_count_vote: Election check: vote from defas1
Feb 5 10:49:18 localhost crmd: [14701]: info: do_election_count_vote: Election won over defas1
Feb 5 10:49:18 localhost crmd: [14701]: info: do_state_transition: State transition S_INTEGRATION -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Feb 5 10:49:18 localhost crmd: [14701]: info: update_dc: Set DC to <null> (<null>)
Feb 5 10:49:19 localhost crmd: [14701]: info: do_election_count_vote: Updated voted hash for defas2 to vote
Feb 5 10:49:19 localhost crmd: [14701]: info: do_election_count_vote: Election ignore: our vote (defas2)
Feb 5 10:49:19 localhost crmd: [14701]: info: do_election_check: Still waiting on 1 non-votes (2 total)
Feb 5 10:49:20 localhost cib: [14697]: info: sync_our_cib: Syncing CIB to defas1
Feb 5 10:49:20 localhost crmd: [14701]: info: do_election_count_vote: Updated voted hash for defas1 to no-vote
Feb 5 10:49:20 localhost crmd: [14701]: info: do_election_count_vote: Election ignore: no-vote from defas1
Feb 5 10:49:20 localhost crmd: [14701]: info: do_state_transition: State transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_election_check ]
Feb 5 10:49:20 localhost crmd: [14701]: info: start_subsystem: Starting sub-system "tengine"
Feb 5 10:49:20 localhost crmd: [14701]: WARN: start_subsystem: Client tengine already running as pid 14704
Feb 5 10:49:20 localhost crmd: [14701]: info: start_subsystem: Starting sub-system "pengine"
Feb 5 10:49:20 localhost crmd: [14701]: WARN: start_subsystem: Client pengine already running as pid 14705
Feb 5 10:49:20 localhost crmd: [14701]: info: do_dc_takeover: Taking over DC status for this partition
Feb 5 10:49:20 localhost cib: [14697]: info: cib_process_readwrite: We are now in R/O mode
Feb 5 10:49:20 localhost cib: [14697]: info: cib_process_readwrite: We are now in R/W mode
Feb 5 10:49:20 localhost crmd: [14701]: info: update_dc: Set DC to <null> (<null>)
Feb 5 10:49:20 localhost crmd: [14701]: info: do_dc_join_offer_all: join-2: Waiting on 2 outstanding join acks
Feb 5 10:49:21 localhost crmd: [14701]: info: update_dc: Set DC to defas2 (1.0.9)
Feb 5 10:49:24 localhost crmd: [14701]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ]
Feb 5 10:49:24 localhost crmd: [14701]: info: do_state_transition: All 2 cluster nodes responded to the join offer.
Feb 5 10:49:24 localhost cib: [14697]: info: sync_our_cib: Syncing CIB to all peers
Feb 5 10:49:24 localhost crmd: [14701]: info: update_attrd: Connecting to attrd...
Feb 5 10:49:24 localhost attrd: [14700]: info: attrd_local_callback: Sending full refresh
Feb 5 10:49:25 localhost crmd: [14701]: info: update_dc: Set DC to defas2 (1.0.9)
Feb 5 10:49:25 localhost crmd: [14701]: info: do_dc_join_ack: join-2: Updating node state to member for defas2
Feb 5 10:49:28 localhost crmd: [14701]: info: do_dc_join_ack: join-2: Updating node state to member for defas1
Feb 5 10:49:28 localhost crmd: [14701]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE [ input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state ]
Feb 5 10:49:28 localhost crmd: [14701]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Feb 5 10:49:28 localhost tengine: [14704]: info: update_abort_priority: Abort priority upgraded to 1000000
Feb 5 10:49:28 localhost tengine: [14704]: info: update_abort_priority: 'DC Takeover' abort superceeded
Feb 5 10:49:28 localhost pengine: [14705]: notice: cluster_option: Using default value '60s' for cluster option 'cluster-delay'
Feb 5 10:49:28 localhost pengine: [14705]: notice: cluster_option: Using default value '-1' for cluster option 'pe-error-series-max'
Feb 5 10:49:28 localhost pengine: [14705]: notice: cluster_option: Using default value '-1' for cluster option 'pe-warn-series-max'
Feb 5 10:49:28 localhost pengine: [14705]: notice: cluster_option: Using default value '-1' for cluster option 'pe-input-series-max'
Feb 5 10:49:28 localhost pengine: [14705]: notice: cluster_option: Using default value 'true' for cluster option 'startup-fencing'
Feb 5 10:49:28 localhost pengine: [14705]: info: determine_online_status: Node defas2 is online
Feb 5 10:49:28 localhost pengine: [14705]: info: determine_online_status: Node defas1 is online
Feb 5 10:49:28 localhost pengine: [14705]: info: group_print: Resource Group: group_1
Feb 5 10:49:28 localhost pengine: [14705]: info: native_print: ldirectord_1 (heartbeat:ldirectord): Stopped
Feb 5 10:49:28 localhost pengine: [14705]: info: native_print: LVSSyncDaemonSwap_2 (heartbeat:LVSSyncDaemonSwap): Stopped
Feb 5 10:49:28 localhost pengine: [14705]: info: native_print: IPaddr_eth (heartbeat::ocf:IPaddr): Stopped
Feb 5 10:49:28 localhost pengine: [14705]: notice: StartRsc: defas1 Start ldirectord_1
Feb 5 10:49:28 localhost pengine: [14705]: notice: RecurringOp: defas1 ldirectord_1_monitor_120000
Feb 5 10:49:28 localhost pengine: [14705]: notice: StartRsc: defas1 Start LVSSyncDaemonSwap_2
Feb 5 10:49:28 localhost pengine: [14705]: notice: RecurringOp: defas1 LVSSyncDaemonSwap_2_monitor_120000
Feb 5 10:49:28 localhost pengine: [14705]: notice: StartRsc: defas1 Start IPaddr_eth
Feb 5 10:49:28 localhost pengine: [14705]: notice: RecurringOp: defas1 IPaddr_eth_monitor_5000
Feb 5 10:49:29 localhost pengine: [14705]: info: process_pe_message: Transition 0: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-122.bz2
Feb 5 10:49:29 localhost crmd: [14701]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 5 10:49:29 localhost tengine: [14704]: info: unpack_graph: Unpacked transition 0: 17 actions in 17 synapses
Feb 5 10:49:29 localhost tengine: [14704]: info: send_rsc_command: Initiating action 3: ldirectord_1_monitor_0 on defas2
Feb 5 10:49:29 localhost tengine: [14704]: info: send_rsc_command: Initiating action 7: ldirectord_1_monitor_0 on defas1
Feb 5 10:49:29 localhost crmd: [14701]: info: do_lrm_rsc_op: Performing op=ldirectord_1_monitor_0 key=3:0:acb11936-6402-42e7-8834-d94a3c9a86fc)
Feb 5 10:49:29 localhost tengine: [14704]: info: send_rsc_command: Initiating action 4: LVSSyncDaemonSwap_2_monitor_0 on defas2
Feb 5 10:49:29 localhost tengine: [14704]: info: send_rsc_command: Initiating action 8: LVSSyncDaemonSwap_2_monitor_0 on defas1
Feb 5 10:49:29 localhost crmd: [14701]: info: do_lrm_rsc_op: Performing op=LVSSyncDaemonSwap_2_monitor_0 key=4:0:acb11936-6402-42e7-8834-d94a3c9a86fc)
Feb 5 10:49:29 localhost tengine: [14704]: info: send_rsc_command: Initiating action 5: IPaddr_eth_monitor_0 on defas2
Feb 5 10:49:29 localhost tengine: [14704]: info: send_rsc_command: Initiating action 9: IPaddr_eth_monitor_0 on defas1
Feb 5 10:49:29 localhost crmd: [14701]: info: do_lrm_rsc_op: Performing op=IPaddr_eth_monitor_0 key=5:0:acb11936-6402-42e7-8834-d94a3c9a86fc)
Feb 5 10:49:29 localhost crmd: [14701]: info: process_lrm_event: LRM operation LVSSyncDaemonSwap_2_monitor_0 (call=3, rc=7) complete
Feb 5 10:49:30 localhost tengine: [14704]: info: match_graph_event: Action LVSSyncDaemonSwap_2_monitor_0 (4) confirmed on defas2
Feb 5 10:49:30 localhost crmd: [14701]: info: process_lrm_event: LRM operation IPaddr_eth_monitor_0 (call=4, rc=7) complete
Feb 5 10:49:30 localhost tengine: [14704]: info: match_graph_event: Action IPaddr_eth_monitor_0 (5) confirmed on defas2
Feb 5 10:49:32 localhost tengine: [14704]: info: match_graph_event: Action LVSSyncDaemonSwap_2_monitor_0 (8) confirmed on defas1
Feb 5 10:49:32 localhost crmd: [14701]: info: process_lrm_event: LRM operation ldirectord_1_monitor_0 (call=2, rc=7) complete
Feb 5 10:49:33 localhost tengine: [14704]: info: match_graph_event: Action ldirectord_1_monitor_0 (3) confirmed on defas2
Feb 5 10:49:33 localhost tengine: [14704]: info: send_rsc_command: Initiating action 2: probe_complete on defas2
Feb 5 10:49:33 localhost tengine: [14704]: info: extract_event: Aborting on transient_attributes changes for 8d083aa4-14cc-4cca-9a8c-e7308fb2743c
Feb 5 10:49:33 localhost tengine: [14704]: info: update_abort_priority: Abort priority upgraded to 1000000
Feb 5 10:49:33 localhost tengine: [14704]: info: update_abort_priority: Abort action 0 superceeded by 2
Feb 5 10:49:33 localhost tengine: [14704]: info: match_graph_event: Action ldirectord_1_monitor_0 (7) confirmed on defas1
Feb 5 10:49:33 localhost tengine: [14704]: info: match_graph_event: Action IPaddr_eth_monitor_0 (9) confirmed on defas1
Feb 5 10:49:33 localhost tengine: [14704]: info: send_rsc_command: Initiating action 6: probe_complete on defas1
Feb 5 10:49:33 localhost tengine: [14704]: info: run_graph: ====================================================
Feb 5 10:49:33 localhost tengine: [14704]: notice: run_graph: Transition 0: (Complete=8, Pending=0, Fired=0, Skipped=9, Incomplete=0)
Feb 5 10:49:33 localhost crmd: [14701]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
Feb 5 10:49:33 localhost crmd: [14701]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Feb 5 10:49:33 localhost pengine: [14705]: notice: cluster_option: Using default value '60s' for cluster option 'cluster-delay'
Feb 5 10:49:33 localhost pengine: [14705]: notice: cluster_option: Using default value '-1' for cluster option 'pe-error-series-max'
Feb 5 10:49:33 localhost pengine: [14705]: notice: cluster_option: Using default value '-1' for cluster option 'pe-warn-series-max'
Feb 5 10:49:33 localhost pengine: [14705]: notice: cluster_option: Using default value '-1' for cluster option 'pe-input-series-max'
Feb 5 10:49:33 localhost pengine: [14705]: notice: cluster_option: Using default value 'true' for cluster option 'startup-fencing'
Feb 5 10:49:33 localhost pengine: [14705]: info: determine_online_status: Node defas2 is online
Feb 5 10:49:33 localhost pengine: [14705]: info: determine_online_status: Node defas1 is online
Feb 5 10:49:33 localhost pengine: [14705]: info: group_print: Resource Group: group_1
Feb 5 10:49:33 localhost pengine: [14705]: info: native_print: ldirectord_1 (heartbeat:ldirectord): Stopped
Feb 5 10:49:33 localhost pengine: [14705]: info: native_print: LVSSyncDaemonSwap_2 (heartbeat:LVSSyncDaemonSwap): Stopped
Feb 5 10:49:33 localhost pengine: [14705]: info: native_print: IPaddr_eth (heartbeat::ocf:IPaddr): Stopped
Feb 5 10:49:33 localhost pengine: [14705]: notice: StartRsc: defas1 Start ldirectord_1
Feb 5 10:49:33 localhost pengine: [14705]: notice: RecurringOp: defas1 ldirectord_1_monitor_120000
Feb 5 10:49:34 localhost pengine: [14705]: notice: StartRsc: defas1 Start LVSSyncDaemonSwap_2
Feb 5 10:49:34 localhost pengine: [14705]: notice: RecurringOp: defas1 LVSSyncDaemonSwap_2_monitor_120000
Feb 5 10:49:34 localhost pengine: [14705]: notice: StartRsc: defas1 Start IPaddr_eth
Feb 5 10:49:34 localhost pengine: [14705]: notice: RecurringOp: defas1 IPaddr_eth_monitor_5000
Feb 5 10:49:34 localhost crmd: [14701]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 5 10:49:34 localhost tengine: [14704]: info: unpack_graph: Unpacked transition 1: 9 actions in 9 synapses
Feb 5 10:49:34 localhost tengine: [14704]: info: te_pseudo_action: Pseudo action 10 fired and confirmed
Feb 5 10:49:34 localhost pengine: [14705]: info: process_pe_message: Transition 1: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-123.bz2
Feb 5 10:49:34 localhost tengine: [14704]: info: send_rsc_command: Initiating action 4: ldirectord_1_start_0 on defas1
Feb 5 10:49:34 localhost tengine: [14704]: info: send_rsc_command: Initiating action 3: probe_complete on defas1
Feb 5 10:49:40 localhost tengine: [14704]: info: extract_event: Aborting on transient_attributes changes for 6ca8e370-6b00-4b8c-85fe-a67da4c562ba
Feb 5 10:49:40 localhost tengine: [14704]: info: update_abort_priority: Abort priority upgraded to 1000000
Feb 5 10:49:40 localhost tengine: [14704]: info: update_abort_priority: Abort action 0 superceeded by 2
Feb 5 10:49:41 localhost tengine: [14704]: info: match_graph_event: Action ldirectord_1_start_0 (4) confirmed on defas1
Feb 5 10:49:41 localhost tengine: [14704]: info: run_graph: ====================================================
Feb 5 10:49:41 localhost tengine: [14704]: notice: run_graph: Transition 1: (Complete=3, Pending=0, Fired=0, Skipped=6, Incomplete=0)
Feb 5 10:49:41 localhost crmd: [14701]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
Feb 5 10:49:41 localhost crmd: [14701]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Feb 5 10:49:41 localhost pengine: [14705]: notice: cluster_option: Using default value '60s' for cluster option 'cluster-delay'
Feb 5 10:49:41 localhost pengine: [14705]: notice: cluster_option: Using default value '-1' for cluster option 'pe-error-series-max'
Feb 5 10:49:41 localhost pengine: [14705]: notice: cluster_option: Using default value '-1' for cluster option 'pe-warn-series-max'
Feb 5 10:49:41 localhost pengine: [14705]: notice: cluster_option: Using default value '-1' for cluster option 'pe-input-series-max'
Feb 5 10:49:41 localhost pengine: [14705]: notice: cluster_option: Using default value 'true' for cluster option 'startup-fencing'
Feb 5 10:49:41 localhost pengine: [14705]: info: determine_online_status: Node defas2 is online
Feb 5 10:49:41 localhost pengine: [14705]: info: determine_online_status: Node defas1 is online
Feb 5 10:49:41 localhost pengine: [14705]: info: group_print: Resource Group: group_1
Feb 5 10:49:41 localhost pengine: [14705]: info: native_print: ldirectord_1 (heartbeat:ldirectord): Started defas1
Feb 5 10:49:41 localhost pengine: [14705]: info: native_print: LVSSyncDaemonSwap_2 (heartbeat:LVSSyncDaemonSwap): Stopped
Feb 5 10:49:41 localhost pengine: [14705]: info: native_print: IPaddr_eth (heartbeat::ocf:IPaddr): Stopped
Feb 5 10:49:41 localhost pengine: [14705]: notice: NoRoleChange: Leave resource ldirectord_1 (defas1)
Feb 5 10:49:41 localhost pengine: [14705]: notice: RecurringOp: defas1 ldirectord_1_monitor_120000
Feb 5 10:49:41 localhost pengine: [14705]: notice: StartRsc: defas1 Start LVSSyncDaemonSwap_2
Feb 5 10:49:42 localhost pengine: [14705]: notice: RecurringOp: defas1 LVSSyncDaemonSwap_2_monitor_120000
Feb 5 10:49:42 localhost pengine: [14705]: notice: StartRsc: defas1 Start IPaddr_eth
Feb 5 10:49:42 localhost pengine: [14705]: notice: RecurringOp: defas1 IPaddr_eth_monitor_5000
Feb 5 10:49:42 localhost crmd: [14701]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 5 10:49:42 localhost tengine: [14704]: info: unpack_graph: Unpacked transition 2: 8 actions in 8 synapses
Feb 5 10:49:42 localhost tengine: [14704]: info: te_pseudo_action: Pseudo action 11 fired and confirmed
Feb 5 10:49:42 localhost pengine: [14705]: info: process_pe_message: Transition 2: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-124.bz2
Feb 5 10:49:42 localhost tengine: [14704]: info: send_rsc_command: Initiating action 5: ldirectord_1_start_0 on defas1
Feb 5 10:49:46 localhost tengine: [14704]: info: match_graph_event: Action ldirectord_1_start_0 (5) confirmed on defas1
Feb 5 10:49:46 localhost tengine: [14704]: info: send_rsc_command: Initiating action 6: ldirectord_1_monitor_120000 on defas1
Feb 5 10:49:46 localhost tengine: [14704]: info: send_rsc_command: Initiating action 7: LVSSyncDaemonSwap_2_start_0 on defas1
Feb 5 10:49:51 localhost tengine: [14704]: WARN: status_from_rc: Action start on defas1 failed (target: <null> vs. rc: 255): Error
Feb 5 10:49:51 localhost tengine: [14704]: info: update_abort_priority: Abort priority upgraded to 1
Feb 5 10:49:51 localhost tengine: [14704]: info: update_abort_priority: Abort action 0 superceeded by 2
Feb 5 10:49:51 localhost tengine: [14704]: info: match_graph_event: Action LVSSyncDaemonSwap_2_start_0 (7) confirmed on defas1
Feb 5 10:49:51 localhost tengine: [14704]: info: match_graph_event: Action ldirectord_1_monitor_120000 (6) confirmed on defas1
Feb 5 10:49:51 localhost tengine: [14704]: info: run_graph: ====================================================
Feb 5 10:49:51 localhost tengine: [14704]: notice: run_graph: Transition 2: (Complete=4, Pending=0, Fired=0, Skipped=4, Incomplete=0)
Feb 5 10:49:51 localhost crmd: [14701]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
Feb 5 10:49:51 localhost crmd: [14701]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Feb 5 10:49:51 localhost pengine: [14705]: notice: cluster_option: Using default value '60s' for cluster option 'cluster-delay'
Feb 5 10:49:51 localhost pengine: [14705]: notice: cluster_option: Using default value '-1' for cluster option 'pe-error-series-max'
Feb 5 10:49:51 localhost pengine: [14705]: notice: cluster_option: Using default value '-1' for cluster option 'pe-warn-series-max'
Feb 5 10:49:51 localhost pengine: [14705]: notice: cluster_option: Using default value '-1' for cluster option 'pe-input-series-max'
Feb 5 10:49:51 localhost pengine: [14705]: notice: cluster_option: Using default value 'true' for cluster option 'startup-fencing'
Feb 5 10:49:51 localhost pengine: [14705]: info: determine_online_status: Node defas2 is online
Feb 5 10:49:51 localhost pengine: [14705]: info: determine_online_status: Node defas1 is online
Feb 5 10:49:51 localhost pengine: [14705]: WARN: unpack_rsc_op: Processing failed op (LVSSyncDaemonSwap_2_start_0) on defas1
Feb 5 10:49:51 localhost pengine: [14705]: WARN: unpack_rsc_op: Handling failed start for LVSSyncDaemonSwap_2 on defas1
Feb 5 10:49:51 localhost pengine: [14705]: WARN: unpack_rsc_op: Processing failed op (ldirectord_1_monitor_120000) on defas1
Feb 5 10:49:51 localhost pengine: [14705]: info: group_print: Resource Group: group_1
Feb 5 10:49:51 localhost pengine: [14705]: info: native_print: ldirectord_1 (heartbeat:ldirectord): Started defas1 FAILED
Feb 5 10:49:51 localhost pengine: [14705]: info: native_print: LVSSyncDaemonSwap_2 (heartbeat:LVSSyncDaemonSwap): Started defas1 FAILED
Feb 5 10:49:51 localhost pengine: [14705]: info: native_print: IPaddr_eth (heartbeat::ocf:IPaddr): Stopped
Feb 5 10:49:52 localhost pengine: [14705]: notice: NoRoleChange: Recover resource ldirectord_1 (defas2)
Feb 5 10:49:52 localhost pengine: [14705]: notice: StopRsc: defas1 Stop ldirectord_1
Feb 5 10:49:52 localhost pengine: [14705]: notice: StartRsc: defas2 Start ldirectord_1
Feb 5 10:49:52 localhost pengine: [14705]: notice: RecurringOp: defas2 ldirectord_1_monitor_120000
Feb 5 10:49:52 localhost pengine: [14705]: notice: NoRoleChange: Recover resource LVSSyncDaemonSwap_2 (defas2)
Feb 5 10:49:52 localhost pengine: [14705]: notice: StopRsc: defas1 Stop LVSSyncDaemonSwap_2
Feb 5 10:49:52 localhost pengine: [14705]: notice: StartRsc: defas2 Start LVSSyncDaemonSwap_2
Feb 5 10:49:52 localhost pengine: [14705]: notice: RecurringOp: defas2 LVSSyncDaemonSwap_2_monitor_120000
Feb 5 10:49:52 localhost pengine: [14705]: notice: StartRsc: defas2 Start IPaddr_eth
Feb 5 10:49:52 localhost pengine: [14705]: notice: RecurringOp: defas2 IPaddr_eth_monitor_5000
Feb 5 10:49:52 localhost crmd: [14701]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 5 10:49:52 localhost pengine: [14705]: info: process_pe_message: Transition 3: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-125.bz2
Feb 5 10:49:52 localhost tengine: [14704]: info: unpack_graph: Unpacked transition 3: 13 actions in 13 synapses
Feb 5 10:49:52 localhost tengine: [14704]: info: te_pseudo_action: Pseudo action 15 fired and confirmed
Feb 5 10:49:52 localhost tengine: [14704]: info: te_pseudo_action: Pseudo action 17 fired and confirmed
Feb 5 10:49:52 localhost tengine: [14704]: info: send_rsc_command: Initiating action 1: LVSSyncDaemonSwap_2_stop_0 on defas1
Feb 5 10:49:56 localhost tengine: [14704]: WARN: status_from_rc: Action stop on defas1 failed (target: <null> vs. rc: 255): Error
Feb 5 10:49:56 localhost tengine: [14704]: info: update_abort_priority: Abort priority upgraded to 1
Feb 5 10:49:56 localhost tengine: [14704]: info: update_abort_priority: Abort action 0 superceeded by 2
Feb 5 10:49:56 localhost tengine: [14704]: info: match_graph_event: Action LVSSyncDaemonSwap_2_stop_0 (1) confirmed on defas1
Feb 5 10:49:56 localhost tengine: [14704]: info: run_graph: ====================================================
Feb 5 10:49:56 localhost tengine: [14704]: notice: run_graph: Transition 3: (Complete=3, Pending=0, Fired=0, Skipped=10, Incomplete=0)
Feb 5 10:49:56 localhost crmd: [14701]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
Feb 5 10:49:56 localhost crmd: [14701]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Feb 5 10:49:56 localhost pengine: [14705]: notice: cluster_option: Using default value '60s' for cluster option 'cluster-delay'
Feb 5 10:49:56 localhost pengine: [14705]: notice: cluster_option: Using default value '-1' for cluster option 'pe-error-series-max'
Feb 5 10:49:56 localhost pengine: [14705]: notice: cluster_option: Using default value '-1' for cluster option 'pe-warn-series-max'
Feb 5 10:49:56 localhost pengine: [14705]: notice: cluster_option: Using default value '-1' for cluster option 'pe-input-series-max'
Feb 5 10:49:56 localhost pengine: [14705]: notice: cluster_option: Using default value 'true' for cluster option 'startup-fencing'
Feb 5 10:49:56 localhost pengine: [14705]: info: determine_online_status: Node defas2 is online
Feb 5 10:49:56 localhost pengine: [14705]: info: determine_online_status: Node defas1 is online
Feb 5 10:49:56 localhost pengine: [14705]: WARN: unpack_rsc_op: Processing failed op (LVSSyncDaemonSwap_2_start_0) on defas1
Feb 5 10:49:56 localhost pengine: [14705]: WARN: unpack_rsc_op: Handling failed start for LVSSyncDaemonSwap_2 on defas1
Feb 5 10:49:56 localhost pengine: [14705]: WARN: unpack_rsc_op: Processing failed op (LVSSyncDaemonSwap_2_stop_0) on defas1
Feb 5 10:49:56 localhost pengine: [14705]: WARN: unpack_rsc_op: Handling failed stop for LVSSyncDaemonSwap_2 on defas1
Feb 5 10:49:56 localhost pengine: [14705]: WARN: unpack_rsc_op: Processing failed op (ldirectord_1_monitor_120000) on defas1
Feb 5 10:49:56 localhost pengine: [14705]: info: group_print: Resource Group: group_1
Feb 5 10:49:56 localhost pengine: [14705]: info: native_print: ldirectord_1 (heartbeat:ldirectord): Started defas1 FAILED
Feb 5 10:49:56 localhost pengine: [14705]: info: native_print: LVSSyncDaemonSwap_2 (heartbeat:LVSSyncDaemonSwap): Started defas1 (unmanaged) FAILED
Feb 5 10:49:56 localhost pengine: [14705]: info: native_print: IPaddr_eth (heartbeat::ocf:IPaddr): Stopped
Feb 5 10:49:56 localhost pengine: [14705]: notice: NoRoleChange: Recover resource ldirectord_1 (defas2)
Feb 5 10:49:56 localhost pengine: [14705]: notice: StopRsc: defas1 Stop ldirectord_1
Feb 5 10:49:56 localhost pengine: [14705]: notice: StartRsc: defas2 Start ldirectord_1
Feb 5 10:49:56 localhost pengine: [14705]: notice: RecurringOp: defas2 ldirectord_1_monitor_120000
Feb 5 10:49:56 localhost pengine: [14705]: notice: NoRoleChange: Recover resource LVSSyncDaemonSwap_2 (defas2)
Feb 5 10:49:56 localhost pengine: [14705]: WARN: custom_action: Action LVSSyncDaemonSwap_2_stop_0 (unmanaged)
Feb 5 10:49:57 localhost pengine: [14705]: WARN: custom_action: Action LVSSyncDaemonSwap_2_start_0 (unmanaged)
Feb 5 10:49:57 localhost pengine: [14705]: notice: StartRsc: defas2 Start LVSSyncDaemonSwap_2
Feb 5 10:49:57 localhost pengine: [14705]: notice: StartRsc: defas2 Start IPaddr_eth
Feb 5 10:49:57 localhost pengine: [14705]: notice: RecurringOp: defas2 IPaddr_eth_monitor_5000
Feb 5 10:49:57 localhost pengine: [14705]: WARN: should_dump_action: action 8 (LVSSyncDaemonSwap_2_stop_0) was for an unmanaged resource (LVSSyncDaemonSwap_2)
Feb 5 10:49:57 localhost pengine: [14705]: WARN: should_dump_action: action 9 (LVSSyncDaemonSwap_2_start_0) was for an unmanaged resource (LVSSyncDaemonSwap_2)
Feb 5 10:49:57 localhost pengine: [14705]: WARN: should_dump_action: action 8 (LVSSyncDaemonSwap_2_stop_0) was for an unmanaged resource (LVSSyncDaemonSwap_2)
Feb 5 10:49:57 localhost pengine: [14705]: WARN: should_dump_action: action 9 (LVSSyncDaemonSwap_2_start_0) was for an unmanaged resource (LVSSyncDaemonSwap_2)
Feb 5 10:49:57 localhost crmd: [14701]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 5 10:49:57 localhost pengine: [14705]: WARN: process_pe_message: Transition 4: WARNINGs found during PE processing. PEngine Input stored in: /var/lib/heartbeat/pengine/pe-warn-2167.bz2
Feb 5 10:49:57 localhost tengine: [14704]: info: unpack_graph: Unpacked transition 4: 10 actions in 10 synapses
Feb 5 10:49:57 localhost pengine: [14705]: info: process_pe_message: Configuration WARNINGs found during PE processing. Please run "crm_verify -L" to identify issues.
Feb 5 10:49:57 localhost tengine: [14704]: info: te_pseudo_action: Pseudo action 14 fired and confirmed
Feb 5 10:49:57 localhost tengine: [14704]: info: te_pseudo_action: Pseudo action 16 fired and confirmed
Feb 5 10:49:57 localhost tengine: [14704]: notice: run_graph: ====================================================
Feb 5 10:49:57 localhost tengine: [14704]: WARN: run_graph: Transition 4: (Complete=2, Pending=0, Fired=0, Skipped=0, Incomplete=8)
Feb 5 10:49:57 localhost tengine: [14704]: ERROR: te_graph_trigger: Transition failed: terminated
Feb 5 10:49:57 localhost tengine: [14704]: WARN: print_graph: Synapse 0 is pending (priority: 0)
Feb 5 10:49:57 localhost tengine: [14704]: WARN: print_elem: [Action 12]: Pending (id: group_1_start_0, type: pseduo, priority: 0)
Feb 5 10:49:57 localhost tengine: [14704]: WARN: print_elem: * [Input 14]: Completed (id: group_1_stop_0, type: pseduo, priority: 0)
Feb 5 10:49:57 localhost tengine: [14704]: WARN: print_elem: * [Input 15]: Pending (id: group_1_stopped_0, type: pseduo, priority: 0)
Feb 5 10:49:57 localhost tengine: [14704]: WARN: print_graph: Synapse 1 is pending (priority: 0)
Feb 5 10:49:57 localhost tengine: [14704]: WARN: print_elem: [Action 13]: Pending (id: group_1_running_0, type: pseduo, priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_elem: * [Input 6]: Pending (id: ldirectord_1_start_0, loc: defas2, priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_elem: * [Input 9]: Pending (id: LVSSyncDaemonSwap_2_start_0, loc: defas2, priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_elem: * [Input 10]: Pending (id: IPaddr_eth_start_0, loc: defas2, priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_elem: * [Input 12]: Pending (id: group_1_start_0, type: pseduo, priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_graph: Synapse 2 was confirmed (priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_graph: Synapse 3 is pending (priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_elem: [Action 15]: Pending (id: group_1_stopped_0, type: pseduo, priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_elem: * [Input 2]: Pending (id: ldirectord_1_stop_0, loc: defas1, priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_elem: * [Input 8]: Pending (id: LVSSyncDaemonSwap_2_stop_0, loc: defas1, priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_elem: * [Input 14]: Completed (id: group_1_stop_0, type: pseduo, priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_elem: * [Input 16]: Completed (id: IPaddr_eth_stop_0, type: pseduo, priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_graph: Synapse 4 is pending (priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_elem: [Action 2]: Pending (id: ldirectord_1_stop_0, loc: defas1, priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_elem: * [Input 8]: Pending (id: LVSSyncDaemonSwap_2_stop_0, loc: defas1, priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_elem: * [Input 14]: Completed (id: group_1_stop_0, type: pseduo, priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_graph: Synapse 5 is pending (priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_elem: [Action 6]: Pending (id: ldirectord_1_start_0, loc: defas2, priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_elem: * [Input 2]: Pending (id: ldirectord_1_stop_0, loc: defas1, priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_elem: * [Input 12]: Pending (id: group_1_start_0, type: pseduo, priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_graph: Synapse 6 is pending (priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_elem: [Action 7]: Pending (id: ldirectord_1_monitor_120000, loc: defas2, priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_elem: * [Input 6]: Pending (id: ldirectord_1_start_0, loc: defas2, priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_graph: Synapse 7 is pending (priority: 0)
Feb 5 10:49:58 localhost tengine: [14704]: WARN: print_elem: [Action 10]: Pending (id: IPaddr_eth_start_0, loc: defas2, priority: 0)
Feb 5 10:49:59 localhost tengine: [14704]: WARN: print_elem: * [Input 9]: Pending (id: LVSSyncDaemonSwap_2_start_0, loc: defas2, priority: 0)
Feb 5 10:49:59 localhost tengine: [14704]: WARN: print_graph: Synapse 8 is pending (priority: 0)
Feb 5 10:49:59 localhost tengine: [14704]: WARN: print_elem: [Action 11]: Pending (id: IPaddr_eth_monitor_5000, loc: defas2, priority: 0)
Feb 5 10:49:59 localhost tengine: [14704]: WARN: print_elem: * [Input 10]: Pending (id: IPaddr_eth_start_0, loc: defas2, priority: 0)
Feb 5 10:49:59 localhost tengine: [14704]: WARN: print_graph: Synapse 9 was confirmed (priority: 0)
Feb 5 10:49:59 localhost tengine: [14704]: info: notify_crmd: Transition 4 status: te_complete - <null>
Feb 5 10:49:59 localhost crmd: [14701]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
-------------- next part --------------
<cib admin_epoch="0" epoch="1" have_quorum="true" ignore_dtd="false" num_peers="2" cib_feature_revision="1.3" ccm_transition="2" generated="true" dc_uuid="8d083aa4-14cc-4cca-9a8c-e7308fb2743c" num_updates="23">
<configuration>
<crm_config>
<cluster_property_set id="cib-bootstrap-options">
<attributes>
<nvpair id="cib-bootstrap-options-symmetric-cluster" name="symmetric-cluster" value="true"/>
<nvpair id="cib-bootstrap-options-no-quorum-policy" name="no-quorum-policy" value="stop"/>
<nvpair id="cib-bootstrap-options-default-resource-stickiness" name="default-resource-stickiness" value="0"/>
<nvpair id="cib-bootstrap-options-default-resource-failure-stickiness" name="default-resource-failure-stickiness" value="0"/>
<nvpair id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" value="false"/>
<nvpair id="cib-bootstrap-options-stonith-action" name="stonith-action" value="reboot"/>
<nvpair id="cib-bootstrap-options-stop-orphan-resources" name="stop-orphan-resources" value="true"/>
<nvpair id="cib-bootstrap-options-stop-orphan-actions" name="stop-orphan-actions" value="true"/>
<nvpair id="cib-bootstrap-options-remove-after-stop" name="remove-after-stop" value="false"/>
<nvpair id="cib-bootstrap-options-short-resource-names" name="short-resource-names" value="true"/>
<nvpair id="cib-bootstrap-options-transition-idle-timeout" name="transition-idle-timeout" value="5min"/>
<nvpair id="cib-bootstrap-options-default-action-timeout" name="default-action-timeout" value="15s"/>
<nvpair id="cib-bootstrap-options-is-managed-default" name="is-managed-default" value="true"/>
</attributes>
</cluster_property_set>
</crm_config>
<nodes>
<node id="8d083aa4-14cc-4cca-9a8c-e7308fb2743c" uname="defas2" type="normal"/>
<node id="6ca8e370-6b00-4b8c-85fe-a67da4c562ba" uname="defas1" type="normal"/>
</nodes>
<resources>
<group id="group_1">
<primitive class="heartbeat" id="ldirectord_1" provider="heartbeat" type="ldirectord">
<operations>
<op id="ldirectord_1_mon" interval="120s" name="monitor" timeout="60s"/>
</operations>
<instance_attributes id="ldirectord_1_inst_attr">
<attributes>
<nvpair id="ldirectord_1_attr_1" name="1" value="ldirectord.cf"/>
</attributes>
</instance_attributes>
</primitive>
<primitive class="heartbeat" id="LVSSyncDaemonSwap_2" provider="heartbeat" type="LVSSyncDaemonSwap">
<operations>
<op id="LVSSyncDaemonSwap_2_mon" interval="120s" name="monitor" timeout="60s"/>
</operations>
<instance_attributes id="LVSSyncDaemonSwap_2_inst_attr">
<attributes>
<nvpair id="LVSSyncDaemonSwap_2_attr_1" name="1" value="master"/>
</attributes>
</instance_attributes>
</primitive>
<primitive class="ocf" id="IPaddr_eth" provider="heartbeat" type="IPaddr">
<operations>
<op id="IPaddr_eth_mon" interval="5s" name="monitor" timeout="5s"/>
</operations>
<instance_attributes id="IPaddr_eth_inst_attr">
<attributes>
<nvpair id="IPaddr_eth_attr_0" name="ip" value="172.30.4.170"/>
<nvpair id="IPaddr_eth_attr_1" name="netmask" value="16"/>
<nvpair id="IPaddr_eth_attr_2" name="nic" value="eth0"/>
</attributes>
</instance_attributes>
</primitive>
</group>
</resources>
<constraints>
<rsc_location id="rsc_location_group_1" rsc="group_1">
<rule id="prefered_location_group_1" score="100">
<expression attribute="#uname" id="prefered_location_group_1_expr" operation="eq" value="defas1"/>
</rule>
</rsc_location>
</constraints>
</configuration>
<status>
<node_state id="8d083aa4-14cc-4cca-9a8c-e7308fb2743c" uname="defas2" crmd="online" crm-debug-origin="do_ccm_update_cache" shutdown="0" in_ccm="true" ha="active" join="member" expected="member">
<lrm id="8d083aa4-14cc-4cca-9a8c-e7308fb2743c">
<lrm_resources>
<lrm_resource id="LVSSyncDaemonSwap_2" type="LVSSyncDaemonSwap" class="heartbeat" provider="heartbeat">
<lrm_rsc_op id="LVSSyncDaemonSwap_2_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" transition_key="4:0:acb11936-6402-42e7-8834-d94a3c9a86fc" transition_magic="0:7;4:0:acb11936-6402-42e7-8834-d94a3c9a86fc" call_id="3" crm_feature_set="1.0.9" rc_code="7" op_status="0" interval="0" op_digest="71cb1bc7f37b5a41e950dbe46794a64f"/>
</lrm_resource>
<lrm_resource id="IPaddr_eth" type="IPaddr" class="ocf" provider="heartbeat">
<lrm_rsc_op id="IPaddr_eth_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" transition_key="5:0:acb11936-6402-42e7-8834-d94a3c9a86fc" transition_magic="0:7;5:0:acb11936-6402-42e7-8834-d94a3c9a86fc" call_id="4" crm_feature_set="1.0.9" rc_code="7" op_status="0" interval="0" op_digest="301eb3ba9d598a369561f1c27e2db486"/>
</lrm_resource>
<lrm_resource id="ldirectord_1" type="ldirectord" class="heartbeat" provider="heartbeat">
<lrm_rsc_op id="ldirectord_1_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" transition_key="3:0:acb11936-6402-42e7-8834-d94a3c9a86fc" transition_magic="0:7;3:0:acb11936-6402-42e7-8834-d94a3c9a86fc" call_id="2" crm_feature_set="1.0.9" rc_code="7" op_status="0" interval="0" op_digest="9369160ba28ec8f79eb02de50b45f82d"/>
</lrm_resource>
</lrm_resources>
</lrm>
<transient_attributes id="8d083aa4-14cc-4cca-9a8c-e7308fb2743c">
<instance_attributes id="status-8d083aa4-14cc-4cca-9a8c-e7308fb2743c">
<attributes>
<nvpair id="status-8d083aa4-14cc-4cca-9a8c-e7308fb2743c-probe_complete" name="probe_complete" value="true"/>
</attributes>
</instance_attributes>
</transient_attributes>
</node_state>
<node_state uname="defas1" crmd="online" in_ccm="true" ha="active" join="member" expected="member" shutdown="0" id="6ca8e370-6b00-4b8c-85fe-a67da4c562ba">
<lrm id="6ca8e370-6b00-4b8c-85fe-a67da4c562ba">
<lrm_resources>
<lrm_resource id="LVSSyncDaemonSwap_2" type="LVSSyncDaemonSwap" class="heartbeat" provider="heartbeat">
<lrm_rsc_op id="LVSSyncDaemonSwap_2_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" transition_key="8:0:acb11936-6402-42e7-8834-d94a3c9a86fc" transition_magic="0:7;8:0:acb11936-6402-42e7-8834-d94a3c9a86fc" call_id="3" crm_feature_set="1.0.9" rc_code="7" op_status="0" interval="0" op_digest="71cb1bc7f37b5a41e950dbe46794a64f"/>
<lrm_rsc_op id="LVSSyncDaemonSwap_2_start_0" operation="start" crm-debug-origin="do_update_resource" transition_key="7:2:acb11936-6402-42e7-8834-d94a3c9a86fc" transition_magic="4:255;7:2:acb11936-6402-42e7-8834-d94a3c9a86fc" call_id="8" crm_feature_set="1.0.9" rc_code="255" op_status="4" interval="0" op_digest="71cb1bc7f37b5a41e950dbe46794a64f"/>
<lrm_rsc_op id="LVSSyncDaemonSwap_2_stop_0" operation="stop" crm-debug-origin="do_update_resource" transition_key="1:3:acb11936-6402-42e7-8834-d94a3c9a86fc" transition_magic="4:255;1:3:acb11936-6402-42e7-8834-d94a3c9a86fc" call_id="9" crm_feature_set="1.0.9" rc_code="255" op_status="4" interval="0" op_digest="71cb1bc7f37b5a41e950dbe46794a64f"/>
</lrm_resource>
<lrm_resource type="ldirectord" class="heartbeat" provider="heartbeat" id="ldirectord_1">
<lrm_rsc_op id="ldirectord_1_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" transition_key="7:0:acb11936-6402-42e7-8834-d94a3c9a86fc" transition_magic="0:7;7:0:acb11936-6402-42e7-8834-d94a3c9a86fc" call_id="2" crm_feature_set="1.0.9" rc_code="7" op_status="0" interval="0" op_digest="9369160ba28ec8f79eb02de50b45f82d"/>
<lrm_rsc_op operation="start" crm_feature_set="1.0.9" rc_code="0" op_status="0" interval="0" op_digest="9369160ba28ec8f79eb02de50b45f82d" id="ldirectord_1_start_0" transition_key="5:2:acb11936-6402-42e7-8834-d94a3c9a86fc" transition_magic="0:0;5:2:acb11936-6402-42e7-8834-d94a3c9a86fc" call_id="6"/>
<lrm_rsc_op id="ldirectord_1_monitor_120000" operation="monitor" crm-debug-origin="do_update_resource" transition_key="6:2:acb11936-6402-42e7-8834-d94a3c9a86fc" transition_magic="0:7;6:2:acb11936-6402-42e7-8834-d94a3c9a86fc" call_id="7" crm_feature_set="1.0.9" rc_code="7" op_status="0" interval="120000" op_digest="9369160ba28ec8f79eb02de50b45f82d"/>
</lrm_resource>
<lrm_resource id="IPaddr_eth" type="IPaddr" class="ocf" provider="heartbeat">
<lrm_rsc_op id="IPaddr_eth_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" transition_key="9:0:acb11936-6402-42e7-8834-d94a3c9a86fc" transition_magic="0:7;9:0:acb11936-6402-42e7-8834-d94a3c9a86fc" call_id="4" crm_feature_set="1.0.9" rc_code="7" op_status="0" interval="0" op_digest="301eb3ba9d598a369561f1c27e2db486"/>
</lrm_resource>
</lrm_resources>
</lrm>
<transient_attributes id="6ca8e370-6b00-4b8c-85fe-a67da4c562ba">
<instance_attributes id="status-6ca8e370-6b00-4b8c-85fe-a67da4c562ba">
<attributes>
<nvpair id="status-6ca8e370-6b00-4b8c-85fe-a67da4c562ba-probe_complete" name="probe_complete" value="true"/>
</attributes>
</instance_attributes>
</transient_attributes>
</node_state>
</status>
</cib>
More information about the Linux-HA
mailing list