[Linux-HA] Help understand an incident
Peter Kruse
pk at q-leap.com
Tue Jul 3 05:27:14 MDT 2007
Hello list!
today in one of our clusters a failover occured. Good news: it
succeeded. But... while looking through the logs we found
that messages are missing on one node so we can not say exactly
what happened. Attached is the syslog from node-2 from the
time where there are no messages on node-1. Is it possible
to say from that log what happened on node-1?
Especially there are messages like this:
Jul 3 11:22:59 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff):
+ <lrm_rsc_op id="nfs:maillastnfs_stop_0" operation="stop"
crm-debug-origin="do_update_resource"
transition_key="6:ad6f57b8-295b-4c20-8e0f-e01494577dfb"
transition_magic="2:152;6:ad6f57b8-295b-4c20-8e0f-e01494577dfb"
call_id="45" rc_code="152" op_status="2" interval="0"
__crm_diff_marker__="added:top"/>
Does that mean the action maillastnfs_stop_0 was run but returned
the status 2? Or is it possible that the action never was run
on node 1?
Thanks in advance.
Peter
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cib.xml
Type: text/xml
Size: 12139 bytes
Desc: not available
Url : http://lists.community.tummy.com/pipermail/linux-ha/attachments/20070703/919315bb/cib-0001.bin
-------------- next part --------------
Jul 3 11:20:52 beosrv-c-2 cib: [16503]: info: mask(notify.c:cib_diff_notify): Update (client: 15627, call:77): 0.10.318 -> 0.10.319 (ok)
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cibmon.c:cibmon_diff): [cib_diff_notify] cib_apply_diff confirmed
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: --- 0.10.318
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: +++ 0.10.319
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <cib num_updates="318">
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <status>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_resources>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_resource id="DoFencing_fence2:fence2:0">
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_rsc_op transition_magic="0:0;4:ad6f57b8-295b-4c20-8e0f-e01494577dfb" rc_code="0" op_status="0" id="DoFencing_fence2:fence2:0_monitor_20000"/>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm_resource>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm_resources>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </node_state>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </status>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </cib>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <cib num_updates="319">
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <status>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resources>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resource id="DoFencing_fence2:fence2:0">
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_rsc_op transition_magic="4:1;4:ad6f57b8-295b-4c20-8e0f-e01494577dfb" rc_code="1" op_status="4" id="DoFencing_fence2:fence2:0_monitor_20000"/>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resource>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resources>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </node_state>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </status>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </cib>
Jul 3 11:20:52 beosrv-c-2 cib: [29511]: info: mask(io.c:write_cib_contents): Wrote version 0.10.319 of the CIB to disk (digest: 811b44971e8f4499cefba9b6f6c756fb)
Jul 3 11:20:52 beosrv-c-2 cib: [16503]: info: mask(notify.c:cib_diff_notify): Update (client: 16356, call:2): 0.10.319 -> 0.10.320 (ok)
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cibmon.c:cibmon_diff): [cib_diff_notify] cib_apply_diff confirmed
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: --- 0.10.319
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: +++ 0.10.320
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <cib num_updates="319"/>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <cib num_updates="320">
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <status>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <transient_attributes id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <instance_attributes id="status-b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <attributes>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <nvpair id="status-b1576e33-5096-426b-856b-2b228f276dde-fail-count-DoFencing_fence2:fence2:0" name="fail-count-DoFencing_fence2:fence2:0" value="1" __crm_diff_marker__="added:top"/>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </attributes>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </instance_attributes>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </transient_attributes>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </node_state>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </status>
Jul 3 11:20:52 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </cib>
Jul 3 11:20:52 beosrv-c-2 cib: [29529]: info: mask(io.c:write_cib_contents): Wrote version 0.10.320 of the CIB to disk (digest: 549520e331aa170fc98daa4347dd154b)
Jul 3 11:20:55 beosrv-c-2 cib: [16503]: info: mask(notify.c:cib_diff_notify): Update (client: 15627, call:81): 0.10.320 -> 0.10.321 (ok)
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cibmon.c:cibmon_diff): [cib_diff_notify] cib_apply_diff confirmed
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: --- 0.10.320
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: +++ 0.10.321
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <cib num_updates="320">
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <status>
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_resources>
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_resource id="DoFencing_fence2:fence2:0">
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_rsc_op transition_key="4:ad6f57b8-295b-4c20-8e0f-e01494577dfb" transition_magic="0:0;4:ad6f57b8-295b-4c20-8e0f-e01494577dfb" call_id="40" rc_code="0" op_status="0" id="DoFencing_fence2:fence2:0_stop_0"/>
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm_resource>
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm_resources>
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm>
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </node_state>
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </status>
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </cib>
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <cib num_updates="321">
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <status>
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resources>
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resource id="DoFencing_fence2:fence2:0">
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_rsc_op transition_key="5:ad6f57b8-295b-4c20-8e0f-e01494577dfb" transition_magic="4:1;5:ad6f57b8-295b-4c20-8e0f-e01494577dfb" call_id="44" rc_code="1" op_status="4" id="DoFencing_fence2:fence2:0_stop_0"/>
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resource>
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resources>
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm>
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </node_state>
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </status>
Jul 3 11:20:55 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </cib>
Jul 3 11:20:56 beosrv-c-2 cib: [29530]: info: mask(io.c:write_cib_contents): Wrote version 0.10.321 of the CIB to disk (digest: 0c91db96f945e000e676f0e16721f495)
Jul 3 11:21:01 beosrv-c-2 cib: [16503]: info: mask(notify.c:cib_diff_notify): Update (client: 15627, call:83): 0.10.321 -> 0.10.322 (ok)
Jul 3 11:21:01 beosrv-c-2 cibmon: [16501]: info: mask(cibmon.c:cibmon_diff): [cib_diff_notify] cib_apply_diff confirmed
Jul 3 11:21:01 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: --- 0.10.321
Jul 3 11:21:01 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: +++ 0.10.322
Jul 3 11:21:01 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <cib num_updates="321"/>
Jul 3 11:21:01 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <cib num_updates="322">
Jul 3 11:21:01 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <status>
Jul 3 11:21:01 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:21:01 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:21:01 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resources>
Jul 3 11:21:01 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resource id="DoFencing_fence1:fence1:0">
Jul 3 11:21:01 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_rsc_op id="DoFencing_fence1:fence1:0_stop_0" operation="stop" crm-debug-origin="do_update_resource" transition_key="6:ad6f57b8-295b-4c20-8e0f-e01494577dfb" transition_magic="0:0;6:ad6f57b8-295b-4c20-8e0f-e01494577dfb" call_id="47" rc_code="0" op_status="0" interval="0" __crm_diff_marker__="added:top"/>
Jul 3 11:21:01 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resource>
Jul 3 11:21:01 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resources>
Jul 3 11:21:01 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm>
Jul 3 11:21:01 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </node_state>
Jul 3 11:21:01 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </status>
Jul 3 11:21:01 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </cib>
Jul 3 11:21:01 beosrv-c-2 cib: [29553]: info: mask(io.c:write_cib_contents): Wrote version 0.10.322 of the CIB to disk (digest: 06e3912e44ad0e974bfa70a7624c2bb8)
Jul 3 11:22:59 beosrv-c-2 cib: [16503]: info: mask(notify.c:cib_diff_notify): Update (client: 15627, call:84): 0.10.322 -> 0.10.323 (ok)
Jul 3 11:22:59 beosrv-c-2 cibmon: [16501]: info: mask(cibmon.c:cibmon_diff): [cib_diff_notify] cib_apply_diff confirmed
Jul 3 11:22:59 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: --- 0.10.322
Jul 3 11:22:59 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: +++ 0.10.323
Jul 3 11:22:59 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <cib num_updates="322"/>
Jul 3 11:22:59 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <cib num_updates="323">
Jul 3 11:22:59 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <status>
Jul 3 11:22:59 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:22:59 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:22:59 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resources>
Jul 3 11:22:59 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resource id="nfs:maillastnfs">
Jul 3 11:22:59 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_rsc_op id="nfs:maillastnfs_stop_0" operation="stop" crm-debug-origin="do_update_resource" transition_key="6:ad6f57b8-295b-4c20-8e0f-e01494577dfb" transition_magic="2:152;6:ad6f57b8-295b-4c20-8e0f-e01494577dfb" call_id="45" rc_code="152" op_status="2" interval="0" __crm_diff_marker__="added:top"/>
Jul 3 11:22:59 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resource>
Jul 3 11:22:59 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resources>
Jul 3 11:22:59 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm>
Jul 3 11:22:59 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </node_state>
Jul 3 11:22:59 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </status>
Jul 3 11:22:59 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </cib>
Jul 3 11:22:59 beosrv-c-2 cib: [30831]: info: mask(io.c:write_cib_contents): Wrote version 0.10.323 of the CIB to disk (digest: b61203a209a612397f5c3e729a3d67d3)
Jul 3 11:23:04 beosrv-c-2 cib: [16503]: info: mask(main.c:cib_stats): Processed 9 operations (0.00us average, 0% utilization) in the last 10min
Jul 3 11:24:14 beosrv-c-2 cib: [16503]: info: mask(notify.c:cib_diff_notify): Update (client: 15627, call:86): 0.10.323 -> 0.10.324 (ok)
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cibmon.c:cibmon_diff): [cib_diff_notify] cib_apply_diff confirmed
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: --- 0.10.323
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: +++ 0.10.324
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <cib num_updates="323">
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <status>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_resources>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_resource id="nfs:nfsapps">
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_rsc_op transition_magic="0:0;1:ad6f57b8-295b-4c20-8e0f-e01494577dfb" rc_code="0" op_status="0" id="nfs:nfsapps_monitor_300000"/>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm_resource>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm_resources>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </node_state>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </status>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </cib>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <cib num_updates="324">
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <status>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resources>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resource id="nfs:nfsapps">
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_rsc_op transition_magic="2:152;1:ad6f57b8-295b-4c20-8e0f-e01494577dfb" rc_code="152" op_status="2" id="nfs:nfsapps_monitor_300000"/>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resource>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resources>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </node_state>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </status>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </cib>
Jul 3 11:24:14 beosrv-c-2 cib: [31538]: info: mask(io.c:write_cib_contents): Wrote version 0.10.324 of the CIB to disk (digest: 2dd24985081ec0da7bf7b76486ca4020)
Jul 3 11:24:14 beosrv-c-2 cib: [16503]: info: mask(notify.c:cib_diff_notify): Update (client: 15627, call:87): 0.10.324 -> 0.10.325 (ok)
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cibmon.c:cibmon_diff): [cib_diff_notify] cib_apply_diff confirmed
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: --- 0.10.324
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: +++ 0.10.325
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <cib num_updates="324">
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <status>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_resources>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_resource id="nfs:fdata">
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_rsc_op transition_magic="0:0;1:ad6f57b8-295b-4c20-8e0f-e01494577dfb" rc_code="0" op_status="0" id="nfs:fdata_monitor_300000"/>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm_resource>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm_resources>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </node_state>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </status>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </cib>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <cib num_updates="325">
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <status>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resources>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resource id="nfs:fdata">
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_rsc_op transition_magic="2:152;1:ad6f57b8-295b-4c20-8e0f-e01494577dfb" rc_code="152" op_status="2" id="nfs:fdata_monitor_300000"/>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resource>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resources>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </node_state>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </status>
Jul 3 11:24:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </cib>
Jul 3 11:24:14 beosrv-c-2 cib: [31539]: info: mask(io.c:write_cib_contents): Wrote version 0.10.325 of the CIB to disk (digest: f90c87cf351e8738d0bd6b506ad7ed06)
Jul 3 11:24:25 beosrv-c-2 cib: [16503]: info: mask(notify.c:cib_diff_notify): Update (client: 15627, call:88): 0.10.325 -> 0.10.326 (ok)
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cibmon.c:cibmon_diff): [cib_diff_notify] cib_apply_diff confirmed
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: --- 0.10.325
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: +++ 0.10.326
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <cib num_updates="325">
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <status>
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_resources>
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_resource id="nfs:fapps">
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_rsc_op transition_magic="0:0;1:ad6f57b8-295b-4c20-8e0f-e01494577dfb" rc_code="0" op_status="0" id="nfs:fapps_monitor_300000"/>
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm_resource>
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm_resources>
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm>
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </node_state>
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </status>
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </cib>
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <cib num_updates="326">
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <status>
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resources>
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resource id="nfs:fapps">
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_rsc_op transition_magic="2:152;1:ad6f57b8-295b-4c20-8e0f-e01494577dfb" rc_code="152" op_status="2" id="nfs:fapps_monitor_300000"/>
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resource>
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resources>
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm>
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </node_state>
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </status>
Jul 3 11:24:25 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </cib>
Jul 3 11:24:25 beosrv-c-2 cib: [31787]: info: mask(io.c:write_cib_contents): Wrote version 0.10.326 of the CIB to disk (digest: d656cdd91e141f055e38a22b6cf485fe)
Jul 3 11:24:48 beosrv-c-2 cib: [16503]: info: mask(notify.c:cib_diff_notify): Update (client: 15627, call:89): 0.10.326 -> 0.10.327 (ok)
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cibmon.c:cibmon_diff): [cib_diff_notify] cib_apply_diff confirmed
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: --- 0.10.326
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: +++ 0.10.327
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <cib num_updates="326">
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <status>
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_resources>
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_resource id="nfs:nfsdata">
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_rsc_op transition_magic="0:0;1:ad6f57b8-295b-4c20-8e0f-e01494577dfb" rc_code="0" op_status="0" id="nfs:nfsdata_monitor_300000"/>
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm_resource>
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm_resources>
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm>
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </node_state>
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </status>
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </cib>
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <cib num_updates="327">
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <status>
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resources>
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resource id="nfs:nfsdata">
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_rsc_op transition_magic="2:152;1:ad6f57b8-295b-4c20-8e0f-e01494577dfb" rc_code="152" op_status="2" id="nfs:nfsdata_monitor_300000"/>
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resource>
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resources>
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm>
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </node_state>
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </status>
Jul 3 11:24:48 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </cib>
Jul 3 11:24:48 beosrv-c-2 cib: [31956]: info: mask(io.c:write_cib_contents): Wrote version 0.10.327 of the CIB to disk (digest: 22803ed66e5a311251c53d596528a663)
Jul 3 11:25:03 beosrv-c-2 cib: [16503]: info: mask(notify.c:cib_diff_notify): Update (client: 15627, call:90): 0.10.327 -> 0.10.328 (ok)
Jul 3 11:25:03 beosrv-c-2 cibmon: [16501]: info: mask(cibmon.c:cibmon_diff): [cib_diff_notify] cib_apply_diff confirmed
Jul 3 11:25:03 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: --- 0.10.327
Jul 3 11:25:03 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: +++ 0.10.328
Jul 3 11:25:03 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <cib num_updates="327"/>
Jul 3 11:25:03 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <cib num_updates="328">
Jul 3 11:25:03 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <status>
Jul 3 11:25:03 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:25:03 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:25:03 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resources>
Jul 3 11:25:03 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resource id="nfs:pbs1">
Jul 3 11:25:03 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_rsc_op id="nfs:pbs1_stop_0" operation="stop" crm-debug-origin="do_update_resource" transition_key="7:ad6f57b8-295b-4c20-8e0f-e01494577dfb" transition_magic="2:152;7:ad6f57b8-295b-4c20-8e0f-e01494577dfb" call_id="49" rc_code="152" op_status="2" interval="0" __crm_diff_marker__="added:top"/>
Jul 3 11:25:03 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resource>
Jul 3 11:25:03 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resources>
Jul 3 11:25:03 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm>
Jul 3 11:25:03 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </node_state>
Jul 3 11:25:03 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </status>
Jul 3 11:25:03 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </cib>
Jul 3 11:25:03 beosrv-c-2 cib: [32171]: info: mask(io.c:write_cib_contents): Wrote version 0.10.328 of the CIB to disk (digest: 2e881e6a17e66485e31e023a9d102ad8)
Jul 3 11:27:06 beosrv-c-2 cib: [16503]: info: mask(notify.c:cib_diff_notify): Update (client: 15627, call:92): 0.10.328 -> 0.10.329 (ok)
Jul 3 11:27:06 beosrv-c-2 cibmon: [16501]: info: mask(cibmon.c:cibmon_diff): [cib_diff_notify] cib_apply_diff confirmed
Jul 3 11:27:06 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: --- 0.10.328
Jul 3 11:27:06 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: +++ 0.10.329
Jul 3 11:27:06 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <cib num_updates="328"/>
Jul 3 11:27:06 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <cib num_updates="329">
Jul 3 11:27:06 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <status>
Jul 3 11:27:06 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:27:06 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:27:06 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resources>
Jul 3 11:27:06 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resource id="nfs:IPaddr2">
Jul 3 11:27:06 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_rsc_op id="nfs:IPaddr2_stop_0" operation="stop" crm-debug-origin="do_update_resource" transition_key="8:ad6f57b8-295b-4c20-8e0f-e01494577dfb" transition_magic="2:152;8:ad6f57b8-295b-4c20-8e0f-e01494577dfb" call_id="51" rc_code="152" op_status="2" interval="0" __crm_diff_marker__="added:top"/>
Jul 3 11:27:06 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resource>
Jul 3 11:27:06 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resources>
Jul 3 11:27:06 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm>
Jul 3 11:27:06 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </node_state>
Jul 3 11:27:06 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </status>
Jul 3 11:27:06 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </cib>
Jul 3 11:27:07 beosrv-c-2 cib: [1128]: info: mask(io.c:write_cib_contents): Wrote version 0.10.329 of the CIB to disk (digest: 1b1b9d80632df66a6624f1498d9ed631)
Jul 3 11:29:09 beosrv-c-2 cib: [16503]: info: mask(notify.c:cib_diff_notify): Update (client: 15627, call:94): 0.10.329 -> 0.10.330 (ok)
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cibmon.c:cibmon_diff): [cib_diff_notify] cib_apply_diff confirmed
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: --- 0.10.329
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: +++ 0.10.330
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <cib num_updates="329">
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <status>
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_resources>
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_resource id="nfs:drbdapps">
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <lrm_rsc_op transition_magic="0:0;1:ad6f57b8-295b-4c20-8e0f-e01494577dfb" rc_code="0" op_status="0" id="nfs:drbdapps_monitor_300000"/>
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm_resource>
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm_resources>
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </lrm>
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </node_state>
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </status>
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - </cib>
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <cib num_updates="330">
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <status>
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resources>
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resource id="nfs:drbdapps">
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_rsc_op transition_magic="2:152;1:ad6f57b8-295b-4c20-8e0f-e01494577dfb" rc_code="152" op_status="2" id="nfs:drbdapps_monitor_300000"/>
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resource>
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resources>
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm>
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </node_state>
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </status>
Jul 3 11:29:09 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </cib>
Jul 3 11:29:09 beosrv-c-2 cib: [2448]: info: mask(io.c:write_cib_contents): Wrote version 0.10.330 of the CIB to disk (digest: e60feb25cb8a7e4ca3eb79cbed5a8c8f)
Jul 3 11:29:10 beosrv-c-2 cib: [16503]: info: mask(notify.c:cib_diff_notify): Update (client: 15627, call:95): 0.10.330 -> 0.10.331 (ok)
Jul 3 11:29:10 beosrv-c-2 cibmon: [16501]: info: mask(cibmon.c:cibmon_diff): [cib_diff_notify] cib_apply_diff confirmed
Jul 3 11:29:10 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: --- 0.10.330
Jul 3 11:29:10 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: +++ 0.10.331
Jul 3 11:29:10 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <cib num_updates="330"/>
Jul 3 11:29:10 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <cib num_updates="331">
Jul 3 11:29:10 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <status>
Jul 3 11:29:10 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:29:10 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:29:10 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resources>
Jul 3 11:29:10 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resource id="nfs:IPaddr1">
Jul 3 11:29:10 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_rsc_op id="nfs:IPaddr1_stop_0" operation="stop" crm-debug-origin="do_update_resource" transition_key="9:ad6f57b8-295b-4c20-8e0f-e01494577dfb" transition_magic="2:152;9:ad6f57b8-295b-4c20-8e0f-e01494577dfb" call_id="53" rc_code="152" op_status="2" interval="0" __crm_diff_marker__="added:top"/>
Jul 3 11:29:10 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resource>
Jul 3 11:29:10 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resources>
Jul 3 11:29:10 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm>
Jul 3 11:29:10 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </node_state>
Jul 3 11:29:10 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </status>
Jul 3 11:29:10 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </cib>
Jul 3 11:29:10 beosrv-c-2 cib: [2449]: info: mask(io.c:write_cib_contents): Wrote version 0.10.331 of the CIB to disk (digest: 089072df29b55d51adde781d21868bb0)
Jul 3 11:31:13 beosrv-c-2 cib: [16503]: info: mask(notify.c:cib_diff_notify): Update (client: 15627, call:97): 0.10.331 -> 0.10.332 (ok)
Jul 3 11:31:13 beosrv-c-2 cibmon: [16501]: info: mask(cibmon.c:cibmon_diff): [cib_diff_notify] cib_apply_diff confirmed
Jul 3 11:31:13 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: --- 0.10.331
Jul 3 11:31:13 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: +++ 0.10.332
Jul 3 11:31:13 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <cib num_updates="331"/>
Jul 3 11:31:13 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <cib num_updates="332"/>
Jul 3 11:31:13 beosrv-c-2 cib: [3653]: info: mask(io.c:write_cib_contents): Wrote version 0.10.332 of the CIB to disk (digest: 692a5fe6eafc3d09a740cae779c5d443)
Jul 3 11:31:14 beosrv-c-2 cib: [16503]: info: mask(notify.c:cib_diff_notify): Update (client: 15627, call:98): 0.10.332 -> 0.10.333 (ok)
Jul 3 11:31:14 beosrv-c-2 cibmon: [16501]: info: mask(cibmon.c:cibmon_diff): [cib_diff_notify] cib_apply_diff confirmed
Jul 3 11:31:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: --- 0.10.332
Jul 3 11:31:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: +++ 0.10.333
Jul 3 11:31:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <cib num_updates="332"/>
Jul 3 11:31:14 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <cib num_updates="333"/>
Jul 3 11:31:14 beosrv-c-2 cib: [3654]: info: mask(io.c:write_cib_contents): Wrote version 0.10.333 of the CIB to disk (digest: 371c704a571c41028e53a94cfea2d436)
Jul 3 11:31:15 beosrv-c-2 automount[3659]: >> pmap_getmaps rpc problem: RPC: Unable to receive; errno = Connection reset by peer
Jul 3 11:31:24 beosrv-c-2 cib: [16503]: info: mask(notify.c:cib_diff_notify): Update (client: 15627, call:99): 0.10.333 -> 0.10.334 (ok)
Jul 3 11:31:24 beosrv-c-2 cibmon: [16501]: info: mask(cibmon.c:cibmon_diff): [cib_diff_notify] cib_apply_diff confirmed
Jul 3 11:31:24 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: --- 0.10.333
Jul 3 11:31:24 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: +++ 0.10.334
Jul 3 11:31:24 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <cib num_updates="333"/>
Jul 3 11:31:24 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <cib num_updates="334"/>
Jul 3 11:31:24 beosrv-c-2 cib: [3908]: info: mask(io.c:write_cib_contents): Wrote version 0.10.334 of the CIB to disk (digest: a1b983ec849ee8ece1f53888c571bfbf)
Jul 3 11:31:37 beosrv-c-2 automount[3659]: >> mount: RPC: Timed out
Jul 3 11:31:37 beosrv-c-2 automount[3659]: mount(nfs): nfs: mount failure beosrv-c:/c/data/etc on /data/etc
Jul 3 11:31:37 beosrv-c-2 automount[3659]: failed to mount /data/etc
Jul 3 11:31:49 beosrv-c-2 cib: [16503]: info: mask(notify.c:cib_diff_notify): Update (client: 15627, call:100): 0.10.334 -> 0.10.335 (ok)
Jul 3 11:31:49 beosrv-c-2 cibmon: [16501]: info: mask(cibmon.c:cibmon_diff): [cib_diff_notify] cib_apply_diff confirmed
Jul 3 11:31:49 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: --- 0.10.334
Jul 3 11:31:49 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: +++ 0.10.335
Jul 3 11:31:49 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <cib num_updates="334"/>
Jul 3 11:31:49 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <cib num_updates="335"/>
Jul 3 11:31:50 beosrv-c-2 cib: [4193]: info: mask(io.c:write_cib_contents): Wrote version 0.10.335 of the CIB to disk (digest: 06d85d773fcbc97223a48c74b65cf77a)
Jul 3 11:33:03 beosrv-c-2 cib: [16503]: info: mask(main.c:cib_stats): Processed 13 operations (3076.00us average, 0% utilization) in the last 10min
Jul 3 11:34:15 beosrv-c-2 cib: [16503]: info: mask(notify.c:cib_diff_notify): Update (client: 15627, call:101): 0.10.335 -> 0.10.336 (ok)
Jul 3 11:34:15 beosrv-c-2 cibmon: [16501]: info: mask(cibmon.c:cibmon_diff): [cib_diff_notify] cib_apply_diff confirmed
Jul 3 11:34:15 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: --- 0.10.335
Jul 3 11:34:15 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: +++ 0.10.336
Jul 3 11:34:15 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <cib num_updates="335"/>
Jul 3 11:34:15 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <cib num_updates="336">
Jul 3 11:34:15 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <status>
Jul 3 11:34:15 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:34:15 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:34:15 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resources>
Jul 3 11:34:15 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resource id="nfs:drbddata">
Jul 3 11:34:15 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_rsc_op id="nfs:drbddata_stop_0" operation="stop" crm-debug-origin="do_update_resource" transition_key="10:ad6f57b8-295b-4c20-8e0f-e01494577dfb" transition_magic="2:152;10:ad6f57b8-295b-4c20-8e0f-e01494577dfb" call_id="55" rc_code="152" op_status="2" interval="0" __crm_diff_marker__="added:top"/>
Jul 3 11:34:15 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resource>
Jul 3 11:34:15 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resources>
Jul 3 11:34:15 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm>
Jul 3 11:34:15 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </node_state>
Jul 3 11:34:15 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </status>
Jul 3 11:34:15 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </cib>
Jul 3 11:34:15 beosrv-c-2 cib: [5643]: info: mask(io.c:write_cib_contents): Wrote version 0.10.336 of the CIB to disk (digest: 6b4c455f42c1bfd13de49e65b7e25002)
Jul 3 11:36:18 beosrv-c-2 cib: [16503]: info: mask(notify.c:cib_diff_notify): Update (client: 15627, call:103): 0.10.336 -> 0.10.337 (ok)
Jul 3 11:36:18 beosrv-c-2 cibmon: [16501]: info: mask(cibmon.c:cibmon_diff): [cib_diff_notify] cib_apply_diff confirmed
Jul 3 11:36:18 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: --- 0.10.336
Jul 3 11:36:18 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): Diff: +++ 0.10.337
Jul 3 11:36:18 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): - <cib num_updates="336"/>
Jul 3 11:36:18 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <cib num_updates="337">
Jul 3 11:36:18 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <status>
Jul 3 11:36:18 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <node_state id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:36:18 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm id="b1576e33-5096-426b-856b-2b228f276dde">
Jul 3 11:36:18 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resources>
Jul 3 11:36:18 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_resource id="nfs:mailfirstnfs">
Jul 3 11:36:18 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + <lrm_rsc_op id="nfs:mailfirstnfs_stop_0" operation="stop" crm-debug-origin="do_update_resource" transition_key="11:ad6f57b8-295b-4c20-8e0f-e01494577dfb" transition_magic="2:152;11:ad6f57b8-295b-4c20-8e0f-e01494577dfb" call_id="56" rc_code="152" op_status="2" interval="0" __crm_diff_marker__="added:top"/>
Jul 3 11:36:18 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resource>
Jul 3 11:36:18 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm_resources>
Jul 3 11:36:18 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </lrm>
Jul 3 11:36:18 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </node_state>
Jul 3 11:36:18 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </status>
Jul 3 11:36:18 beosrv-c-2 cibmon: [16501]: info: mask(cib_apply_diff): + </cib>
Jul 3 11:36:18 beosrv-c-2 cib: [7068]: info: mask(io.c:write_cib_contents): Wrote version 0.10.337 of the CIB to disk (digest: faeeea93db13f231480b562f26561936)
Jul 3 11:36:21 beosrv-c-2 stonithd: [16505]: info: stonith_operate_locally::2174: sending fencing op (1) for beosrv-c-1 to device external (rsc_id=DoFencing_fence1:fence1:1, pid=7069)
Jul 3 11:36:21 beosrv-c-2 stonithd: [16505]: info: Node beosrv-c-2 try to help node beosrv-c-1 to fence node beosrv-c-1.
Jul 3 11:36:21 beosrv-c-2 apc[7073]: resetting beosrv-c-1
More information about the Linux-HA
mailing list