[Linux-HA] [UPDATE]Re: VM never come back to it's original node
Rene Purcell
rene.purcell at gmail.com
Tue May 15 08:05:26 MDT 2007
I had downloaded a copy of SLES10 SP1-RC4 and everything work fine... even
with the live migration..
maybe there was a problem with the RA ...
On 5/11/07, Rene Purcell <rene.purcell at gmail.com> wrote:
>
> here's the output of "cibadmin -Ql" when my ressource become unmanaged
>
> # cibadmin -Ql
>
> <cib generated="true" admin_epoch="0" have_quorum="true" num_peers="2"
> cib_feature_revision=" 1.3" ignore_dtd="false" epoch="110"
> num_updates="1817" cib-last-written="Fri May 11 14:38:23 2007"
> ccm_transition="4" dc_uuid="8b61d090-aa45-46af-9828-7b228c070e81">
> <configuration>
> <crm_config>
> <cluster_property_set id="cibbootstrap">
> <attributes>
> <nvpair id="cibbootstrap-01" name="transition_idle_timeout"
> value="60"/>
> <nvpair id="cibbootstrap-02" name="default_resource_stickiness"
> value="-INFINITY"/>
> <nvpair id="cibbootstrap-03"
> name="default_resource_failure_stickiness" value="-500"/>
> <nvpair id="cibbootstrap-04" name="stonith_enabled"
> value="false"/>
> <nvpair id="cibbootstrap-05" name="stonith_action"
> value="reboot"/>
> <nvpair id="cibbootstrap-06" name="symmetric_cluster"
> value="true"/>
> <nvpair id="cibbootstrap-07" name="no_quorum_policy"
> value="stop"/>
> <nvpair id="cibbootstrap-08" name="stop_orphan_resources"
> value="true"/>
> <nvpair id="cibbootstrap-09" name="stop_orphan_actions"
> value="true"/>
> <nvpair id="cibbootstrap-10" name="is_managed_default"
> value="true"/>
> </attributes>
> </cluster_property_set>
> </crm_config>
> <nodes>
> <node id="8b61d090-aa45-46af-9828-7b228c070e81" uname="qclsles02"
> type="normal"/>
> <node id="1ea9fa99-d5f7-42fb-a7ef-a466f96d3347" uname="qclsles01"
> type="normal"/>
> </nodes>
> <resources>
> <primitive id="qclvmsles01" class="ocf" type="Xen"
> provider="heartbeat" multiple_active="stop_start">
> <operations>
> <op name="monitor" interval="10s" timeout="60s"
> prereq="nothing" id="xen-op-01" start_delay="0" disabled="false"
> role="Started"/>
> </operations>
> <instance_attributes id="qclvmsles01">
> <attributes>
> <nvpair id="xen-01" name="xmfile"
> value="/etc/xen/vm/qclvmsles01"/>
> </attributes>
> </instance_attributes>
> </primitive>
> </resources>
> <constraints>
> <rsc_location id="qclvmsles01_location" rsc="qclvmsles01">
> <rule id="pref_qclvmsles01_location" score="INFINITY">
> <expression attribute="#uname" operation="eq" value="qclsles01"
> id="40f3ef1b-429e-41ca-873e-731e6eb21bb0"/>
> </rule>
> </rsc_location>
> </constraints>
> </configuration>
> <status>
> <node_state id="8b61d090-aa45-46af-9828-7b228c070e81"
> uname="qclsles02" crmd="online" crm-debug-origin="do_update_resource"
> shutdown="0" in_ccm="true" ha="active" join="member" expected="member">
> <transient_attributes id="8b61d090-aa45-46af-9828-7b228c070e81">
> <instance_attributes
> id="status-8b61d090-aa45-46af-9828-7b228c070e81">
> <attributes>
> <nvpair
> id="status-8b61d090-aa45-46af-9828-7b228c070e81-probe_complete"
> name="probe_complete" value="true"/>
> </attributes>
> </instance_attributes>
> </transient_attributes>
> <lrm id="8b61d090-aa45-46af-9828-7b228c070e81">
> <lrm_resources>
> <lrm_resource id="qclvmsles01" type="Xen" class="ocf"
> provider="heartbeat">
> <lrm_rsc_op id="qclvmsles01_monitor_0" operation="monitor"
> crm-debug-origin="build_active_RAs"
> transition_key="3:4:bb9813e5-387f-477a-8051-606c0b063dbe"
> transition_magic="4:7;3:4:bb9813e5-387f-477a-8051-606c0b063dbe" call_id="3"
> crm_feature_set=" 1.0.7" rc_code="7" op_status="4" interval="0"
> op_digest="382cb81041e4b9b54816aac250648525"/>
> <lrm_rsc_op id="qclvmsles01_start_0" operation="start"
> crm-debug-origin="build_active_RAs"
> transition_key="3:6:bb9813e5-387f-477a-8051-606c0b063dbe"
> transition_magic="0:0;3:6:bb9813e5-387f-477a-8051-606c0b063dbe" call_id="4"
> crm_feature_set=" 1.0.7" rc_code="0" op_status="0" interval="0"
> op_digest="382cb81041e4b9b54816aac250648525"/>
> <lrm_rsc_op id="qclvmsles01_monitor_10000"
> operation="monitor" crm-debug-origin="build_active_RAs"
> transition_key="4:6:bb9813e5-387f-477a-8051-606c0b063dbe"
> transition_magic="0:0;4:6:bb9813e5-387f-477a-8051-606c0b063dbe" call_id="5"
> crm_feature_set=" 1.0.7" rc_code="0" op_status="0" interval="10000"
> op_digest="382cb81041e4b9b54816aac250648525"/>
> <lrm_rsc_op id="qclvmsles01_stop_0" operation="stop"
> crm-debug-origin="do_update_resource"
> transition_key="6:7:bb9813e5-387f-477a-8051-606c0b063dbe"
> transition_magic="4:4;6:7:bb9813e5-387f-477a-8051-606c0b063dbe" call_id="7"
> crm_feature_set=" 1.0.7" rc_code="4" op_status="4" interval="0"
> op_digest="382cb81041e4b9b54816aac250648525"/>
> </lrm_resource>
> </lrm_resources>
> </lrm>
> </node_state>
> <node_state id="1ea9fa99-d5f7-42fb-a7ef-a466f96d3347"
> uname="qclsles01" crmd="online" crm-debug-origin="do_update_resource"
> ha="active" shutdown="0" in_ccm="true" join="member" expected="member">
> <lrm id="1ea9fa99-d5f7-42fb-a7ef-a466f96d3347">
> <lrm_resources>
> <lrm_resource id="qclvmsles01" type="Xen" class="ocf"
> provider="heartbeat">
> <lrm_rsc_op id="qclvmsles01_monitor_0" operation="monitor"
> crm-debug-origin="do_update_resource"
> transition_key="5:7:bb9813e5-387f-477a-8051-606c0b063dbe"
> transition_magic="0:7;5:7:bb9813e5-387f-477a-8051-606c0b063dbe" call_id="2"
> crm_feature_set=" 1.0.7" rc_code="7" op_status="0" interval="0"
> op_digest="382cb81041e4b9b54816aac250648525"/>
> </lrm_resource>
> </lrm_resources>
> </lrm>
> <transient_attributes id="1ea9fa99-d5f7-42fb-a7ef-a466f96d3347">
> <instance_attributes
> id="status-1ea9fa99-d5f7-42fb-a7ef-a466f96d3347">
> <attributes>
> <nvpair
> id="status-1ea9fa99-d5f7-42fb-a7ef-a466f96d3347-probe_complete"
> name="probe_complete" value="true"/>
> </attributes>
> </instance_attributes>
> </transient_attributes>
> </node_state>
> </status>
> </cib>
>
>
> On 5/10/07, Rene Purcell < rene.purcell at gmail.com> wrote:
> >
> > Hi all, I still unable to find anwsers ... and maybe someone will be
> > able to help me here..
> >
> >
> >
> > This is the configuration I'm using:
> > file: cibbootstrap.xml
> >
> > <cluster_property_set id="cibbootstrap">
> >
> > <attributes>
> >
> > <nvpair id="cibbootstrap-01" name="transition_idle_timeout"
> > value="60"/>
> >
> > <nvpair id="cibbootstrap-02" name="default_resource_stickiness"
> > value="INFINITY"/>
> >
> > <nvpair id="cibbootstrap-03"
> > name="default_resource_failure_stickiness" value="-500"/>
> >
> > <nvpair id="cibbootstrap-04" name="stonith_enabled" value="false"/>
> >
> > <nvpair id="cibbootstrap-05" name="stonith_action" value="reboot"/>
> >
> > <nvpair id="cibbootstrap-06" name="symmetric_cluster" value="true"/>
> >
> > <nvpair id="cibbootstrap-07" name="no_quorum_policy" value="stop"/>
> >
> > <nvpair id="cibbootstrap-08" name="stop_orphan_resources"
> > value="true"/>
> >
> > <nvpair id="cibbootstrap-09" name="stop_orphan_actions" value="true"/>
> >
> > <nvpair id="cibbootstrap-10" name="is_managed_default" value="true"/>
> >
> > </attributes>
> >
> > </cluster_property_set
> >
> > I added this configuration using "cibadmin -C -o crm_config -x
> > ./cibbootstrap.xml"
> >
> > file: qclvmsles01location.xml
> >
> > <rsc_location id="qclvmsles01_location" rsc="qclvmsles01">
> >
> > <rule id="pref_qclvmsles01_location" score="INFINITY">
> >
> > <expression attribute="#uname" operation="eq" value="qclsles01"/>
> >
> > </rule>
> >
> > </rsc_location
> >
> > I used "cibadmin -C -o constraints -x ./qclvmsles01location.xml" to add
> > the conf into the cib..
> >
> > file:qclvmsles01.xml
> >
> > <primitive id="qclvmsles01" class="ocf" type="Xen" provider="heartbeat">
> >
> > <operations>
> >
> > <op name="monitor" interval="10s" timeout="60s" prereq="nothing"
> > id="xen-op-01"/>
> > <op name="stop" timeout="60s" id="xen-op-03"/>
> >
> > </operations>
> >
> > <instance_attributes id="qclvmsles01">
> >
> > <attributes>
> >
> > <nvpair id="xen-01" name="xmfile" value="/etc/xen/vm/qclvmsles01"/>
> >
> > </attributes>
> >
> > </instance_attributes>
> >
> > </primitive
> >
> > and I used "cibadmin -C -o resources -x ./qclvmsles01.xml"
> >
> >
> > There's the problem.. When my VM start, it start on the right node
> > (qclsles01) if this node become unavailable the VM start on the second node
> > (qclsles02). The problem occur when my qclsles01 comeback online, heartbeat
> > shutdown the vm on qclsles02 but he never start the vm on qclsles01 !!! the
> > resource become unmanaged... Am I wrong or he's supposed to restart the vm
> > on qclsles01, this is why I put a constraints right ?
> >
> > there's the log on qclsles02, because nothing happend on qclsles01..
> > there's nothing trying to start a VM..
> >
> > I'll try to help you figure out..
> >
> > 10h53:30 qclsles02 detect qclsles01 is down and around 10h53:33 he start
> > the vm on qclsles02
> >
> > 10h55:56 the heartbeat service on qclsles01 is restarted
> >
> > 10h56:35 qclsles01 is online!
> >
> > 10h56:50 VM STOP on qclsles02 but never start on qclsles01... as you can
> > see there's error about non existing VM ... but everything is ok If I clean
> > the ressource it will start on qclsles01...
> >
> > May 10 10:53:30 qclsles02 heartbeat: [13248]: WARN: node qclsles01: is
> > dead
> > May 10 10:53:30 qclsles02 heartbeat: [13248]: info: Link qclsles01:eth0
> > dead.
> > May 10 10:53:30 qclsles02 ccm: [13256]: debug: quorum plugin: majority
> > May 10 10:53:30 qclsles02 ccm: [13256]: debug: cluster:linux-ha,
> > member_count=1, member_quorum_votes=100
> > May 10 10:53:30 qclsles02 ccm: [13256]: debug: total_node_count=2,
> > total_quorum_votes=200
> > May 10 10:53:30 qclsles02 ccm: [13256]: debug: quorum plugin: twonodes
> > May 10 10:53:30 qclsles02 ccm: [13256]: debug: cluster:linux-ha,
> > member_count=1, member_quorum_votes=100
> > May 10 10:53:30 qclsles02 ccm: [13256]: debug: total_node_count=2,
> > total_quorum_votes=200
> > May 10 10:53:30 qclsles02 ccm: [13256]: info: Break tie for 2 nodes
> > cluster
> > May 10 10:53:30 qclsles02 cib: [13257]: info: mem_handle_event: Got an
> > event OC_EV_MS_INVALID from ccm
> > May 10 10:53:30 qclsles02 crmd: [13261]: notice:
> > crmd_ha_status_callback:callbacks.c Status update: Node qclsles01 now
> > has status [dead]
> > May 10 10:53:30 qclsles02 cib: [13257]: info: mem_handle_event: no
> > mbr_track info
> > May 10 10:53:30 qclsles02 crmd: [13261]: info: mem_handle_event: Got an
> > event OC_EV_MS_INVALID from ccm
> > May 10 10:53:30 qclsles02 cib: [13257]: info: mem_handle_event: Got an
> > event OC_EV_MS_NEW_MEMBERSHIP from ccm
> > May 10 10:53:30 qclsles02 crmd: [13261]: info: mem_handle_event: no
> > mbr_track info
> > May 10 10:53:30 qclsles02 cib: [13257]: info: mem_handle_event:
> > instance=5, nodes=1, new=0, lost=1, n_idx=0, new_idx=1, old_idx=3
> > May 10 10:53:30 qclsles02 crmd: [13261]: info: mem_handle_event: Got an
> > event OC_EV_MS_NEW_MEMBERSHIP from ccm
> > May 10 10:53:30 qclsles02 cib: [13257]: info: cib_ccm_msg_callback:
> > callbacks.c LOST: qclsles01
> > May 10 10:53:30 qclsles02 crmd: [13261]: info: mem_handle_event:
> > instance=5, nodes=1, new=0, lost=1, n_idx=0, new_idx=1, old_idx=3
> > May 10 10:53:30 qclsles02 cib: [13257]: info: cib_ccm_msg_callback:
> > callbacks.c PEER: qclsles02
> > May 10 10:53:30 qclsles02 crmd: [13261]: info: crmd_ccm_msg_callback:
> > callbacks.c Quorum (re)attained after event=NEW MEMBERSHIP (id=5)
> > May 10 10:53:30 qclsles02 crmd: [13261]: info: ccm_event_detail:ccm.cNEW MEMBERSHIP: trans=5, nodes=1, new=0, lost=1 n_idx=0, new_idx=1,
> > old_idx=3
> > May 10 10:53:30 qclsles02 crmd: [13261]: info: ccm_event_detail:ccm.c
> > CURRENT: qclsles02 [nodeid=1, born=5]
> > May 10 10:53:30 qclsles02 crmd: [13261]: info: ccm_event_detail:ccm.c
> > LOST: qclsles01 [nodeid=0, born=4]
> > May 10 10:53:30 qclsles02 cib: [13257]: info: activateCibXml:io.c CIB
> > size is 67792 bytes (was 72616)
> > May 10 10:53:30 qclsles02 tengine: [13270]: info: te_update_diff:
> > callbacks.c Processing diff (cib_update): 0.86.1483 -> 0.86.1483
> > May 10 10:53:30 qclsles02 cib: [13257]: info: cib_diff_notify: notify.cLocal-only Change (client:13261, call: 77):
> > 0.86.1483 (ok)
> > May 10 10:53:30 qclsles02 tengine: [13270]: WARN: match_down_event:
> > events.c No match for shutdown action on
> > 1ea9fa99-d5f7-42fb-a7ef-a466f96d3347
> > May 10 10:53:30 qclsles02 tengine: [13270]: info: extract_event:
> > events.c Stonith/shutdown event not matched
> > May 10 10:53:30 qclsles02 tengine: [13270]: info: update_abort_priority:
> > utils.c Abort priority upgraded to 1000000
> > May 10 10:53:30 qclsles02 crmd: [13261]: info: do_state_transition:
> > fsa.c qclsles02: State transition S_IDLE -> S_POLICY_ENGINE [
> > input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
> > May 10 10:53:30 qclsles02 tengine: [13270]: info: te_update_diff:
> > callbacks.c Aborting on transient_attributes deletions
> > May 10 10:53:30 qclsles02 crmd: [13261]: info: do_state_transition:fsa.cAll 1 cluster nodes are eligable to run resources.
> > May 10 10:53:30 qclsles02 cib: [13257]: info: cib_diff_notify:notify.cLocal-only Change (client:13261, call: 78):
> > 0.86.1483 (ok)
> > May 10 10:53:30 qclsles02 tengine: [13270]: info: te_update_diff:
> > callbacks.c Processing diff (cib_update): 0.86.1483 -> 0.86.1483
> > May 10 10:53:30 qclsles02 cib: [14292]: info: write_cib_contents:io.cWrote version
> > 0.86.1483 of the CIB to disk (digest: d8898f172a80d47c1f3637f78d67ba6a)
> > May 10 10:53:30 qclsles02 pengine: [13271]: info: process_pe_message:
> > [generation] <cib generated="true" admin_epoch="0" have_quorum="true"
> > num_peers="2" cib_feature_revision=" 1.3" epoch="86" num_updates="1483"
> > cib-last-written="Thu May 10 10:19:04 2007" ccm_transition="5"
> > dc_uuid="8b61d090-aa45-46af-9828-7b228c070e81"/>
> > May 10 10:53:30 qclsles02 pengine: [13271]: WARN: unpack_config:
> > unpack.c No value specified for cluster preference:
> > default_action_timeout
> > May 10 10:53:30 qclsles02 pengine: [13271]: info: unpack_config:unpack.cDefault stickiness: 1000000
> > May 10 10:53:30 qclsles02 pengine: [13271]: info: unpack_config:
> > unpack.c Default failure stickiness: -500
> > May 10 10:53:30 qclsles02 pengine: [13271]: info: unpack_config:unpack.cSTONITH of failed nodes is disabled
> > May 10 10:53:30 qclsles02 pengine: [13271]: info: unpack_config:
> > unpack.c STONITH will reboot nodes
> > May 10 10:53:30 qclsles02 pengine: [13271]: info: unpack_config:unpack.cCluster is symmetric - resources can run anywhere by default
> > May 10 10:53:30 qclsles02 pengine: [13271]: info: unpack_config:
> > unpack.c On loss of CCM Quorum: Stop ALL resources
> > May 10 10:53:30 qclsles02 pengine: [13271]: info: unpack_config:unpack.cOrphan resources are stopped
> > May 10 10:53:30 qclsles02 pengine: [13271]: info: unpack_config:
> > unpack.c Orphan resource actions are stopped
> > May 10 10:53:30 qclsles02 pengine: [13271]: WARN: unpack_config:unpack.cNo value specified for cluster preference: remove_after_stop
> > May 10 10:53:30 qclsles02 pengine: [13271]: info: unpack_config:
> > unpack.c Stopped resources are removed from the status section: false
> > May 10 10:53:30 qclsles02 pengine: [13271]: info: unpack_config:unpack.cBy default resources are managed
> > May 10 10:53:30 qclsles02 pengine: [13271]: info:
> > determine_online_status: unpack.c Node qclsles02 is online
> > May 10 10:53:30 qclsles02 pengine: [13271]: info: qclvmsles01
> > (heartbeat::ocf:Xen): Stopped
> > May 10 10:53:30 qclsles02 pengine: [13271]: notice: StartRsc:native.c
> > qclsles02 Start qclvmsles01
> > May 10 10:53:30 qclsles02 pengine: [13271]: notice: Recurring:native.cqclsles02 qclvmsles01_monitor_10000
> > May 10 10:53:30 qclsles02 pengine: [13271]: notice: stage8:allocate.cCreated transition graph 13.
> > May 10 10:53:30 qclsles02 pengine: [13271]: WARN: process_pe_message:
> > pengine.c No value specified for cluster preference: pe-input-series-max
> > May 10 10:53:30 qclsles02 crmd: [13261]: info: do_state_transition:fsa.cqclsles02: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [
> > input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
> > May 10 10:53:30 qclsles02 pengine: [13271]: info: process_pe_message:
> > pengine.c Transition 13: PEngine Input stored in:
> > /var/lib/heartbeat/pengine/pe- input-145.bz2
> > May 10 10:53:30 qclsles02 tengine: [13270]: info: unpack_graph:unpack.cUnpacked transition 13: 2 actions in 2 synapses
> > May 10 10:53:30 qclsles02 tengine: [13270]: info: send_rsc_command:
> > actions.c Initiating action 3: qclvmsles01_start_0 on qclsles02
> > May 10 10:53:30 qclsles02 crmd: [13261]: info: do_lrm_rsc_op:lrm.cPerforming op start on qclvmsles01 (interval=0ms,
> > key=13:0e2fe36b-9665-430d-985c-4a17a04c835b)
> > May 10 10:53:30 qclsles02 lrmd: [13258]: info: RA output:
> > (qclvmsles01:start:stderr) Error: the domain 'qclvmsles01' does not exist.
> > May 10 10:53:30 qclsles02 lrmd: [13258]: info: RA output:
> > (qclvmsles01:start:stderr)
> > May 10 10:53:30 qclsles02 lrmd: [13258]: info: RA output:
> > (qclvmsles01:start:stderr) /dev/sr0: No medium found
> > May 10 10:53:30 qclsles02 kernel: EXT3-fs: INFO: recovery required on
> > readonly filesystem.
> > May 10 10:53:30 qclsles02 kernel: EXT3-fs: write access will be enabled
> > during recovery.
> > May 10 10:53:31 qclsles02 kernel: (fs/jbd/recovery.c, 255):
> > journal_recover: JBD: recovery, exit status 0, recovered transactions 3982
> > to 4099
> > May 10 10:53:31 qclsles02 kernel: (fs/jbd/recovery.c, 257):
> > journal_recover: JBD: Replayed 1534 and revoked 8/25 blocks
> > May 10 10:53:31 qclsles02 kernel: kjournald starting. Commit interval 5
> > seconds
> > May 10 10:53:31 qclsles02 kernel: EXT3-fs: dm-0: orphan cleanup on
> > readonly fs
> > May 10 10:53:31 qclsles02 kernel: ext3_orphan_cleanup: deleting
> > unreferenced inode 738808
> > May 10 10:53:31 qclsles02 kernel: EXT3-fs: dm-0: 1 orphan inode deleted
> > May 10 10:53:31 qclsles02 kernel: EXT3-fs: recovery complete.
> > May 10 10:53:31 qclsles02 kernel: EXT3-fs: mounted filesystem with
> > ordered data mode.
> > May 10 10:53:33 qclsles02 logger: /etc/xen/scripts/block: add
> > XENBUS_PATH=backend/vbd/10/768
> > May 10 10:53:33 qclsles02 logger: /etc/xen/scripts/block: add
> > XENBUS_PATH=backend/vbd/10/5632
> > May 10 10:53:33 qclsles02 logger: /etc/xen/scripts/block: add
> > XENBUS_PATH=backend/vbd/10/832
> > May 10 10:53:33 qclsles02 logger: /etc/xen/scripts/vif-bridge: online
> > XENBUS_PATH=backend/vif/10/0
> > May 10 10:53:33 qclsles02 kernel: device vif10.0 entered promiscuous
> > mode
> > May 10 10:53:33 qclsles02 kernel: xenbr0: port 3(vif10.0) entering
> > learning state
> > May 10 10:53:33 qclsles02 kernel: xenbr0: topology change detected,
> > propagating
> > May 10 10:53:33 qclsles02 kernel: xenbr0: port 3( vif10.0) entering
> > forwarding state
> > May 10 10:53:33 qclsles02 logger: /etc/xen/scripts/vif-bridge:
> > Successful vif-bridge online for vif10.0, bridge xenbr0.
> > May 10 10:53:33 qclsles02 logger: /etc/xen/scripts/vif-bridge: Writing
> > backend/vif/10/0/hotplug-status connected to xenstore.
> > May 10 10:53:33 qclsles02 ifup: vif10.0
> > May 10 10:53:33 qclsles02 ifup: No configuration found for
> > vif10.0
> > May 10 10:53:33 qclsles02 logger: /etc/xen/scripts/block: Writing
> > backend/vbd/10/768/node /dev/loop0 to xenstore.
> > May 10 10:53:33 qclsles02 logger: /etc/xen/scripts/block: Writing
> > backend/vbd/10/768/physical-device 7:0 to xenstore.
> > May 10 10:53:33 qclsles02 logger: /etc/xen/scripts/block: Writing
> > backend/vbd/10/768/hotplug-status connected to xenstore.
> > May 10 10:53:33 qclsles02 logger: /etc/xen/scripts/block: Writing
> > backend/vbd/10/832/node /dev/loop1 to xenstore.
> > May 10 10:53:33 qclsles02 logger: /etc/xen/scripts/block: Writing
> > backend/vbd/10/832/physical-device 7:1 to xenstore.
> > May 10 10:53:33 qclsles02 logger: /etc/xen/scripts/block: Writing
> > backend/vbd/10/832/hotplug-status connected to xenstore.
> > May 10 10:53:34 qclsles02 logger: /etc/xen/scripts/block: Writing
> > backend/vbd/10/5632/physical-device b:0 to xenstore.
> > May 10 10:53:34 qclsles02 logger: /etc/xen/scripts/block: Writing
> > backend/vbd/10/5632/hotplug-status connected to xenstore.
> > May 10 10:53:34 qclsles02 kernel: vbd vbd-10-5632: 2 creating vbd
> > structure
> > May 10 10:53:34 qclsles02 lrmd: [13258]: info: RA output:
> > (qclvmsles01:start:stdout) Using config file "/etc/xen/vm/qclvm32sles01".
> > Started domain qclvmsles01
> > May 10 10:53:34 qclsles02 crmd: [13261]: info: process_lrm_event:lrm.cLRM operation (8) start_0 on qclvmsles01 complete
> > May 10 10:53:34 qclsles02 cib: [13257]: info: activateCibXml:io.c CIB
> > size is 70084 bytes (was 67792)
> > May 10 10:53:34 qclsles02 crmd: [13261]: info: do_lrm_rsc_op:lrm.cPerforming op monitor on qclvmsles01 (interval=10000ms,
> > key=13:0e2fe36b-9665-430d-985c-4a17a04c835b)
> > May 10 10:53:34 qclsles02 tengine: [13270]: info: te_update_diff:
> > callbacks.c Processing diff (cib_update): 0.86.1483 -> 0.86.1484
> > May 10 10:53:34 qclsles02 cib: [13257]: info: cib_diff_notify:notify.cUpdate (client: 13261, call:81):
> > 0.86.1483 -> 0.86.1484 (ok)
> > May 10 10:53:34 qclsles02 tengine: [13270]: info: match_graph_event:
> > events.c Action qclvmsles01_start_0 (3) confirmed
> > May 10 10:53:34 qclsles02 tengine: [13270]: info: send_rsc_command:
> > actions.c Initiating action 4: qclvmsles01_monitor_10000 on qclsles02
> > May 10 10:53:34 qclsles02 cib: [14689]: info: write_cib_contents: io.cWrote version
> > 0.86.1484 of the CIB to disk (digest: 010bf09fd9ffd13814a471d4e59f593f)
> > May 10 10:53:34 qclsles02 crmd: [13261]: info: process_lrm_event:lrm.cLRM operation (9) monitor_10000 on qclvmsles01 complete
> > May 10 10:53:34 qclsles02 cib: [13257]: info: activateCibXml:io.c CIB
> > size is 72376 bytes (was 70084)
> > May 10 10:53:34 qclsles02 cib: [13257]: info: cib_diff_notify:notify.cUpdate (client: 13261, call:82):
> > 0.86.1484 -> 0.86.1485 (ok)
> > May 10 10:53:34 qclsles02 tengine: [13270]: info: te_update_diff:
> > callbacks.c Processing diff (cib_update): 0.86.1484 -> 0.86.1485
> > May 10 10:53:34 qclsles02 tengine: [13270]: info: match_graph_event:
> > events.c Action qclvmsles01_monitor_10000 (4) confirmed
> > May 10 10:53:34 qclsles02 tengine: [13270]: info: run_graph:graph.cTransition 13: (Complete=2, Pending=0, Fired=0, Skipped=0, Incomplete=0)
> > May 10 10:53:34 qclsles02 crmd: [13261]: info: do_state_transition:
> > fsa.c qclsles02: State transition S_TRANSITION_ENGINE -> S_IDLE [
> > input=I_TE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
> > May 10 10:53:34 qclsles02 tengine: [13270]: info: notify_crmd:actions.cTransition 13 status: te_complete - (null)
> > May 10 10:53:34 qclsles02 cib: [14695]: info: write_cib_contents:io.cWrote version
> > 0.86.1485 of the CIB to disk (digest: 0a61d5132f247554b336456b28603627)
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > May 10 10:55:56 qclsles02 heartbeat: [13248]: info: Heartbeat restart on
> > node qclsles01
> > May 10 10:55:56 qclsles02 heartbeat: [13248]: info: Link qclsles01:eth0
> > up.
> > May 10 10:55:56 qclsles02 heartbeat: [13248]: info: Status update for
> > node qclsles01: status init
> > May 10 10:55:56 qclsles02 crmd: [13261]: notice:
> > crmd_ha_status_callback: callbacks.c Status update: Node qclsles01 now
> > has status [init]
> > May 10 10:55:56 qclsles02 heartbeat: [13248]: info: Status update for
> > node qclsles01: status up
> > May 10 10:55:56 qclsles02 crmd: [13261]: notice:
> > crmd_ha_status_callback: callbacks.c Status update: Node qclsles01 now
> > has status [up]
> > May 10 10:55:57 qclsles02 heartbeat: [13248]: info: all clients are now
> > paused
> > May 10 10:55:57 qclsles02 heartbeat: [13248]: debug: hist->ackseq =2241
> > May 10 10:55:57 qclsles02 heartbeat: [13248]: debug: hist->lowseq =2240,
> > hist->hiseq=2342
> > May 10 10:55:57 qclsles02 heartbeat: [13248]: debug:
> > May 10 10:55:58 qclsles02 heartbeat: [13248]: debug: hist->ackseq =2241
> > May 10 10:55:58 qclsles02 heartbeat: [13248]: debug: hist->lowseq =2240,
> > hist->hiseq=2343
> > May 10 10:55:58 qclsles02 heartbeat: [13248]: debug:
> > May 10 10:55:59 qclsles02 heartbeat: [13248]: debug: hist->ackseq =2241
> > May 10 10:55:59 qclsles02 heartbeat: [13248]: debug: hist->lowseq =2240,
> > hist->hiseq=2344
> > May 10 10:55:59 qclsles02 heartbeat: [13248]: debug:
> > May 10 10:56:00 qclsles02 heartbeat: [13248]: debug: hist->ackseq =2241
> > May 10 10:56:00 qclsles02 heartbeat: [13248]: debug: hist->lowseq =2240,
> > hist->hiseq=2345
> > May 10 10:56:00 qclsles02 heartbeat: [13248]: debug: expecting from
> > qclsles01
> > May 10 10:56:00 qclsles02 heartbeat: [13248]: debug: it's ackseq=0
> > May 10 10:56:00 qclsles02 heartbeat: [13248]: debug:
> > May 10 10:56:00 qclsles02 heartbeat: [13248]: info: all clients are now
> > resumed
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > May 10 10:56:25 qclsles02 heartbeat: [13248]: debug: get_delnodelist:
> > delnodelist=
> > May 10 10:56:25 qclsles02 heartbeat: [13248]: info: Status update for
> > node qclsles01: status active
> > May 10 10:56:25 qclsles02 crmd: [13261]: notice:
> > crmd_ha_status_callback:callbacks.c Status update: Node qclsles01 now
> > has status [active]
> > May 10 10:56:25 qclsles02 cib: [13257]: info: cib_diff_notify:notify.cLocal-only Change (client:13261, call: 83):
> > 0.86.1485 (ok)
> > May 10 10:56:25 qclsles02 cib: [13257]: info:
> > cib_client_status_callback:callbacks.c Status update: Client
> > qclsles01/cib now has status [join]
> > May 10 10:56:25 qclsles02 cib: [14779]: info: write_cib_contents:io.cWrote version
> > 0.86.1485 of the CIB to disk (digest: 42256113194a2c186ff6cf7bda6bb4a8)
> > May 10 10:56:25 qclsles02 tengine: [13270]: info: te_update_diff:
> > callbacks.c Processing diff (cib_update): 0.86.1485 -> 0.86.1485
> > May 10 10:56:27 qclsles02 heartbeat: [13248]: WARN: 1 lost packet(s) for
> > [qclsles01] [60:62]
> > May 10 10:56:27 qclsles02 heartbeat: [13248]: info: No pkts missing from
> > qclsles01!
> > May 10 10:56:27 qclsles02 crmd: [13261]: notice:
> > crmd_client_status_callback:callbacks.c Status update: Client
> > qclsles01/crmd now has status [online]
> > May 10 10:56:27 qclsles02 crmd: [13261]: info:
> > crmd_client_status_callback:callbacks.c Uncaching UUID for qclsles01
> > May 10 10:56:27 qclsles02 cib: [14785]: info: write_cib_contents:io.cWrote version
> > 0.86.1485 of the CIB to disk (digest: 554009bfdad70e34bb9d4cca2fdf8834)
> > May 10 10:56:28 qclsles02 heartbeat: [13248]: WARN: 1 lost packet(s) for
> > [qclsles01] [65:67]
> > May 10 10:56:28 qclsles02 heartbeat: [13248]: info: No pkts missing from
> > qclsles01!
> > May 10 10:56:29 qclsles02 ccm: [13256]: debug: quorum plugin: majority
> > May 10 10:56:29 qclsles02 ccm: [13256]: debug: cluster:linux-ha,
> > member_count=2, member_quorum_votes=200
> > May 10 10:56:29 qclsles02 crmd: [13261]: info: mem_handle_event: Got an
> > event OC_EV_MS_INVALID from ccm
> > May 10 10:56:29 qclsles02 ccm: [13256]: debug: total_node_count=2,
> > total_quorum_votes=200
> > May 10 10:56:29 qclsles02 crmd: [13261]: info: mem_handle_event: no
> > mbr_track info
> > May 10 10:56:29 qclsles02 crmd: [13261]: info: mem_handle_event: Got an
> > event OC_EV_MS_NEW_MEMBERSHIP from ccm
> > May 10 10:56:29 qclsles02 crmd: [13261]: info: mem_handle_event:
> > instance=6, nodes=2, new=1, lost=0, n_idx=0, new_idx=2, old_idx=4
> > May 10 10:56:29 qclsles02 cib: [13257]: info: mem_handle_event: Got an
> > event OC_EV_MS_INVALID from ccm
> > May 10 10:56:29 qclsles02 crmd: [13261]: info: crmd_ccm_msg_callback:
> > callbacks.c Quorum (re)attained after event=NEW MEMBERSHIP (id=6)
> > May 10 10:56:29 qclsles02 cib: [13257]: info: mem_handle_event: no
> > mbr_track info
> > May 10 10:56:29 qclsles02 crmd: [13261]: info: ccm_event_detail:ccm.cNEW MEMBERSHIP: trans=6, nodes=2, new=1, lost=0 n_idx=0, new_idx=2,
> > old_idx=4
> > May 10 10:56:29 qclsles02 cib: [13257]: info: mem_handle_event: Got an
> > event OC_EV_MS_NEW_MEMBERSHIP from ccm
> > May 10 10:56:29 qclsles02 crmd: [13261]: info: ccm_event_detail:ccm.c
> > CURRENT: qclsles02 [nodeid=1, born=1]
> > May 10 10:56:29 qclsles02 cib: [13257]: info: mem_handle_event:
> > instance=6, nodes=2, new=1, lost=0, n_idx=0, new_idx=2, old_idx=4
> > May 10 10:56:29 qclsles02 crmd: [13261]: info: ccm_event_detail:ccm.c
> > CURRENT: qclsles01 [nodeid=0, born=6]
> > May 10 10:56:29 qclsles02 cib: [13257]: info: cib_ccm_msg_callback:
> > callbacks.c PEER: qclsles02
> > May 10 10:56:29 qclsles02 crmd: [13261]: info: ccm_event_detail:ccm.c
> > NEW: qclsles01 [nodeid=0, born=6]
> > May 10 10:56:29 qclsles02 cib: [13257]: info: cib_ccm_msg_callback:
> > callbacks.c PEER: qclsles01
> > May 10 10:56:29 qclsles02 cib: [13257]: info: cib_diff_notify:notify.cLocal-only Change (client:13261, call: 85):
> > 0.86.1485 (ok)
> > May 10 10:56:29 qclsles02 tengine: [13270]: info: te_update_diff:
> > callbacks.c Processing diff (cib_update): 0.86.1485 -> 0.86.1485
> > May 10 10:56:29 qclsles02 cib: [14786]: info: write_cib_contents:io.cWrote version
> > 0.86.1485 of the CIB to disk (digest: bb5126a9c96453b5e33b20ee1bf4d74b)
> > May 10 10:56:32 qclsles02 crmd: [13261]: info: do_state_transition:fsa.cqclsles02: State transition S_IDLE -> S_INTEGRATION [ input=I_NODE_JOIN
> > cause=C_HA_MESSAGE origin=route_message ]
> > May 10 10:56:32 qclsles02 crmd: [13261]: info: update_dc: utils.c Set DC
> > to <null> (<null>)
> > May 10 10:56:32 qclsles02 tengine: [13270]: info: update_abort_priority:
> > utils.c Abort priority upgraded to 1000000
> > May 10 10:56:32 qclsles02 crmd: [13261]: info:
> > do_dc_join_offer_all:join_dc.c join-5: Waiting on 2 outstanding join acks
> > May 10 10:56:32 qclsles02 crmd: [13261]: info: update_dc:utils.c Set DC
> > to qclsles02 (1.0.6)
> > May 10 10:56:33 qclsles02 crmd: [13261]: info: do_state_transition:fsa.cqclsles02: State transition S_INTEGRATION -> S_FINALIZE_JOIN [
> > input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ]
> > May 10 10:56:33 qclsles02 crmd: [13261]: info: do_state_transition:fsa.cAll 2 cluster nodes responded to the join offer.
> > May 10 10:56:33 qclsles02 cib: [13257]: info: sync_our_cib:messages.cSyncing CIB to all peers
> > May 10 10:56:33 qclsles02 attrd: [13260]: info: attrd_local_callback:
> > attrd.c Sending full refresh
> > May 10 10:56:33 qclsles02 cib: [13257]: info: cib_diff_notify:notify.cUpdate (client: 13261, call:88):
> > 0.86.1485 -> 0.86.1486 (ok)
> > May 10 10:56:33 qclsles02 tengine: [13270]: info: te_update_diff:
> > callbacks.c Processing diff (cib_update): 0.86.1485 -> 0.86.1486
> > May 10 10:56:33 qclsles02 cib: [13257]: info: cib_diff_notify:notify.cUpdate (client: 13261, call:89):
> > 0.86.1486 -> 0.87.1487 (ok)
> > May 10 10:56:33 qclsles02 crmd: [13261]: info: update_dc:utils.c Set DC
> > to qclsles02 (1.0.6)
> > May 10 10:56:33 qclsles02 tengine: [13270]: info: te_update_diff:
> > callbacks.c Processing diff (cib_bump): 0.86.1486 -> 0.87.1487
> > May 10 10:56:33 qclsles02 cib: [13257]: info: cib_diff_notify:notify.cUpdate (client: 13261, call:90):
> > 0.87.1487 -> 0.87.1488 (ok)
> > May 10 10:56:33 qclsles02 tengine: [13270]: info: te_update_diff:
> > callbacks.c Processing diff (cib_update): 0.87.1487 -> 0.87.1488
> > May 10 10:56:33 qclsles02 cib: [13257]: info: cib_diff_notify:notify.cUpdate (client: 13261, call:91):
> > 0.87.1488 -> 0.87.1489 (ok)
> > May 10 10:56:33 qclsles02 tengine: [13270]: info: te_update_diff:
> > callbacks.c Processing diff (cib_update): 0.87.1488 -> 0.87.1489
> > May 10 10:56:33 qclsles02 cib: [14787]: info: write_cib_contents:io.cWrote version
> > 0.87.1489 of the CIB to disk (digest: 3ec92be96aef12af63563bffe02488bc)
> > May 10 10:56:33 qclsles02 crmd: [13261]: info: do_dc_join_ack:join_dc.c
> > join-5: Updating node state to member for qclsles02)
> > May 10 10:56:33 qclsles02 cib: [13257]: info: cib_diff_notify:notify.cUpdate (client: 13261, call:92):
> > 0.87.1489 -> 0.87.1490 (ok)
> > May 10 10:56:33 qclsles02 tengine: [13270]: info: te_update_diff:
> > callbacks.c Processing diff (cib_update): 0.87.1489 -> 0.87.1490
> > May 10 10:56:33 qclsles02 cib: [14788]: info: write_cib_contents: io.cWrote version
> > 0.87.1490 of the CIB to disk (digest: f73cd23d04ca932db2dda02428da956d)
> > May 10 10:56:34 qclsles02 crmd: [13261]: info: do_dc_join_ack:join_dc.c
> > join-5: Updating node state to member for qclsles01)
> > May 10 10:56:34 qclsles02 cib: [13257]: info: cib_diff_notify:notify.cUpdate (client: 13261, call:93):
> > 0.87.1490 -> 0.87.1491 (ok)
> > May 10 10:56:34 qclsles02 tengine: [13270]: info: te_update_diff:
> > callbacks.c Processing diff (cib_update): 0.87.1490 -> 0.87.1491
> > May 10 10:56:34 qclsles02 crmd: [13261]: info: do_state_transition:fsa.cqclsles02: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE [
> > input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state ]
> > May 10 10:56:34 qclsles02 crmd: [13261]: info: do_state_transition:fsa.cAll 2 cluster nodes are eligable to run resources.
> > May 10 10:56:34 qclsles02 cib: [14789]: info: write_cib_contents:io.cWrote version
> > 0.87.1491 of the CIB to disk (digest: e71abb0623f45a22564d4a722bf9fea1)
> > May 10 10:56:34 qclsles02 pengine: [13271]: info: process_pe_message:
> > [generation] <cib generated="true" admin_epoch="0" have_quorum="true"
> > num_peers="2" cib_feature_revision=" 1.3" epoch="87" num_updates="1491"
> > cib-last-written="Thu May 10 10:19:04 2007" ccm_transition="6"
> > dc_uuid="8b61d090-aa45-46af-9828-7b228c070e81"/>
> > May 10 10:56:34 qclsles02 pengine: [13271]: WARN: unpack_config:
> > unpack.c No value specified for cluster preference:
> > default_action_timeout
> > May 10 10:56:34 qclsles02 pengine: [13271]: info: unpack_config:unpack.cDefault stickiness: 1000000
> > May 10 10:56:34 qclsles02 pengine: [13271]: info: unpack_config:
> > unpack.c Default failure stickiness: -500
> > May 10 10:56:34 qclsles02 pengine: [13271]: info: unpack_config:unpack.cSTONITH of failed nodes is disabled
> > May 10 10:56:34 qclsles02 pengine: [13271]: info: unpack_config:
> > unpack.c STONITH will reboot nodes
> > May 10 10:56:34 qclsles02 pengine: [13271]: info: unpack_config:unpack.cCluster is symmetric - resources can run anywhere by default
> > May 10 10:56:34 qclsles02 pengine: [13271]: info: unpack_config:
> > unpack.c On loss of CCM Quorum: Stop ALL resources
> > May 10 10:56:34 qclsles02 pengine: [13271]: info: unpack_config:unpack.cOrphan resources are stopped
> > May 10 10:56:34 qclsles02 pengine: [13271]: info: unpack_config:
> > unpack.c Orphan resource actions are stopped
> > May 10 10:56:34 qclsles02 pengine: [13271]: WARN: unpack_config:unpack.cNo value specified for cluster preference: remove_after_stop
> > May 10 10:56:34 qclsles02 pengine: [13271]: info: unpack_config:
> > unpack.c Stopped resources are removed from the status section: false
> > May 10 10:56:34 qclsles02 pengine: [13271]: info: unpack_config:unpack.cBy default resources are managed
> > May 10 10:56:34 qclsles02 pengine: [13271]: info:
> > determine_online_status: unpack.c Node qclsles02 is online
> > May 10 10:56:34 qclsles02 pengine: [13271]: info:
> > determine_online_status:unpack.c Node qclsles01 is online
> > May 10 10:56:34 qclsles02 pengine: [13271]: ERROR: native_add_running:
> > native.c Resource ocf::Xen:qclvmsles01 appears to be active on 2 nodes.
> > May 10 10:56:34 qclsles02 pengine: [13271]: ERROR: See http://linux-ha.org/v2/faq/resource_too_active
> > for more information.
> > May 10 10:56:34 qclsles02 pengine: [13271]: info: qclvmsles01
> > (heartbeat::ocf:Xen)
> > May 10 10:56:34 qclsles02 pengine: [13271]: info: 0 : qclsles02
> > May 10 10:56:34 qclsles02 pengine: [13271]: info: 1 : qclsles01
> > May 10 10:56:34 qclsles02 pengine: [13271]: WARN: choose_node_from_list:
> > allocate.c 2 nodes with equal score (+INFINITY) for running the listed
> > resources (chose qclsles02):
> > May 10 10:56:34 qclsles02 pengine: [13271]: WARN: qclvmsles01
> > (heartbeat::ocf:Xen)
> > May 10 10:56:34 qclsles02 pengine: [13271]: ERROR:
> > native_create_actions:native.c Attempting recovery of resource
> > qclvmsles01
> > May 10 10:56:34 qclsles02 pengine: [13271]: notice: StopRsc:native.c
> > qclsles02 Stop qclvmsles01
> > May 10 10:56:34 qclsles02 pengine: [13271]: notice: StopRsc:native.c
> > qclsles01 Stop qclvmsles01
> > May 10 10:56:34 qclsles02 pengine: [13271]: notice: StartRsc:native.c
> > qclsles02 Start qclvmsles01
> > May 10 10:56:34 qclsles02 pengine: [13271]: notice: Recurring:native.cqclsles02 qclvmsles01_monitor_10000
> > May 10 10:56:34 qclsles02 pengine: [13271]: notice: stage8:allocate.cCreated transition graph 14.
> > May 10 10:56:34 qclsles02 pengine: [13271]: WARN: process_pe_message:
> > pengine.c No value specified for cluster preference: pe-error-series-max
> > May 10 10:56:34 qclsles02 pengine: [13271]: ERROR: process_pe_message:
> > pengine.c Transition 14: ERRORs found during PE processing. PEngine
> > Input stored in: /var/lib/heartbeat/pengine/pe-error-3.bz2
> > May 10 10:56:34 qclsles02 crmd: [13261]: info: do_state_transition:fsa.cqclsles02: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [
> > input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
> > May 10 10:56:34 qclsles02 tengine: [13270]: info: unpack_graph:unpack.cUnpacked transition 14: 5 actions in 5 synapses
> > May 10 10:56:34 qclsles02 tengine: [13270]: info: send_rsc_command:
> > actions.c Initiating action 6: qclvmsles01_stop_0 on qclsles02
> > May 10 10:56:34 qclsles02 tengine: [13270]: info: send_rsc_command:
> > actions.c Initiating action 7: qclvmsles01_stop_0 on qclsles01
> > May 10 10:56:34 qclsles02 tengine: [13270]: info: send_rsc_command:
> > actions.c Initiating action 5: probe_complete on qclsles01
> > May 10 10:56:34 qclsles02 crmd: [13261]: info: do_lrm_rsc_op:lrm.cPerforming op stop on qclvmsles01 (interval=0ms,
> > key=14:0e2fe36b-9665-430d-985c-4a17a04c835b)
> > May 10 10:56:34 qclsles02 crmd: [13261]: WARN: process_lrm_event: lrm.cLRM operation (9) monitor_10000 on qclvmsles01 Cancelled
> > May 10 10:56:34 qclsles02 lrmd: [13258]: info: RA output:
> > (qclvmsles01:stop:stdout) Name ID Mem(MiB)
> > VCPUs State Time(s) qclvmsles01 10 1024 1
> > -b---- 14.6
> > May 10 10:56:34 qclsles02 mgmtd: [13262]: ERROR: native_add_running:
> > native.c Resource ocf::Xen:qclvmsles01 appears to be active on 2 nodes.
> > May 10 10:56:34 qclsles02 mgmtd: [13262]: ERROR: See
> > http://linux-ha.org/v2/faq/resource_too_active for more information.
> > May 10 10:56:35 qclsles02 cib: [13257]: info: activateCibXml:io.c CIB
> > size is 77200 bytes (was 72376)
> > May 10 10:56:35 qclsles02 tengine: [13270]: info: te_update_diff:
> > callbacks.c Processing diff (cib_update): 0.87.1491 -> 0.87.1492
> > May 10 10:56:35 qclsles02 cib: [13257]: info: cib_diff_notify:notify.cUpdate (client: 8142, call:13):
> > 0.87.1491 -> 0.87.1492 (ok)
> > May 10 10:56:35 qclsles02 tengine: [13270]: info: extract_event:
> > events.c Aborting on transient_attributes changes
> > May 10 10:56:35 qclsles02 tengine: [13270]: info: update_abort_priority:
> > utils.c Abort priority upgraded to 1000000
> > May 10 10:56:35 qclsles02 tengine: [13270]: info: update_abort_priority:
> > utils.c Abort action 0 superceeded by 2
> > May 10 10:56:35 qclsles02 cib: [13257]: info: activateCibXml:io.c CIB
> > size is 79492 bytes (was 77200)
> > May 10 10:56:35 qclsles02 cib: [13257]: info: cib_diff_notify:notify.cUpdate (client: 8142, call:14):
> > 0.87.1492 -> 0.87.1493 (ok)
> > May 10 10:56:35 qclsles02 tengine: [13270]: info: te_update_diff:
> > callbacks.c Processing diff (cib_update): 0.87.1492 -> 0.87.1493
> > May 10 10:56:35 qclsles02 tengine: [13270]: info: match_graph_event:
> > events.c Action qclvmsles01_stop_0 (7) confirmed
> > May 10 10:56:35 qclsles02 cib: [14800]: info: write_cib_contents:io.cWrote version
> > 0.87.1493 of the CIB to disk (digest: 154a1fde74efaf02b46ae3e112385e80)
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > May 10 10:56:49 qclsles02 kernel: xenbr0: port 3(vif10.0) entering
> > disabled state
> > May 10 10:56:49 qclsles02 kernel: device vif10.0 left promiscuous mode
> > May 10 10:56:49 qclsles02 kernel: xenbr0: port 3( vif10.0) entering
> > disabled state
> > May 10 10:56:49 qclsles02 logger: /etc/xen/scripts/vif-bridge: offline
> > XENBUS_PATH=backend/vif/10/0
> > May 10 10:56:49 qclsles02 logger: /etc/xen/scripts/block: remove
> > XENBUS_PATH=backend/vbd/10/5632
> > May 10 10:56:49 qclsles02 logger: /etc/xen/scripts/block: remove
> > XENBUS_PATH=backend/vbd/10/832
> > May 10 10:56:49 qclsles02 logger: /etc/xen/scripts/block: remove
> > XENBUS_PATH=backend/vbd/10/768
> > May 10 10:56:49 qclsles02 logger: /etc/xen/scripts/vif-bridge: brctl
> > delif xenbr0 vif10.0 failed
> > May 10 10:56:49 qclsles02 logger: /etc/xen/scripts/vif-bridge: ifconfig
> > vif10.0 down failed
> > May 10 10:56:49 qclsles02 logger: /etc/xen/scripts/xen-hotplug-cleanup:
> > XENBUS_PATH=backend/vbd/10/5632
> > May 10 10:56:49 qclsles02 logger: /etc/xen/scripts/xen-hotplug-cleanup:
> > XENBUS_PATH=backend/vbd/10/832
> > May 10 10:56:49 qclsles02 logger: /etc/xen/scripts/vif-bridge:
> > Successful vif-bridge offline for vif10.0, bridge xenbr0.
> > May 10 10:56:49 qclsles02 ifdown: vif10.0
> > May 10 10:56:49 qclsles02 logger: /etc/xen/scripts/xen-hotplug-cleanup:
> > XENBUS_PATH=backend/vif/10/0
> > May 10 10:56:49 qclsles02 ifdown: Interface not available and no
> > configuration found.
> > May 10 10:56:49 qclsles02 logger: /etc/xen/scripts/xen-hotplug-cleanup:
> > XENBUS_PATH=backend/vbd/10/768
> > May 10 10:56:50 qclsles02 lrmd: [13258]: info: RA output:
> > (qclvmsles01:stop:stderr) Error: the domain 'qclvmsles01' does not exist.
> > May 10 10:56:50 qclsles02 lrmd: [13258]: info: RA output:
> > (qclvmsles01:stop:stdout) Domain qclvmsles01 terminated
> > May 10 10:56:50 qclsles02 crmd: [13261]: WARN: process_lrm_event:lrm.cLRM operation (11) stop_0 on qclvmsles01 Error: (4) insufficient privileges
> > May 10 10:56:50 qclsles02 cib: [13257]: info: activateCibXml:io.c CIB
> > size is 81784 bytes (was 79492)
> > May 10 10:56:50 qclsles02 cib: [13257]: info: cib_diff_notify:notify.cUpdate (client: 13261, call:95):
> > 0.87.1493 -> 0.87.1494 (ok)
> > May 10 10:56:50 qclsles02 tengine: [13270]: info: te_update_diff:
> > callbacks.c Processing diff (cib_update): 0.87.1493 -> 0.87.1494
> > May 10 10:56:50 qclsles02 tengine: [13270]: ERROR: match_graph_event:
> > events.c Action qclvmsles01_stop_0 on qclsles02 failed (target: 0 vs.
> > rc: 4): Error
> > May 10 10:56:50 qclsles02 tengine: [13270]: info: match_graph_event:
> > events.c Action qclvmsles01_stop_0 (6) confirmed
> > May 10 10:56:50 qclsles02 tengine: [13270]: info: run_graph: graph.c====================================================
> > May 10 10:56:50 qclsles02 tengine: [13270]: notice: run_graph:graph.cTransition 14: (Complete=3, Pending=0, Fired=0, Skipped=2, Incomplete=0)
> > May 10 10:56:50 qclsles02 crmd: [13261]: info: do_state_transition:
> > fsa.c qclsles02: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE
> > [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
> > May 10 10:56:50 qclsles02 crmd: [13261]: info: do_state_transition:fsa.cAll 2 cluster nodes are eligable to run resources.
> > May 10 10:56:50 qclsles02 cib: [15075]: info: write_cib_contents:io.cWrote version
> > 0.87.1494 of the CIB to disk (digest: f0bfce8e6e3fc3b7ee40ba76a841395a)
> > May 10 10:56:50 qclsles02 pengine: [13271]: info: process_pe_message:
> > [generation] <cib generated="true" admin_epoch="0" have_quorum="true"
> > num_peers="2" cib_feature_revision=" 1.3" epoch="87" num_updates="1494"
> > cib-last-written="Thu May 10 10:19:04 2007" ccm_transition="6"
> > dc_uuid="8b61d090-aa45-46af-9828-7b228c070e81"/>
> > May 10 10:56:50 qclsles02 pengine: [13271]: WARN: unpack_config:
> > unpack.c No value specified for cluster preference:
> > default_action_timeout
> > May 10 10:56:50 qclsles02 pengine: [13271]: info: unpack_config:unpack.cDefault stickiness: 1000000
> > May 10 10:56:50 qclsles02 pengine: [13271]: info: unpack_config:
> > unpack.c Default failure stickiness: -500
> > May 10 10:56:50 qclsles02 pengine: [13271]: info: unpack_config:unpack.cSTONITH of failed nodes is disabled
> > May 10 10:56:50 qclsles02 pengine: [13271]: info: unpack_config:
> > unpack.c STONITH will reboot nodes
> > May 10 10:56:50 qclsles02 pengine: [13271]: info: unpack_config:unpack.cCluster is symmetric - resources can run anywhere by default
> > May 10 10:56:50 qclsles02 pengine: [13271]: info: unpack_config:
> > unpack.c On loss of CCM Quorum: Stop ALL resources
> > May 10 10:56:50 qclsles02 pengine: [13271]: info: unpack_config:unpack.cOrphan resources are stopped
> > May 10 10:56:50 qclsles02 pengine: [13271]: info: unpack_config:
> > unpack.c Orphan resource actions are stopped
> > May 10 10:56:50 qclsles02 pengine: [13271]: WARN: unpack_config:unpack.cNo value specified for cluster preference: remove_after_stop
> > May 10 10:56:50 qclsles02 pengine: [13271]: info: unpack_config:
> > unpack.c Stopped resources are removed from the status section: false
> > May 10 10:56:50 qclsles02 pengine: [13271]: info: unpack_config:unpack.cBy default resources are managed
> > May 10 10:56:50 qclsles02 pengine: [13271]: info:
> > determine_online_status: unpack.c Node qclsles02 is online
> > May 10 10:56:50 qclsles02 pengine: [13271]: WARN: unpack_rsc_op:unpack.cProcessing failed op (qclvmsles01_stop_0) for qclvmsles01 on qclsles02May 10
> > 10:56:50 qclsles02 pengine: [13271]: WARN: unpack_rsc_op: unpack.cHandling failed stop for qclvmsles01 on qclsles02
> > May 10 10:56:50 qclsles02 pengine: [13271]: info:
> > determine_online_status:unpack.c Node qclsles01 is online
> > May 10 10:56:50 qclsles02 pengine: [13271]: info: qclvmsles01
> > (heartbeat::ocf:Xen): Started qclsles02 (unmanaged) FAILED
> > May 10 10:56:50 qclsles02 pengine: [13271]: notice: NoRoleChange:
> > native.c Move resource qclvmsles01 (qclsles02 -> qclsles01)
> > May 10 10:56:50 qclsles02 pengine: [13271]: WARN: custom_action:utils.cAction qclvmsles01_stop_0 stop is for qclvmsles01 (unmanaged)
> > May 10 10:56:50 qclsles02 pengine: [13271]: WARN: custom_action:utils.cAction qclvmsles01_start_0 start is for qclvmsles01 (unmanaged)
> > May 10 10:56:50 qclsles02 pengine: [13271]: notice: stage8:allocate.cCreated transition graph 15.
> > May 10 10:56:50 qclsles02 pengine: [13271]: WARN: process_pe_message:
> > pengine.c No value specified for cluster preference: pe-input-series-max
> > May 10 10:56:50 qclsles02 pengine: [13271]: info: process_pe_message:
> > pengine.c Transition 15: PEngine Input stored in:
> > /var/lib/heartbeat/pengine/pe-input-146.bz2
> > May 10 10:56:50 qclsles02 crmd: [13261]: info: do_state_transition:fsa.cqclsles02: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [
> > input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
> > May 10 10:56:50 qclsles02 tengine: [13270]: info: unpack_graph:unpack.cUnpacked transition 15: 0 actions in 0 synapses
> > May 10 10:56:50 qclsles02 tengine: [13270]: info: run_graph:graph.cTransition 15: (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0)
> > May 10 10:56:50 qclsles02 crmd: [13261]: info: do_state_transition:fsa.cqclsles02: State transition S_TRANSITION_ENGINE -> S_IDLE [
> > input=I_TE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
> > May 10 10:56:50 qclsles02 tengine: [13270]: info: notify_crmd: actions.cTransition 15 status: te_complete - (null)
> >
> > EOF
> >
> >
> >
> > someone understand where I'm wrong ?!?!
> >
> >
> > --
> > René Jr Purcell
> > Chargé de projet, sécurité et sytèmes
> > Techno Centre Logiciels Libres, http://www.tc2l.ca/
> > Téléphone : (418) 681-2929 #124
>
>
>
>
> --
> René Jr Purcell
> Chargé de projet, sécurité et sytèmes
> Techno Centre Logiciels Libres, http://www.tc2l.ca/
> Téléphone : (418) 681-2929 #124
>
--
René Jr Purcell
Chargé de projet, sécurité et sytèmes
Techno Centre Logiciels Libres, http://www.tc2l.ca/
Téléphone : (418) 681-2929 #124
More information about the Linux-HA
mailing list