[Linux-HA] Resource sticking on wrong node.
Birju Prajapati
floating.buddha at gmail.com
Wed Feb 7 07:17:45 MST 2007
Hi all,
I have a a simple virtual ip resource in an active/passive config.
However, after a failover, it is sticking on the wrong node. I have
stopped the resource and run
crm_resource -C -r WebServerIP
The restarted.
But this does not do anything - it still sticks on the wrong node.
I've tried it with 'auto_failback on' too. Could anyone give any
pointers?
Many thanks,
Birju
It seems to be with a bit in the logs saying NoRoleChange. Here are my
config and primary log files when I start heartbeat on both the nodes
(passive after active):
-----------------------------------------------------------------------------------------------
/etc/ha.d/ha.cf:
logfacility local0
bcast eth0 # Linux
auto_failback off
crm on
node etch1 #where the virtual ip should going to
node etch2
-----------------------------------------------------------------------------------------------
bprajapati at etch1:~$ sudo cibadmin -Q
Password:
<cib admin_epoch="0" have_quorum="true" num_peers="2"
cib_feature_revision="1.3" ccm_transition="10" generated="true"
dc_uuid="7009cce3-2480-4c31-8a61-523ed335b4d2" epoch="28"
num_updates="317">
<configuration>
<crm_config>
<cluster_property_set id="cib-bootstrap-options">
<attributes>
<nvpair name="last-lrm-refresh"
id="cib-bootstrap-options-last-lrm-refresh" value="1170853879"/>
</attributes>
</cluster_property_set>
</crm_config>
<nodes>
<node id="7009cce3-2480-4c31-8a61-523ed335b4d2" uname="etch2"
type="normal"/>
<node id="5198e43b-d32a-4361-8808-2bd4e2cab731" uname="etch1"
type="normal"/>
</nodes>
<resources>
<primitive class="ocf" type="IPaddr" provider="heartbeat"
id="WebServerIP">
<instance_attributes id="fae89f69-03ce-4fe8-adbc-663439c3da71">
<attributes>
<nvpair id="WebServerIP_ip" name="ip" value="172.16.28.128"/>
<nvpair name="target_role" id="wsid-role" value="Started"/>
</attributes>
</instance_attributes>
</primitive>
</resources>
<constraints>
<rsc_location rsc="ip_resource" id="run_ip_resource">
<rule score="100" id="pref_run_ip_resource">
<expression attribute="#uname" operation="eq"
id="bea57266-4aa5-4e4a-8349-ef4383c070a8" value="etch1"/>
</rule>
</rsc_location>
</constraints>
</configuration>
<status>
<node_state uname="etch1" crmd="online" in_ccm="true" ha="active"
join="member" id="5198e43b-d32a-4361-8808-2bd4e2cab731" shutdown="0"
expected="member" crm-debug-origin="do_update_resource">
<transient_attributes id="5198e43b-d32a-4361-8808-2bd4e2cab731">
<instance_attributes id="status-5198e43b-d32a-4361-8808-2bd4e2cab731">
<attributes>
<nvpair
id="status-5198e43b-d32a-4361-8808-2bd4e2cab731-probe_complete"
name="probe_complete" value="true"/>
</attributes>
</instance_attributes>
</transient_attributes>
<lrm id="5198e43b-d32a-4361-8808-2bd4e2cab731">
<lrm_resources>
<lrm_resource id="WebServerIP" type="IPaddr" class="ocf"
provider="heartbeat">
<lrm_rsc_op id="WebServerIP_monitor_0"
operation="monitor" crm-debug-origin="do_update_resource"
transition_key="8:9bbde5e8-591a-4ab2-95ba-b4182aadcd27"
transition_magic="4:7;8:9bbde5e8-591a-4ab2-95ba-b4182aadcd27"
call_id="4" crm_feature_set="1.0.6" rc_code="7" op_status="4"
interval="0" op_digest="11e41d6fe9dfd0cab0fc9e14f65b2589"/>
</lrm_resource>
</lrm_resources>
</lrm>
</node_state>
<node_state uname="etch2" ha="active" crmd="online" in_ccm="true"
join="member" shutdown="0" id="7009cce3-2480-4c31-8a61-523ed335b4d2"
expected="member" crm-debug-origin="do_update_resource">
<transient_attributes id="7009cce3-2480-4c31-8a61-523ed335b4d2">
<instance_attributes id="status-7009cce3-2480-4c31-8a61-523ed335b4d2">
<attributes>
<nvpair
id="status-7009cce3-2480-4c31-8a61-523ed335b4d2-probe_complete"
name="probe_complete" value="true"/>
</attributes>
</instance_attributes>
</transient_attributes>
<lrm id="7009cce3-2480-4c31-8a61-523ed335b4d2">
<lrm_resources>
<lrm_resource id="WebServerIP" type="IPaddr" class="ocf"
provider="heartbeat">
<lrm_rsc_op id="WebServerIP_monitor_0"
operation="monitor" crm-debug-origin="do_update_resource"
transition_key="8:9bbde5e8-591a-4ab2-95ba-b4182aadcd27"
transition_magic="4:7;8:9bbde5e8-591a-4ab2-95ba-b4182aadcd27"
call_id="8" crm_feature_set="1.0.6" rc_code="7" op_status="4"
interval="0" op_digest="11e41d6fe9dfd0cab0fc9e14f65b2589"/>
<lrm_rsc_op id="WebServerIP_start_0" operation="start"
crm-debug-origin="do_update_resource"
transition_key="9:9bbde5e8-591a-4ab2-95ba-b4182aadcd27"
transition_magic="0:0;9:9bbde5e8-591a-4ab2-95ba-b4182aadcd27"
call_id="9" crm_feature_set="1.0.6" rc_code="0" op_status="0"
interval="0" op_digest="e9b2d366a1de167cd6ba98615ea0d42c"/>
</lrm_resource>
</lrm_resources>
</lrm>
</node_state>
</status>
</cib>
------------------------------------------------------------------------------------
syslog on etch1 after starting heartbeat:
Feb 7 14:09:20 etch1 logd: [11553]: info: logd started with default
configuration.
Feb 7 14:09:20 etch1 logd: [11553]: info: G_main_add_SignalHandler:
Added signal handler for signal 15
Feb 7 14:09:20 etch1 logd: [11554]: info: G_main_add_SignalHandler:
Added signal handler for signal 15
Feb 7 14:09:21 etch1 heartbeat: [11574]: WARN: Logging daemon is
disabled --enabling logging daemon is recommended
Feb 7 14:09:21 etch1 heartbeat: [11574]: info: **************************
Feb 7 14:09:21 etch1 heartbeat: [11574]: info: Configuration
validated. Starting heartbeat 2.0.7
Feb 7 14:09:21 etch1 heartbeat: [11575]: info: heartbeat: version 2.0.7
Feb 7 14:09:21 etch1 heartbeat: [11575]: info: Heartbeat generation: 24
Feb 7 14:09:21 etch1 heartbeat: [11575]: info:
G_main_add_TriggerHandler: Added signal manual handler
Feb 7 14:09:21 etch1 heartbeat: [11575]: info:
G_main_add_TriggerHandler: Added signal manual handler
Feb 7 14:09:21 etch1 heartbeat: [11575]: info: Removing
/var/run/heartbeat/rsctmp failed, recreating.
Feb 7 14:09:21 etch1 heartbeat: [11575]: info: glib: UDP Broadcast
heartbeat started on port 694 (694) interface eth0
Feb 7 14:09:21 etch1 heartbeat: [11575]: info: glib: UDP Broadcast
heartbeat closed on port 694 interface eth0 - Status: 1
Feb 7 14:09:21 etch1 heartbeat: [11575]: info:
G_main_add_SignalHandler: Added signal handler for signal 17
Feb 7 14:09:21 etch1 heartbeat: [11575]: info: Local status now set to: 'up'
Feb 7 14:09:22 etch1 heartbeat: [11575]: info: Link etch1:eth0 up.
Feb 7 14:09:51 etch1 heartbeat: [11575]: WARN: node etch2: is dead
Feb 7 14:09:51 etch1 heartbeat: [11575]: info: Comm_now_up():
updating status to active
Feb 7 14:09:51 etch1 heartbeat: [11575]: info: Local status now set
to: 'active'
Feb 7 14:09:51 etch1 heartbeat: [11575]: info: Starting child client
"/usr/lib/heartbeat/ccm" (104,106)
Feb 7 14:09:51 etch1 heartbeat: [11575]: info: Starting child client
"/usr/lib/heartbeat/cib" (104,106)
Feb 7 14:09:51 etch1 heartbeat: [11575]: info: Starting child client
"/usr/lib/heartbeat/lrmd" (0,0)
Feb 7 14:09:51 etch1 heartbeat: [11575]: info: Starting child client
"/usr/lib/heartbeat/stonithd" (0,0)
Feb 7 14:09:51 etch1 heartbeat: [11575]: info: Starting child client
"/usr/lib/heartbeat/attrd" (104,106)
Feb 7 14:09:51 etch1 heartbeat: [11575]: info: Starting child client
"/usr/lib/heartbeat/crmd" (104,106)
Feb 7 14:09:51 etch1 heartbeat: [11575]: info: Starting child client
"/usr/lib/heartbeat/mgmtd -v" (0,0)
Feb 7 14:09:51 etch1 heartbeat: [11583]: info: Starting
"/usr/lib/heartbeat/ccm" as uid 104 gid 106 (pid 11583)
Feb 7 14:09:51 etch1 heartbeat: [11584]: info: Starting
"/usr/lib/heartbeat/cib" as uid 104 gid 106 (pid 11584)
Feb 7 14:09:51 etch1 cib: [11584]: info: G_main_add_SignalHandler:
Added signal handler for signal 15
Feb 7 14:09:51 etch1 cib: [11584]: info: G_main_add_TriggerHandler:
Added signal manual handler
Feb 7 14:09:51 etch1 cib: [11584]: info: G_main_add_SignalHandler:
Added signal handler for signal 17
Feb 7 14:09:51 etch1 cib: [11584]: info: main:main.c Retrieval of a
per-action CIB: disabled
Feb 7 14:09:51 etch1 cib: [11584]: info: cib_register_ha:main.c
Signing in with Heartbeat
Feb 7 14:09:51 etch1 cib: [11584]: info: cib_register_ha:main.c FSA
Hostname: etch1
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile:io.c Reading
cluster configuration from: /var/lib/heartbeat/crm/cib.xml
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
<cib admin_epoch="0" have_quorum="false" num_peers="2"
cib_feature_revision="1.3" generated="true" epoch="30"
num_updates="346" cib-last-written="Wed Feb 7 14:01:10 2007"
ccm_transition="2">
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
<configuration>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
<crm_config>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
<cluster_property_set id="cib-bootstrap-options">
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
<attributes>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
<nvpair name="last-lrm-refresh"
id="cib-bootstrap-options-last-lrm-refresh" value="1170853879"/>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
</attributes>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
</cluster_property_set>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
</crm_config>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk] <nodes>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
<node id="7009cce3-2480-4c31-8a61-523ed335b4d2" uname="etch2"
type="normal"/>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
<node id="5198e43b-d32a-4361-8808-2bd4e2cab731" uname="etch1"
type="normal"/>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk] </nodes>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
<resources>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
<primitive class="ocf" type="IPaddr" provider="heartbeat"
id="WebServerIP">
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
<instance_attributes id="fae89f69-03ce-4fe8-adbc-663439c3da71">
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
<attributes>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
<nvpair id="WebServerIP_ip" name="ip" value="172.16.28.128"/>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
<nvpair name="target_role" id="wsid-role" value="Started"/>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
</attributes>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
</instance_attributes>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
</primitive>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
</resources>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
<constraints>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
<rsc_location rsc="ip_resource" id="run_ip_resource">
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
<rule score="100" id="pref_run_ip_resource">
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
<expression attribute="#uname" operation="eq"
id="bea57266-4aa5-4e4a-8349-ef4383c070a8" value="etch1"/>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
</rule>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
</rsc_location>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
</constraints>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk]
</configuration>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk] <status/>
Feb 7 14:09:51 etch1 cib: [11584]: info: readCibXmlFile: [on-disk] </cib>
Feb 7 14:09:51 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 24996 bytes (was 0)
Feb 7 14:09:51 etch1 cib: [11584]: info: startCib:main.c CIB
Initialization completed successfully
Feb 7 14:09:51 etch1 cib: [11584]: WARN: init_start:main.c CCM
Activation failed
Feb 7 14:09:51 etch1 cib: [11584]: WARN: init_start:main.c CCM
Connection failed 1 times (30 max)
Feb 7 14:09:51 etch1 heartbeat: [11585]: info: Starting
"/usr/lib/heartbeat/lrmd" as uid 0 gid 0 (pid 11585)
Feb 7 14:09:51 etch1 lrmd: [11585]: info: G_main_add_SignalHandler:
Added signal handler for signal 15
Feb 7 14:09:51 etch1 heartbeat: [11586]: info: Starting
"/usr/lib/heartbeat/stonithd" as uid 0 gid 0 (pid 11586)
Feb 7 14:09:51 etch1 heartbeat: [11587]: info: Starting
"/usr/lib/heartbeat/attrd" as uid 104 gid 106 (pid 11587)
Feb 7 14:09:51 etch1 heartbeat: [11588]: info: Starting
"/usr/lib/heartbeat/crmd" as uid 104 gid 106 (pid 11588)
Feb 7 14:09:51 etch1 crmd: [11588]: info: init_start:main.c Starting crmd
Feb 7 14:09:51 etch1 crmd: [11588]: info: G_main_add_SignalHandler:
Added signal handler for signal 15
Feb 7 14:09:51 etch1 crmd: [11588]: info: G_main_add_TriggerHandler:
Added signal manual handler
Feb 7 14:09:51 etch1 crmd: [11588]: info: G_main_add_SignalHandler:
Added signal handler for signal 17
Feb 7 14:09:51 etch1 heartbeat: [11589]: info: Starting
"/usr/lib/heartbeat/mgmtd -v" as uid 0 gid 0 (pid 11589)
Feb 7 14:09:51 etch1 mgmtd: [11589]: info: G_main_add_SignalHandler:
Added signal handler for signal 15
Feb 7 14:09:51 etch1 mgmtd: [11589]: debug: Enabling coredumps
Feb 7 14:09:51 etch1 lrmd: [11585]: info: G_main_add_SignalHandler:
Added signal handler for signal 17
Feb 7 14:09:51 etch1 lrmd: [11585]: info: G_main_add_SignalHandler:
Added signal handler for signal 10
Feb 7 14:09:51 etch1 lrmd: [11585]: info: G_main_add_SignalHandler:
Added signal handler for signal 12
Feb 7 14:09:51 etch1 lrmd: [11585]: info: Started.
Feb 7 14:09:52 etch1 stonithd: [11586]: info:
G_main_add_SignalHandler: Added signal handler for signal 10
Feb 7 14:09:52 etch1 stonithd: [11586]: info:
G_main_add_SignalHandler: Added signal handler for signal 12
Feb 7 14:09:52 etch1 stonithd: [11586]: info: Signing in with heartbeat.
Feb 7 14:09:52 etch1 attrd: [11587]: info: G_main_add_SignalHandler:
Added signal handler for signal 15
Feb 7 14:09:52 etch1 attrd: [11587]: info: register_with_ha:attrd.c
Hostname: etch1
Feb 7 14:09:52 etch1 mgmtd: [11589]: info: G_main_add_SignalHandler:
Added signal handler for signal 10
Feb 7 14:09:52 etch1 mgmtd: [11589]: info: G_main_add_SignalHandler:
Added signal handler for signal 12
Feb 7 14:09:52 etch1 ccm: [11583]: info: Hostname: etch1
Feb 7 14:09:52 etch1 stonithd: [11586]: notice:
/usr/lib/heartbeat/stonithd start up successfully.
Feb 7 14:09:52 etch1 stonithd: [11586]: info:
G_main_add_SignalHandler: Added signal handler for signal 17
Feb 7 14:09:52 etch1 attrd: [11587]: info: register_with_ha:attrd.c
UUID: 5198e43b-d32a-4361-8808-2bd4e2cab731
Feb 7 14:09:52 etch1 mgmtd: [11589]: info: init_crm
Feb 7 14:09:52 etch1 cib: [11584]: WARN: init_start:main.c CCM
Activation failed
Feb 7 14:09:52 etch1 cib: [11584]: WARN: init_start:main.c CCM
Connection failed 2 times (30 max)
Feb 7 14:09:53 etch1 cib: [11584]: WARN: init_start:main.c CCM
Activation failed
Feb 7 14:09:53 etch1 cib: [11584]: WARN: init_start:main.c CCM
Connection failed 3 times (30 max)
Feb 7 14:09:54 etch1 ccm: [11583]: debug: quorum plugin: majority
Feb 7 14:09:54 etch1 ccm: [11583]: debug: cluster:linux-ha,
member_count=1, member_quorum_votes=100
Feb 7 14:09:54 etch1 ccm: [11583]: debug: total_node_count=2,
total_quorum_votes=200
Feb 7 14:09:54 etch1 ccm: [11583]: debug: quorum plugin: twonodes
Feb 7 14:09:54 etch1 ccm: [11583]: debug: cluster:linux-ha,
member_count=1, member_quorum_votes=100
Feb 7 14:09:54 etch1 ccm: [11583]: debug: total_node_count=2,
total_quorum_votes=200
Feb 7 14:09:54 etch1 ccm: [11583]: info: Break tie for 2 nodes cluster
Feb 7 14:09:54 etch1 ccm: [11583]: info: G_main_add_SignalHandler:
Added signal handler for signal 15
Feb 7 14:09:55 etch1 cib: [11584]: info: init_start:main.c Starting
cib mainloop
Feb 7 14:09:55 etch1 cib: [11584]: info: mem_handle_event: Got an
event OC_EV_MS_NEW_MEMBERSHIP from ccm
Feb 7 14:09:55 etch1 cib: [11584]: info: mem_handle_event:
instance=1, nodes=1, new=1, lost=0, n_idx=0, new_idx=0, old_idx=3
Feb 7 14:09:55 etch1 cib: [11584]: info:
cib_ccm_msg_callback:callbacks.c PEER: etch1
Feb 7 14:09:55 etch1 cib: [11584]: info:
cib_client_status_callback:callbacks.c Status update: Client etch1/cib
now has status [join]
Feb 7 14:09:55 etch1 cib: [11584]: info:
cib_client_status_callback:callbacks.c Status update: Client etch1/cib
now has status [online]
Feb 7 14:09:55 etch1 cib: [11590]: info: write_cib_contents:io.c
Wrote version 0.30.346 of the CIB to disk (digest:
ec1922626a12dd79b9d2e96602b1da22)
Feb 7 14:09:55 etch1 crmd: [11588]: info: do_cib_control:cib.c CIB
connection established
Feb 7 14:09:55 etch1 crmd: [11588]: info: register_with_ha:control.c
Hostname: etch1
Feb 7 14:09:55 etch1 cib: [11584]: info:
cib_null_callback:callbacks.c Setting cib_refresh_notify callbacks for
crmd: on
Feb 7 14:09:55 etch1 cib: [11584]: info:
cib_null_callback:callbacks.c Setting cib_diff_notify callbacks for
mgmtd: on
Feb 7 14:09:55 etch1 crmd: [11588]: info: register_with_ha:control.c
UUID: 5198e43b-d32a-4361-8808-2bd4e2cab731
Feb 7 14:09:56 etch1 crmd: [11588]: info:
populate_cib_nodes:control.c Requesting the list of configured nodes
Feb 7 14:09:57 etch1 mgmtd: [11589]: debug: main: run the loop...
Feb 7 14:09:57 etch1 mgmtd: [11589]: info: Started.
Feb 7 14:09:57 etch1 crmd: [11588]: WARN: get_uuid:utils.c Could not
calculate UUID for etch2
Feb 7 14:09:57 etch1 crmd: [11588]: WARN:
populate_cib_nodes:control.c Node etch2: no uuid found
Feb 7 14:09:57 etch1 crmd: [11588]: notice:
populate_cib_nodes:control.c Node: etch1 (uuid:
5198e43b-d32a-4361-8808-2bd4e2cab731)
Feb 7 14:09:57 etch1 crmd: [11588]: info: do_ha_control:control.c
Connected to Heartbeat
Feb 7 14:09:57 etch1 crmd: [11588]: info: do_ccm_control:ccm.c CCM
connection established... waiting for first callback
Feb 7 14:09:57 etch1 crmd: [11588]: info: do_started:control.c
Delaying start, CCM (0000000000100000) not connected
Feb 7 14:09:57 etch1 crmd: [11588]: info: init_start:main.c Starting
crmd's mainloop
Feb 7 14:09:57 etch1 crmd: [11588]: notice:
crmd_client_status_callback:callbacks.c Status update: Client
etch1/crmd now has status [online]
Feb 7 14:09:57 etch1 crmd: [11588]: info:
crmd_client_status_callback:callbacks.c Uncaching UUID for etch1
Feb 7 14:09:57 etch1 cib: [11591]: info: write_cib_contents:io.c
Wrote version 0.30.346 of the CIB to disk (digest:
ae4618713a38efc227d3a1447e9da6af)
Feb 7 14:09:57 etch1 crmd: [11588]: notice:
crmd_client_status_callback:callbacks.c Status update: Client
etch1/crmd now has status [online]
Feb 7 14:09:57 etch1 crmd: [11588]: info:
crmd_client_status_callback:callbacks.c Uncaching UUID for etch1
Feb 7 14:09:57 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 26704 bytes (was 25100)
Feb 7 14:09:57 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Local-only Change (client:11588, call: 5): 0.30.346 (ok)
Feb 7 14:09:57 etch1 cib: [11592]: info: write_cib_contents:io.c
Wrote version 0.30.346 of the CIB to disk (digest:
ae4618713a38efc227d3a1447e9da6af)
Feb 7 14:09:58 etch1 crmd: [11588]: info: do_started:control.c
Delaying start, CCM (0000000000100000) not connected
Feb 7 14:09:58 etch1 crmd: [11588]: info: mem_handle_event: Got an
event OC_EV_MS_NEW_MEMBERSHIP from ccm
Feb 7 14:09:58 etch1 crmd: [11588]: info: mem_handle_event:
instance=1, nodes=1, new=1, lost=0, n_idx=0, new_idx=0, old_idx=3
Feb 7 14:09:58 etch1 crmd: [11588]: info:
crmd_ccm_msg_callback:callbacks.c Quorum (re)attained after event=NEW
MEMBERSHIP (id=1)
Feb 7 14:09:58 etch1 crmd: [11588]: info: ccm_event_detail:ccm.c NEW
MEMBERSHIP: trans=1, nodes=1, new=1, lost=0 n_idx=0, new_idx=0,
old_idx=3
Feb 7 14:09:58 etch1 crmd: [11588]: info: ccm_event_detail:ccm.c
^ICURRENT: etch1 [nodeid=0, born=1]
Feb 7 14:09:58 etch1 crmd: [11588]: info: ccm_event_detail:ccm.c
^INEW: etch1 [nodeid=0, born=1]
Feb 7 14:09:58 etch1 crmd: [11588]: info: do_started:control.c The
local CRM is operational
Feb 7 14:09:58 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_STARTING -> S_PENDING [ input=I_PENDING
cause=C_CCM_CALLBACK origin=do_started ]
Feb 7 14:09:58 etch1 crmd: [11588]: info: update_dc:utils.c Set DC to
<null> (<null>)
Feb 7 14:09:58 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 26808 bytes (was 26704)
Feb 7 14:09:58 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Local-only Change (client:11588, call: 7): 0.30.346 (ok)
Feb 7 14:09:58 etch1 cib: [11593]: info: write_cib_contents:io.c
Wrote version 0.30.346 of the CIB to disk (digest:
5631d9743a093e8c1e4aac4203a7985b)
Feb 7 14:10:00 etch1 attrd: [11587]: info: main:attrd.c Starting mainloop...
Feb 7 14:10:14 etch1 crmd: [11588]: info: crm_timer_popped:utils.c
Election Trigger (I_DC_TIMEOUT) just popped!
Feb 7 14:10:14 etch1 crmd: [11588]: WARN: do_log:misc.c [[FSA]] Input
I_DC_TIMEOUT from crm_timer_popped() received in state (S_PENDING)
Feb 7 14:10:14 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_PENDING -> S_ELECTION [ input=I_DC_TIMEOUT
cause=C_TIMER_POPPED origin=crm_timer_popped ]
Feb 7 14:10:14 etch1 crmd: [11588]: info: update_dc:utils.c Set DC to
<null> (<null>)
Feb 7 14:10:14 etch1 crmd: [11588]: info:
do_election_count_vote:election.c Updated voted hash for etch1 to vote
Feb 7 14:10:14 etch1 crmd: [11588]: info:
do_election_count_vote:election.c Election ignore: our vote (etch1)
Feb 7 14:10:14 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_ELECTION -> S_INTEGRATION [
input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_election_check ]
Feb 7 14:10:14 etch1 crmd: [11588]: info:
start_subsystem:subsystems.c Starting sub-system "tengine"
Feb 7 14:10:14 etch1 crmd: [11588]: info:
start_subsystem:subsystems.c Starting sub-system "pengine"
Feb 7 14:10:14 etch1 crmd: [11588]: info: do_dc_takeover:election.c
Taking over DC status for this partition
Feb 7 14:10:14 etch1 cib: [11584]: info:
cib_process_readwrite:messages.c We are now in R/W mode
Feb 7 14:10:14 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:11): 0.30.346 -> 0.30.347 (ok)
Feb 7 14:10:14 etch1 crmd: [11588]: info: update_dc:utils.c Set DC to
<null> (<null>)
Feb 7 14:10:14 etch1 crmd: [11588]: info:
do_dc_join_offer_all:join_dc.c join-1: Waiting on 1 outstanding join
acks
Feb 7 14:10:14 etch1 tengine: [11595]: info:
G_main_add_SignalHandler: Added signal handler for signal 15
Feb 7 14:10:14 etch1 tengine: [11595]: info:
G_main_add_TriggerHandler: Added signal manual handler
Feb 7 14:10:14 etch1 cib: [11584]: info:
cib_null_callback:callbacks.c Setting cib_diff_notify callbacks for
tengine: on
Feb 7 14:10:14 etch1 tengine: [11595]: info: init_start:main.c
Registering TE UUID: 7c75cc38-5cef-461d-a799-0528641cefad
Feb 7 14:10:14 etch1 tengine: [11595]: info:
set_graph_functions:utils.c Setting custom graph functions
Feb 7 14:10:14 etch1 tengine: [11595]: info: unpack_graph:unpack.c
Unpacked transition -1: 0 actions in 0 synapses
Feb 7 14:10:14 etch1 tengine: [11595]: info: init_start:main.c Starting tengine
Feb 7 14:10:14 etch1 pengine: [11596]: info:
G_main_add_SignalHandler: Added signal handler for signal 15
Feb 7 14:10:14 etch1 pengine: [11596]: info: init_start:main.c Starting pengine
Feb 7 14:10:14 etch1 cib: [11597]: info: write_cib_contents:io.c
Wrote version 0.30.347 of the CIB to disk (digest:
bcdbe277a08051dfecd0e55b6471336c)
Feb 7 14:10:14 etch1 crmd: [11588]: info: update_dc:utils.c Set DC to
etch1 (1.0.6)
Feb 7 14:10:15 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_INTEGRATION -> S_FINALIZE_JOIN [
input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ]
Feb 7 14:10:15 etch1 crmd: [11588]: info: do_state_transition:fsa.c
All 1 cluster nodes responded to the join offer.
Feb 7 14:10:15 etch1 cib: [11584]: info: sync_our_cib:messages.c
Syncing CIB to all peers
Feb 7 14:10:15 etch1 crmd: [11588]: info: update_attrd:join_dc.c
Connecting to attrd...
Feb 7 14:10:15 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 26944 bytes (was 26808)
Feb 7 14:10:15 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:14): 0.30.347 -> 0.30.348 (ok)
Feb 7 14:10:15 etch1 attrd: [11587]: info:
attrd_local_callback:attrd.c Sending full refresh
Feb 7 14:10:15 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.30.347 ->
0.30.348
Feb 7 14:10:15 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:15): 0.30.348 -> 0.31.349 (ok)
Feb 7 14:10:15 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:16): 0.31.349 -> 0.31.350 (ok)
Feb 7 14:10:15 etch1 cib: [11598]: info: write_cib_contents:io.c
Wrote version 0.31.350 of the CIB to disk (digest:
c8f12e656f857581ab6c0a98d2f5f37b)
Feb 7 14:10:15 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_bump): 0.30.348 ->
0.31.349
Feb 7 14:10:15 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.31.349 ->
0.31.350
Feb 7 14:10:15 etch1 crmd: [11588]: info: update_dc:utils.c Set DC to
etch1 (1.0.6)
Feb 7 14:10:15 etch1 crmd: [11588]: info: do_dc_join_ack:join_dc.c
join-1: Updating node state to member for etch1)
Feb 7 14:10:15 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 29496 bytes (was 26944)
Feb 7 14:10:15 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:17): 0.31.350 -> 0.31.351 (ok)
Feb 7 14:10:15 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE [
input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state ]
Feb 7 14:10:15 etch1 crmd: [11588]: info: do_state_transition:fsa.c
All 1 cluster nodes are eligable to run resources.
Feb 7 14:10:15 etch1 tengine: [11595]: info:
update_abort_priority:utils.c Abort priority upgraded to 1000000
Feb 7 14:10:15 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.31.350 ->
0.31.351
Feb 7 14:10:15 etch1 cib: [11599]: info: write_cib_contents:io.c
Wrote version 0.31.351 of the CIB to disk (digest:
897b004fe714425d4a8b9f06d67debb4)
Feb 7 14:10:15 etch1 pengine: [11596]: info: process_pe_message:
[generation] <cib admin_epoch="0" have_quorum="true" num_peers="1"
cib_feature_revision="1.3" generated="true" epoch="31"
num_updates="351" cib-last-written="Wed Feb 7 14:01:10 2007"
ccm_transition="1" dc_uuid="5198e43b-d32a-4361-8808-2bd4e2cab731"/>
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: default_action_timeout
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: transition_idle_timeout
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: default_resource_stickiness
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
Default stickiness: 0
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference:
default_resource_failure_stickiness
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
Default failure stickiness: 0
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stonith_enabled
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
STONITH of failed nodes is disabled
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stonith_action
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
STONITH will reboot nodes
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: symmetric_cluster
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
Cluster is symmetric - resources can run anywhere by default
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: no_quorum_policy
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
On loss of CCM Quorum: Stop ALL resources
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stop_orphan_resources
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
Orphan resources are stopped
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stop_orphan_actions
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
Orphan resource actions are stopped
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: remove_after_stop
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
Stopped resources are removed from the status section: false
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: is_managed_default
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
By default resources are managed
Feb 7 14:10:15 etch1 pengine: [11596]: info:
determine_online_status:unpack.c Node etch1 is online
Feb 7 14:10:15 etch1 pengine: [11596]: WARN:
unpack_rsc_location:allocate.c No resource (con=run_ip_resource,
rsc=ip_resource)
Feb 7 14:10:15 etch1 pengine: [11596]: info:
WebServerIP^I(heartbeat::ocf:IPaddr):^IStopped
Feb 7 14:10:15 etch1 pengine: [11596]: notice:
native_create_probe:native.c etch1: Created probe for WebServerIP
Feb 7 14:10:15 etch1 pengine: [11596]: notice: StartRsc:native.c
etch1^IStart WebServerIP
Feb 7 14:10:15 etch1 pengine: [11596]: notice: stage8:allocate.c
Created transition graph 0.
Feb 7 14:10:15 etch1 pengine: [11596]: WARN:
process_pe_message:pengine.c No value specified for cluster
preference: pe-input-series-max
Feb 7 14:10:15 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [
input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 7 14:10:15 etch1 tengine: [11595]: info: unpack_graph:unpack.c
Unpacked transition 0: 4 actions in 4 synapses
Feb 7 14:10:15 etch1 tengine: [11595]: info:
send_rsc_command:actions.c Initiating action 3: WebServerIP_monitor_0
on etch1
Feb 7 14:10:15 etch1 pengine: [11596]: info:
process_pe_message:pengine.c Transition 0: PEngine Input stored in:
/var/lib/heartbeat/pengine/pe-input-71.bz2
Feb 7 14:10:15 etch1 crmd: [11588]: info: do_lrm_rsc_op:lrm.c
Performing op monitor on WebServerIP (interval=0ms,
key=0:7c75cc38-5cef-461d-a799-0528641cefad)
Feb 7 14:10:15 etch1 crmd: [11588]: info: process_lrm_event:lrm.c LRM
operation (2) monitor_0 on WebServerIP Error: (7) not running
Feb 7 14:10:15 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 33256 bytes (was 29496)
Feb 7 14:10:15 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:19): 0.31.351 -> 0.31.352 (ok)
Feb 7 14:10:15 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.31.351 ->
0.31.352
Feb 7 14:10:15 etch1 tengine: [11595]: info:
match_graph_event:events.c Action WebServerIP_monitor_0 (3) confirmed
Feb 7 14:10:15 etch1 tengine: [11595]: info:
send_rsc_command:actions.c Initiating action 2: probe_complete on
etch1
Feb 7 14:10:15 etch1 tengine: [11595]: info:
te_pseudo_action:actions.c Pseudo action 1 confirmed
Feb 7 14:10:15 etch1 tengine: [11595]: info:
send_rsc_command:actions.c Initiating action 4: WebServerIP_start_0 on
etch1
Feb 7 14:10:15 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 38080 bytes (was 33256)
Feb 7 14:10:15 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:20): 0.31.352 -> 0.31.353 (ok)
Feb 7 14:10:15 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.31.352 ->
0.31.353
Feb 7 14:10:15 etch1 tengine: [11595]: info: extract_event:events.c
Aborting on transient_attributes changes
Feb 7 14:10:15 etch1 tengine: [11595]: info:
update_abort_priority:utils.c Abort priority upgraded to 1000000
Feb 7 14:10:15 etch1 tengine: [11595]: info:
update_abort_priority:utils.c Abort action 0 superceeded by 2
Feb 7 14:10:15 etch1 cib: [11609]: info: write_cib_contents:io.c
Wrote version 0.31.353 of the CIB to disk (digest:
715f4c18211747959de1d7c2fd6cf66f)
Feb 7 14:10:15 etch1 crmd: [11588]: info: do_lrm_rsc_op:lrm.c
Performing op start on WebServerIP (interval=0ms,
key=0:7c75cc38-5cef-461d-a799-0528641cefad)
Feb 7 14:10:15 etch1 IPaddr[11610]: INFO: eval /sbin/ifconfig eth0:0
172.16.28.128 netmask 255.255.255.0 broadcast 172.16.28.255
Feb 7 14:10:15 etch1 IPaddr[11610]: INFO: Sending Gratuitous Arp for
172.16.28.128 on eth0:0 [eth0]
Feb 7 14:10:15 etch1 IPaddr[11610]: INFO: /usr/lib/heartbeat/send_arp
-i 500 -r 10 -p
/var/run/heartbeat/rsctmp/send_arp/send_arp-172.16.28.128 eth0
172.16.28.128 auto 172.16.28.128 ffffffffffff
Feb 7 14:10:15 etch1 crmd: [11588]: info: process_lrm_event:lrm.c LRM
operation (3) start_0 on WebServerIP complete
Feb 7 14:10:15 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 40372 bytes (was 38080)
Feb 7 14:10:15 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:21): 0.31.353 -> 0.31.354 (ok)
Feb 7 14:10:15 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.31.353 ->
0.31.354
Feb 7 14:10:15 etch1 tengine: [11595]: info:
match_graph_event:events.c Action WebServerIP_start_0 (4) confirmed
Feb 7 14:10:15 etch1 tengine: [11595]: info: run_graph:graph.c
Transition 0: (Complete=4, Pending=0, Fired=0, Skipped=0,
Incomplete=0)
Feb 7 14:10:15 etch1 cib: [11681]: info: write_cib_contents:io.c
Wrote version 0.31.354 of the CIB to disk (digest:
c67eaf943c23675ee7f6c9f35b455981)
Feb 7 14:10:15 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [
input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
Feb 7 14:10:15 etch1 crmd: [11588]: info: do_state_transition:fsa.c
All 1 cluster nodes are eligable to run resources.
Feb 7 14:10:15 etch1 pengine: [11596]: info: process_pe_message:
[generation] <cib admin_epoch="0" have_quorum="true" num_peers="1"
cib_feature_revision="1.3" generated="true" epoch="31"
num_updates="354" cib-last-written="Wed Feb 7 14:01:10 2007"
ccm_transition="1" dc_uuid="5198e43b-d32a-4361-8808-2bd4e2cab731"/>
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: default_action_timeout
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: transition_idle_timeout
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: default_resource_stickiness
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
Default stickiness: 0
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference:
default_resource_failure_stickiness
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
Default failure stickiness: 0
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stonith_enabled
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
STONITH of failed nodes is disabled
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stonith_action
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
STONITH will reboot nodes
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: symmetric_cluster
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
Cluster is symmetric - resources can run anywhere by default
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: no_quorum_policy
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
On loss of CCM Quorum: Stop ALL resources
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stop_orphan_resources
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
Orphan resources are stopped
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stop_orphan_actions
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
Orphan resource actions are stopped
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: remove_after_stop
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
Stopped resources are removed from the status section: false
Feb 7 14:10:15 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: is_managed_default
Feb 7 14:10:15 etch1 pengine: [11596]: info: unpack_config:unpack.c
By default resources are managed
Feb 7 14:10:15 etch1 pengine: [11596]: info:
determine_online_status:unpack.c Node etch1 is online
Feb 7 14:10:15 etch1 pengine: [11596]: WARN:
unpack_rsc_location:allocate.c No resource (con=run_ip_resource,
rsc=ip_resource)
Feb 7 14:10:15 etch1 pengine: [11596]: info:
WebServerIP^I(heartbeat::ocf:IPaddr):^IStarted etch1
Feb 7 14:10:15 etch1 pengine: [11596]: notice: NoRoleChange:native.c
Leave resource WebServerIP^I(etch1)
Feb 7 14:10:15 etch1 pengine: [11596]: notice: stage8:allocate.c
Created transition graph 1.
Feb 7 14:10:15 etch1 pengine: [11596]: WARN:
process_pe_message:pengine.c No value specified for cluster
preference: pe-input-series-max
Feb 7 14:10:15 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [
input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 7 14:10:15 etch1 tengine: [11595]: info: unpack_graph:unpack.c
Unpacked transition 1: 0 actions in 0 synapses
Feb 7 14:10:15 etch1 tengine: [11595]: info: run_graph:graph.c
Transition 1: (Complete=0, Pending=0, Fired=0, Skipped=0,
Incomplete=0)
Feb 7 14:10:15 etch1 tengine: [11595]: info: notify_crmd:actions.c
Transition 1 status: te_complete - (null)
Feb 7 14:10:15 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_TRANSITION_ENGINE -> S_IDLE [
input=I_TE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 7 14:10:15 etch1 pengine: [11596]: info:
process_pe_message:pengine.c Transition 1: PEngine Input stored in:
/var/lib/heartbeat/pengine/pe-input-72.bz2
Feb 7 14:14:22 etch1 heartbeat: [11575]: info: Link etch2:eth0 up.
Feb 7 14:14:22 etch1 heartbeat: [11575]: info: Status update for node
etch2: status init
Feb 7 14:14:22 etch1 heartbeat: [11575]: info: Status update for node
etch2: status up
Feb 7 14:14:22 etch1 heartbeat: [11575]: debug: get_delnodelist: delnodelist=
Feb 7 14:14:22 etch1 heartbeat: [11575]: info: all clients are now paused
Feb 7 14:14:22 etch1 heartbeat: [11575]: debug: hist->ackseq =246
Feb 7 14:14:22 etch1 heartbeat: [11575]: debug: hist->lowseq =245,
hist->hiseq=347
Feb 7 14:14:22 etch1 heartbeat: [11575]: debug:
Feb 7 14:14:22 etch1 crmd: [11588]: notice:
crmd_ha_status_callback:callbacks.c Status update: Node etch2 now has
status [init]
Feb 7 14:14:22 etch1 crmd: [11588]: notice:
crmd_ha_status_callback:callbacks.c Status update: Node etch2 now has
status [up]
Feb 7 14:14:22 etch1 heartbeat: [11575]: debug: hist->ackseq =246
Feb 7 14:14:22 etch1 heartbeat: [11575]: debug: hist->lowseq =245,
hist->hiseq=348
Feb 7 14:14:22 etch1 heartbeat: [11575]: debug:
Feb 7 14:14:22 etch1 heartbeat: [11575]: info: Status update for node
etch2: status active
Feb 7 14:14:22 etch1 crmd: [11588]: notice:
crmd_ha_status_callback:callbacks.c Status update: Node etch2 now has
status [active]
Feb 7 14:14:22 etch1 heartbeat: [11575]: info: all clients are now resumed
Feb 7 14:14:22 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 41872 bytes (was 40372)
Feb 7 14:14:22 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Local-only Change (client:11588, call: 23): 0.31.354 (ok)
Feb 7 14:14:22 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.31.354 ->
0.31.354
Feb 7 14:14:22 etch1 cib: [11693]: info: write_cib_contents:io.c
Wrote version 0.31.354 of the CIB to disk (digest:
ec922622986d22965bbbf694b268710d)
Feb 7 14:14:23 etch1 cib: [11584]: info:
cib_client_status_callback:callbacks.c Status update: Client etch2/cib
now has status [join]
Feb 7 14:14:27 etch1 heartbeat: [11575]: WARN: 1 lost packet(s) for
[etch2] [17:19]
Feb 7 14:14:27 etch1 heartbeat: [11575]: info: No pkts missing from etch2!
Feb 7 14:14:27 etch1 crmd: [11588]: notice:
crmd_client_status_callback:callbacks.c Status update: Client
etch2/crmd now has status [online]
Feb 7 14:14:27 etch1 crmd: [11588]: info:
crmd_client_status_callback:callbacks.c Uncaching UUID for etch2
Feb 7 14:14:27 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 42080 bytes (was 41872)
Feb 7 14:14:27 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Local-only Change (client:11588, call: 24): 0.31.354 (ok)
Feb 7 14:14:27 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.31.354 ->
0.31.354
Feb 7 14:14:27 etch1 cib: [11694]: info: write_cib_contents:io.c
Wrote version 0.31.354 of the CIB to disk (digest:
739754876dd5c43ef348c5c411eee3a9)
Feb 7 14:14:28 etch1 heartbeat: [11575]: WARN: 1 lost packet(s) for
[etch2] [22:24]
Feb 7 14:14:28 etch1 heartbeat: [11575]: info: No pkts missing from etch2!
Feb 7 14:14:29 etch1 ccm: [11583]: debug: quorum plugin: majority
Feb 7 14:14:29 etch1 ccm: [11583]: debug: cluster:linux-ha,
member_count=2, member_quorum_votes=200
Feb 7 14:14:29 etch1 ccm: [11583]: debug: total_node_count=2,
total_quorum_votes=200
Feb 7 14:14:29 etch1 crmd: [11588]: info: mem_handle_event: Got an
event OC_EV_MS_INVALID from ccm
Feb 7 14:14:29 etch1 crmd: [11588]: info: mem_handle_event: no mbr_track info
Feb 7 14:14:29 etch1 crmd: [11588]: info: mem_handle_event: Got an
event OC_EV_MS_NEW_MEMBERSHIP from ccm
Feb 7 14:14:29 etch1 crmd: [11588]: info: mem_handle_event:
instance=2, nodes=2, new=1, lost=0, n_idx=0, new_idx=2, old_idx=4
Feb 7 14:14:29 etch1 crmd: [11588]: info:
crmd_ccm_msg_callback:callbacks.c Quorum (re)attained after event=NEW
MEMBERSHIP (id=2)
Feb 7 14:14:29 etch1 crmd: [11588]: info: ccm_event_detail:ccm.c NEW
MEMBERSHIP: trans=2, nodes=2, new=1, lost=0 n_idx=0, new_idx=2,
old_idx=4
Feb 7 14:14:29 etch1 crmd: [11588]: info: ccm_event_detail:ccm.c
^ICURRENT: etch1 [nodeid=0, born=1]
Feb 7 14:14:29 etch1 crmd: [11588]: info: ccm_event_detail:ccm.c
^ICURRENT: etch2 [nodeid=1, born=2]
Feb 7 14:14:29 etch1 crmd: [11588]: info: ccm_event_detail:ccm.c
^INEW: etch2 [nodeid=1, born=2]
Feb 7 14:14:29 etch1 cib: [11584]: info: mem_handle_event: Got an
event OC_EV_MS_INVALID from ccm
Feb 7 14:14:29 etch1 cib: [11584]: info: mem_handle_event: no mbr_track info
Feb 7 14:14:29 etch1 cib: [11584]: info: mem_handle_event: Got an
event OC_EV_MS_NEW_MEMBERSHIP from ccm
Feb 7 14:14:29 etch1 cib: [11584]: info: mem_handle_event:
instance=2, nodes=2, new=1, lost=0, n_idx=0, new_idx=2, old_idx=4
Feb 7 14:14:29 etch1 cib: [11584]: info:
cib_ccm_msg_callback:callbacks.c PEER: etch1
Feb 7 14:14:29 etch1 cib: [11584]: info:
cib_ccm_msg_callback:callbacks.c PEER: etch2
Feb 7 14:14:29 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 42184 bytes (was 42080)
Feb 7 14:14:29 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Local-only Change (client:11588, call: 25): 0.31.354 (ok)
Feb 7 14:14:29 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.31.354 ->
0.31.354
Feb 7 14:14:29 etch1 cib: [11695]: info: write_cib_contents:io.c
Wrote version 0.31.354 of the CIB to disk (digest:
35222de32c72bd6dbb2f0ca8777a90e2)
Feb 7 14:14:32 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_IDLE -> S_INTEGRATION [ input=I_NODE_JOIN
cause=C_HA_MESSAGE origin=route_message ]
Feb 7 14:14:32 etch1 crmd: [11588]: info: update_dc:utils.c Set DC to
<null> (<null>)
Feb 7 14:14:32 etch1 crmd: [11588]: info:
do_dc_join_offer_all:join_dc.c join-2: Waiting on 2 outstanding join
acks
Feb 7 14:14:32 etch1 tengine: [11595]: info:
update_abort_priority:utils.c Abort priority upgraded to 1000000
Feb 7 14:14:32 etch1 crmd: [11588]: info: update_dc:utils.c Set DC to
etch1 (1.0.6)
Feb 7 14:14:33 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_INTEGRATION -> S_FINALIZE_JOIN [
input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ]
Feb 7 14:14:33 etch1 crmd: [11588]: info: do_state_transition:fsa.c
All 2 cluster nodes responded to the join offer.
Feb 7 14:14:33 etch1 crmd: [11588]: info:
do_dc_join_finalize:join_dc.c join-2: Asking etch2 for its copy of the
CIB
Feb 7 14:14:34 etch1 cib: [11584]: info: cib_replace_notify:notify.c
Replaced: 0.31.354 -> 0.31.356 from (null)
Feb 7 14:14:34 etch1 crmd: [11588]: info:
populate_cib_nodes:control.c Requesting the list of configured nodes
Feb 7 14:14:34 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 28412 bytes (was 42184)
Feb 7 14:14:34 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:27): 0.31.354 -> 0.31.356 (ok)
Feb 7 14:14:34 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_replace): 0.31.354 ->
0.31.356
Feb 7 14:14:34 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Aborting on transient_attributes deletions
Feb 7 14:14:34 etch1 cib: [11696]: info: write_cib_contents:io.c
Wrote version 0.31.356 of the CIB to disk (digest:
94b0ac9624bc68de565c070236dcecfa)
Feb 7 14:14:34 etch1 crmd: [11588]: notice:
populate_cib_nodes:control.c Node: etch2 (uuid:
7009cce3-2480-4c31-8a61-523ed335b4d2)
Feb 7 14:14:35 etch1 crmd: [11588]: notice:
populate_cib_nodes:control.c Node: etch1 (uuid:
5198e43b-d32a-4361-8808-2bd4e2cab731)
Feb 7 14:14:35 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_FINALIZE_JOIN -> S_ELECTION [
input=I_ELECTION cause=C_FSA_INTERNAL origin=do_cib_replaced ]
Feb 7 14:14:35 etch1 crmd: [11588]: info: update_dc:utils.c Set DC to
<null> (<null>)
Feb 7 14:14:35 etch1 attrd: [11587]: info:
attrd_local_callback:attrd.c Sending full refresh
Feb 7 14:14:35 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 28620 bytes (was 28412)
Feb 7 14:14:35 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:29): 0.31.356 -> 0.31.357 (ok)
Feb 7 14:14:35 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 28756 bytes (was 28620)
Feb 7 14:14:35 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:30): 0.31.357 -> 0.31.358 (ok)
Feb 7 14:14:35 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:31): 0.31.358 -> 0.32.359 (ok)
Feb 7 14:14:35 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:32): 0.32.359 -> 0.32.360 (ok)
Feb 7 14:14:35 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.31.356 ->
0.31.357
Feb 7 14:14:35 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.31.357 ->
0.31.358
Feb 7 14:14:35 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_bump): 0.31.358 ->
0.32.359
Feb 7 14:14:35 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.32.359 ->
0.32.360
Feb 7 14:14:35 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:33): 0.32.360 -> 0.32.361 (ok)
Feb 7 14:14:35 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.32.360 ->
0.32.361
Feb 7 14:14:35 etch1 cib: [11697]: info: write_cib_contents:io.c
Wrote version 0.32.361 of the CIB to disk (digest:
aa5db70ddb8bb9b8deaa92e5fb92b417)
Feb 7 14:14:35 etch1 crmd: [11588]: info:
do_election_count_vote:election.c Updated voted hash for etch1 to vote
Feb 7 14:14:35 etch1 crmd: [11588]: info:
do_election_count_vote:election.c Election ignore: our vote (etch1)
Feb 7 14:14:35 etch1 crmd: [11588]: info:
do_election_check:election.c Still waiting on 1 non-votes (2 total)
Feb 7 14:14:36 etch1 crmd: [11588]: info:
do_election_count_vote:election.c Updated voted hash for etch2 to
no-vote
Feb 7 14:14:36 etch1 crmd: [11588]: info:
do_election_count_vote:election.c Election ignore: no-vote from etch2
Feb 7 14:14:36 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_ELECTION -> S_INTEGRATION [
input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_election_check ]
Feb 7 14:14:36 etch1 crmd: [11588]: info:
start_subsystem:subsystems.c Starting sub-system "tengine"
Feb 7 14:14:36 etch1 crmd: [11588]: WARN:
start_subsystem:subsystems.c Client tengine already running as pid
11595
Feb 7 14:14:36 etch1 crmd: [11588]: info:
start_subsystem:subsystems.c Starting sub-system "pengine"
Feb 7 14:14:36 etch1 crmd: [11588]: WARN:
start_subsystem:subsystems.c Client pengine already running as pid
11596
Feb 7 14:14:36 etch1 crmd: [11588]: info: do_dc_takeover:election.c
Taking over DC status for this partition
Feb 7 14:14:36 etch1 crmd: [11588]: info: update_dc:utils.c Set DC to
<null> (<null>)
Feb 7 14:14:36 etch1 crmd: [11588]: info:
do_dc_join_offer_all:join_dc.c join-3: Waiting on 2 outstanding join
acks
Feb 7 14:14:36 etch1 cib: [11584]: info:
cib_process_readwrite:messages.c We are now in R/O mode
Feb 7 14:14:36 etch1 cib: [11584]: info:
cib_process_readwrite:messages.c We are now in R/W mode
Feb 7 14:14:36 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:36): 0.32.361 -> 0.32.362 (ok)
Feb 7 14:14:36 etch1 cib: [11698]: info: write_cib_contents:io.c
Wrote version 0.32.362 of the CIB to disk (digest:
4a0ffff7a2ef0b94bf267594d3c7aa2d)
Feb 7 14:14:36 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.32.361 ->
0.32.362
Feb 7 14:14:36 etch1 crmd: [11588]: info: update_dc:utils.c Set DC to
etch1 (1.0.6)
Feb 7 14:14:37 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_INTEGRATION -> S_FINALIZE_JOIN [
input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ]
Feb 7 14:14:37 etch1 crmd: [11588]: info: do_state_transition:fsa.c
All 2 cluster nodes responded to the join offer.
Feb 7 14:14:37 etch1 cib: [11584]: info: sync_our_cib:messages.c
Syncing CIB to all peers
Feb 7 14:14:37 etch1 attrd: [11587]: info:
attrd_local_callback:attrd.c Sending full refresh
Feb 7 14:14:37 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:39): 0.32.362 -> 0.32.363 (ok)
Feb 7 14:14:37 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:40): 0.32.363 -> 0.33.364 (ok)
Feb 7 14:14:37 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.32.362 ->
0.32.363
Feb 7 14:14:37 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_bump): 0.32.363 ->
0.33.364
Feb 7 14:14:37 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:41): 0.33.364 -> 0.33.365 (ok)
Feb 7 14:14:37 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:42): 0.33.365 -> 0.33.366 (ok)
Feb 7 14:14:37 etch1 cib: [11699]: info: write_cib_contents:io.c
Wrote version 0.33.366 of the CIB to disk (digest:
c08b3f70ed20c6d5fdc1a9e1e04c070c)
Feb 7 14:14:37 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.33.364 ->
0.33.365
Feb 7 14:14:37 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.33.365 ->
0.33.366
Feb 7 14:14:37 etch1 crmd: [11588]: info: update_dc:utils.c Set DC to
etch1 (1.0.6)
Feb 7 14:14:38 etch1 crmd: [11588]: info: do_dc_join_ack:join_dc.c
join-3: Updating node state to member for etch1)
Feb 7 14:14:38 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 37360 bytes (was 28756)
Feb 7 14:14:38 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:43): 0.33.366 -> 0.33.367 (ok)
Feb 7 14:14:38 etch1 cib: [11700]: info: write_cib_contents:io.c
Wrote version 0.33.367 of the CIB to disk (digest:
85a43ddb62f02966a01e2233dec8ca18)
Feb 7 14:14:38 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.33.366 ->
0.33.367
Feb 7 14:14:38 etch1 tengine: [11595]: WARN:
process_graph_event:events.c Event not found.
Feb 7 14:14:38 etch1 tengine: [11595]: info: process_graph_event:
match:not-found <lrm_rsc_op id="WebServerIP_monitor_0"
operation="monitor" crm-debug-origin="build_active_RAs"
transition_key="0:7c75cc38-5cef-461d-a799-0528641cefad"
transition_magic="4:7;0:7c75cc38-5cef-461d-a799-0528641cefad"
call_id="2" crm_feature_set="1.0.6" rc_code="7" op_status="4"
interval="0" op_digest="e9b2d366a1de167cd6ba98615ea0d42c"/>
Feb 7 14:14:38 etch1 tengine: [11595]: WARN:
process_graph_event:events.c Event not found.
Feb 7 14:14:38 etch1 tengine: [11595]: info: process_graph_event:
match:not-found <lrm_rsc_op id="WebServerIP_start_0" operation="start"
crm-debug-origin="build_active_RAs"
transition_key="0:7c75cc38-5cef-461d-a799-0528641cefad"
transition_magic="0:0;0:7c75cc38-5cef-461d-a799-0528641cefad"
call_id="3" crm_feature_set="1.0.6" rc_code="0" op_status="0"
interval="0" op_digest="e9b2d366a1de167cd6ba98615ea0d42c"/>
Feb 7 14:14:38 etch1 crmd: [11588]: info: do_dc_join_ack:join_dc.c
join-3: Updating node state to member for etch2)
Feb 7 14:14:38 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 39808 bytes (was 37360)
Feb 7 14:14:38 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:44): 0.33.367 -> 0.33.368 (ok)
Feb 7 14:14:38 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.33.367 ->
0.33.368
Feb 7 14:14:38 etch1 cib: [11701]: info: write_cib_contents:io.c
Wrote version 0.33.368 of the CIB to disk (digest:
d8ec53464f8d63d4ed44b00931e650a0)
Feb 7 14:14:38 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE [
input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state ]
Feb 7 14:14:38 etch1 crmd: [11588]: info: do_state_transition:fsa.c
All 2 cluster nodes are eligable to run resources.
Feb 7 14:14:38 etch1 pengine: [11596]: info: process_pe_message:
[generation] <cib admin_epoch="0" have_quorum="true" num_peers="2"
cib_feature_revision="1.3" generated="true" epoch="33"
num_updates="368" cib-last-written="Wed Feb 7 14:09:56 2007"
ccm_transition="2" dc_uuid="5198e43b-d32a-4361-8808-2bd4e2cab731"/>
Feb 7 14:14:38 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: default_action_timeout
Feb 7 14:14:38 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: transition_idle_timeout
Feb 7 14:14:38 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: default_resource_stickiness
Feb 7 14:14:38 etch1 pengine: [11596]: info: unpack_config:unpack.c
Default stickiness: 0
Feb 7 14:14:38 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference:
default_resource_failure_stickiness
Feb 7 14:14:38 etch1 pengine: [11596]: info: unpack_config:unpack.c
Default failure stickiness: 0
Feb 7 14:14:38 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stonith_enabled
Feb 7 14:14:38 etch1 pengine: [11596]: info: unpack_config:unpack.c
STONITH of failed nodes is disabled
Feb 7 14:14:38 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stonith_action
Feb 7 14:14:38 etch1 pengine: [11596]: info: unpack_config:unpack.c
STONITH will reboot nodes
Feb 7 14:14:38 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: symmetric_cluster
Feb 7 14:14:38 etch1 pengine: [11596]: info: unpack_config:unpack.c
Cluster is symmetric - resources can run anywhere by default
Feb 7 14:14:38 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: no_quorum_policy
Feb 7 14:14:38 etch1 pengine: [11596]: info: unpack_config:unpack.c
On loss of CCM Quorum: Stop ALL resources
Feb 7 14:14:38 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stop_orphan_resources
Feb 7 14:14:38 etch1 pengine: [11596]: info: unpack_config:unpack.c
Orphan resources are stopped
Feb 7 14:14:38 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stop_orphan_actions
Feb 7 14:14:38 etch1 pengine: [11596]: info: unpack_config:unpack.c
Orphan resource actions are stopped
Feb 7 14:14:38 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: remove_after_stop
Feb 7 14:14:38 etch1 pengine: [11596]: info: unpack_config:unpack.c
Stopped resources are removed from the status section: false
Feb 7 14:14:38 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: is_managed_default
Feb 7 14:14:38 etch1 pengine: [11596]: info: unpack_config:unpack.c
By default resources are managed
Feb 7 14:14:38 etch1 pengine: [11596]: info:
determine_online_status:unpack.c Node etch2 is online
Feb 7 14:14:38 etch1 pengine: [11596]: info:
determine_online_status:unpack.c Node etch1 is online
Feb 7 14:14:38 etch1 pengine: [11596]: WARN:
unpack_rsc_location:allocate.c No resource (con=run_ip_resource,
rsc=ip_resource)
Feb 7 14:14:38 etch1 pengine: [11596]: info:
WebServerIP^I(heartbeat::ocf:IPaddr):^IStarted etch1
Feb 7 14:14:38 etch1 pengine: [11596]: notice:
native_create_probe:native.c etch2: Created probe for WebServerIP
Feb 7 14:14:38 etch1 pengine: [11596]: notice: NoRoleChange:native.c
Move resource WebServerIP^I(etch1 -> etch2)
Feb 7 14:14:38 etch1 pengine: [11596]: notice: stage8:allocate.c
Created transition graph 2.
Feb 7 14:14:38 etch1 pengine: [11596]: WARN:
process_pe_message:pengine.c No value specified for cluster
preference: pe-input-series-max
Feb 7 14:14:38 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [
input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 7 14:14:38 etch1 tengine: [11595]: info: unpack_graph:unpack.c
Unpacked transition 2: 6 actions in 6 synapses
Feb 7 14:14:38 etch1 tengine: [11595]: info:
send_rsc_command:actions.c Initiating action 3: WebServerIP_monitor_0
on etch2
Feb 7 14:14:38 etch1 tengine: [11595]: info:
send_rsc_command:actions.c Initiating action 5: WebServerIP_stop_0 on
etch1
Feb 7 14:14:38 etch1 tengine: [11595]: info:
send_rsc_command:actions.c Initiating action 4: probe_complete on
etch1
Feb 7 14:14:38 etch1 pengine: [11596]: info:
process_pe_message:pengine.c Transition 2: PEngine Input stored in:
/var/lib/heartbeat/pengine/pe-input-73.bz2
Feb 7 14:14:38 etch1 crmd: [11588]: info: do_lrm_rsc_op:lrm.c
Performing op stop on WebServerIP (interval=0ms,
key=2:7c75cc38-5cef-461d-a799-0528641cefad)
Feb 7 14:14:38 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 44632 bytes (was 39808)
Feb 7 14:14:38 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:46): 0.33.368 -> 0.33.369 (ok)
Feb 7 14:14:38 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.33.368 ->
0.33.369
Feb 7 14:14:38 etch1 tengine: [11595]: info: extract_event:events.c
Aborting on transient_attributes changes
Feb 7 14:14:38 etch1 tengine: [11595]: info:
update_abort_priority:utils.c Abort priority upgraded to 1000000
Feb 7 14:14:38 etch1 tengine: [11595]: info:
update_abort_priority:utils.c Abort action 0 superceeded by 2
Feb 7 14:14:38 etch1 cib: [11711]: info: write_cib_contents:io.c
Wrote version 0.33.369 of the CIB to disk (digest:
823b6a63d407033f80499a514da0f22e)
Feb 7 14:14:38 etch1 IPaddr[11702]: INFO: /sbin/route -n del -host
172.16.28.128
Feb 7 14:14:38 etch1 lrmd: [11585]: info: RA output:
(WebServerIP:stop:stderr) SIOCDELRT: No such process
Feb 7 14:14:38 etch1 IPaddr[11702]: INFO: /sbin/ifconfig eth0:0
172.16.28.128 down
Feb 7 14:14:38 etch1 IPaddr[11702]: INFO: IP Address 172.16.28.128 released
Feb 7 14:14:38 etch1 crmd: [11588]: info: process_lrm_event:lrm.c LRM
operation (4) stop_0 on WebServerIP complete
Feb 7 14:14:38 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 46924 bytes (was 44632)
Feb 7 14:14:38 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 11588, call:47): 0.33.369 -> 0.33.370 (ok)
Feb 7 14:14:38 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.33.369 ->
0.33.370
Feb 7 14:14:38 etch1 tengine: [11595]: info:
match_graph_event:events.c Action WebServerIP_stop_0 (5) confirmed
Feb 7 14:14:38 etch1 cib: [11730]: info: write_cib_contents:io.c
Wrote version 0.33.370 of the CIB to disk (digest:
7ea94011f531797b96dbd3d2c1869120)
Feb 7 14:14:39 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 50684 bytes (was 46924)
Feb 7 14:14:39 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 3782, call:14): 0.33.370 -> 0.33.371 (ok)
Feb 7 14:14:39 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.33.370 ->
0.33.371
Feb 7 14:14:39 etch1 tengine: [11595]: info:
match_graph_event:events.c Action WebServerIP_monitor_0 (3) confirmed
Feb 7 14:14:39 etch1 tengine: [11595]: info:
send_rsc_command:actions.c Initiating action 2: probe_complete on
etch2
Feb 7 14:14:39 etch1 tengine: [11595]: info: run_graph:graph.c
====================================================
Feb 7 14:14:39 etch1 tengine: [11595]: notice: run_graph:graph.c
Transition 2: (Complete=4, Pending=0, Fired=0, Skipped=2,
Incomplete=0)
Feb 7 14:14:39 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [
input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
Feb 7 14:14:39 etch1 crmd: [11588]: info: do_state_transition:fsa.c
All 2 cluster nodes are eligable to run resources.
Feb 7 14:14:39 etch1 cib: [11731]: info: write_cib_contents:io.c
Wrote version 0.33.371 of the CIB to disk (digest:
05acd6cd4cf537431b97d02a1a64e5e4)
Feb 7 14:14:39 etch1 pengine: [11596]: info: process_pe_message:
[generation] <cib admin_epoch="0" have_quorum="true" num_peers="2"
cib_feature_revision="1.3" generated="true" epoch="33"
num_updates="371" cib-last-written="Wed Feb 7 14:09:56 2007"
ccm_transition="2" dc_uuid="5198e43b-d32a-4361-8808-2bd4e2cab731"/>
Feb 7 14:14:39 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: default_action_timeout
Feb 7 14:14:39 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: transition_idle_timeout
Feb 7 14:14:39 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: default_resource_stickiness
Feb 7 14:14:39 etch1 pengine: [11596]: info: unpack_config:unpack.c
Default stickiness: 0
Feb 7 14:14:39 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference:
default_resource_failure_stickiness
Feb 7 14:14:39 etch1 pengine: [11596]: info: unpack_config:unpack.c
Default failure stickiness: 0
Feb 7 14:14:39 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stonith_enabled
Feb 7 14:14:39 etch1 pengine: [11596]: info: unpack_config:unpack.c
STONITH of failed nodes is disabled
Feb 7 14:14:39 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stonith_action
Feb 7 14:14:39 etch1 pengine: [11596]: info: unpack_config:unpack.c
STONITH will reboot nodes
Feb 7 14:14:39 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: symmetric_cluster
Feb 7 14:14:39 etch1 pengine: [11596]: info: unpack_config:unpack.c
Cluster is symmetric - resources can run anywhere by default
Feb 7 14:14:39 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: no_quorum_policy
Feb 7 14:14:39 etch1 pengine: [11596]: info: unpack_config:unpack.c
On loss of CCM Quorum: Stop ALL resources
Feb 7 14:14:39 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stop_orphan_resources
Feb 7 14:14:39 etch1 pengine: [11596]: info: unpack_config:unpack.c
Orphan resources are stopped
Feb 7 14:14:39 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stop_orphan_actions
Feb 7 14:14:39 etch1 pengine: [11596]: info: unpack_config:unpack.c
Orphan resource actions are stopped
Feb 7 14:14:39 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: remove_after_stop
Feb 7 14:14:39 etch1 pengine: [11596]: info: unpack_config:unpack.c
Stopped resources are removed from the status section: false
Feb 7 14:14:39 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: is_managed_default
Feb 7 14:14:39 etch1 pengine: [11596]: info: unpack_config:unpack.c
By default resources are managed
Feb 7 14:14:39 etch1 pengine: [11596]: info:
determine_online_status:unpack.c Node etch2 is online
Feb 7 14:14:39 etch1 pengine: [11596]: info:
determine_online_status:unpack.c Node etch1 is online
Feb 7 14:14:39 etch1 pengine: [11596]: WARN:
unpack_rsc_location:allocate.c No resource (con=run_ip_resource,
rsc=ip_resource)
Feb 7 14:14:39 etch1 pengine: [11596]: info:
WebServerIP^I(heartbeat::ocf:IPaddr):^IStopped
Feb 7 14:14:39 etch1 pengine: [11596]: notice: StartRsc:native.c
etch2^IStart WebServerIP
Feb 7 14:14:39 etch1 pengine: [11596]: notice: stage8:allocate.c
Created transition graph 3.
Feb 7 14:14:39 etch1 pengine: [11596]: WARN:
process_pe_message:pengine.c No value specified for cluster
preference: pe-input-series-max
Feb 7 14:14:39 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [
input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 7 14:14:39 etch1 tengine: [11595]: info: unpack_graph:unpack.c
Unpacked transition 3: 2 actions in 2 synapses
Feb 7 14:14:39 etch1 tengine: [11595]: info:
send_rsc_command:actions.c Initiating action 4: WebServerIP_start_0 on
etch2
Feb 7 14:14:39 etch1 tengine: [11595]: info:
send_rsc_command:actions.c Initiating action 2: probe_complete on
etch2
Feb 7 14:14:39 etch1 pengine: [11596]: info:
process_pe_message:pengine.c Transition 3: PEngine Input stored in:
/var/lib/heartbeat/pengine/pe-input-74.bz2
Feb 7 14:14:40 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 55508 bytes (was 50684)
Feb 7 14:14:40 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 3782, call:15): 0.33.371 -> 0.33.372 (ok)
Feb 7 14:14:40 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.33.371 ->
0.33.372
Feb 7 14:14:40 etch1 tengine: [11595]: info: extract_event:events.c
Aborting on transient_attributes changes
Feb 7 14:14:40 etch1 tengine: [11595]: info:
update_abort_priority:utils.c Abort priority upgraded to 1000000
Feb 7 14:14:40 etch1 tengine: [11595]: info:
update_abort_priority:utils.c Abort action 0 superceeded by 2
Feb 7 14:14:40 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 3782, call:16): 0.33.372 -> 0.33.373 (ok)
Feb 7 14:14:40 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.33.372 ->
0.33.373
Feb 7 14:14:40 etch1 cib: [11584]: info: activateCibXml:io.c CIB size
is 57800 bytes (was 55508)
Feb 7 14:14:40 etch1 cib: [11584]: info: cib_diff_notify:notify.c
Update (client: 3782, call:17): 0.33.373 -> 0.33.374 (ok)
Feb 7 14:14:40 etch1 tengine: [11595]: info:
te_update_diff:callbacks.c Processing diff (cib_update): 0.33.373 ->
0.33.374
Feb 7 14:14:40 etch1 tengine: [11595]: info:
match_graph_event:events.c Action WebServerIP_start_0 (4) confirmed
Feb 7 14:14:40 etch1 tengine: [11595]: info: run_graph:graph.c
Transition 3: (Complete=2, Pending=0, Fired=0, Skipped=0,
Incomplete=0)
Feb 7 14:14:40 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [
input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
Feb 7 14:14:40 etch1 crmd: [11588]: info: do_state_transition:fsa.c
All 2 cluster nodes are eligable to run resources.
Feb 7 14:14:40 etch1 pengine: [11596]: info: process_pe_message:
[generation] <cib admin_epoch="0" have_quorum="true" num_peers="2"
cib_feature_revision="1.3" generated="true" epoch="33"
num_updates="374" cib-last-written="Wed Feb 7 14:09:56 2007"
ccm_transition="2" dc_uuid="5198e43b-d32a-4361-8808-2bd4e2cab731"/>
Feb 7 14:14:40 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: default_action_timeout
Feb 7 14:14:40 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: transition_idle_timeout
Feb 7 14:14:40 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: default_resource_stickiness
Feb 7 14:14:40 etch1 pengine: [11596]: info: unpack_config:unpack.c
Default stickiness: 0
Feb 7 14:14:40 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference:
default_resource_failure_stickiness
Feb 7 14:14:40 etch1 pengine: [11596]: info: unpack_config:unpack.c
Default failure stickiness: 0
Feb 7 14:14:40 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stonith_enabled
Feb 7 14:14:40 etch1 pengine: [11596]: info: unpack_config:unpack.c
STONITH of failed nodes is disabled
Feb 7 14:14:40 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stonith_action
Feb 7 14:14:40 etch1 pengine: [11596]: info: unpack_config:unpack.c
STONITH will reboot nodes
Feb 7 14:14:40 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: symmetric_cluster
Feb 7 14:14:40 etch1 pengine: [11596]: info: unpack_config:unpack.c
Cluster is symmetric - resources can run anywhere by default
Feb 7 14:14:40 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: no_quorum_policy
Feb 7 14:14:40 etch1 pengine: [11596]: info: unpack_config:unpack.c
On loss of CCM Quorum: Stop ALL resources
Feb 7 14:14:40 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stop_orphan_resources
Feb 7 14:14:40 etch1 pengine: [11596]: info: unpack_config:unpack.c
Orphan resources are stopped
Feb 7 14:14:40 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: stop_orphan_actions
Feb 7 14:14:40 etch1 pengine: [11596]: info: unpack_config:unpack.c
Orphan resource actions are stopped
Feb 7 14:14:40 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: remove_after_stop
Feb 7 14:14:40 etch1 pengine: [11596]: info: unpack_config:unpack.c
Stopped resources are removed from the status section: false
Feb 7 14:14:40 etch1 pengine: [11596]: WARN: unpack_config:unpack.c
No value specified for cluster preference: is_managed_default
Feb 7 14:14:40 etch1 pengine: [11596]: info: unpack_config:unpack.c
By default resources are managed
Feb 7 14:14:40 etch1 pengine: [11596]: info:
determine_online_status:unpack.c Node etch2 is online
Feb 7 14:14:40 etch1 pengine: [11596]: info:
determine_online_status:unpack.c Node etch1 is online
Feb 7 14:14:40 etch1 pengine: [11596]: WARN:
unpack_rsc_location:allocate.c No resource (con=run_ip_resource,
rsc=ip_resource)
Feb 7 14:14:40 etch1 pengine: [11596]: info:
WebServerIP^I(heartbeat::ocf:IPaddr):^IStarted etch2
Feb 7 14:14:40 etch1 pengine: [11596]: notice: NoRoleChange:native.c
Leave resource WebServerIP^I(etch2)
Feb 7 14:14:40 etch1 pengine: [11596]: notice: stage8:allocate.c
Created transition graph 4.
Feb 7 14:14:40 etch1 pengine: [11596]: WARN:
process_pe_message:pengine.c No value specified for cluster
preference: pe-input-series-max
Feb 7 14:14:40 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [
input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 7 14:14:40 etch1 tengine: [11595]: info: unpack_graph:unpack.c
Unpacked transition 4: 0 actions in 0 synapses
Feb 7 14:14:40 etch1 tengine: [11595]: info: run_graph:graph.c
Transition 4: (Complete=0, Pending=0, Fired=0, Skipped=0,
Incomplete=0)
Feb 7 14:14:40 etch1 tengine: [11595]: info: notify_crmd:actions.c
Transition 4 status: te_complete - (null)
Feb 7 14:14:40 etch1 crmd: [11588]: info: do_state_transition:fsa.c
etch1: State transition S_TRANSITION_ENGINE -> S_IDLE [
input=I_TE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 7 14:14:40 etch1 pengine: [11596]: info:
process_pe_message:pengine.c Transition 4: PEngine Input stored in:
/var/lib/heartbeat/pengine/pe-input-75.bz2
Feb 7 14:14:40 etch1 cib: [11732]: info: write_cib_contents:io.c
Wrote version 0.33.374 of the CIB to disk (digest:
2b1105e69b72c9d147bc62055334f120)
bprajapati at etch1:~$
More information about the Linux-HA
mailing list