[Linux-HA] Score and fail count don't updated after first time

Fri Feb 8 23:54:43 MST 2008

On Feb 8, 2008, at 9:02 PM, Serge Dubrouski wrote:

> Am I understand this bug right that it prevents resources to be failed
> over to other node in a "forced faiilover" situation? CRM will try to
> restart a failed resource endlessly no matter how stickiness is
> configured in cib.xml?

almost

if a resource would have moved nodes after 1 failure, then it will  
continue to fail over.
if a resource required more than 1 failure to move, then it will  
restart on the current node as you described.

>
>
> On Feb 8, 2008 12:23 PM, Andrew Beekhof <beekhof at gmail.com> wrote:
>>
>> On Feb 8, 2008, at 6:28 PM, China wrote:
>>
>>> Hi,
>>> I've a problem with heartbeat 2.1.3
>>> I've a two 64bit node cluster with a IPaddr resource.
>>> Starting cluster make no problems, after that I take down two o more
>>> times
>>> the virtual interface with "ifconfi eth0:1 down".
>>> The first time all is ok, fail count goes to 1, score is lowered of
>>> resource_failure_stickiness (becomes 600 from 1100) and resource
>>> restarted.
>>> The second time (and following) the resource is restarted but the
>>> fail count
>>> remains 1 and score 600.
>>>
>>> Is this a bug?
>>
>>
>> yes
>>
>> http://hg.clusterlabs.org/pacemaker/stable-0.6/rev/ed6dff2d9024
>>
>> this fix will be included in 0.6.1 when it comes out on monday (or
>> tuesday)
>>
>>
>>>
>>>
>>> This is the debug log after reading cib file and DC election:
>>>
>>> ...
>>> crmd[12917]: 2008/02/08_17:09:24 info: do_lrm_rsc_op: Performing
>>> op=squid-proxy-virtualip_monitor_0
>>> key=5:2:1940e09f-288a-48db-8b96-b20889734324)
>>> lrmd[12914]: 2008/02/08_17:09:24 info: rsc:squid-proxy-virtualip:
>>> monitor
>>> cib[12913]: 2008/02/08_17:09:24 info: apply_xml_diff: Digest mis-
>>> match:
>>> expected 3d6a03940246d1cd9ffb1e6ffd0b8d62, calculated
>>> 028a53e6d860e8f58e3a3a5b0f8cd3f0
>>> cib[12913]: 2008/02/08_17:09:24 info: cib_process_diff: Diff 0.7.3  
>>> ->
>>> 0.7.4not applied to
>>> 0.7.3: Failed application of a global update.  Requesting full
>>> refresh.
>>> cib[12913]: 2008/02/08_17:09:24 info: cib_process_diff: Requesting
>>> re-sync
>>> from peer: Failed application of a global update.  Requesting full
>>> refresh.
>>> cib[12913]: 2008/02/08_17:09:24 WARN: do_cib_notify: cib_apply_diff
>>> of <diff
>>>> FAILED: Application of an update diff failed, requesting a full
>>>> refresh
>>> cib[12913]: 2008/02/08_17:09:24 WARN: cib_process_request:
>>> cib_apply_diff
>>> operation failed: Application of an update diff failed, requesting a
>>> full
>>> refresh
>>> cib[12913]: 2008/02/08_17:09:24 WARN: cib_process_diff: Not applying
>>> diff
>>> 0.7.4 -> 0.7.5 (sync in progress)
>>> cib[12913]: 2008/02/08_17:09:24 WARN: do_cib_notify: cib_apply_diff
>>> of <diff
>>>> FAILED: Application of an update diff failed, requesting a full
>>>> refresh
>>> cib[12913]: 2008/02/08_17:09:24 WARN: cib_process_request:
>>> cib_apply_diff
>>> operation failed: Application of an update diff failed, requesting a
>>> full
>>> refresh
>>> cib[12913]: 2008/02/08_17:09:24 WARN: cib_process_diff: Not applying
>>> diff
>>> 0.7.5 -> 0.7.6 (sync in progress)
>>> cib[12913]: 2008/02/08_17:09:24 WARN: do_cib_notify: cib_apply_diff
>>> of <diff
>>>> FAILED: Application of an update diff failed, requesting a full
>>>> refresh
>>> cib[12913]: 2008/02/08_17:09:24 WARN: cib_process_request:
>>> cib_apply_diff
>>> operation failed: Application of an update diff failed, requesting a
>>> full
>>> refresh
>>> RnD-IPaddr[13016][13022]: 2008/02/08_17:09:24 DEBUG: monitoring
>>> virtual IP
>>> 192.168.123.180
>>> RnD-IPaddr[13016][13026]: 2008/02/08_17:09:24 DEBUG: squid-proxy-
>>> virtualip
>>> monitor : 7
>>> crmd[12917]: 2008/02/08_17:09:24 info: process_lrm_event: LRM
>>> operation
>>> squid-proxy-virtualip_monitor_0 (call=2, rc=7) complete
>>> attrd[12916]: 2008/02/08_17:09:25 info: main: Starting mainloop...
>>> cib[12913]: 2008/02/08_17:09:25 info: cib_replace_notify: Replaced:
>>> 0.7.3 ->
>>> 0.7.6 from <null>
>>> crmd[12917]: 2008/02/08_17:09:25 info: do_lrm_rsc_op: Performing
>>> op=squid-proxy-virtualip_start_0
>>> key=8:2:1940e09f-288a-48db-8b96-b20889734324)
>>> lrmd[12914]: 2008/02/08_17:09:25 info: rsc:squid-proxy-virtualip:
>>> start
>>> crmd[12917]: 2008/02/08_17:09:25 info: populate_cib_nodes:
>>> Requesting the
>>> list of configured nodes
>>> RnD-IPaddr[13041][13047]: 2008/02/08_17:09:25 DEBUG: starting
>>> squid-proxy-virtualip
>>> RnD-IPaddr[13041][13053]: 2008/02/08_17:09:25 DEBUG: squid-proxy-
>>> virtualip
>>> start : 0
>>> crmd[12917]: 2008/02/08_17:09:26 notice: populate_cib_nodes: Node:
>>> proxyre2.mmfg.it (uuid: 26ad35b2-cb0e-4671-999a-0ec1a262f16f)
>>> crmd[12917]: 2008/02/08_17:09:26 notice: populate_cib_nodes: Node:
>>> proxyre1.mmfg.it (uuid: a13e4a9b-5aea-4daf-82be-17a9553ce409)
>>> crmd[12917]: 2008/02/08_17:09:26 info: process_lrm_event: LRM
>>> operation
>>> squid-proxy-virtualip_start_0 (call=3, rc=0) complete
>>> crmd[12917]: 2008/02/08_17:09:27 info: do_lrm_rsc_op: Performing
>>> op=squid-proxy-virtualip_monitor_5000
>>> key=7:3:1940e09f-288a-48db-8b96-b20889734324)
>>> RnD-IPaddr[13093][13099]: 2008/02/08_17:09:27 DEBUG: monitoring
>>> virtual IP
>>> 192.168.123.180
>>> RnD-IPaddr[13093][13105]: 2008/02/08_17:09:27 DEBUG: squid-proxy-
>>> virtualip
>>> monitor : 0
>>> crmd[12917]: 2008/02/08_17:09:27 info: process_lrm_event: LRM
>>> operation
>>> squid-proxy-virtualip_monitor_5000 (call=4, rc=0) complete
>>> RnD-IPaddr[13106][13112]: 2008/02/08_17:09:32 DEBUG: monitoring
>>> virtual IP
>>> 192.168.123.180
>>> RnD-IPaddr[13106][13118]: 2008/02/08_17:09:32 DEBUG: squid-proxy-
>>> virtualip
>>> monitor : 0
>>> RnD-IPaddr[13119][13125]: 2008/02/08_17:09:37 DEBUG: monitoring
>>> virtual IP
>>> 192.168.123.180
>>> RnD-IPaddr[13119][13131]: 2008/02/08_17:09:37 DEBUG: squid-proxy-
>>> virtualip
>>> monitor : 0
>>> RnD-IPaddr[13132][13138]: 2008/02/08_17:09:42 DEBUG: monitoring
>>> virtual IP
>>> 192.168.123.180
>>> RnD-IPaddr[13132][13144]: 2008/02/08_17:09:42 DEBUG: squid-proxy-
>>> virtualip
>>> monitor : 0
>>> RnD-IPaddr[13145][13151]: 2008/02/08_17:09:47 DEBUG: monitoring
>>> virtual IP
>>> 192.168.123.180
>>> RnD-IPaddr[13145][13157]: 2008/02/08_17:09:47 DEBUG: squid-proxy-
>>> virtualip
>>> monitor : 0
>>> RnD-IPaddr[13158][13164]: 2008/02/08_17:09:52 DEBUG: monitoring
>>> virtual IP
>>> 192.168.123.180
>>> RnD-IPaddr[13158][13170]: 2008/02/08_17:09:52 DEBUG: squid-proxy-
>>> virtualip
>>> monitor : 0
>>> RnD-IPaddr[13171][13177]: 2008/02/08_17:09:57 DEBUG: monitoring
>>> virtual IP
>>> 192.168.123.180
>>> RnD-IPaddr[13171][13183]: 2008/02/08_17:09:57 DEBUG: squid-proxy-
>>> virtualip
>>> monitor : 0
>>> RnD-IPaddr[13202][13208]: 2008/02/08_17:10:02 DEBUG: monitoring
>>> virtual IP
>>> 192.168.123.180
>>> RnD-IPaddr[13202][13212]: 2008/02/08_17:10:02 DEBUG: squid-proxy-
>>> virtualip
>>> monitor : 7
>>> crmd[12917]: 2008/02/08_17:10:02 info: process_lrm_event: LRM
>>> operation
>>> squid-proxy-virtualip_monitor_5000 (call=4, rc=7) complete
>>> crmd[12917]: 2008/02/08_17:10:02 info: do_lrm_rsc_op: Performing
>>> op=squid-proxy-virtualip_stop_0
>>> key=2:5:1940e09f-288a-48db-8b96-b20889734324)
>>> lrmd[12914]: 2008/02/08_17:10:02 info: rsc:squid-proxy-virtualip:  
>>> stop
>>> crmd[12917]: 2008/02/08_17:10:02 info: process_lrm_event: LRM
>>> operation
>>> squid-proxy-virtualip_monitor_5000 (call=4, rc=-2) Cancelled
>>> RnD-IPaddr[13220][13229]: 2008/02/08_17:10:02 DEBUG: stopping
>>> squid-proxy-virtualip
>>> RnD-IPaddr[13220][13240]: 2008/02/08_17:10:03 DEBUG: squid-proxy-
>>> virtualip
>>> stop : 0
>>> crmd[12917]: 2008/02/08_17:10:03 info: process_lrm_event: LRM
>>> operation
>>> squid-proxy-virtualip_stop_0 (call=5, rc=0) complete
>>> crmd[12917]: 2008/02/08_17:10:03 info: do_lrm_rsc_op: Performing
>>> op=squid-proxy-virtualip_start_0
>>> key=7:5:1940e09f-288a-48db-8b96-b20889734324)
>>> lrmd[12914]: 2008/02/08_17:10:03 info: rsc:squid-proxy-virtualip:
>>> start
>>> RnD-IPaddr[13248][13261]: 2008/02/08_17:10:03 DEBUG: starting
>>> squid-proxy-virtualip
>>> RnD-IPaddr[13248][13287]: 2008/02/08_17:10:04 DEBUG: squid-proxy-
>>> virtualip
>>> start : 0
>>> crmd[12917]: 2008/02/08_17:10:04 info: process_lrm_event: LRM
>>> operation
>>> squid-proxy-virtualip_start_0 (call=6, rc=0) complete
>>> crmd[12917]: 2008/02/08_17:10:04 info: do_lrm_rsc_op: Performing
>>> op=squid-proxy-virtualip_monitor_5000
>>> key=1:5:1940e09f-288a-48db-8b96-b20889734324)
>>> RnD-IPaddr[13288][13294]: 2008/02/08_17:10:04 DEBUG: monitoring
>>> virtual IP
>>> 192.168.123.180
>>> RnD-IPaddr[13288][13300]: 2008/02/08_17:10:04 DEBUG: squid-proxy-
>>> virtualip
>>> monitor : 0
>>> crmd[12917]: 2008/02/08_17:10:04 info: process_lrm_event: LRM
>>> operation
>>> squid-proxy-virtualip_monitor_5000 (call=7, rc=0) complete
>>> RnD-IPaddr[13390][13396]: 2008/02/08_17:10:09 DEBUG: monitoring
>>> virtual IP
>>> 192.168.123.180
>>> RnD-IPaddr[13390][13400]: 2008/02/08_17:10:09 DEBUG: squid-proxy-
>>> virtualip
>>> monitor : 7
>>> crmd[12917]: 2008/02/08_17:10:09 info: process_lrm_event: LRM
>>> operation
>>> squid-proxy-virtualip_monitor_5000 (call=7, rc=7) complete
>>> crmd[12917]: 2008/02/08_17:10:10 info: do_lrm_rsc_op: Performing
>>> op=squid-proxy-virtualip_stop_0
>>> key=2:6:1940e09f-288a-48db-8b96-b20889734324)
>>> lrmd[12914]: 2008/02/08_17:10:10 info: rsc:squid-proxy-virtualip:  
>>> stop
>>> crmd[12917]: 2008/02/08_17:10:10 info: process_lrm_event: LRM
>>> operation
>>> squid-proxy-virtualip_monitor_5000 (call=7, rc=-2) Cancelled
>>> RnD-IPaddr[13445][13451]: 2008/02/08_17:10:10 DEBUG: stopping
>>> squid-proxy-virtualip
>>> RnD-IPaddr[13445][13462]: 2008/02/08_17:10:11 DEBUG: squid-proxy-
>>> virtualip
>>> stop : 0
>>> crmd[12917]: 2008/02/08_17:10:11 info: process_lrm_event: LRM
>>> operation
>>> squid-proxy-virtualip_stop_0 (call=8, rc=0) complete
>>> crmd[12917]: 2008/02/08_17:10:11 info: do_lrm_rsc_op: Performing
>>> op=squid-proxy-virtualip_start_0
>>> key=7:6:1940e09f-288a-48db-8b96-b20889734324)
>>> lrmd[12914]: 2008/02/08_17:10:11 info: rsc:squid-proxy-virtualip:
>>> start
>>> RnD-IPaddr[13470][13479]: 2008/02/08_17:10:11 DEBUG: starting
>>> squid-proxy-virtualip
>>> RnD-IPaddr[13470][13485]: 2008/02/08_17:10:12 DEBUG: squid-proxy-
>>> virtualip
>>> start : 0
>>> crmd[12917]: 2008/02/08_17:10:12 info: process_lrm_event: LRM
>>> operation
>>> squid-proxy-virtualip_start_0 (call=9, rc=0) complete
>>> crmd[12917]: 2008/02/08_17:10:12 info: do_lrm_rsc_op: Performing
>>> op=squid-proxy-virtualip_monitor_5000
>>> key=1:6:1940e09f-288a-48db-8b96-b20889734324)
>>> RnD-IPaddr[13520][13526]: 2008/02/08_17:10:12 DEBUG: monitoring
>>> virtual IP
>>> 192.168.123.180
>>> RnD-IPaddr[13520][13532]: 2008/02/08_17:10:12 DEBUG: squid-proxy-
>>> virtualip
>>> monitor : 0
>>> crmd[12917]: 2008/02/08_17:10:12 info: process_lrm_event: LRM
>>> operation
>>> squid-proxy-virtualip_monitor_5000 (call=10, rc=0) complete
>>> RnD-IPaddr[13621][13627]: 2008/02/08_17:10:17 DEBUG: monitoring
>>> virtual IP
>>> 192.168.123.180
>>> RnD-IPaddr[13621][13633]: 2008/02/08_17:10:17 DEBUG: squid-proxy-
>>> virtualip
>>> monitor : 0
>>> RnD-IPaddr[13677][13683]: 2008/02/08_17:10:22 DEBUG: monitoring
>>> virtual IP
>>> 192.168.123.180
>>> RnD-IPaddr[13677][13689]: 2008/02/08_17:10:22 DEBUG: squid-proxy-
>>> virtualip
>>> monitor : 0
>>> ....
>>>
>>>
>>>
>>> --
>>>
>>> Davide Belloni
>>> _______________________________________________
>>> Linux-HA mailing list
>>> Linux-HA at lists.linux-ha.org
>>> http://lists.linux-ha.org/mailman/listinfo/linux-ha
>>> See also: http://linux-ha.org/ReportingProblems
>>
>> _______________________________________________
>> Linux-HA mailing list
>> Linux-HA at lists.linux-ha.org
>> http://lists.linux-ha.org/mailman/listinfo/linux-ha
>> See also: http://linux-ha.org/ReportingProblems
>>
>
>
>
> -- 
> Serge Dubrouski.
> _______________________________________________
> Linux-HA mailing list
> Linux-HA at lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems