[Linux-HA] resource monitoring and restarting

[Andrew Beekhof]

On Jul 11, 2008, at 1:23 AM, Steve Wray wrote:

> I am wondering if the Debian Etch nfs init scripts are LSB  
> compliant...
>
> After some experimenting I was shocked to find that /etc/init.d/nfs- 
> kernel-server stop returned status 0 yet did not stop anything at all.

then thats a no :)

i documented a series of tests to check for compliance a while back:
    http://linux-ha.org/LSBResourceAgent

>
>
> After running the stop command, status shows that its still running.
>
>
> Steve Wray wrote:
>> Raghuram Bondalapati wrote:
>>> I think if you set Resource Failure Stickiness to -INFINITY on  
>>> mysql service
>>> it should failover.
>>>
>>> In general if you set Default Resource Failure Stickiness to - 
>>> INFINITY, it
>>> will failover any failed service to the other node instead of  
>>> trying to
>>> restart.
>> This is interesting.
>> Its working for mysql but not for nfs...
>> Somehow after the nfs server processes are killed off heartbeat  
>> does respond in a way; the drbd devices end up secondary/secondary  
>> and it gets stuck there.
>> It appears that heartbeat does notice that nfs is dead and does try  
>> to migrate it but fails somehow, not obvious to me from the logs.
>> The server where nfs was killed unmounts its drbd disk and sets its  
>> status to secondary.
>> It looks as if the nfs init scripts on the failover node failed in  
>> some way and heartbeat then backs out. Heartbeat does not try to  
>> fail back to the original node (where nfs was forcibly killed off  
>> by me).
>> I'm wondering if the nfs init scripts are being run before the drbd  
>> device is made primary and mounted... this would result in their  
>> failing.
>> However, given that it works absolutely fine when:
>> 1. With stickiness set to 0 it restarts nfs on the node where I  
>> manually killed it.
>> 2. With stickiness 0, if I manually kill the server itself instead  
>> of just killing nfs its fine; fails over.
>> but not fine when...
>> 3. With stickiness -INFINITY, if I manually kill the server itself  
>> instead of just killing nfs, then failover fails. This is bad.
>> 4. With stickiness -INFINITY, if I kill the nfs processes then  
>> failover fails.
>> :(
>>> --Raghu
>>>
>>>
>>> On 7/9/08, Steve Wray <steve.wray at cwa.co.nz> wrote:
>>>> Hi there,
>>>>
>>>> I've gotten the v2 config working with monitoring on NFS and  
>>>> MySQL on a
>>>> couple of pairs of servers.
>>>>
>>>> Its not going quite as I had expected though.
>>>>
>>>> If, on the current primary I forcably kill mysqld, heartbeat does  
>>>> detect
>>>> this but it restarts it on that node.
>>>>
>>>> What I was hoping for was that the death of mysqld would result  
>>>> in a
>>>> failover to the other node.
>>>>
>>>> My reasoning is that (in real life) mysqld died for a *reason* on  
>>>> this host
>>>> and that rather than constantly restarting it on the same host  
>>>> perhaps it
>>>> would be best to just give up on trying to run mysql on that host  
>>>> and run it
>>>> on another?
>>>>
>>>> Any ideas?
>>>>
>>>> Thanks!
>>>> _______________________________________________
>>>> Linux-HA mailing list
>>>> Linux-HA at lists.linux-ha.org
>>>> http://lists.linux-ha.org/mailman/listinfo/linux-ha
>>>> See also: http://linux-ha.org/ReportingProblems
>>>>
>>> _______________________________________________
>>> Linux-HA mailing list
>>> Linux-HA at lists.linux-ha.org
>>> http://lists.linux-ha.org/mailman/listinfo/linux-ha
>>> See also: http://linux-ha.org/ReportingProblems
>> _______________________________________________
>> Linux-HA mailing list
>> Linux-HA at lists.linux-ha.org
>> http://lists.linux-ha.org/mailman/listinfo/linux-ha
>> See also: http://linux-ha.org/ReportingProblems
>
> _______________________________________________
> Linux-HA mailing list
> Linux-HA at lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems