[Linux-HA] Heartbeat: weird problem

Joe Abdo abdojoe at hotmail.com
Fri Oct 26 05:54:51 MDT 2007


Hi!
I have this strange heartbeat problem that is complicating my life
Im running heartbeat 2.0.8 on debian 2.6
Im using crm
Ve4 starting heartbeat on both nodes, i have cib.xml well configured using the python script.
After i start heartbeat, node 2 starts normally but i have problems with node 1: ACTUALLY IF I CHECK cib.xml ON NODE1 after starting heartbeat, I FIND IT EMPTY ... ALTHOUGH IT WAS GOOD BE4 STARTING HEARBEAT

In addition, always talking about node1, although the right permissions are set to /var/run/heartbeat/ccm/ccm  and var/run/heartbeat/ccm/crm as 777 and hacluster:haclient, when i try to access /var/run/heartbeat/ccm/ccm i get a "permission denied" in "vi" on node2.(P.S: on node 1 that is starting normally i have acess to /var/run/heartbeat/ccm/ccm)
i dunno if this has any relation to my problem but thought to mention it

below u can find my ha.cf, and some logs from ha-log
I hope you can help
Thanks for your time
Joe Abdo

ha.cf

ucast eth1 192.168.1.62
keepalive 1
deadtime 15
warntime 5
initdead 120 # depend on your hardware
udpport 694
ping 192.168.1.4
auto_failback off
node    DATADOMAIN-BDC
node    DATADOMAIN-PDC
use_logd yes
compression     bz2
compression_threshold 2
crm yes


ha-log

logd[24090]: 2007/10/26_11:10:27 info: logd started with /etc/logd.cf.
logd[24094]: 2007/10/26_11:10:27 info: G_main_add_SignalHandler: Added signal handler for signal 15
logd[24090]: 2007/10/26_11:10:27 info: G_main_add_SignalHandler: Added signal handler for signal 15
heartbeat[24111]: 2007/10/26_11:10:27 info: Enabling logging daemon
heartbeat[24111]: 2007/10/26_11:10:27 info: logfile and debug file are those specified in logd config file (default /etc/logd.cf)
heartbeat[24111]: 2007/10/26_11:10:27 WARN: File /etc/ha.d/haresources exists.
heartbeat[24111]: 2007/10/26_11:10:27 WARN: This file is not used because crm is enabled
heartbeat[24111]: 2007/10/26_11:10:27 WARN: logd is enabled but logfile/debugfile is still configured in ha.cf
heartbeat[24111]: 2007/10/26_11:10:27 info: **************************
heartbeat[24111]: 2007/10/26_11:10:27 info: Configuration validated. Starting heartbeat 2.0.7
heartbeat[24112]: 2007/10/26_11:10:27 info: heartbeat: version 2.0.7
heartbeat[24112]: 2007/10/26_11:10:27 info: Heartbeat generation: 9
heartbeat[24112]: 2007/10/26_11:10:27 info: G_main_add_TriggerHandler: Added signal manual handler
heartbeat[24112]: 2007/10/26_11:10:27 info: G_main_add_TriggerHandler: Added signal manual handler
heartbeat[24112]: 2007/10/26_11:10:27 info: Removing /var/run/heartbeat/rsctmp failed, recreating.
heartbeat[24112]: 2007/10/26_11:10:27 info: glib: ucast: write socket priority set to IPTOS_LOWDELAY on eth1
heartbeat[24112]: 2007/10/26_11:10:27 info: glib: ucast: bound send socket to device: eth1
heartbeat[24112]: 2007/10/26_11:10:27 info: glib: ucast: bound receive socket to device: eth1
heartbeat[24112]: 2007/10/26_11:10:27 info: glib: ucast: started on port 694 interface eth1 to 192.168.1.62
heartbeat[24112]: 2007/10/26_11:10:27 info: glib: ping heartbeat started.
heartbeat[24112]: 2007/10/26_11:10:27 info: G_main_add_SignalHandler: Added signal handler for signal 17
heartbeat[24112]: 2007/10/26_11:10:27 info: Local status now set to: 'up'
heartbeat[24112]: 2007/10/26_11:10:28 info: Link 192.168.1.4:192.168.1.4 up.
heartbeat[24112]: 2007/10/26_11:10:28 info: Status update for node 192.168.1.4: status ping
heartbeat[24112]: 2007/10/26_11:10:39 info: Link datadomain-bdc:eth1 up.
heartbeat[24112]: 2007/10/26_11:10:39 info: Status update for node datadomain-bdc: status up
heartbeat[24112]: 2007/10/26_11:10:40 info: Comm_now_up(): updating status to active
heartbeat[24112]: 2007/10/26_11:10:40 info: Local status now set to: 'active'
heartbeat[24112]: 2007/10/26_11:10:40 info: Starting child client "/usr/lib/heartbeat/ccm" (106,110)
heartbeat[24112]: 2007/10/26_11:10:40 info: Starting child client "/usr/lib/heartbeat/cib" (106,110)
heartbeat[24112]: 2007/10/26_11:10:40 info: Starting child client "/usr/lib/heartbeat/lrmd" (0,0)
heartbeat[24112]: 2007/10/26_11:10:40 info: Starting child client "/usr/lib/heartbeat/stonithd" (0,0)
heartbeat[24112]: 2007/10/26_11:10:40 info: Starting child client "/usr/lib/heartbeat/attrd" (106,110)
heartbeat[24112]: 2007/10/26_11:10:40 info: Starting child client "/usr/lib/heartbeat/crmd" (106,110)
heartbeat[24112]: 2007/10/26_11:10:40 info: Starting child client "/usr/lib/heartbeat/mgmtd -v" (0,0)
heartbeat[24122]: 2007/10/26_11:10:40 info: Starting "/usr/lib/heartbeat/ccm" as uid 106  gid 110 (pid 24122)
heartbeat[24123]: 2007/10/26_11:10:40 info: Starting "/usr/lib/heartbeat/cib" as uid 106  gid 110 (pid 24123)
heartbeat[24124]: 2007/10/26_11:10:40 info: Starting "/usr/lib/heartbeat/lrmd" as uid 0  gid 0 (pid 24124)
heartbeat[24125]: 2007/10/26_11:10:40 info: Starting "/usr/lib/heartbeat/stonithd" as uid 0  gid 0 (pid 24125)
heartbeat[24126]: 2007/10/26_11:10:40 info: Starting "/usr/lib/heartbeat/attrd" as uid 106  gid 110 (pid 24126)
heartbeat[24127]: 2007/10/26_11:10:40 info: Starting "/usr/lib/heartbeat/crmd" as uid 106  gid 110 (pid 24127)
heartbeat[24128]: 2007/10/26_11:10:40 info: Starting "/usr/lib/heartbeat/mgmtd -v" as uid 0  gid 0 (pid 24128)
lrmd[24124]: 2007/10/26_11:10:40 info: G_main_add_SignalHandler: Added signal handler for signal 15
lrmd[24124]: 2007/10/26_11:10:40 info: G_main_add_SignalHandler: Added signal handler for signal 17
lrmd[24124]: 2007/10/26_11:10:40 info: G_main_add_SignalHandler: Added signal handler for signal 10
lrmd[24124]: 2007/10/26_11:10:40 info: G_main_add_SignalHandler: Added signal handler for signal 12
lrmd[24124]: 2007/10/26_11:10:40 info: Started.
mgmtd[24128]: 2007/10/26_11:10:40 info: G_main_add_SignalHandler: Added signal handler for signal 15
mgmtd[24128]: 2007/10/26_11:10:40 info: G_main_add_SignalHandler: Added signal handler for signal 10
mgmtd[24128]: 2007/10/26_11:10:40 info: G_main_add_SignalHandler: Added signal handler for signal 12
heartbeat[24112]: 2007/10/26_11:10:40 info: Status update for node datadomain-bdc: status active
attrd[24126]: 2007/10/26_11:10:40 info: G_main_add_SignalHandler: Added signal handler for signal 15
ccm[24122]: 2007/10/26_11:10:40 info: Hostname: datadomain-pdc
attrd[24126]: 2007/10/26_11:10:40 info: register_with_ha:attrd.c Hostname: datadomain-pdc
cib[24123]: 2007/10/26_11:10:40 info: G_main_add_SignalHandler: Added signal handler for signal 15
cib[24123]: 2007/10/26_11:10:40 info: G_main_add_TriggerHandler: Added signal manual handler
cib[24123]: 2007/10/26_11:10:40 info: G_main_add_SignalHandler: Added signal handler for signal 17
cib[24123]: 2007/10/26_11:10:40 info: main:main.c Retrieval of a per-action CIB: disabled
cib[24123]: 2007/10/26_11:10:40 info: cib_register_ha:main.c Signing in with Heartbeat
cib[24123]: 2007/10/26_11:10:40 info: cib_register_ha:main.c FSA Hostname: datadomain-pdc
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile:io.c Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml
cib[24123]: 2007/10/26_11:10:40 WARN: validate_cib_digest:io.c No on-disk digest present
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk] 
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]   
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]     
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]       
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]         
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]         
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]       
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]     
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]     
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]     
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]       
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]         
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]             
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]             
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]               
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]             
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]         
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]         
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]             
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]         
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]       
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]     
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]     
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]       
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]         
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]           
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]         
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]       
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]     
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]   
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk]   
cib[24123]: 2007/10/26_11:10:40 info: readCibXmlFile: [on-disk] 
cib[24123]: 2007/10/26_11:10:40 info: activateCibXml:io.c CIB size is 44912 bytes (was 0)
cib[24123]: 2007/10/26_11:10:40 info: startCib:main.c CIB Initialization completed successfully
cib[24123]: 2007/10/26_11:10:40 WARN: init_start:main.c CCM Activation failed
cib[24123]: 2007/10/26_11:10:40 WARN: init_start:main.c CCM Connection failed 1 times (30 max)
attrd[24126]: 2007/10/26_11:10:40 info: register_with_ha:attrd.c UUID: fe7e083d-b165-495e-bcd9-97f394f5bff2
stonithd[24125]: 2007/10/26_11:10:40 info: G_main_add_SignalHandler: Added signal handler for signal 10
stonithd[24125]: 2007/10/26_11:10:40 info: G_main_add_SignalHandler: Added signal handler for signal 12
stonithd[24125]: 2007/10/26_11:10:40 info: Signing in with heartbeat.
crmd[24127]: 2007/10/26_11:10:40 info: init_start:main.c Starting crmd
crmd[24127]: 2007/10/26_11:10:40 info: G_main_add_SignalHandler: Added signal handler for signal 15
crmd[24127]: 2007/10/26_11:10:40 info: G_main_add_TriggerHandler: Added signal manual handler
crmd[24127]: 2007/10/26_11:10:40 info: G_main_add_SignalHandler: Added signal handler for signal 17
mgmtd[24128]: 2007/10/26_11:10:40 info: init_crm
stonithd[24125]: 2007/10/26_11:10:40 notice: /usr/lib/heartbeat/stonithd start up successfully.
stonithd[24125]: 2007/10/26_11:10:40 info: G_main_add_SignalHandler: Added signal handler for signal 17
cib[24123]: 2007/10/26_11:10:41 WARN: init_start:main.c CCM Activation failed
cib[24123]: 2007/10/26_11:10:41 WARN: init_start:main.c CCM Connection failed 2 times (30 max)
cib[24123]: 2007/10/26_11:10:42 WARN: init_start:main.c CCM Activation failed
cib[24123]: 2007/10/26_11:10:42 WARN: init_start:main.c CCM Connection failed 3 times (30 max)
ccm[24122]: 2007/10/26_11:10:43 info: G_main_add_SignalHandler: Added signal handler for signal 15
cib[24123]: 2007/10/26_11:10:43 info: init_start:main.c Starting cib mainloop
cib[24129]: 2007/10/26_11:10:43 WARN: validate_cib_digest:io.c No on-disk digest present
cib[24129]: 2007/10/26_11:10:43 info: write_cib_contents:io.c Wrote version 0.0.0 of the CIB to disk (digest: 41480864a95aa900ca2f7b570e67a99c)
cib[24123]: 2007/10/26_11:10:43 info: cib_client_status_callback:callbacks.c Status update: Client datadomain-pdc/cib now has status [join]
cib[24123]: 2007/10/26_11:10:43 info: cib_client_status_callback:callbacks.c Status update: Client datadomain-pdc/cib now has status [online]
crmd[24127]: 2007/10/26_11:10:43 info: do_cib_control:cib.c CIB connection established
crmd[24127]: 2007/10/26_11:10:43 info: register_with_ha:control.c Hostname: datadomain-pdc
cib[24123]: 2007/10/26_11:10:43 info: cib_null_callback:callbacks.c Setting cib_refresh_notify callbacks for crmd: on
cib[24123]: 2007/10/26_11:10:43 info: cib_null_callback:callbacks.c Setting cib_diff_notify callbacks for mgmtd: on
cib[24123]: 2007/10/26_11:10:44 info: cib_client_status_callback:callbacks.c Status update: Client datadomain-bdc/cib now has status [online]
crmd[24127]: 2007/10/26_11:10:44 info: register_with_ha:control.c UUID: fe7e083d-b165-495e-bcd9-97f394f5bff2
crmd[24127]: 2007/10/26_11:10:45 info: populate_cib_nodes:control.c Requesting the list of configured nodes
heartbeat[24112]: 2007/10/26_11:10:45 WARN: 1 lost packet(s) for [datadomain-bdc] [19:21]
heartbeat[24112]: 2007/10/26_11:10:45 info: No pkts missing from datadomain-bdc!
mgmtd[24128]: 2007/10/26_11:10:45 info: Started.
crmd[24127]: 2007/10/26_11:10:45 notice: populate_cib_nodes:control.c Node: datadomain-pdc (uuid: fe7e083d-b165-495e-bcd9-97f394f5bff2)
ccm[24122]: 2007/10/26_11:10:45 info: Break tie for 2 nodes cluster
cib[24123]: 2007/10/26_11:10:45 info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm
cib[24123]: 2007/10/26_11:10:45 info: mem_handle_event: instance=1, nodes=1, new=1, lost=0, n_idx=0, new_idx=0, old_idx=3
cib[24123]: 2007/10/26_11:10:45 info: cib_ccm_msg_callback:callbacks.c PEER: datadomain-pdc
heartbeat[24112]: 2007/10/26_11:10:46 WARN: 1 lost packet(s) for [datadomain-bdc] [25:27]
heartbeat[24112]: 2007/10/26_11:10:46 info: No pkts missing from datadomain-bdc!
crmd[24127]: 2007/10/26_11:10:46 notice: populate_cib_nodes:control.c Node: datadomain-bdc (uuid: ede73bcf-bf29-4a84-a560-554211996f59)
crmd[24127]: 2007/10/26_11:10:46 info: do_ha_control:control.c Connected to Heartbeat
cib[24123]: 2007/10/26_11:10:46 info: mem_handle_event: Got an event OC_EV_MS_INVALID from ccm
cib[24123]: 2007/10/26_11:10:46 info: mem_handle_event: no mbr_track info
cib[24123]: 2007/10/26_11:10:46 info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm
cib[24123]: 2007/10/26_11:10:46 info: mem_handle_event: instance=2, nodes=2, new=1, lost=0, n_idx=0, new_idx=2, old_idx=4
cib[24123]: 2007/10/26_11:10:46 info: cib_ccm_msg_callback:callbacks.c PEER: datadomain-pdc
cib[24123]: 2007/10/26_11:10:46 info: cib_ccm_msg_callback:callbacks.c PEER: datadomain-bdc
cib[24123]: 2007/10/26_11:10:46 info: activateCibXml:io.c CIB size is 47912 bytes (was 45120)
cib[24123]: 2007/10/26_11:10:46 info: cib_diff_notify:notify.c Local-only Change (client:24127, call: 3): 0.0.0 (ok)
cib[24130]: 2007/10/26_11:10:46 WARN: file2xml:xml.c File contained no XML
cib[24130]: 2007/10/26_11:10:46 ERROR: validate_cib_digest:io.c Digest comparision failed:  vs. (null)
cib[24130]: 2007/10/26_11:10:46 ERROR: write_cib_contents:io.c /var/lib/heartbeat/crm/cib.xml was manually modified while Heartbeat was active!
cib[24123]: 2007/10/26_11:10:46 ERROR: cib_diskwrite_complete:main.c Disk write failed: status=256, signo=0, exitcode=1
cib[24123]: 2007/10/26_11:10:46 ERROR: cib_diskwrite_complete:main.c Disabling disk writes after write failure
crmd[24127]: 2007/10/26_11:10:46 info: do_ccm_control:ccm.c CCM connection established... waiting for first callback
crmd[24127]: 2007/10/26_11:10:46 info: do_started:control.c Delaying start, CCM (0000000000100000) not connected
crmd[24127]: 2007/10/26_11:10:46 info: init_start:main.c Starting crmd's mainloop
crmd[24127]: 2007/10/26_11:10:46 notice: crmd_client_status_callback:callbacks.c Status update: Client datadomain-pdc/crmd now has status [online]
crmd[24127]: 2007/10/26_11:10:46 info: crmd_client_status_callback:callbacks.c Uncaching UUID for datadomain-pdc
crmd[24127]: 2007/10/26_11:10:47 notice: crmd_client_status_callback:callbacks.c Status update: Client datadomain-bdc/crmd now has status [online]
crmd[24127]: 2007/10/26_11:10:47 info: crmd_client_status_callback:callbacks.c Uncaching UUID for datadomain-bdc
cib[24123]: 2007/10/26_11:10:47 info: cib_diff_notify:notify.c Local-only Change (client:24127, call: 5): 0.0.0 (ok)
crmd[24127]: 2007/10/26_11:10:47 notice: crmd_client_status_callback:callbacks.c Status update: Client datadomain-pdc/crmd now has status [online]
crmd[24127]: 2007/10/26_11:10:47 info: crmd_client_status_callback:callbacks.c Uncaching UUID for datadomain-pdc
cib[24123]: 2007/10/26_11:10:47 info: cib_diff_notify:notify.c Local-only Change (client:24127, call: 6): 0.0.0 (ok)
crmd[24127]: 2007/10/26_11:10:47 notice: crmd_client_status_callback:callbacks.c Status update: Client datadomain-bdc/crmd now has status [online]
crmd[24127]: 2007/10/26_11:10:47 info: crmd_client_status_callback:callbacks.c Uncaching UUID for datadomain-bdc
crmd[24127]: 2007/10/26_11:10:48 info: do_started:control.c Delaying start, CCM (0000000000100000) not connected
crmd[24127]: 2007/10/26_11:10:48 info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm
crmd[24127]: 2007/10/26_11:10:48 info: mem_handle_event: instance=2, nodes=2, new=2, lost=0, n_idx=0, new_idx=0, old_idx=4
crmd[24127]: 2007/10/26_11:10:48 info: crmd_ccm_msg_callback:callbacks.c Quorum (re)attained after event=NEW MEMBERSHIP (id=2)
crmd[24127]: 2007/10/26_11:10:48 info: ccm_event_detail:ccm.c NEW MEMBERSHIP: trans=2, nodes=2, new=2, lost=0 n_idx=0, new_idx=0, old_idx=4
crmd[24127]: 2007/10/26_11:10:48 info: ccm_event_detail:ccm.c   CURRENT: datadomain-pdc [nodeid=1, born=1]
crmd[24127]: 2007/10/26_11:10:48 info: ccm_event_detail:ccm.c   CURRENT: datadomain-bdc [nodeid=0, born=2]
crmd[24127]: 2007/10/26_11:10:48 info: ccm_event_detail:ccm.c   NEW:     datadomain-pdc [nodeid=1, born=1]
crmd[24127]: 2007/10/26_11:10:48 info: ccm_event_detail:ccm.c   NEW:     datadomain-bdc [nodeid=0, born=2]
crmd[24127]: 2007/10/26_11:10:48 info: do_started:control.c The local CRM is operational
crmd[24127]: 2007/10/26_11:10:48 info: do_state_transition:fsa.c datadomain-pdc: State transition S_STARTING -> S_PENDING [ input=I_PENDING cause=C_CCM_CALLBACK origin=do_started ]
crmd[24127]: 2007/10/26_11:10:48 info: update_dc:utils.c Set DC to  ()
cib[24123]: 2007/10/26_11:10:48 info: cib_diff_notify:notify.c Local-only Change (client:24127, call: 9): 0.0.0 (ok)
attrd[24126]: 2007/10/26_11:10:48 info: main:attrd.c Starting mainloop...
crmd[24127]: 2007/10/26_11:11:49 info: crm_timer_popped:utils.c Election Trigger (I_DC_TIMEOUT) just popped!
crmd[24127]: 2007/10/26_11:11:49 WARN: do_log:misc.c [[FSA]] Input I_DC_TIMEOUT from crm_timer_popped() received in state (S_PENDING)
crmd[24127]: 2007/10/26_11:11:49 info: do_state_transition:fsa.c datadomain-pdc: State transition S_PENDING -> S_ELECTION [ input=I_DC_TIMEOUT cause=C_TIMER_POPPED origin=crm_timer_popped ]
crmd[24127]: 2007/10/26_11:11:49 info: update_dc:utils.c Set DC to  ()
crmd[24127]: 2007/10/26_11:11:49 info: do_election_count_vote:election.c Election check: vote from datadomain-bdc
crmd[24127]: 2007/10/26_11:11:49 info: do_election_count_vote:election.c Election won over datadomain-bdc
crmd[24127]: 2007/10/26_11:11:49 info: do_election_check:election.c Still waiting on 2 non-votes (2 total)
crmd[24127]: 2007/10/26_11:11:49 info: do_election_count_vote:election.c Updated voted hash for datadomain-pdc to vote
crmd[24127]: 2007/10/26_11:11:49 info: do_election_count_vote:election.c Election ignore: our vote (datadomain-pdc)
crmd[24127]: 2007/10/26_11:11:49 info: do_election_check:election.c Still waiting on 1 non-votes (2 total)
crmd[24127]: 2007/10/26_11:11:49 info: do_election_count_vote:election.c Updated voted hash for datadomain-pdc to vote
crmd[24127]: 2007/10/26_11:11:49 info: do_election_count_vote:election.c Election ignore: our vote (datadomain-pdc)
crmd[24127]: 2007/10/26_11:11:49 info: do_election_check:election.c Still waiting on 1 non-votes (2 total)
crmd[24127]: 2007/10/26_11:11:50 info: do_election_count_vote:election.c Updated voted hash for datadomain-bdc to no-vote
crmd[24127]: 2007/10/26_11:11:50 info: do_election_count_vote:election.c Election ignore: no-vote from datadomain-bdc
crmd[24127]: 2007/10/26_11:11:50 info: do_election_check:election.c Still waiting on 1 non-votes (2 total)
crmd[24127]: 2007/10/26_11:11:50 info: do_state_transition:fsa.c datadomain-pdc: State transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_election_check ]
crmd[24127]: 2007/10/26_11:11:50 info: start_subsystem:subsystems.c Starting sub-system "tengine"
crmd[24127]: 2007/10/26_11:11:50 info: start_subsystem:subsystems.c Starting sub-system "pengine"
tengine[24151]: 2007/10/26_11:11:50 info: G_main_add_SignalHandler: Added signal handler for signal 15
crmd[24127]: 2007/10/26_11:11:50 info: do_dc_takeover:election.c Taking over DC status for this partition
tengine[24151]: 2007/10/26_11:11:50 info: G_main_add_TriggerHandler: Added signal manual handler
crmd[24127]: 2007/10/26_11:11:50 info: update_dc:utils.c Set DC to  ()
crmd[24127]: 2007/10/26_11:11:50 info: do_dc_join_offer_all:join_dc.c join-1: Waiting on 2 outstanding join acks
cib[24123]: 2007/10/26_11:11:50 info: cib_process_readwrite:messages.c We are now in R/W mode
cib[24123]: 2007/10/26_11:11:50 info: revision_check:messages.c Updating CIB revision to 1.3
cib[24123]: 2007/10/26_11:11:50 info: cib_diff_notify:notify.c Update (client: 24127, call:13): 0.0.0 -> 0.0.1 (ok)
crmd[24127]: 2007/10/26_11:11:50 info: update_dc:utils.c Set DC to datadomain-pdc (1.0.6)
cib[24123]: 2007/10/26_11:11:50 info: cib_null_callback:callbacks.c Setting cib_diff_notify callbacks for tengine: on
tengine[24151]: 2007/10/26_11:11:50 info: init_start:main.c Registering TE UUID: df9db025-8ad3-4239-8417-ea52c343fbeb
tengine[24151]: 2007/10/26_11:11:50 info: set_graph_functions:utils.c Setting custom graph functions
tengine[24151]: 2007/10/26_11:11:50 info: unpack_graph:unpack.c Unpacked transition -1: 0 actions in 0 synapses
tengine[24151]: 2007/10/26_11:11:50 info: init_start:main.c Starting tengine
pengine[24152]: 2007/10/26_11:11:50 info: G_main_add_SignalHandler: Added signal handler for signal 15
pengine[24152]: 2007/10/26_11:11:50 info: init_start:main.c Starting pengine
crmd[24127]: 2007/10/26_11:11:51 info: do_state_transition:fsa.c datadomain-pdc: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ]
crmd[24127]: 2007/10/26_11:11:51 info: do_state_transition:fsa.c All 2 cluster nodes responded to the join offer.
crmd[24127]: 2007/10/26_11:11:51 info: update_attrd:join_dc.c Connecting to attrd...
attrd[24126]: 2007/10/26_11:11:51 info: attrd_local_callback:attrd.c Sending full refresh
cib[24123]: 2007/10/26_11:11:51 info: sync_our_cib:messages.c Syncing CIB to all peers
cib[24123]: 2007/10/26_11:11:51 info: cib_diff_notify:notify.c Update (client: 24127, call:16): 0.0.1 -> 0.0.2 (ok)
tengine[24151]: 2007/10/26_11:11:51 info: te_update_diff:callbacks.c Processing diff (cib_update): 0.0.1 -> 0.0.2
cib[24123]: 2007/10/26_11:11:51 info: cib_diff_notify:notify.c Update (client: 24127, call:17): 0.0.2 -> 0.1.3 (ok)
tengine[24151]: 2007/10/26_11:11:51 info: te_update_diff:callbacks.c Processing diff (cib_bump): 0.0.2 -> 0.1.3
cib[24123]: 2007/10/26_11:11:52 info: cib_diff_notify:notify.c Update (client: 24127, call:18): 0.1.3 -> 0.1.4 (ok)
tengine[24151]: 2007/10/26_11:11:52 info: te_update_diff:callbacks.c Processing diff (cib_update): 0.1.3 -> 0.1.4
cib[24123]: 2007/10/26_11:11:52 info: cib_diff_notify:notify.c Update (client: 24127, call:19): 0.1.4 -> 0.1.5 (ok)
tengine[24151]: 2007/10/26_11:11:52 info: te_update_diff:callbacks.c Processing diff (cib_update): 0.1.4 -> 0.1.5
crmd[24127]: 2007/10/26_11:11:52 info: update_dc:utils.c Set DC to datadomain-pdc (1.0.6)
crmd[24127]: 2007/10/26_11:11:52 info: do_dc_join_ack:join_dc.c join-1: Updating node state to member for datadomain-pdc)
cib[24123]: 2007/10/26_11:11:52 info: cib_diff_notify:notify.c Update (client: 24127, call:20): 0.1.5 -> 0.1.6 (ok)
tengine[24151]: 2007/10/26_11:11:52 info: te_update_diff:callbacks.c Processing diff (cib_update): 0.1.5 -> 0.1.6
crmd[24127]: 2007/10/26_11:11:53 info: do_dc_join_ack:join_dc.c join-1: Updating node state to member for datadomain-bdc)
cib[24123]: 2007/10/26_11:11:54 info: cib_diff_notify:notify.c Update (client: 24127, call:21): 0.1.6 -> 0.1.7 (ok)
crmd[24127]: 2007/10/26_11:11:54 info: do_state_transition:fsa.c datadomain-pdc: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE [ input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state ]
crmd[24127]: 2007/10/26_11:11:54 info: do_state_transition:fsa.c All 2 cluster nodes are eligable to run resources.
tengine[24151]: 2007/10/26_11:11:54 info: te_update_diff:callbacks.c Processing diff (cib_update): 0.1.6 -> 0.1.7
tengine[24151]: 2007/10/26_11:11:54 WARN: process_graph_event:events.c Event not found.
tengine[24151]: 2007/10/26_11:11:54 info: process_graph_event: match:not-found 
tengine[24151]: 2007/10/26_11:11:54 info: update_abort_priority:utils.c Abort priority upgraded to 1000000
tengine[24151]: 2007/10/26_11:11:54 WARN: process_graph_event:events.c Event not found.
tengine[24151]: 2007/10/26_11:11:54 info: process_graph_event: match:not-found 
pengine[24152]: 2007/10/26_11:11:56 info: get_last_sequence:utils.c /var/lib/heartbeat/pengine/pe-input.last was not valid
pengine[24152]: 2007/10/26_11:11:56 ERROR: write_xml_file:xml.c bzWriteClose() failed: -6
pengine[24152]: 2007/10/26_11:11:56 ERROR: Cannot write output to /var/lib/heartbeat/pengine/pe-input-0.bz2: No space left on device
pengine[24152]: 2007/10/26_11:11:56 info: process_pe_message:pengine.c Transition 1: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-0.bz2
crmd[24127]: 2007/10/26_11:11:56 info: process_lrm_event:lrm.c LRM operation (3) monitor_0 on exim4_2 Error: (7) not running
cib[24123]: 2007/10/26_11:11:56 info: cib_diff_notify:notify.c Update (client: 24127, call:53): 0.1.7 -> 0.1.8 (ok)
tengine[24151]: 2007/10/26_11:11:56 info: te_update_diff:callbacks.c Processing diff (cib_update): 0.1.7 -> 0.1.8
tengine[24151]: 2007/10/26_11:11:56 info: match_graph_event:events.c Action exim4_2_monitor_0 (4) confirmed
crmd[24127]: 2007/10/26_11:11:56 info: process_lrm_event:lrm.c LRM operation (2) monitor_0 on IPaddr_192_168_1_11 Error: (7) not running
cib[24123]: 2007/10/26_11:11:56 info: cib_diff_notify:notify.c Update (client: 24127, call:54): 0.1.8 -> 0.1.9 (ok)
tengine[24151]: 2007/10/26_11:11:56 info: te_update_diff:callbacks.c Processing diff (cib_update): 0.1.8 -> 0.1.9
tengine[24151]: 2007/10/26_11:11:56 info: match_graph_event:events.c Action IPaddr_192_168_1_11_monitor_0 (3) confirmed
tengine[24151]: 2007/10/26_11:11:56 info: send_rsc_command:actions.c Initiating action 2: probe_complete on datadomain-pdc
tengine[24151]: 2007/10/26_11:11:56 info: te_pseudo_action:actions.c Pseudo action 1 confirmed
tengine[24151]: 2007/10/26_11:11:56 info: te_pseudo_action:actions.c Pseudo action 10 confirmed


_________________________________________________________________
Discover the new Windows Vista
http://search.msn.com/results.aspx?q=windows+vista&mkt=en-US&form=QBRE


More information about the Linux-HA mailing list