[Linux-HA] crm_abort: get_lrm_resource: Triggered non-fatal assert at lrm.c:864

Andrew Beekhof beekhof at gmail.com
Mon Mar 5 04:02:16 MST 2007 

On 3/4/07, Sebastian Reitenbach <sebastia at l00-bugdead-prods.de> wrote:
> Hi,
> >
> > First question: what type of resource is PPS_CACHE_1?
> I got these again,

with the patch applied?

> on a grouped resource, made of an IP and a Postgresql
> resource. see below
> how.
>
> > Second question: were you doing anything related to it with the GUI
> I had the group of the resources marked in the GUI, and clicked on the cleanup
> button in the
> top menu.
>
> > before it happened?
> > And lastly, the next line scares the crap out of me:
> >    <rsc_op transition_key="mgmtd-7264">
> >
> > where that came from I have NO idea.  this is what a transition key looks
> like:
> >    transition_key="12:4:a5b1d497-63e6-4ad8-9a58-907adc887a82"
> >
> > can you attach the logs from the node that was the DC at the time (ie.
> > running the tengine and pengine) by any chance?
> >
> > >
> > > I use heartbeat 2.0.8, on SLES 10, x86_64.
> > >
> logfile from this time with the group attached.
>
>
> Sebastian
>
>
> In the gui, on the menu, clicking the cleanup button:
>
>
> Mar  3 12:23:46 ppsnfs102 cib: [6946]: info: MSG[11] : [client_gen=3]
> Mar  3 12:23:46 ppsnfs102 cib: [6946]: info: MSG[12] : [src=ppsnfs101]
> Mar  3 12:23:46 ppsnfs102 cib: [6946]: info: MSG[13] : [(1)srcuuid=0x682218(3627)]
> Mar  3 12:23:46 ppsnfs102 cib: [6946]: info: MSG[14] : [seq=131b7]
> Mar  3 12:23:46 ppsnfs102 cib: [6946]: info: MSG[15] : [hg=1]
> Mar  3 12:23:46 ppsnfs102 cib: [6946]: info: MSG[16] : [ts=45e95ac2]
> Mar  3 12:23:46 ppsnfs102 cib: [6946]: info: MSG[17] : [ld=0.00 0.00 0.00 3/2658123]
> Mar  3 12:23:46 ppsnfs102 cib: [6946]: info: MSG[18] : [ttl=6]
> Mar  3 12:23:46 ppsnfs102 cib: [6946]: info: MSG[19] : [_compression_algorithm=zlib]
> Mar  3 12:23:46 ppsnfs102 cib: [6946]: WARN: cib_process_request: Request not broadcast: call failed: The object/attribute does not exist
> Mar  3 12:24:10 ppsnfs102 crmd: [6950]: ERROR: crm_abort: get_lrm_resource: Triggered non-fatal assert at lrm.c:864 : class != NULL
> Mar  3 12:24:10 ppsnfs102 crmd: [6950]: ERROR: do_lrm_invoke: Invalid resource definition
> Mar  3 12:24:10 ppsnfs102 crmd: [6950]: WARN: log_data_element: do_lrm_invoke: Bad command <rsc_op transition_key="mgmtd-6906">
> Mar  3 12:24:10 ppsnfs102 crmd: [6950]: WARN: log_data_element: do_lrm_invoke: Bad command   <primitive id="PGSQL_FIS"/>
> Mar  3 12:24:10 ppsnfs102 crmd: [6950]: WARN: log_data_element: do_lrm_invoke: Bad command   <attributes crm_feature_set="1.0.8"/>
> Mar  3 12:24:10 ppsnfs102 crmd: [6950]: WARN: log_data_element: do_lrm_invoke: Bad command </rsc_op>
> Mar  3 12:24:10 ppsnfs102 crmd: [6950]: info: append_restart_list: Resource PGSQL_MGMT does not support reloads
> Mar  3 12:24:10 ppsnfs102 crmd: [6950]: info: append_restart_list: Resource IP_MGMT does not support reloads
> Mar  3 12:24:10 ppsnfs102 crmd: [6950]: info: append_restart_list: Resource Stonith:0 does not support reloads
> Mar  3 12:24:10 ppsnfs102 crmd: [6950]: info: do_lrm_invoke: Forcing a local LRM refresh
> Mar  3 12:24:10 ppsnfs102 cib: [6946]: info: cib_diff_notify: Update (client: 6950, call:1312): 0.2.2787 -> 0.2.2788 (ok)
> Mar  3 12:24:10 ppsnfs102 tengine: [6962]: info: te_update_diff: Processing diff (cib_update): 0.2.2787 -> 0.2.2788
> Mar  3 12:24:10 ppsnfs102 cib: [18622]: info: write_cib_contents: Wrote version 0.2.2788 of the CIB to disk (digest: 7f8cb5205900f08da71444b613ac930d)
> Mar  3 12:24:11 ppsnfs102 cib: [6946]: info: cib_diff_notify: Update (client: 6905, call:472): 0.2.2788 -> 0.2.2789 (ok)
> Mar  3 12:24:11 ppsnfs102 tengine: [6962]: info: te_update_diff: Processing diff (cib_update): 0.2.2788 -> 0.2.2789
> Mar  3 12:24:11 ppsnfs102 cib: [6946]: info: cib_diff_notify: Update (client: 7171, call:647): 0.2.2789 -> 0.2.2790 (ok)
> Mar  3 12:24:11 ppsnfs102 tengine: [6962]: info: te_update_diff: Processing diff (cib_update): 0.2.2789 -> 0.2.2790
> Mar  3 12:24:11 ppsnfs102 cib: [18623]: info: write_cib_contents: Wrote version 0.2.2790 of the CIB to disk (digest: b4060eac9102ec83963509dcce3659e7)
> Mar  3 12:24:11 ppsnfs102 cib: [6946]: info: cib_diff_notify: Update (client: 7931, call:639): 0.2.2790 -> 0.2.2791 (ok)
> Mar  3 12:24:11 ppsnfs102 tengine: [6962]: info: te_update_diff: Processing diff (cib_update): 0.2.2790 -> 0.2.2791
> Mar  3 12:24:11 ppsnfs102 cib: [6946]: info: cib_diff_notify: Update (client: 5751, call:109): 0.2.2791 -> 0.2.2792 (ok)
> Mar  3 12:24:11 ppsnfs102 tengine: [6962]: info: te_update_diff: Processing diff (cib_update): 0.2.2791 -> 0.2.2792
> Mar  3 12:24:11 ppsnfs102 cib: [18624]: info: write_cib_contents: Wrote version 0.2.2792 of the CIB to disk (digest: fdfd0d62cf8c254f8ba48c9a5b82fc06)
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: WARN: do_cib_notify: cib_modify of <cluster_property_set > FAILED: The object/attribute does not exist
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: ERROR: cib_process_request: cib_modify operation failed: The object/attribute does not exist
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: crm_log_message_adv: #========= Input message message start ==========#
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG: Dumping message with 20 fields
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[0] : [t=cib]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[1] : [cib_clientid=cdc4ae03-4dcf-4c6c-b743-6ab51962cdee]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[2] : [cib_callopt=1052672]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[3] : [cib_callid=137]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[4] : [cib_op=cib_modify]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[5] : [cib_section=crm_config]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[6] : [cib_clientname=6906]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[7] : [(5)cib_calldata=0x5530f8(242 288)]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info:  <cluster_property_set id="cib-bootstrap-options">
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info:    <attributes>
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info:      <nvpair id="cib-bootstrap-options-last-lrm-refresh" name="last-lrm-refresh" value="1172921050"/>
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info:    </attributes>
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info:  </cluster_property_set>
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[8] : [cib_delegated_from=ppsnfs101]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[9] : [from_id=cib]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[10] : [to_id=cib]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[11] : [client_gen=3]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[12] : [src=ppsnfs101]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[13] : [(1)srcuuid=0x6708c8(3627)]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[14] : [seq=131d8]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[15] : [hg=1]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[16] : [ts=45e95adf]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[17] : [ld=0.00 0.00 0.00 3/2658173]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[18] : [ttl=6]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: info: MSG[19] : [_compression_algorithm=zlib]
> Mar  3 12:24:16 ppsnfs102 cib: [6946]: WARN: cib_process_request: Request not broadcast: call failed: The object/attribute does not exist
> Mar  3 12:24:17 ppsnfs102 crmd: [6950]: ERROR: crm_abort: get_lrm_resource: Triggered non-fatal assert at lrm.c:864 : class != NULL
> Mar  3 12:24:17 ppsnfs102 crmd: [6950]: ERROR: do_lrm_invoke: Invalid resource definition
> Mar  3 12:24:17 ppsnfs102 crmd: [6950]: WARN: log_data_element: do_lrm_invoke: Bad command <rsc_op transition_key="mgmtd-6906">
> Mar  3 12:24:17 ppsnfs102 crmd: [6950]: WARN: log_data_element: do_lrm_invoke: Bad command   <primitive id="IP_FIS"/>
> Mar  3 12:24:17 ppsnfs102 crmd: [6950]: WARN: log_data_element: do_lrm_invoke: Bad command   <attributes crm_feature_set="1.0.8"/>
> Mar  3 12:24:17 ppsnfs102 crmd: [6950]: WARN: log_data_element: do_lrm_invoke: Bad command </rsc_op>
> Mar  3 12:24:17 ppsnfs102 crmd: [6950]: info: append_restart_list: Resource PGSQL_MGMT does not support reloads
> Mar  3 12:24:17 ppsnfs102 crmd: [6950]: info: append_restart_list: Resource IP_MGMT does not support reloads
> Mar  3 12:24:17 ppsnfs102 crmd: [6950]: info: append_restart_list: Resource Stonith:0 does not support reloads
> Mar  3 12:24:17 ppsnfs102 crmd: [6950]: info: do_lrm_invoke: Forcing a local LRM refresh
> Mar  3 12:24:17 ppsnfs102 cib: [6946]: info: cib_diff_notify: Update (client: 7171, call:648): 0.2.2792 -> 0.2.2793 (ok)
> Mar  3 12:24:17 ppsnfs102 tengine: [6962]: info: te_update_diff: Processing diff (cib_update): 0.2.2792 -> 0.2.2793
> Mar  3 12:24:17 ppsnfs102 cib: [6946]: info: cib_diff_notify: Update (client: 6905, call:473): 0.2.2793 -> 0.2.2794 (ok)
> Mar  3 12:24:17 ppsnfs102 tengine: [6962]: info: te_update_diff: Processing diff (cib_update): 0.2.2793 -> 0.2.2794
> Mar  3 12:24:17 ppsnfs102 cib: [6946]: info: cib_diff_notify: Update (client: 6950, call:1313): 0.2.2794 -> 0.2.2795 (ok)
> Mar  3 12:24:17 ppsnfs102 tengine: [6962]: info: te_update_diff: Processing diff (cib_update): 0.2.2794 -> 0.2.2795
> Mar  3 12:24:17 ppsnfs102 cib: [6946]: info: cib_diff_notify: Update (client: 7931, call:640): 0.2.2795 -> 0.2.2796 (ok)
> Mar  3 12:24:17 ppsnfs102 tengine: [6962]: info: te_update_diff: Processing diff (cib_update): 0.2.2795 -> 0.2.2796
> Mar  3 12:24:17 ppsnfs102 cib: [6946]: info: cib_diff_notify: Update (client: 5751, call:110): 0.2.2796 -> 0.2.2797 (ok)
> Mar  3 12:24:17 ppsnfs102 tengine: [6962]: info: te_update_diff: Processing diff (cib_update): 0.2.2796 -> 0.2.2797
> Mar  3 12:24:17 ppsnfs102 cib: [18653]: info: write_cib_contents: Wrote version 0.2.2797 of the CIB to disk (digest: 6e38f86312b99a14f6a482040d0e6695)
> Mar  3 12:24:21 ppsnfs102 crmd: [6950]: info: do_state_transition: ppsnfs102: State
> transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
> Mar  3 12:24:21 ppsnfs102 tengine: [6962]: WARN: global_timer_callback: Timer popped (abort_level=1000000, complete=false)
> Mar  3 12:24:21 ppsnfs102 crmd: [6950]: info: do_state_transition: All 5 cluster nodes are eligable to run resources.
> Mar  3 12:24:21 ppsnfs102 tengine: [6962]: info: unconfirmed_actions: Waiting on 1 unconfirmed actions
> Mar  3 12:24:21 ppsnfs102 tengine: [6962]: WARN: global_timer_callback: Transition abort timeout reached... marking transition complete.
> Mar  3 12:24:21 ppsnfs102 tengine: [6962]: WARN: global_timer_callback: Writing 1 unconfirmed actions to the CIB
> Mar  3 12:24:21 ppsnfs102 tengine: [6962]: ERROR: unconfirmed_actions: Action 3 unconfirmed from peer
> Mar  3 12:24:21 ppsnfs102 tengine: [6962]: WARN: cib_action_update: rsc_op 3: PGSQL_FIS_monitor_20000 on ppsnfs102 timed out
> Mar  3 12:24:21 ppsnfs102 tengine: [6962]: info: unconfirmed_actions: Waiting on 1 unconfirmed actions
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: log_data_element: process_pe_message: [generation] <cib generated="true" admin_epoch="0" epoch="2" num_updates="2797" have_quorum="true" ignore_dtd="false" num_peers="5" ccm_transition="6" cib_feature_revision="1.3" dc_uuid="828cae3f-a6ea-4666-9e9c-769a839002d8"/>
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: notice: cluster_option: Using default value '(null)' for cluster option 'cluster-delay'
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: unpack_nodes: Node ppsbackup101 is in standby-mode
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: unpack_nodes: Node ppsdb102 is in standby-mode
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: unpack_nodes: Node ppsdb101 is in standby-mode
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: determine_online_status: Node ppsnfs102 is online
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: determine_online_status: Node ppsdb101 is online
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: unpack_find_resource: Internally renamed Stonith:0 on ppsdb101 to Stonith:1
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: determine_online_status: Node ppsnfs101 is online
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: unpack_find_resource: Internally renamed Stonith:0 on ppsnfs101 to Stonith:1
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: determine_online_status: Node ppsdb102 is online
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: unpack_find_resource: Internally renamed Stonith:0 on ppsdb102 to Stonith:2
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: unpack_find_resource: Internally renamed Stonith:1 on ppsdb102 to Stonith:2
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: determine_online_status: Node ppsbackup101 is online
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: unpack_find_resource: Internally renamed Stonith:0 on ppsbackup101 to Stonith:2
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: unpack_find_resource: Internally renamed Stonith:1 on ppsbackup101 to Stonith:2
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: clone_print: Clone Set: Clone_Stonith
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: native_print:     Stonith:0    (stonith:external/ilo): Started ppsnfs102
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: native_print:     Stonith:1    (stonith:external/ilo): Started ppsnfs101
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: native_print:     Stonith:2    (stonith:external/ilo): Stopped
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: native_print:     Stonith:3    (stonith:external/ilo): Stopped
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: native_print:     Stonith:4    (stonith:external/ilo): Stopped
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: group_print: Resource Group: Group_MGMT
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: native_print:     PGSQL_MGMT   (heartbeat::ocf:pgsql): Started ppsnfs102
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: native_print:     IP_MGMT      (heartbeat::ocf:IPaddr2):       Started ppsnfs102
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: group_print: Resource Group:Group_FIS
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: native_print:     PGSQL_FIS    (heartbeat::ocf:pgsql): Stopped
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: native_print:     IP_FIS       (heartbeat::ocf:IPaddr2):       Stopped
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: native_print: IP_NFS1  (heartbeat::ocf:IPaddr2):       Started ppsnfs101
> Mar  3 12:24:21 ppsnfs102 pengine: [6963]: info: native_print: IP_NFS2  (heartbeat::ocf:IPaddr2):       Started ppsnfs102
> _______________________________________________
> Linux-HA mailing list
> Linux-HA at lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
>

More information about the Linux-HA mailing list