[prev in list] [next in list] [prev in thread] [next in thread]
List: linux-ha
Subject: [Linux-HA] Problem: DRBD 8.3 + Pacemaker 1.1.2 + SLES11SP1 on
From: "Thomas Fachtan [motiondrive AG]" <thomas.fachtan () motiondrive ! ag>
Date: 2011-01-25 9:54:12
Message-ID: B37E40AEA106114A92DC3AD580EDECC30EF70692 () EXCH01 ! domain ! local
[Download RAW message or body]
Hi there,
I am currently working on a 2 node drbd+pacemaker mysql active / passive cluster \
under SLES11 SP1 + HAE on VMware vSphere 4. The Clusters working quite well when I`m \
testing failover situations etc., but irregularly reboots both nodes without logging \
the reason for the reboots. As nothing like stonith shutdown commands or error \
notices appear in the system logs, it looks like the virtual machine got powered off \
suddenly. Other VMs (yet SLES10 with Heartbeat + Stonith) running without any \
problems on the virtual infrastructure.. Yesterday I set up 2 clones of the virtual \
machines and configured them running without stonith device. These 2 clones survived \
the night. The ones I`m talking about right now shut down at different time (one at \
7:57pm, one at 05:19 am this morning). Could be a coincidence but to be honest I \
don`t think so, because I got the same problems for weeks now, even after completely \
configuring the VMs from OS Installation until cluster configuration.. I will monitor \
the clones without stonith device over the weekend to see if theyre running without \
rebooting. Anyone got similar problems or can tell me where to look for restricting \
what could be the problem?
Thanks in advance!
In order to help you understand my configuration, some facts:
Summary of setup:
- 2 x SLES11 SP1 + HAE virtual machines on VMware vSphere4
- both VMs running on same ESX
- VMware Tools installed
- 2 x vNICs: 1 for Public Interface, 1 for Private Interconnect
- 3 vDisks: 1 for OS, 1 for DRBD, 1 shared disc for Stonith SBD
- pacemaker-1.1.2-0.2.1
- openais-1.1.2-0.5.19
- drbd-8.3.7-0.4.15
- Active / Passive Configuration
lsmod | grep softdog
softdog 4055 2
/etc/sysconfig/sbd
SBD_DEVICE="/dev/sdc"
SBD_OPTS="-W"
/var/log/warn
...
...
Jan 21 09:50:09 mysql-node1 stonith: external/sbd device OK.
Jan 21 09:51:10 mysql-node1 stonith: external/sbd device OK.
Jan 21 09:52:11 mysql-node1 stonith: external/sbd device OK.
...
...
CRM Configure
node mysql-node1 \
attributes standby="off"
node mysql-node2 \
attributes standby="off"
primitive drbd_mysql ocf:linbit:drbd \
params drbd_resource="mysql" \
op monitor interval="15s"
primitive fs_mysql ocf:heartbeat:Filesystem \
params device="/dev/drbd0" directory="/var/lib/mysql" fstype="ext3"
primitive ip_mysql ocf:heartbeat:IPaddr2 \
params ip="192.168.96.10" nic="eth0"
primitive mysqld lsb:mysql
primitive notification ocf:heartbeat:MailTo \
meta target-role="Started" \
operations $id="notification-operations" \
op monitor interval="10" timeout="10" \
params email="xxx@xxx.xx" subject="Cluster Takeover on mysql-cluster"
primitive ping ocf:pacemaker:ping \
op monitor interval="15" timeout="5" \
params host_list="192.168.96.2"
primitive stonith-device stonith:external/sbd \
params sbd_device="/dev/sdc" \
op monitor interval="60s"
group mysql fs_mysql ip_mysql mysqld
ms ms_drbd_mysql drbd_mysql \
meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" \
notify="true" clone notificationclone notification
clone pingclone ping \
meta globally-unique="false"
clone stonith stonith-device
location location_mysql mysql \
rule $id="location_mysql-rule" -inf: not_defined pingd or pingd lte 0
colocation mysql_on_drbd inf: mysql ms_drbd_mysql:Master
order mysql_after_drbd inf: ms_drbd_mysql:promote mysql:start
property $id="cib-bootstrap-options" \
dc-version="1.1.2-2e096a41a5f9e184a1c1537c82c6da1093698eb5" \
cluster-infrastructure="openais" \
expected-quorum-votes="2" \
last-lrm-refresh="1294994548" \
no-quorum-policy="ignore" \
stonith-enabled="true"
Corosync.conf
compatibility: whitetank
aisexec {
user: root
group: root
}
totem {
version: 2
token: 3000
join: 1000
token_retransmits_before_loss_const: 20
consensus: 3600
max_messages: 20
clear_node_high_bit: yes
secauth: off
threads: 0
interface {
ringnumber: 0
bindnetaddr: 10.0.0.0
mcastaddr: 226.94.1.1
mcastport: 5405
}
}
logging {
fileline: off
to_stderr: no
to_logfile: no
to_syslog: yes
syslog_facility: daemon
debug: on
timestamp: on
}
amf {
mode: disabled
}
service {
name: pacemaker
ver: 0
use_mgmtd: yes
use_logd: yes
}
/etc/drbd.d/mysql.res
resource mysql {
protocol C;
startup {
wfc-timeout 0;
degr-wfc-timeout 120;
}
disk {
on-io-error detach;
}
net {
max-buffers 2048;
ko-count 4;
after-sb-0pri disconnect;
after-sb-1pri disconnect;
after-sb-2pri disconnect;
rr-conflict disconnect;
}
syncer {
rate 100M;
al-extents 257;
}
on mysql-node1 {
device /dev/drbd0;
disk /dev/sdb;
address 10.0.0.101:7788;
meta-disk internal;
}
on mysql-node2 {
device /dev/drbd0;
disk /dev/sdb;
address 10.0.0.102:7788;
meta-disk internal;
}
}
/etc/drbd.d/global_common.conf
global {
# usage-count yes;
usage-count no;
# minor-count dialog-refresh disable-ip-verification
}
common {
protocol C;
handlers {
pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; \
/usr/lib/drbd/notify-emergency-reboot.sh; echo b > \
/proc/sysrq-trigger ; reboot -f";
pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; \
/usr/lib/drbd/notify-emergency-reboot.sh; echo b > \
/proc/sysrq-trigger ; reboot -f";
local-io-error "/usr/lib/drbd/notify-io-error.sh; \
/usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f";
}
startup {
}
disk {
}
net {
}
syncer {
rate 100M;
}
}
Node1 debug log:
Jan 22 10:24:17 mysql-node1 attrd_updater: [14240]: info: attrd_lazy_update: \
Connecting to cluster... 5 retries remaining
Jan 22 10:24:17 mysql-node1 attrd_updater: [14240]: debug: \
init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Jan 22 10:24:17 mysql-node1 attrd: [9245]: debug: attrd_local_callback: update \
message from attrd_updater: pingd=1
Jan 22 10:24:17 mysql-node1 attrd: [9245]: debug: attrd_local_callback: Supplied: 1, \
Current: 1, Stored: 1
Jan 22 10:24:17 mysql-node1 attrd_updater: [14240]: debug: attrd_update: Sent update: \
pingd=1 for localhost
Jan 22 10:24:18 mysql-node1 lrmd: [9244]: debug: rsc:notification:0:41: monitor
Jan 22 10:24:18 mysql-node1 lrmd: [14241]: debug: perform_ra_op: resetting scheduler \
class to SCHED_OTHER
Jan 22 10:24:18 mysql-node1 lrmd: [9244]: debug: RA output: \
(notification:0:monitor:stdout) running
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: crm_timer_popped: PEngine Recheck \
Timer (I_PE_CALC) just popped!
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: s_crmd_fsa: Processing I_PE_CALC: [ \
state=S_IDLE cause=C_TIMER_POPPED origin=crm_timer_popped ]
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: do_state_transition: State transition \
S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED \
origin=crm_timer_popped ]
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: do_state_transition: Progressed to \
state S_POLICY_ENGINE after C_TIMER_POPPED
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: do_state_transition: All 2 cluster \
nodes are eligible to run resources.
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace: // \
A_DC_TIMER_STOP
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace: // \
A_INTEGRATE_TIMER_STOP
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace: // \
A_FINALIZE_TIMER_STOP
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace: // \
A_PE_INVOKE
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: do_pe_invoke: Query 207: Requesting \
the current CIB: S_POLICY_ENGINE
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: do_pe_invoke_callback: Invoking the \
PE: query=207, ref=pe_calc-dc-1295688261-214, seq=220, quorate=1
Jan 22 10:24:21 mysql-node1 pengine: [9246]: info: unpack_config: Startup probes: \
enabled
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: s_crmd_fsa: Processing I_PE_SUCCESS: \
[ state=S_POLICY_ENGINE cause=C_IPC_MESSAGE origin=handle_response ]
Jan 22 10:24:21 mysql-node1 pengine: [9246]: debug: unpack_config: STONITH timeout: \
60000
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace: // \
A_LOG
Jan 22 10:24:21 mysql-node1 pengine: [9246]: debug: unpack_config: STONITH of failed \
nodes is enabled
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: do_state_transition: State transition \
S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE \
origin=handle_response ]
Jan 22 10:24:21 mysql-node1 pengine: [9246]: debug: unpack_config: Stop all active \
resources: false
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace: // \
A_DC_TIMER_STOP
Jan 22 10:24:21 mysql-node1 pengine: [9246]: debug: unpack_config: Cluster is \
symmetric - resources can run anywhere by default
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace: // \
A_INTEGRATE_TIMER_STOP
Jan 22 10:24:21 mysql-node1 pengine: [9246]: debug: unpack_config: Default \
stickiness: 0
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace: // \
A_FINALIZE_TIMER_STOP
Jan 22 10:24:21 mysql-node1 pengine: [9246]: notice: unpack_config: On loss of CCM \
Quorum: Ignore
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace: // \
A_TE_INVOKE
Jan 22 10:24:21 mysql-node1 pengine: [9246]: info: unpack_config: Node scores: 'red' \
= -INFINITY, 'yellow' = 0, 'green' = 0
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: unpack_graph: Unpacked transition \
114: 0 actions in 0 synapses
Jan 22 10:24:21 mysql-node1 pengine: [9246]: info: unpack_domains: Unpacking domains
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: do_te_invoke: Processing graph 114 \
(ref=pe_calc-dc-1295688261-214) derived from /var/lib/pengine/pe-input-659.bz2
Jan 22 10:24:21 mysql-node1 pengine: [9246]: info: determine_online_status: Node \
mysql-node1 is online
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: print_graph: ## Empty transition \
graph ##
Jan 22 10:24:21 mysql-node1 pengine: [9246]: info: determine_online_status: Node \
mysql-node2 is online
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: run_graph: \
====================================================
Jan 22 10:24:21 mysql-node1 pengine: [9246]: info: find_clone: Internally renamed \
drbd_mysql:0 on mysql-node2 to drbd_mysql:1
Jan 22 10:24:21 mysql-node1 crmd: [9247]: notice: run_graph: Transition 114 \
(Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, \
Source=/var/lib/pengine/pe-input-659.bz2): Complete
Jan 22 10:24:21 mysql-node1 pengine: [9246]: info: find_clone: Internally renamed \
stonith-device:0 on mysql-node2 to stonith-device:1
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: print_graph: ## Empty transition \
graph ##
Jan 22 10:24:21 mysql-node1 pengine: [9246]: info: find_clone: Internally renamed \
notification:0 on mysql-node2 to notification:1
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: te_graph_trigger: Transition 114 is \
now complete
Jan 22 10:24:21 mysql-node1 pengine: [9246]: info: find_clone: Internally renamed \
ping:0 on mysql-node2 to ping:1
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: notify_crmd: Processing transition \
completion in state S_TRANSITION_ENGINE
Jan 22 10:24:21 mysql-node1 pengine: [9246]: notice: clone_print: Master/Slave Set: \
ms_drbd_mysql
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: notify_crmd: Transition 114 status: \
done - <null>
Jan 22 10:24:21 mysql-node1 pengine: [9246]: debug: native_active: Resource \
drbd_mysql:0 active on mysql-node1
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: s_crmd_fsa: Processing I_TE_SUCCESS: \
[ state=S_TRANSITION_ENGINE cause=C_FSA_INTERNAL origin=notify_crmd ]
Jan 22 10:24:21 mysql-node1 pengine: [9246]: debug: native_active: Resource \
drbd_mysql:0 active on mysql-node1
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace: // \
A_LOG
Jan 22 10:24:21 mysql-node1 pengine: [9246]: debug: native_active: Resource \
drbd_mysql:1 active on mysql-node2
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: do_state_transition: State transition \
S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL \
origin=notify_crmd ]
Jan 22 10:24:21 mysql-node1 pengine: [9246]: debug: native_active: Resource \
drbd_mysql:1 active on mysql-node2
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: do_state_transition: Starting PEngine \
Recheck Timer
Jan 22 10:24:21 mysql-node1 pengine: [9246]: notice: short_print: Masters: [ \
mysql-node2 ]
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: crm_timer_start: Started PEngine \
Recheck Timer (I_PE_CALC:900000ms), src=400
Jan 22 10:24:21 mysql-node1 pengine: [9246]: notice: short_print: Slaves: [ \
mysql-node1 ]
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace: // \
A_DC_TIMER_STOP
Jan 22 10:24:21 mysql-node1 pengine: [9246]: notice: group_print: Resource Group: \
mysql
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace: // \
A_INTEGRATE_TIMER_STOP
Jan 22 10:24:21 mysql-node1 pengine: [9246]: notice: native_print: fs_mysql \
(ocf::heartbeat:Filesystem): Started mysql-node2
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace: // \
A_FINALIZE_TIMER_STOP
Jan 22 10:24:21 mysql-node1 pengine: [9246]: notice: native_print: ip_mysql \
(ocf::heartbeat:IPaddr2): Started mysql-node2
Jan 22 10:24:21 mysql-node1 pengine: [9246]: notice: native_print: mysqld \
(lsb:mysql): Started mysql-node2
....
...
Jan 22 10:25:46 mysql-node1 rchal: CPU frequency scaling is not supported by your \
processor.
Jan 22 10:25:46 mysql-node1 rchal: boot with 'CPUFREQ=no' in to avoid this warning.
Jan 22 10:25:47 mysql-node1 rchal: Cannot load cpufreq governors - No cpufreq driver \
available
Jan 22 10:25:48 mysql-node1 ifup: lo
Jan 22 10:25:48 mysql-node1 ifup: lo
Jan 22 10:25:48 mysql-node1 ifup: IP address: 127.0.0.1/8
Jan 22 10:25:48 mysql-node1 ifup:
Jan 22 10:25:48 mysql-node1 ifup:
Jan 22 10:25:48 mysql-node1 ifup: IP address: 127.0.0.2/8
Jan 22 10:25:48 mysql-node1 ifup:
Jan 22 10:25:48 mysql-node1 ifup: eth0 device: Intel Corporation 82545EM \
Gigabit Ethernet Controller (Copper) (rev 01)
Jan 22 10:25:48 mysql-node1 ifup: eth0
Jan 22 10:25:48 mysql-node1 ifup: IP address: 192.168.96.11/24
Node2 debug log:
Jan 22 10:25:13 mysql-node2 attrd_updater: [21677]: info: attrd_lazy_update: \
Connecting to cluster... 5 retries remaining
Jan 22 10:25:13 mysql-node2 attrd_updater: [21677]: debug: \
init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Jan 22 10:25:13 mysql-node2 attrd_updater: [21677]: debug: attrd_update: Sent update: \
pingd=1 for localhost
Jan 22 10:25:13 mysql-node2 attrd: [27805]: debug: attrd_local_callback: update \
message from attrd_updater: pingd=1
Jan 22 10:25:13 mysql-node2 attrd: [27805]: debug: attrd_local_callback: Supplied: 1, \
Current: 1, Stored: 1
Jan 22 10:25:17 mysql-node2 kernel: [104450.241909] block drbd0: PingAck did not \
arrive in time.
Jan 22 10:25:17 mysql-node2 kernel: [104450.241919] block drbd0: peer( Secondary -> \
Unknown ) conn( Connected -> NetworkFailure ) pdsk( UpToDate -> DUnknown )
Jan 22 10:25:17 mysql-node2 kernel: [104450.241940] block drbd0: asender terminated
Jan 22 10:25:17 mysql-node2 kernel: [104450.241943] block drbd0: Terminating asender \
thread
Jan 22 10:25:17 mysql-node2 kernel: [104450.241989] block drbd0: short read expecting \
header on sock: r=-512
Jan 22 10:25:17 mysql-node2 kernel: [104450.242055] block drbd0: Creating new current \
UUID
Jan 22 10:25:17 mysql-node2 kernel: [104450.242316] block drbd0: Connection closed
Jan 22 10:25:17 mysql-node2 kernel: [104450.242322] block drbd0: conn( NetworkFailure \
-> Unconnected )
Jan 22 10:25:17 mysql-node2 kernel: [104450.242326] block drbd0: receiver terminated
Jan 22 10:25:17 mysql-node2 kernel: [104450.242328] block drbd0: Restarting receiver \
thread
Jan 22 10:25:17 mysql-node2 kernel: [104450.242330] block drbd0: receiver (re)started
Jan 22 10:25:17 mysql-node2 kernel: [104450.242334] block drbd0: conn( Unconnected -> \
WFConnection )
Jan 22 10:25:18 mysql-node2 corosync[27796]: [TOTEM ] The token was lost in the \
OPERATIONAL state.
Jan 22 10:25:18 mysql-node2 corosync[27796]: [TOTEM ] A processor failed, forming \
new configuration.
Jan 22 10:25:18 mysql-node2 corosync[27796]: [TOTEM ] Receive multicast socket recv \
buffer size (262142 bytes).
Jan 22 10:25:18 mysql-node2 corosync[27796]: [TOTEM ] Transmit multicast socket \
send buffer size (262142 bytes).
Jan 22 10:25:18 mysql-node2 corosync[27796]: [TOTEM ] entering GATHER state from 2.
Jan 22 10:25:23 mysql-node2 lrmd: [27804]: debug: rsc:notification:1:14: monitor
Jan 22 10:25:23 mysql-node2 lrmd: [21678]: debug: perform_ra_op: resetting scheduler \
class to SCHED_OTHER
Jan 22 10:25:23 mysql-node2 lrmd: [27804]: debug: RA output: \
(notification:1:monitor:stdout) running
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] entering GATHER state from 0.
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] Creating commit token because \
I am the rep.
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] Saving state aru 1d2 high seq \
received 1d2
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] Storing new sequence id for \
ring e0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] entering COMMIT state.
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] got commit token
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] entering RECOVERY state.
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] TRANS [0] member 10.0.0.102:
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] position [0] member \
10.0.0.102:
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] previous ring seq 220 rep \
10.0.0.101
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] aru 1d2 high delivered 1d2 \
received flag 1
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] Did not need to originate any \
messages in recovery.
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] got commit token
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] Sending initial ORF token
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] token retrans flag is 0 my \
set retrans flag0 retrans queue empty 1 count 0, aru 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] install seq 0 aru 0 high seq \
received 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] token retrans flag is 0 my \
set retrans flag0 retrans queue empty 1 count 1, aru 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] install seq 0 aru 0 high seq \
received 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] token retrans flag is 0 my \
set retrans flag0 retrans queue empty 1 count 2, aru 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] install seq 0 aru 0 high seq \
received 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] token retrans flag is 0 my \
set retrans flag0 retrans queue empty 1 count 3, aru 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] install seq 0 aru 0 high seq \
received 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] retrans flag count 4 token \
aru 0 install seq 0 aru 0 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] Resetting old ring state
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] recovery to regular 1-0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] Delivering to app 1d3 to 1d2
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] CLM CONFIGURATION CHANGE
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] New Configuration:
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] r(0) ip(10.0.0.102)
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] Members Left:
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] r(0) ip(10.0.0.101)
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] Members Joined:
Jan 22 10:25:26 mysql-node2 corosync[27796]: [EVT ] Evt conf change 1
Jan 22 10:25:26 mysql-node2 corosync[27796]: [EVT ] m 1, j 0 l 1
Jan 22 10:25:26 mysql-node2 corosync[27796]: [LCK ] [DEBUG]: lck_confchg_fn
Jan 22 10:25:26 mysql-node2 corosync[27796]: [MSG ] [DEBUG]: msg_confchg_fn
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] notice: pcmk_peer_update: \
Transitional membership event on ring 224: memb=1, new=0, lost=1
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] info: pcmk_peer_update: memb: \
mysql-node2 1711276042
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] info: pcmk_peer_update: lost: \
mysql-node1 1694498826
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] CLM CONFIGURATION CHANGE
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] New Configuration:
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] r(0) ip(10.0.0.102)
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] Members Left:
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] Members Joined:
Jan 22 10:25:26 mysql-node2 corosync[27796]: [EVT ] Evt conf change 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [EVT ] m 1, j 0 l 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [LCK ] [DEBUG]: lck_confchg_fn
Jan 22 10:25:26 mysql-node2 corosync[27796]: [MSG ] [DEBUG]: msg_confchg_fn
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] notice: pcmk_peer_update: \
Stable membership event on ring 224: memb=1, new=0, lost=0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] info: pcmk_peer_update: MEMB: \
mysql-node2 1711276042
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] info: \
ais_mark_unseen_peer_dead: Node mysql-node1 was not seen in the previous transition
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] info: update_member: Node \
1694498826/mysql-node1 is now: lost
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] debug: pcmk_peer_update: 1 \
nodes changed
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] info: \
send_member_notification: Sending membership update 224 to 2 children
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] debug: send_cluster_id: Local \
update: id=1711276042, born=220, seq=224
Jan 22 10:25:26 mysql-node2 corosync[27796]: [SYNC ] This node is within the \
primary component and will provide service.
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] entering OPERATIONAL state.
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] A processor joined or left \
the membership and a new membership was formed.
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] mcasted message added to \
pending queue
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] mcasted message added to \
pending queue
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] Delivering 0 to 2
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] Delivering MCAST message with \
seq 1 to pending delivery queue
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] debug: \
pcmk_cluster_id_callback: Node update: mysql-node2 (1.1.2)
Jan 22 10:25:26 mysql-node2 corosync[27796]: [SYNC ] confchg entries 1
Jan 22 10:25:26 mysql-node2 corosync[27796]: [SYNC ] Barrier Start Received From \
1711276042
Jan 22 10:25:26 mysql-node2 corosync[27796]: [SYNC ] Barrier completion status for \
nodeid 1711276042 = 1.
Jan 22 10:25:26 mysql-node2 corosync[27796]: [SYNC ] Synchronization barrier \
completed
Jan 22 10:25:26 mysql-node2 corosync[27796]: [SYNC ] Synchronization actions \
starting for (openais cluster membership service B.01.01)
...
...
Viele Grüße aus Neustadt,
Thomas Fachtan
Database Administrator
Fon: +49 9602 9442-301
Fax: +49 9602 9442-309
E-Mail: thomas.fachtan@motiondrive.ag
.................................................
motiondrive AG
Im Wiesengrund 1
92660 Neustadt
http://www.motiondrive.ag
Vorstandsvorsitzender: Werner A. Krachtus
Aufsichtsratsvorsitzender: Dipl.-Kfm. Ralf Barth
Handelsregister Weiden: HRB 2817
_______________________________________________
Linux-HA mailing list
Linux-HA@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic