[prev in list] [next in list] [prev in thread] [next in thread] 

List:       linux-ha-dev
Subject:    Re: [Linux-ha-dev] stonithd received sigabt when testing by CTS.
From:       AOSHIMA Kentaro <aoshima.kentaro () nttcom ! co ! jp>
Date:       2008-07-28 4:17:57
Message-ID: 488D4875.9020900 () nttcom ! co ! jp
[Download RAW message or body]

Hi,all

We would like to inform you that we've completed
500 times CTS Test for "changeset: 12158: a8b2fc037b29"
on RHEL5.2(x86-64).

That changeset includes some patches(changeset 12153) for LRM
by Dejan. The issue we had reported before this did not
happened and the log attached showed "success" on all tests.

we are not familiar with CTS and I wonder if CTS test on SUSE or any
other OS shows the same results as we had.
if anyone did run CTS, please let me know the result.

Best regards

> Hi,all
> 
> I'm Aoshima. I'd like to ask about CTS for 2.1.4.
> We try to run CTS on RHEL5.2(x86-64) for 2.1.4 whose
> changeset is "12148:e902ad7642fd" .
> 
> heartbeat and resource agent seemed to fail to start
> after kicking CTSlab.py .
> According to ha-log, when the lrmd procces tried to start
> STONITH resource, something wrong had occured and  stonithd had
> received SIGABT.
> 
> When using stonith(ex: external/ssh) for CTS,
> any extra configurations are needed?
> 
> best regards
> 
> our testing environment are indicated below.
> ---ha.cf---
> crm on
> logfile  /var/log/ha-log
> logfacility local7
> keepalive 2
> deadtime 30
> warntime 10
> initdead 60
> udpport 651
> auto_failback legacy
> bcast eth1
> bcast eth3
> 
> node cgl49
> node cgl50
> 
> --ha-log--
> lrmd[8899]: 2008/07/22_14:12:04 info: rsc:ocf_msdummy:0: start
> lrmd[9361]: 2008/07/22_14:12:04 info: Try to start STONITH resource
> <rsc_id=child_DoFencing:0> : Device=external/ssh
> stonithd[8900]: 2008/07/22_14:12:04 ERROR: ipc_bufpool_update: magic
> number in head does not match.Something very bad happened, abor
> t now, farside pid =9361
> stonithd[8900]: 2008/07/22_14:12:04 ERROR: magic=6c754a3e, expected
> value=abcd
> stonithd[8900]: 2008/07/22_14:12:04 info: pool: refcount=1,
> startpos=0x17dc4ba8, currpos=0x17dc4c8d,consumepos=0x17dc4c14, endpos=0x
> 17dc5b78, size=4096
> stonithd[8900]: 2008/07/22_14:12:04 info: nmsgs=0
> lrmd[8899]: 2008/07/22_14:12:04 info: rsc:ocf_msdummy:1: start
> lrmd[9361]: 2008/07/22_14:12:04 ERROR: stonithd_virtual_stonithRA_ops:
> failed to fetch reply
> tengine[8951]: 2008/07/22_14:12:04 ERROR: stonithd_op_result_ready: not
> signed on
> heartbeat[8887]: 2008/07/22_14:12:04 WARN: Managed
> /usr/lib64/heartbeat/stonithd process 8900 killed by signal 6 [SIGABRT -
> Abort].
> lrmd[9361]: 2008/07/22_14:12:04 ERROR: sending stonithRA op to stonithd
> failed.
> tengine[8951]: 2008/07/22_14:12:04 ERROR:
> tengine_stonith_connection_destroy: Fencing daemon has left us
> heartbeat[8887]: 2008/07/22_14:12:04 ERROR: Managed
> /usr/lib64/heartbeat/stonithd process 8900 dumped core
> lrmd[9361]: 2008/07/22_14:12:04 notice: Not currently connected.
> tengine[8951]: 2008/07/22_14:12:04 info: te_connect_stonith: Attempting
> connection to fencing daemon...
> heartbeat[8887]: 2008/07/22_14:12:04 ERROR: Respawning client
> "/usr/lib64/heartbeat/stonithd":
> heartbeat[8887]: 2008/07/22_14:12:04 info: Starting child client
> "/usr/lib64/heartbeat/stonithd" (0,0)
> lrmd[8899]: 2008/07/22_14:12:04 WARN: mapped the invalid return code 254.
> crmd[8902]: 2008/07/22_14:12:04 info: process_lrm_event: LRM operation
> child_DoFencing:0_start_0 (call=19, rc=1) complete
> heartbeat[9368]: 2008/07/22_14:12:04 info: Starting
> "/usr/lib64/heartbeat/stonithd" as uid 0  gid 0 (pid 9368)
> _______________________________________________________
> Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
> Home Page: http://linux-ha.org/
> 
> 


["ctslab_20080728.log" (text/plain)]

Jul 25 17:43:12	Random seed is: 1216975392
Jul 25 17:43:12	>>>>>>>>>>>>>>>> BEGINNING 500 TESTS
Jul 25 17:43:12	HA configuration directory: /etc/ha.d
Jul 25 17:43:12	System log files: /var/log/ha-log-local7
Jul 25 17:43:12	Enable Stonith:   1
Jul 25 17:43:12	Enable Fencing:   1
Jul 25 17:43:12	Enable Standby:   1
Jul 25 17:43:12	Enable Resources: 1
Jul 25 17:43:13	Cluster nodes:
Jul 25 17:43:13	* cgl49: 7ada022c-796d-478d-956a-6fb37ca665a2
Jul 25 17:43:13	* cgl50: a87e2535-2bf8-41b0-b0dd-9f894a6af11f
Jul 25 17:43:14	Stopping Cluster Manager on all nodes
Jul 25 17:43:22	Starting Cluster Manager on all nodes.
Jul 25 17:46:42	Running test SpecialTest1 (cgl50) 	[1]
Jul 25 17:49:38	Running test SpecialTest1 (cgl49) 	[2]
Jul 25 17:52:40	Running test Stonithd (cgl49) 	[3]
Jul 25 17:55:57	Running test StartOnebyOne (cgl50) 	[4]
Jul 25 17:58:58	Running test ResourceRecover (cgl50) 	[5]
Jul 25 17:59:11	Running test StopOnebyOne (cgl50) 	[6]
Jul 25 17:59:34	Running test Stonithd (cgl50) 	[7]
Jul 25 18:04:14	Running test Stonithd (cgl49) 	[8]
Jul 25 18:07:21	Running test ComponentFail (cgl49) 	[9]
Jul 25 18:07:21	...component crmd (dc=0,boot=1)
Jul 25 18:10:08	Running test StartOnebyOne (cgl50) 	[10]
Jul 25 18:13:13	Running test SpecialTest1 (cgl49) 	[11]
Jul 25 18:16:11	Running test SimulStop (cgl49) 	[12]
Jul 25 18:16:25	Running test SimulStart (cgl50) 	[13]
Jul 25 18:18:04	Running test SimulStart (cgl49) 	[14]
Jul 25 18:19:53	Running test RestartOnebyOne (cgl50) 	[15]
Jul 25 18:21:23	Running test ComponentFail (cgl49) 	[16]
Jul 25 18:21:24	...component tengine (dc=1,boot=1)
Jul 25 18:24:20	Running test ResourceRecover (cgl50) 	[17]
Jul 25 18:24:30	Running test Flip (cgl49) 	[18]
Jul 25 18:25:21	Running test StartOnebyOne (cgl50) 	[19]
Jul 25 18:28:13	Running test SimulStop (cgl50) 	[20]
Jul 25 18:28:28	Running test SimulStart (cgl49) 	[21]
Jul 25 18:30:02	Running test Restart (cgl50) 	[22]
Jul 25 18:30:53	Running test Stonithd (cgl49) 	[23]
Jul 25 18:34:02	Running test Flip (cgl50) 	[24]
Jul 25 18:34:57	Running test ResourceRecover (cgl50) 	[25]
Jul 25 18:35:35	Running test RestartOnebyOne (cgl50) 	[26]
Jul 25 18:37:09	Running test PartialStart (cgl50) 	[27]
Jul 25 18:38:33	Running test StartOnebyOne (cgl50) 	[28]
Jul 25 18:41:24	Running test SpecialTest1 (cgl49) 	[29]
Jul 25 18:44:24	Running test ComponentFail (cgl50) 	[30]
Jul 25 18:44:24	...component cib (dc=0,boot=1)
Jul 25 18:47:15	Running test PartialStart (cgl50) 	[31]
Jul 25 18:48:36	Running test StopOnebyOne (cgl49) 	[32]
Jul 25 18:50:21	Running test StopOnebyOne (cgl50) 	[33]
Jul 25 18:52:09	Running test NearQuorumPoint (cgl49) 	[34]
Jul 25 18:52:09	Running test ComponentFail (cgl50) 	[35]
Jul 25 18:53:36	...component lrmd (dc=1,boot=1)
Jul 25 18:56:23	Running test RestartOnebyOne (cgl49) 	[36]
Jul 25 18:57:55	Running test SimulStop (cgl50) 	[37]
Jul 25 18:58:08	Running test Stonithd (cgl50) 	[38]
Jul 25 19:02:33	Running test PartialStart (cgl49) 	[39]
Jul 25 19:03:54	Running test Flip (cgl49) 	[40]
Jul 25 19:06:13	Running test NearQuorumPoint (cgl49) 	[41]
Jul 25 19:06:45	Running test SimulStart (cgl49) 	[42]
Jul 25 19:08:31	Running test Stonithd (cgl50) 	[43]
Jul 25 19:11:34	Running test NearQuorumPoint (cgl49) 	[44]
Jul 25 19:11:58	Running test ComponentFail (cgl49) 	[45]
Jul 25 19:12:22	...component lrmd (dc=0,boot=1)
Jul 25 19:15:12	Running test standby2 (cgl50) 	[46]
Jul 25 19:16:27	Running test SimulStop (cgl50) 	[47]
Jul 25 19:16:43	Running test ComponentFail (cgl50) 	[48]
Jul 25 19:18:09	...component stonithd (dc=1,boot=0)
Jul 25 19:18:19	Running test SimulStop (cgl49) 	[49]
Jul 25 19:18:35	Running test standby2 (cgl49) 	[50]
Jul 25 19:21:18	Running test ResourceRecover (cgl50) 	[51]
Jul 25 19:21:30	Running test SpecialTest1 (cgl49) 	[52]
Jul 25 19:24:34	Running test SimulStop (cgl50) 	[53]
Jul 25 19:24:47	Running test SpecialTest1 (cgl49) 	[54]
Jul 25 19:27:34	Running test standby2 (cgl49) 	[55]
Jul 25 19:28:49	Running test StopOnebyOne (cgl50) 	[56]
Jul 25 19:29:12	Running test SpecialTest1 (cgl50) 	[57]
Jul 25 19:32:03	Running test PartialStart (cgl49) 	[58]
Jul 25 19:33:27	Running test StartOnebyOne (cgl49) 	[59]
Jul 25 19:36:13	Running test ResourceRecover (cgl49) 	[60]
Jul 25 19:36:23	Running test ComponentFail (cgl49) 	[61]
Jul 25 19:36:24	...component stonithd (dc=1,boot=0)
Jul 25 19:36:34	Running test Restart (cgl50) 	[62]
Jul 25 19:37:19	Running test standby2 (cgl49) 	[63]
Jul 25 19:38:34	Running test ResourceRecover (cgl49) 	[64]
Jul 25 19:38:45	Running test NearQuorumPoint (cgl49) 	[65]
Jul 25 19:38:53	Running test StartOnebyOne (cgl50) 	[66]
Jul 25 19:41:58	Running test PartialStart (cgl49) 	[67]
Jul 25 19:43:22	Running test PartialStart (cgl50) 	[68]
Jul 25 19:44:33	Running test SimulStop (cgl49) 	[69]
Jul 25 19:46:13	Running test ComponentFail (cgl50) 	[70]
Jul 25 19:47:40	...component tengine (dc=1,boot=1)
Jul 25 19:50:31	Running test StopOnebyOne (cgl50) 	[71]
Jul 25 19:50:55	Running test Flip (cgl50) 	[72]
Jul 25 19:53:13	Running test Restart (cgl49) 	[73]
Jul 25 19:53:46	Running test Flip (cgl50) 	[74]
Jul 25 19:54:40	Running test Flip (cgl50) 	[75]
Jul 25 19:55:20	Running test SpecialTest1 (cgl49) 	[76]
Jul 25 19:58:24	Running test NearQuorumPoint (cgl50) 	[77]
Jul 25 19:58:32	Running test Restart (cgl50) 	[78]
Jul 25 19:59:15	Running test PartialStart (cgl49) 	[79]
Jul 25 20:00:37	Running test standby2 (cgl50) 	[80]
Jul 25 20:03:18	Running test SimulStop (cgl49) 	[81]
Jul 25 20:03:34	Running test SpecialTest1 (cgl50) 	[82]
Jul 25 20:06:22	Running test ResourceRecover (cgl49) 	[83]
Jul 25 20:06:36	Running test standby2 (cgl49) 	[84]
Jul 25 20:07:53	Running test SimulStop (cgl50) 	[85]
Jul 25 20:08:08	Running test SimulStop (cgl49) 	[86]
Jul 25 20:09:51	Running test SpecialTest1 (cgl49) 	[87]
Jul 25 20:12:39	Running test NearQuorumPoint (cgl49) 	[88]
Jul 25 20:13:03	Running test Stonithd (cgl50) 	[89]
Jul 25 20:16:39	Running test Restart (cgl50) 	[90]
Jul 25 20:17:28	Running test SpecialTest1 (cgl50) 	[91]
Jul 25 20:20:31	Running test SpecialTest1 (cgl50) 	[92]
Jul 25 20:23:38	Running test ResourceRecover (cgl49) 	[93]
Jul 25 20:23:59	Running test RestartOnebyOne (cgl50) 	[94]
Jul 25 20:25:33	Running test RestartOnebyOne (cgl49) 	[95]
Jul 25 20:27:12	Running test RestartOnebyOne (cgl49) 	[96]
Jul 25 20:28:45	Running test SpecialTest1 (cgl49) 	[97]
Jul 25 20:31:47	Running test SpecialTest1 (cgl49) 	[98]
Jul 25 20:34:50	Running test Stonithd (cgl49) 	[99]
Jul 25 20:38:04	Running test StartOnebyOne (cgl49) 	[100]
Jul 25 20:41:11	Running test Flip (cgl50) 	[101]
Jul 25 20:41:58	Running test ComponentFail (cgl49) 	[102]
Jul 25 20:42:23	...component ccm (dc=1,boot=1)
Jul 25 20:45:13	Running test NearQuorumPoint (cgl49) 	[103]
Jul 25 20:45:38	Running test NearQuorumPoint (cgl49) 	[104]
Jul 25 20:45:42	Running test SimulStart (cgl49) 	[105]
Jul 25 20:47:15	Running test RestartOnebyOne (cgl50) 	[106]
Jul 25 20:48:48	Running test ComponentFail (cgl49) 	[107]
Jul 25 20:48:48	...component lrmd (dc=1,boot=1)
Jul 25 20:51:44	Running test SimulStart (cgl49) 	[108]
Jul 25 20:53:35	Running test ComponentFail (cgl49) 	[109]
Jul 25 20:53:36	...component crmd (dc=0,boot=1)
Jul 25 20:56:33	Running test ResourceRecover (cgl50) 	[110]
Jul 25 20:56:43	Running test ResourceRecover (cgl50) 	[111]
Jul 25 20:56:55	Running test ComponentFail (cgl50) 	[112]
Jul 25 20:56:56	...component cib (dc=1,boot=1)
Jul 25 20:59:47	Running test StartOnebyOne (cgl50) 	[113]
Jul 25 21:02:45	Running test SimulStart (cgl50) 	[114]
Jul 25 21:04:31	Running test SimulStop (cgl49) 	[115]
Jul 25 21:04:46	Running test Flip (cgl49) 	[116]
Jul 25 21:07:05	Running test RestartOnebyOne (cgl50) 	[117]
Jul 25 21:09:11	Running test ComponentFail (cgl49) 	[118]
Jul 25 21:09:11	...component ccm (dc=1,boot=1)
Jul 25 21:12:06	Running test standby2 (cgl50) 	[119]
Jul 25 21:13:20	Running test ResourceRecover (cgl50) 	[120]
Jul 25 21:13:31	Running test PartialStart (cgl50) 	[121]
Jul 25 21:14:56	Running test RestartOnebyOne (cgl50) 	[122]
Jul 25 21:17:51	Running test StopOnebyOne (cgl50) 	[123]
Jul 25 21:18:18	Running test Flip (cgl49) 	[124]
Jul 25 21:20:36	Running test Flip (cgl50) 	[125]
Jul 25 21:21:09	Running test Flip (cgl50) 	[126]
Jul 25 21:21:57	Running test Stonithd (cgl50) 	[127]
Jul 25 21:25:26	Running test Restart (cgl50) 	[128]
Jul 25 21:26:13	Running test NearQuorumPoint (cgl49) 	[129]
Jul 25 21:26:31	Running test ResourceRecover (cgl50) 	[130]
Jul 25 21:27:07	Running test NearQuorumPoint (cgl49) 	[131]
Jul 25 21:27:23	Running test ResourceRecover (cgl50) 	[132]
Jul 25 21:28:01	Running test standby2 (cgl49) 	[133]
Jul 25 21:29:16	Running test Restart (cgl49) 	[134]
Jul 25 21:30:09	Running test SpecialTest1 (cgl50) 	[135]
Jul 25 21:33:08	Running test SimulStop (cgl50) 	[136]
Jul 25 21:33:24	Running test SpecialTest1 (cgl50) 	[137]
Jul 25 21:36:11	Running test Stonithd (cgl50) 	[138]
Jul 25 21:39:18	Running test Restart (cgl49) 	[139]
Jul 25 21:40:12	Running test NearQuorumPoint (cgl50) 	[140]
Jul 25 21:40:20	Running test NearQuorumPoint (cgl50) 	[141]
Jul 25 21:40:45	Running test ComponentFail (cgl50) 	[142]
Jul 25 21:41:09	...component stonithd (dc=0,boot=0)
Jul 25 21:41:18	Running test ComponentFail (cgl50) 	[143]
Jul 25 21:41:19	...component lrmd (dc=0,boot=1)
Jul 25 21:44:09	Running test ComponentFail (cgl49) 	[144]
Jul 25 21:44:09	...component pengine (dc=1,boot=1)
Jul 25 21:47:05	Running test Stonithd (cgl49) 	[145]
Jul 25 21:50:16	Running test ComponentFail (cgl50) 	[146]
Jul 25 21:50:17	...component ccm (dc=1,boot=1)
Jul 25 21:53:03	Running test SimulStart (cgl49) 	[147]
Jul 25 21:54:52	Running test RestartOnebyOne (cgl50) 	[148]
Jul 25 21:56:21	Running test StopOnebyOne (cgl50) 	[149]
Jul 25 21:56:45	Running test standby2 (cgl49) 	[150]
Jul 25 21:59:28	Running test RestartOnebyOne (cgl49) 	[151]
Jul 25 22:01:07	Running test Flip (cgl50) 	[152]
Jul 25 22:01:56	Running test Restart (cgl49) 	[153]
Jul 25 22:04:20	Running test standby2 (cgl49) 	[154]
Jul 25 22:05:59	Running test RestartOnebyOne (cgl49) 	[155]
Jul 25 22:07:36	Running test SpecialTest1 (cgl49) 	[156]
Jul 25 22:10:34	Running test NearQuorumPoint (cgl49) 	[157]
Jul 25 22:10:50	Running test standby2 (cgl49) 	[158]
Jul 25 22:13:34	Running test StopOnebyOne (cgl49) 	[159]
Jul 25 22:13:56	Running test standby2 (cgl49) 	[160]
Jul 25 22:16:42	Running test SpecialTest1 (cgl50) 	[161]
Jul 25 22:19:48	Running test SimulStart (cgl50) 	[162]
Jul 25 22:21:35	Running test standby2 (cgl49) 	[163]
Jul 25 22:22:52	Running test SpecialTest1 (cgl50) 	[164]
Jul 25 22:25:53	Running test Stonithd (cgl49) 	[165]
Jul 25 22:29:03	Running test Restart (cgl49) 	[166]
Jul 25 22:29:55	Running test Restart (cgl50) 	[167]
Jul 25 22:30:49	Running test Stonithd (cgl49) 	[168]
Jul 25 22:33:59	Running test Restart (cgl50) 	[169]
Jul 25 22:34:51	Running test StopOnebyOne (cgl49) 	[170]
Jul 25 22:35:15	Running test standby2 (cgl49) 	[171]
Jul 25 22:38:00	Running test NearQuorumPoint (cgl49) 	[172]
Jul 25 22:38:17	Running test RestartOnebyOne (cgl49) 	[173]
Jul 25 22:40:16	Running test SimulStop (cgl49) 	[174]
Jul 25 22:40:31	Running test RestartOnebyOne (cgl50) 	[175]
Jul 25 22:43:30	Running test RestartOnebyOne (cgl50) 	[176]
Jul 25 22:45:05	Running test SpecialTest1 (cgl49) 	[177]
Jul 25 22:48:05	Running test Restart (cgl49) 	[178]
Jul 25 22:48:55	Running test SpecialTest1 (cgl50) 	[179]
Jul 25 22:51:57	Running test Flip (cgl50) 	[180]
Jul 25 22:52:50	Running test SpecialTest1 (cgl50) 	[181]
Jul 25 22:55:40	Running test standby2 (cgl49) 	[182]
Jul 25 22:56:57	Running test NearQuorumPoint (cgl50) 	[183]
Jul 25 22:57:21	Running test ComponentFail (cgl50) 	[184]
Jul 25 22:57:46	...component crmd (dc=0,boot=1)
Jul 25 23:00:39	Running test Restart (cgl49) 	[185]
Jul 25 23:01:28	Running test RestartOnebyOne (cgl49) 	[186]
Jul 25 23:03:02	Running test PartialStart (cgl49) 	[187]
Jul 25 23:04:25	Running test SimulStart (cgl49) 	[188]
Jul 25 23:06:00	Running test RestartOnebyOne (cgl49) 	[189]
Jul 25 23:07:33	Running test NearQuorumPoint (cgl50) 	[190]
Jul 25 23:07:41	Running test Restart (cgl49) 	[191]
Jul 25 23:08:31	Running test SpecialTest1 (cgl50) 	[192]
Jul 25 23:11:31	Running test standby2 (cgl49) 	[193]
Jul 25 23:12:49	Running test SimulStart (cgl49) 	[194]
Jul 25 23:14:35	Running test standby2 (cgl49) 	[195]
Jul 25 23:15:52	Running test RestartOnebyOne (cgl50) 	[196]
Jul 25 23:17:25	Running test SpecialTest1 (cgl49) 	[197]
Jul 25 23:20:24	Running test Flip (cgl50) 	[198]
Jul 25 23:21:13	Running test ComponentFail (cgl50) 	[199]
Jul 25 23:21:38	...component lrmd (dc=0,boot=1)
Jul 25 23:24:27	Running test Restart (cgl49) 	[200]
Jul 25 23:25:15	Running test SimulStop (cgl49) 	[201]
Jul 25 23:25:31	Running test ComponentFail (cgl50) 	[202]
Jul 25 23:26:58	...component tengine (dc=1,boot=1)
Jul 25 23:29:48	Running test RestartOnebyOne (cgl50) 	[203]
Jul 25 23:31:21	Running test RestartOnebyOne (cgl50) 	[204]
Jul 25 23:32:56	Running test StopOnebyOne (cgl49) 	[205]
Jul 25 23:33:19	Running test PartialStart (cgl50) 	[206]
Jul 25 23:34:30	Running test SimulStart (cgl50) 	[207]
Jul 25 23:36:08	Running test StartOnebyOne (cgl49) 	[208]
Jul 25 23:39:06	Running test standby2 (cgl49) 	[209]
Jul 25 23:40:21	Running test SimulStart (cgl50) 	[210]
Jul 25 23:42:09	Running test standby2 (cgl49) 	[211]
Jul 25 23:43:27	Running test ComponentFail (cgl49) 	[212]
Jul 25 23:43:27	...component crmd (dc=0,boot=1)
Jul 25 23:46:25	Running test StartOnebyOne (cgl50) 	[213]
Jul 25 23:49:26	Running test StartOnebyOne (cgl50) 	[214]
Jul 25 23:52:26	Running test StopOnebyOne (cgl50) 	[215]
Jul 25 23:52:49	Running test SimulStop (cgl50) 	[216]
Jul 25 23:54:28	Running test StopOnebyOne (cgl50) 	[217]
Jul 25 23:56:11	Running test Stonithd (cgl50) 	[218]
Jul 26 00:00:46	Running test StartOnebyOne (cgl50) 	[219]
Jul 26 00:03:46	Running test Restart (cgl49) 	[220]
Jul 26 00:04:39	Running test standby2 (cgl50) 	[221]
Jul 26 00:05:54	Running test standby2 (cgl50) 	[222]
Jul 26 00:07:08	Running test SpecialTest1 (cgl50) 	[223]
Jul 26 00:10:05	Running test StopOnebyOne (cgl49) 	[224]
Jul 26 00:10:26	Running test ResourceRecover (cgl50) 	[225]
Jul 26 00:12:02	Running test Flip (cgl49) 	[226]
Jul 26 00:12:49	Running test standby2 (cgl49) 	[227]
Jul 26 00:14:34	Running test ComponentFail (cgl50) 	[228]
Jul 26 00:14:35	...component lrmd (dc=1,boot=1)
Jul 26 00:17:21	Running test RestartOnebyOne (cgl49) 	[229]
Jul 26 00:18:53	Running test Restart (cgl50) 	[230]
Jul 26 00:19:37	Running test RestartOnebyOne (cgl50) 	[231]
Jul 26 00:21:15	Running test Stonithd (cgl49) 	[232]
Jul 26 00:24:29	Running test standby2 (cgl50) 	[233]
Jul 26 00:25:44	Running test Flip (cgl49) 	[234]
Jul 26 00:26:36	Running test NearQuorumPoint (cgl50) 	[235]
Jul 26 00:28:31	Running test StartOnebyOne (cgl50) 	[236]
Jul 26 00:31:19	Running test StopOnebyOne (cgl49) 	[237]
Jul 26 00:31:43	Running test SimulStart (cgl49) 	[238]
Jul 26 00:33:18	Running test ResourceRecover (cgl49) 	[239]
Jul 26 00:33:34	Running test Restart (cgl50) 	[240]
Jul 26 00:34:27	Running test StopOnebyOne (cgl50) 	[241]
Jul 26 00:34:50	Running test Restart (cgl50) 	[242]
Jul 26 00:37:09	Running test Restart (cgl50) 	[243]
Jul 26 00:39:32	Running test SimulStart (cgl50) 	[244]
Jul 26 00:41:10	Running test Restart (cgl50) 	[245]
Jul 26 00:41:58	Running test Flip (cgl49) 	[246]
Jul 26 00:42:50	Running test ResourceRecover (cgl50) 	[247]
Jul 26 00:43:25	Running test Flip (cgl50) 	[248]
Jul 26 00:44:18	Running test StartOnebyOne (cgl49) 	[249]
Jul 26 00:47:04	Running test StartOnebyOne (cgl50) 	[250]
Jul 26 00:49:59	Running test SimulStop (cgl49) 	[251]
Jul 26 00:50:12	Running test Restart (cgl49) 	[252]
Jul 26 00:52:33	Running test SimulStart (cgl50) 	[253]
Jul 26 00:54:13	Running test StopOnebyOne (cgl50) 	[254]
Jul 26 00:54:32	Running test StartOnebyOne (cgl50) 	[255]
Jul 26 00:57:20	Running test Restart (cgl50) 	[256]
Jul 26 00:58:06	Running test SimulStart (cgl49) 	[257]
Jul 26 00:59:55	Running test RestartOnebyOne (cgl50) 	[258]
Jul 26 01:01:25	Running test ComponentFail (cgl50) 	[259]
Jul 26 01:01:26	...component cib (dc=0,boot=1)
Jul 26 01:04:14	Running test StopOnebyOne (cgl50) 	[260]
Jul 26 01:04:36	Running test StartOnebyOne (cgl50) 	[261]
Jul 26 01:07:23	Running test ResourceRecover (cgl49) 	[262]
Jul 26 01:07:35	Running test PartialStart (cgl50) 	[263]
Jul 26 01:08:59	Running test ResourceRecover (cgl49) 	[264]
Jul 26 01:10:44	Running test standby2 (cgl50) 	[265]
Jul 26 01:11:59	Running test standby2 (cgl50) 	[266]
Jul 26 01:13:13	Running test ComponentFail (cgl49) 	[267]
Jul 26 01:13:14	...component crmd (dc=0,boot=1)
Jul 26 01:16:09	Running test Stonithd (cgl49) 	[268]
Jul 26 01:19:14	Running test PartialStart (cgl49) 	[269]
Jul 26 01:20:41	Running test NearQuorumPoint (cgl49) 	[270]
Jul 26 01:22:14	Running test SimulStop (cgl50) 	[271]
Jul 26 01:22:29	Running test RestartOnebyOne (cgl49) 	[272]
Jul 26 01:25:30	Running test ComponentFail (cgl49) 	[273]
Jul 26 01:25:31	...component cib (dc=1,boot=1)
Jul 26 01:28:20	Running test Flip (cgl49) 	[274]
Jul 26 01:29:13	Running test PartialStart (cgl49) 	[275]
Jul 26 01:30:24	Running test Stonithd (cgl50) 	[276]
Jul 26 01:34:54	Running test StopOnebyOne (cgl50) 	[277]
Jul 26 01:35:18	Running test SimulStart (cgl50) 	[278]
Jul 26 01:36:52	Running test Restart (cgl50) 	[279]
Jul 26 01:37:45	Running test ComponentFail (cgl49) 	[280]
Jul 26 01:37:45	...component tengine (dc=1,boot=1)
Jul 26 01:40:41	Running test SpecialTest1 (cgl50) 	[281]
Jul 26 01:43:41	Running test Stonithd (cgl49) 	[282]
Jul 26 01:46:50	Running test StopOnebyOne (cgl50) 	[283]
Jul 26 01:47:13	Running test Flip (cgl50) 	[284]
Jul 26 01:49:31	Running test standby2 (cgl49) 	[285]
Jul 26 01:51:14	Running test Stonithd (cgl50) 	[286]
Jul 26 01:54:23	Running test StopOnebyOne (cgl49) 	[287]
Jul 26 01:54:46	Running test standby2 (cgl50) 	[288]
Jul 26 01:57:26	Running test Stonithd (cgl49) 	[289]
Jul 26 02:00:42	Running test SimulStart (cgl49) 	[290]
Jul 26 02:02:27	Running test ComponentFail (cgl50) 	[291]
Jul 26 02:02:28	...component cib (dc=1,boot=1)
Jul 26 02:05:15	Running test Restart (cgl49) 	[292]
Jul 26 02:06:06	Running test ResourceRecover (cgl50) 	[293]
Jul 26 02:06:18	Running test RestartOnebyOne (cgl50) 	[294]
Jul 26 02:07:47	Running test Restart (cgl50) 	[295]
Jul 26 02:08:35	Running test SimulStart (cgl50) 	[296]
Jul 26 02:10:29	Running test StopOnebyOne (cgl50) 	[297]
Jul 26 02:10:49	Running test standby2 (cgl50) 	[298]
Jul 26 02:13:28	Running test NearQuorumPoint (cgl49) 	[299]
Jul 26 02:13:48	Running test StopOnebyOne (cgl50) 	[300]
Jul 26 02:14:31	Running test StopOnebyOne (cgl50) 	[301]
Jul 26 02:16:16	Running test StopOnebyOne (cgl50) 	[302]
Jul 26 02:18:01	Running test NearQuorumPoint (cgl49) 	[303]
Jul 26 02:20:19	Running test Restart (cgl50) 	[304]
Jul 26 02:20:52	Running test ResourceRecover (cgl49) 	[305]
Jul 26 02:21:03	Running test StopOnebyOne (cgl49) 	[306]
Jul 26 02:21:26	Running test Stonithd (cgl50) 	[307]
Jul 26 02:26:01	Running test PartialStart (cgl49) 	[308]
Jul 26 02:27:24	Running test RestartOnebyOne (cgl49) 	[309]
Jul 26 02:30:18	Running test RestartOnebyOne (cgl50) 	[310]
Jul 26 02:31:58	Running test StartOnebyOne (cgl50) 	[311]
Jul 26 02:34:58	Running test ResourceRecover (cgl50) 	[312]
Jul 26 02:35:12	Running test ComponentFail (cgl50) 	[313]
Jul 26 02:35:12	...component crmd (dc=0,boot=1)
Jul 26 02:38:04	Running test StartOnebyOne (cgl50) 	[314]
Jul 26 02:41:01	Running test Flip (cgl50) 	[315]
Jul 26 02:41:49	Running test SimulStop (cgl49) 	[316]
Jul 26 02:42:28	Running test Stonithd (cgl50) 	[317]
Jul 26 02:47:00	Running test Flip (cgl50) 	[318]
Jul 26 02:47:48	Running test Flip (cgl49) 	[319]
Jul 26 02:48:23	Running test SimulStop (cgl49) 	[320]
Jul 26 02:50:03	Running test ComponentFail (cgl49) 	[321]
Jul 26 02:51:27	...component lrmd (dc=0,boot=1)
Jul 26 02:54:21	Running test SimulStart (cgl50) 	[322]
Jul 26 02:56:11	Running test PartialStart (cgl49) 	[323]
Jul 26 02:57:31	Running test Stonithd (cgl49) 	[324]
Jul 26 03:02:06	Running test NearQuorumPoint (cgl49) 	[325]
Jul 26 03:02:31	Running test Flip (cgl50) 	[326]
Jul 26 03:03:07	Running test StartOnebyOne (cgl50) 	[327]
Jul 26 03:06:07	Running test Restart (cgl49) 	[328]
Jul 26 03:06:58	Running test PartialStart (cgl49) 	[329]
Jul 26 03:08:25	Running test standby2 (cgl50) 	[330]
Jul 26 03:11:06	Running test standby2 (cgl50) 	[331]
Jul 26 03:12:20	Running test Stonithd (cgl49) 	[332]
Jul 26 03:15:33	Running test SimulStop (cgl50) 	[333]
Jul 26 03:15:47	Running test Restart (cgl50) 	[334]
Jul 26 03:18:06	Running test StopOnebyOne (cgl50) 	[335]
Jul 26 03:18:53	Running test Restart (cgl50) 	[336]
Jul 26 03:21:12	Running test StopOnebyOne (cgl50) 	[337]
Jul 26 03:21:58	Running test Stonithd (cgl49) 	[338]
Jul 26 03:26:38	Running test ResourceRecover (cgl49) 	[339]
Jul 26 03:26:51	Running test StopOnebyOne (cgl49) 	[340]
Jul 26 03:27:14	Running test PartialStart (cgl49) 	[341]
Jul 26 03:28:23	Running test StopOnebyOne (cgl49) 	[342]
Jul 26 03:30:10	Running test SimulStop (cgl50) 	[343]
Jul 26 03:31:49	Running test NearQuorumPoint (cgl49) 	[344]
Jul 26 03:34:07	Running test StartOnebyOne (cgl49) 	[345]
Jul 26 03:36:56	Running test ComponentFail (cgl49) 	[346]
Jul 26 03:36:57	...component tengine (dc=1,boot=1)
Jul 26 03:39:59	Running test StopOnebyOne (cgl50) 	[347]
Jul 26 03:40:18	Running test ResourceRecover (cgl49) 	[348]
Jul 26 03:42:02	Running test Flip (cgl49) 	[349]
Jul 26 03:42:52	Running test SimulStop (cgl50) 	[350]
Jul 26 03:43:34	Running test SimulStart (cgl49) 	[351]
Jul 26 03:45:06	Running test ResourceRecover (cgl50) 	[352]
Jul 26 03:45:16	Running test ComponentFail (cgl49) 	[353]
Jul 26 03:45:17	...component crmd (dc=0,boot=1)
Jul 26 03:48:15	Running test PartialStart (cgl49) 	[354]
Jul 26 03:49:41	Running test standby2 (cgl49) 	[355]
Jul 26 03:52:29	Running test standby2 (cgl49) 	[356]
Jul 26 03:53:45	Running test NearQuorumPoint (cgl49) 	[357]
Jul 26 03:53:57	Running test StartOnebyOne (cgl50) 	[358]
Jul 26 03:56:47	Running test Restart (cgl49) 	[359]
Jul 26 03:57:37	Running test NearQuorumPoint (cgl49) 	[360]
Jul 26 03:57:45	Running test NearQuorumPoint (cgl50) 	[361]
Jul 26 03:58:05	Running test StartOnebyOne (cgl49) 	[362]
Jul 26 04:00:55	Running test RestartOnebyOne (cgl50) 	[363]
Jul 26 04:02:30	Running test Restart (cgl50) 	[364]
Jul 26 04:03:15	Running test Stonithd (cgl50) 	[365]
Jul 26 04:06:27	Running test ComponentFail (cgl50) 	[366]
Jul 26 04:06:27	...component stonithd (dc=0,boot=0)
Jul 26 04:06:36	Running test StopOnebyOne (cgl50) 	[367]
Jul 26 04:06:58	Running test ComponentFail (cgl50) 	[368]
Jul 26 04:08:25	...component crmd (dc=1,boot=1)
Jul 26 04:11:16	Running test Flip (cgl50) 	[369]
Jul 26 04:12:05	Running test SpecialTest1 (cgl49) 	[370]
Jul 26 04:14:55	Running test SimulStart (cgl50) 	[371]
Jul 26 04:16:42	Running test PartialStart (cgl49) 	[372]
Jul 26 04:18:05	Running test RestartOnebyOne (cgl50) 	[373]
Jul 26 04:21:00	Running test SpecialTest1 (cgl49) 	[374]
Jul 26 04:24:03	Running test Flip (cgl50) 	[375]
Jul 26 04:24:51	Running test SpecialTest1 (cgl50) 	[376]
Jul 26 04:27:42	Running test StartOnebyOne (cgl50) 	[377]
Jul 26 04:30:45	Running test StopOnebyOne (cgl50) 	[378]
Jul 26 04:31:09	Running test Restart (cgl49) 	[379]
Jul 26 04:33:29	Running test SimulStop (cgl50) 	[380]
Jul 26 04:34:07	Running test RestartOnebyOne (cgl50) 	[381]
Jul 26 04:37:04	Running test Stonithd (cgl49) 	[382]
Jul 26 04:40:14	Running test StopOnebyOne (cgl49) 	[383]
Jul 26 04:40:37	Running test SimulStart (cgl50) 	[384]
Jul 26 04:42:11	Running test Restart (cgl50) 	[385]
Jul 26 04:43:02	Running test ComponentFail (cgl50) 	[386]
Jul 26 04:43:02	...component crmd (dc=0,boot=1)
Jul 26 04:45:54	Running test SimulStop (cgl49) 	[387]
Jul 26 04:46:07	Running test NearQuorumPoint (cgl49) 	[388]
Jul 26 04:46:08	Running test SimulStart (cgl50) 	[389]
Jul 26 04:47:45	Running test SimulStart (cgl49) 	[390]
Jul 26 04:49:36	Running test Flip (cgl49) 	[391]
Jul 26 04:50:24	Running test Restart (cgl49) 	[392]
Jul 26 04:50:57	Running test Flip (cgl49) 	[393]
Jul 26 04:51:49	Running test StartOnebyOne (cgl49) 	[394]
Jul 26 04:54:37	Running test ComponentFail (cgl49) 	[395]
Jul 26 04:54:38	...component tengine (dc=1,boot=1)
Jul 26 04:57:27	Running test ResourceRecover (cgl50) 	[396]
Jul 26 04:57:37	Running test ResourceRecover (cgl49) 	[397]
Jul 26 04:57:54	Running test StopOnebyOne (cgl50) 	[398]
Jul 26 04:58:14	Running test Flip (cgl50) 	[399]
Jul 26 05:00:34	Running test SimulStop (cgl49) 	[400]
Jul 26 05:01:18	Running test RestartOnebyOne (cgl49) 	[401]
Jul 26 05:04:15	Running test ResourceRecover (cgl50) 	[402]
Jul 26 05:04:29	Running test StartOnebyOne (cgl50) 	[403]
Jul 26 05:07:31	Running test SimulStop (cgl50) 	[404]
Jul 26 05:07:45	Running test RestartOnebyOne (cgl50) 	[405]
Jul 26 05:10:39	Running test ResourceRecover (cgl50) 	[406]
Jul 26 05:10:53	Running test standby2 (cgl50) 	[407]
Jul 26 05:12:10	Running test PartialStart (cgl49) 	[408]
Jul 26 05:13:32	Running test StartOnebyOne (cgl50) 	[409]
Jul 26 05:16:19	Running test Flip (cgl49) 	[410]
Jul 26 05:17:12	Running test ResourceRecover (cgl50) 	[411]
Jul 26 05:17:46	Running test SimulStart (cgl50) 	[412]
Jul 26 05:19:33	Running test PartialStart (cgl50) 	[413]
Jul 26 05:20:59	Running test PartialStart (cgl50) 	[414]
Jul 26 05:22:09	Running test SimulStart (cgl49) 	[415]
Jul 26 05:23:45	Running test ResourceRecover (cgl49) 	[416]
Jul 26 05:24:05	Running test Flip (cgl50) 	[417]
Jul 26 05:25:00	Running test NearQuorumPoint (cgl49) 	[418]
Jul 26 05:26:57	Running test ComponentFail (cgl50) 	[419]
Jul 26 05:27:24	...component stonithd (dc=1,boot=0)
Jul 26 05:27:34	Running test SimulStop (cgl50) 	[420]
Jul 26 05:27:51	Running test Stonithd (cgl50) 	[421]
Jul 26 05:32:24	Running test ResourceRecover (cgl50) 	[422]
Jul 26 05:32:37	Running test PartialStart (cgl50) 	[423]
Jul 26 05:34:00	Running test SimulStart (cgl49) 	[424]
Jul 26 05:35:33	Running test ComponentFail (cgl49) 	[425]
Jul 26 05:35:33	...component ccm (dc=0,boot=1)
Jul 26 05:38:25	Running test SimulStop (cgl50) 	[426]
Jul 26 05:38:42	Running test standby2 (cgl50) 	[427]
Jul 26 05:41:23	Running test RestartOnebyOne (cgl50) 	[428]
Jul 26 05:42:56	Running test Flip (cgl50) 	[429]
Jul 26 05:43:45	Running test Stonithd (cgl50) 	[430]
Jul 26 05:47:16	Running test Restart (cgl49) 	[431]
Jul 26 05:48:06	Running test SimulStop (cgl50) 	[432]
Jul 26 05:48:22	Running test NearQuorumPoint (cgl49) 	[433]
Jul 26 05:50:39	Running test StopOnebyOne (cgl49) 	[434]
Jul 26 05:51:26	Running test Stonithd (cgl50) 	[435]
Jul 26 05:55:57	Running test SpecialTest1 (cgl50) 	[436]
Jul 26 05:58:55	Running test ComponentFail (cgl50) 	[437]
Jul 26 05:58:55	...component lrmd (dc=1,boot=1)
Jul 26 06:01:41	Running test PartialStart (cgl49) 	[438]
Jul 26 06:03:05	Running test standby2 (cgl49) 	[439]
Jul 26 06:05:50	Running test ResourceRecover (cgl50) 	[440]
Jul 26 06:06:01	Running test ComponentFail (cgl50) 	[441]
Jul 26 06:06:01	...component stonithd (dc=1,boot=0)
Jul 26 06:06:11	Running test SimulStart (cgl50) 	[442]
Jul 26 06:08:03	Running test ResourceRecover (cgl49) 	[443]
Jul 26 06:08:19	Running test Stonithd (cgl49) 	[444]
Jul 26 06:11:28	Running test StartOnebyOne (cgl49) 	[445]
Jul 26 06:14:27	Running test Stonithd (cgl50) 	[446]
Jul 26 06:17:33	Running test standby2 (cgl49) 	[447]
Jul 26 06:18:48	Running test SimulStop (cgl50) 	[448]
Jul 26 06:19:01	Running test SpecialTest1 (cgl50) 	[449]
Jul 26 06:21:49	Running test RestartOnebyOne (cgl49) 	[450]
Jul 26 06:23:23	Running test ResourceRecover (cgl49) 	[451]
Jul 26 06:23:35	Running test standby2 (cgl50) 	[452]
Jul 26 06:24:53	Running test Flip (cgl49) 	[453]
Jul 26 06:25:45	Running test SimulStop (cgl50) 	[454]
Jul 26 06:26:23	Running test RestartOnebyOne (cgl49) 	[455]
Jul 26 06:29:25	Running test Flip (cgl49) 	[456]
Jul 26 06:30:18	Running test StartOnebyOne (cgl50) 	[457]
Jul 26 06:33:05	Running test RestartOnebyOne (cgl49) 	[458]
Jul 26 06:34:42	Running test ComponentFail (cgl49) 	[459]
Jul 26 06:34:43	...component cib (dc=1,boot=1)
Jul 26 06:37:39	Running test Flip (cgl50) 	[460]
Jul 26 06:38:35	Running test Restart (cgl49) 	[461]
Jul 26 06:40:58	Running test NearQuorumPoint (cgl50) 	[462]
Jul 26 06:41:32	Running test Restart (cgl50) 	[463]
Jul 26 06:42:17	Running test standby2 (cgl49) 	[464]
Jul 26 06:43:32	Running test standby2 (cgl49) 	[465]
Jul 26 06:44:47	Running test standby2 (cgl49) 	[466]
Jul 26 06:46:02	Running test NearQuorumPoint (cgl50) 	[467]
Jul 26 06:46:25	Running test Flip (cgl50) 	[468]
Jul 26 06:47:00	Running test StopOnebyOne (cgl50) 	[469]
Jul 26 06:48:43	Running test StartOnebyOne (cgl49) 	[470]
Jul 26 06:51:34	Running test PartialStart (cgl49) 	[471]
Jul 26 06:52:59	Running test Restart (cgl50) 	[472]
Jul 26 06:55:21	Running test SimulStop (cgl50) 	[473]
Jul 26 06:56:00	Running test PartialStart (cgl49) 	[474]
Jul 26 06:57:10	Running test SimulStop (cgl50) 	[475]
Jul 26 06:58:48	Running test StopOnebyOne (cgl50) 	[476]
Jul 26 07:00:35	Running test ResourceRecover (cgl50) 	[477]
Jul 26 07:02:12	Running test standby2 (cgl50) 	[478]
Jul 26 07:03:26	Running test Restart (cgl49) 	[479]
Jul 26 07:04:17	Running test SimulStart (cgl50) 	[480]
Jul 26 07:06:07	Running test NearQuorumPoint (cgl49) 	[481]
Jul 26 07:06:23	Running test StartOnebyOne (cgl49) 	[482]
Jul 26 07:09:11	Running test SimulStart (cgl49) 	[483]
Jul 26 07:11:00	Running test NearQuorumPoint (cgl50) 	[484]
Jul 26 07:11:08	Running test StopOnebyOne (cgl50) 	[485]
Jul 26 07:11:26	Running test StartOnebyOne (cgl50) 	[486]
Jul 26 07:14:11	Running test ResourceRecover (cgl49) 	[487]
Jul 26 07:14:21	Running test standby2 (cgl50) 	[488]
Jul 26 07:15:39	Running test Flip (cgl50) 	[489]
Jul 26 07:16:26	Running test StartOnebyOne (cgl50) 	[490]
Jul 26 07:19:13	Running test StartOnebyOne (cgl50) 	[491]
Jul 26 07:22:13	Running test RestartOnebyOne (cgl50) 	[492]
Jul 26 07:23:46	Running test PartialStart (cgl49) 	[493]
Jul 26 07:25:10	Running test StopOnebyOne (cgl50) 	[494]
Jul 26 07:26:53	Running test ResourceRecover (cgl49) 	[495]
Jul 26 07:28:34	Running test ComponentFail (cgl49) 	[496]
Jul 26 07:28:35	...component cib (dc=0,boot=1)
Jul 26 07:31:26	Running test PartialStart (cgl50) 	[497]
Jul 26 07:32:52	Running test SimulStart (cgl50) 	[498]
Jul 26 07:34:26	Running test ResourceRecover (cgl50) 	[499]
Jul 26 07:34:36	Running test PartialStart (cgl50) 	[500]
Jul 26 07:35:59	Stopping Cluster Manager on all nodes
Jul 26 07:35:59	****************
Jul 26 07:35:59	Overall Results:{'failure': 0, 'success': 500, 'BadNews': 0}
Jul 26 07:35:59	****************
Jul 26 07:35:59	Detailed Results
Jul 26 07:35:59	Test Flip: 	{'elapsed_time': 2196.8458616733551, 'skipped': 0, \
'calls': 38, 'success': 38, 'started': 9, 'down->up': 9, 'auditfail': 0, 'failure': \
0, 'stopped': 29, 'max_time': 134.24374890327454, 'min_time': 25.265329122543335, \
                'up->down': 29}
Jul 26 07:35:59	Test Restart: 	{'elapsed_time': 2503.1073522567749, 'skipped': 0, \
'calls': 41, 'success': 41, 'min_time': 24.928310871124268, 'node:cgl49': 19, \
'auditfail': 0, 'failure': 0, 'max_time': 137.9945809841156, 'WasStopped': 32, \
                'node:cgl50': 22}
Jul 26 07:35:59	Test Stonithd: 	{'elapsed_time': 6719.7714626789093, 'skipped': 0, \
'calls': 32, 'success': 32, 'auditfail': 0, 'failure': 0, 'max_time': \
                272.05326509475708, 'min_time': 175.01992702484131}
Jul 26 07:35:59	Test StartOnebyOne: 	{'elapsed_time': 5659.2619166374207, 'skipped': \
0, 'calls': 34, 'success': 34, 'auditfail': 0, 'failure': 0, 'max_time': \
                165.19714999198914, 'min_time': 155.1442539691925}
Jul 26 07:35:59	Test SimulStart: 	{'elapsed_time': 3123.2426362037659, 'skipped': 0, \
'calls': 33, 'success': 33, 'auditfail': 0, 'failure': 0, 'max_time': \
                105.71168088912964, 'min_time': 84.037414073944092}
Jul 26 07:35:59	Test SimulStop: 	{'elapsed_time': 1133.2050211429596, 'skipped': 0, \
'calls': 33, 'success': 33, 'auditfail': 0, 'failure': 0, 'max_time': \
                102.50121688842773, 'min_time': 12.656371116638184}
Jul 26 07:35:59	Test StopOnebyOne: 	{'elapsed_time': 1691.2769255638123, 'skipped': \
0, 'calls': 39, 'success': 39, 'auditfail': 0, 'failure': 0, 'max_time': \
                26.028330087661743, 'min_time': 17.131213903427124}
Jul 26 07:35:59	Test RestartOnebyOne: 	{'elapsed_time': 3971.4064090251923, \
'skipped': 0, 'calls': 37, 'success': 37, 'auditfail': 0, 'failure': 0, 'max_time': \
                93.03079891204834, 'min_time': 79.468178987503052}
Jul 26 07:35:59	Test PartialStart: 	{'elapsed_time': 2334.4547417163849, 'skipped': \
0, 'calls': 29, 'success': 29, 'auditfail': 0, 'failure': 0, 'max_time': \
                86.133626937866211, 'min_time': 68.318685054779053}
Jul 26 07:35:59	Test standby2: 	{'elapsed_time': 3969.9773828983307, 'skipped': 0, \
'calls': 42, 'success': 42, 'auditfail': 0, 'failure': 0, 'max_time': \
                159.38483786582947, 'min_time': 66.617243051528931}
Jul 26 07:35:59	Test ResourceRecover: 	{'elapsed_time': 745.37951374053955, \
'skipped': 0, 'calls': 38, 'success': 38, 'auditfail': 0, 'failure': 0, 'max_time': \
                97.431318044662476, 'min_time': 2.5022552013397217}
Jul 26 07:35:59	Test ComponentFail: 	{'elapsed_time': 6429.3971040248871, 'skipped': \
0, 'calls': 41, 'success': 41, 'cib': 7, 'stonithd': 6, 'pengine': 1, 'lrmd': 8, \
'auditfail': 0, 'failure': 0, 'ccm': 4, 'crmd': 9, 'tengine': 6, 'max_time': \
                250.20063495635986, 'min_time': 1.3860170841217041}
Jul 26 07:35:59	Test SpecialTest1: 	{'elapsed_time': 5245.4763262271881, 'skipped': \
0, 'calls': 31, 'success': 0, 'auditfail': 0, 'failure': 0, 'max_time': \
                179.44279503822327, 'min_time': 158.35295510292053}
Jul 26 07:35:59	Test NearQuorumPoint: 	{'elapsed_time': 1002.3898875713348, \
'skipped': 8, 'calls': 32, 'success': 24, 'auditfail': 0, 'failure': 0, 'max_time': \
133.40973401069641, 'min_time': 0.27209186553955078} Jul 26 07:35:59	<<<<<<<<<<<<<<<< \
TESTS COMPLETED



_______________________________________________________
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/


[prev in list] [next in list] [prev in thread] [next in thread] 

Configure | About | News | Add a list | Sponsored by KoreLogic