[prev in list] [next in list] [prev in thread] [next in thread]
List: vdsm-devel
Subject: [ovirt-devel] [VDSM] Flaky network test?
From: Nir Soffer <nsoffer () redhat ! com>
Date: 2018-02-26 12:44:32
Message-ID: CAMRbyyvsb3H8+Hyne6TnHL0xg==-B5MSj8pZtyCWK-kVVXB0MQ () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
I had this random failure (one in 3 builds) - can we mark this tests as
broken on ovirt ci
until we make it more reliable?
*00:04:16.904* ======================================================================*00:04:16.905*
ERROR: test_add_ipv6_gateway_given_existing_ipv4_and_ipv6_gateways
(network.ip_address_test.TestAddressSetup)*00:04:16.906*
----------------------------------------------------------------------*00:04:16.906*
Traceback (most recent call last):*00:04:16.906* File
"/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/network/ip_address_test.py",
line 147, in test_add_ipv6_gateway_given_existing_ipv4_and_ipv6_gateways*00:04:16.908*
address.add(nic, ipv4=None, ipv6=ipv6)*00:04:16.908* File
"/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/lib/vdsm/network/ip/address/__init__.py",
line 277, in add*00:04:16.909* _add_ipv6_address(iface,
ipv6)*00:04:16.909* File
"/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/lib/vdsm/network/ip/address/__init__.py",
line 292, in _add_ipv6_address*00:04:16.910*
ipwrapper.addrAdd(iface, ipv6addr, ipv6netmask,
family=6)*00:04:16.911* File
"/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/lib/vdsm/network/ipwrapper.py",
line 561, in addrAdd*00:04:16.911*
_exec_cmd(command)*00:04:16.912* File
"/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/lib/vdsm/network/ipwrapper.py",
line 482, in _exec_cmd*00:04:16.912* raise exc(returnCode,
error.splitlines())*00:04:16.913* IPRoute2Error: (2, ['RTNETLINK
answers: File exists'])*00:04:16.913* -------------------- >> begin
captured logging << --------------------*00:04:16.914* 2018-02-26
12:31:00,244 DEBUG (MainThread) [root] /sbin/ip link add name
dummy_aZGko type dummy (cwd None) (cmdutils:150)*00:04:16.915*
2018-02-26 12:31:00,256 DEBUG (MainThread) [root] SUCCESS: <err> = '';
<rc> = 0 (cmdutils:158)*00:04:16.915* 2018-02-26 12:31:00,260 DEBUG
(netlink/events) [root] START thread <Thread(netlink/events, started
daemon 140705544533760)> (func=<bound method Monitor._scan of
<vdsm.network.netlink.monitor.Monitor object at 0x5c51150>>, args=(),
kwargs={}) (concurrent:192)*00:04:16.917* 2018-02-26 12:31:00,261
DEBUG (MainThread) [root] /sbin/ip link set dev dummy_aZGko up (cwd
None) (cmdutils:150)*00:04:16.918* 2018-02-26 12:31:00,272 DEBUG
(MainThread) [root] SUCCESS: <err> = ''; <rc> = 0
(cmdutils:158)*00:04:16.919* 2018-02-26 12:31:00,275 DEBUG
(netlink/events) [root] FINISH thread <Thread(netlink/events, started
daemon 140705544533760)> (concurrent:195)*00:04:16.920* 2018-02-26
12:31:00,279 DEBUG (MainThread) [root] /sbin/ip route show to
0.0.0.0/0 table main (cwd None) (cmdutils:150)*00:04:16.921*
2018-02-26 12:31:00,293 DEBUG (MainThread) [root] SUCCESS: <err> = '';
<rc> = 0 (cmdutils:158)*00:04:16.921* 2018-02-26 12:31:00,295 DEBUG
(MainThread) [root] /sbin/ip -6 route show to ::/0 table main (cwd
None) (cmdutils:150)*00:04:16.922* 2018-02-26 12:31:00,311 DEBUG
(MainThread) [root] SUCCESS: <err> = ''; <rc> = 0
(cmdutils:158)*00:04:16.923* 2018-02-26 12:31:00,313 DEBUG
(MainThread) [root] /sbin/ip -4 addr add dev dummy_aZGko
192.168.99.1/255.255.255.248 (cwd None) (cmdutils:150)*00:04:16.924*
2018-02-26 12:31:00,328 DEBUG (MainThread) [root] SUCCESS: <err> = '';
<rc> = 0 (cmdutils:158)*00:04:16.925* 2018-02-26 12:31:00,329 DEBUG
(MainThread) [root] /sbin/ip -4 route add default via 192.168.99.6
(cwd None) (cmdutils:150)*00:04:16.925* 2018-02-26 12:31:00,340 DEBUG
(MainThread) [root] SUCCESS: <err> = ''; <rc> = 0
(cmdutils:158)*00:04:16.926* 2018-02-26 12:31:00,342 DEBUG
(MainThread) [root] /sbin/ip -6 addr flush dev dummy_aZGko scope
global (cwd None) (cmdutils:150)*00:04:16.927* 2018-02-26 12:31:00,363
DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0
(cmdutils:158)*00:04:16.928* 2018-02-26 12:31:00,364 DEBUG
(MainThread) [root] /sbin/ip -6 addr add dev dummy_aZGko 2001:99::1/64
(cwd None) (cmdutils:150)*00:04:16.929* 2018-02-26 12:31:00,398 DEBUG
(MainThread) [root] SUCCESS: <err> = ''; <rc> = 0
(cmdutils:158)*00:04:16.929* 2018-02-26 12:31:00,399 DEBUG
(MainThread) [root] /sbin/ip -6 route add default via 2001:99::99 dev
dummy_aZGko (cwd None) (cmdutils:150)*00:04:16.930* 2018-02-26
12:31:00,410 DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0
(cmdutils:158)*00:04:16.931* 2018-02-26 12:31:00,412 DEBUG
(MainThread) [root] /sbin/ip -6 addr flush dev dummy_aZGko scope
global (cwd None) (cmdutils:150)*00:04:16.932* 2018-02-26 12:31:00,441
DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0
(cmdutils:158)*00:04:16.932* 2018-02-26 12:31:00,442 DEBUG
(MainThread) [root] /sbin/ip -6 addr add dev dummy_aZGko 2001:99::1/64
(cwd None) (cmdutils:150)*00:04:17.049* 2018-02-26 12:31:00,473
DCoverage.py warning: Module
/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/vdsm
was never imported. (module-not-imported)*00:04:32.618* EBUG
(MainThread) [root] FAILED: <err> = 'RTNETLINK answers: File
exists\n'; <rc> = 2 (cmdutils:158)*00:04:32.619* 2018-02-26
12:31:00,482 DEBUG (MainThread) [root] /sbin/ip link del dev
dummy_aZGko (cwd None) (cmdutils:150)*00:04:32.619* 2018-02-26
12:31:00,549 DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0
(cmdutils:158)*00:04:32.620* --------------------- >> end captured
logging << ---------------------
[Attachment #5 (text/html)]
<div dir="ltr">I had this random failure (one in 3 builds) - can we mark this tests \
as broken on ovirt ci<div>until we make it more \
reliable?<br><div><div><br></div><div><pre class="inbox-inbox-console-output" \
style="box-sizing:border-box;white-space:pre-wrap;word-wrap:break-word;margin-top:0px;margin-bottom:0px;color:rgb(51,51,51);font-size:14px"><span \
class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.904</b> \
</span>====================================================================== <span \
class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.905</b> </span>ERROR: \
test_add_ipv6_gateway_given_existing_ipv4_and_ipv6_gateways \
(network.ip_address_test.TestAddressSetup) <span class="inbox-inbox-timestamp" \
style="box-sizing:border-box"><b style="box-sizing:border-box">00:04:16.906</b> \
</span>---------------------------------------------------------------------- <span \
class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.906</b> </span>Traceback (most recent call \
last): <span class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.906</b> </span> File \
"/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/network/ip_address_test.py", \
line 147, in test_add_ipv6_gateway_given_existing_ipv4_and_ipv6_gateways <span \
class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.908</b> </span> address.add(nic, ipv4=None, \
ipv6=ipv6) <span class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.908</b> </span> File \
"/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/lib/vdsm/network/ip/address/__init__.py", \
line 277, in add <span class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.909</b> </span> _add_ipv6_address(iface, \
ipv6) <span class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.909</b> </span> File \
"/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/lib/vdsm/network/ip/address/__init__.py", \
line 292, in _add_ipv6_address <span class="inbox-inbox-timestamp" \
style="box-sizing:border-box"><b style="box-sizing:border-box">00:04:16.910</b> \
</span> ipwrapper.addrAdd(iface, ipv6addr, ipv6netmask, family=6) <span \
class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.911</b> </span> File \
"/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/lib/vdsm/network/ipwrapper.py", \
line 561, in addrAdd <span class="inbox-inbox-timestamp" \
style="box-sizing:border-box"><b style="box-sizing:border-box">00:04:16.911</b> \
</span> _exec_cmd(command) <span class="inbox-inbox-timestamp" \
style="box-sizing:border-box"><b style="box-sizing:border-box">00:04:16.912</b> \
</span> File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/lib/vdsm/network/ipwrapper.py", \
line 482, in _exec_cmd <span class="inbox-inbox-timestamp" \
style="box-sizing:border-box"><b style="box-sizing:border-box">00:04:16.912</b> \
</span> raise exc(returnCode, error.splitlines()) <span \
class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.913</b> </span>IPRoute2Error: (2, \
['RTNETLINK answers: File exists']) <span class="inbox-inbox-timestamp" \
style="box-sizing:border-box"><b style="box-sizing:border-box">00:04:16.913</b> \
</span>-------------------- >> begin captured logging << \
-------------------- <span class="inbox-inbox-timestamp" \
style="box-sizing:border-box"><b style="box-sizing:border-box">00:04:16.914</b> \
</span>2018-02-26 12:31:00,244 DEBUG (MainThread) [root] /sbin/ip link add name \
dummy_aZGko type dummy (cwd None) (cmdutils:150) <span class="inbox-inbox-timestamp" \
style="box-sizing:border-box"><b style="box-sizing:border-box">00:04:16.915</b> \
</span>2018-02-26 12:31:00,256 DEBUG (MainThread) [root] SUCCESS: <err> = \
''; <rc> = 0 (cmdutils:158) <span class="inbox-inbox-timestamp" \
style="box-sizing:border-box"><b style="box-sizing:border-box">00:04:16.915</b> \
</span>2018-02-26 12:31:00,260 DEBUG (netlink/events) [root] START thread \
<Thread(netlink/events, started daemon 140705544533760)> (func=<bound method \
Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at \
0x5c51150>>, args=(), kwargs={}) (concurrent:192) <span \
class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.917</b> </span>2018-02-26 12:31:00,261 DEBUG \
(MainThread) [root] /sbin/ip link set dev dummy_aZGko up (cwd None) (cmdutils:150) \
<span class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.918</b> </span>2018-02-26 12:31:00,272 DEBUG \
(MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (cmdutils:158) \
<span class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.919</b> </span>2018-02-26 12:31:00,275 DEBUG \
(netlink/events) [root] FINISH thread <Thread(netlink/events, started daemon \
140705544533760)> (concurrent:195) <span class="inbox-inbox-timestamp" \
style="box-sizing:border-box"><b style="box-sizing:border-box">00:04:16.920</b> \
</span>2018-02-26 12:31:00,279 DEBUG (MainThread) [root] /sbin/ip route show to <a \
href="http://0.0.0.0/0">0.0.0.0/0</a> table main (cwd None) (cmdutils:150) <span \
class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.921</b> </span>2018-02-26 12:31:00,293 DEBUG \
(MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (cmdutils:158) \
<span class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.921</b> </span>2018-02-26 12:31:00,295 DEBUG \
(MainThread) [root] /sbin/ip -6 route show to ::/0 table main (cwd None) \
(cmdutils:150) <span class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.922</b> </span>2018-02-26 12:31:00,311 DEBUG \
(MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (cmdutils:158) \
<span class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.923</b> </span>2018-02-26 12:31:00,313 DEBUG \
(MainThread) [root] /sbin/ip -4 addr add dev dummy_aZGko <a \
href="http://192.168.99.1/255.255.255.248">192.168.99.1/255.255.255.248</a> (cwd \
None) (cmdutils:150) <span class="inbox-inbox-timestamp" \
style="box-sizing:border-box"><b style="box-sizing:border-box">00:04:16.924</b> \
</span>2018-02-26 12:31:00,328 DEBUG (MainThread) [root] SUCCESS: <err> = \
''; <rc> = 0 (cmdutils:158) <span class="inbox-inbox-timestamp" \
style="box-sizing:border-box"><b style="box-sizing:border-box">00:04:16.925</b> \
</span>2018-02-26 12:31:00,329 DEBUG (MainThread) [root] /sbin/ip -4 route add \
default via 192.168.99.6 (cwd None) (cmdutils:150) <span \
class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.925</b> </span>2018-02-26 12:31:00,340 DEBUG \
(MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (cmdutils:158) \
<span class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.926</b> </span>2018-02-26 12:31:00,342 DEBUG \
(MainThread) [root] /sbin/ip -6 addr flush dev dummy_aZGko scope global (cwd None) \
(cmdutils:150) <span class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.927</b> </span>2018-02-26 12:31:00,363 DEBUG \
(MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (cmdutils:158) \
<span class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.928</b> </span>2018-02-26 12:31:00,364 DEBUG \
(MainThread) [root] /sbin/ip -6 addr add dev dummy_aZGko 2001:99::1/64 (cwd None) \
(cmdutils:150) <span class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.929</b> </span>2018-02-26 12:31:00,398 DEBUG \
(MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (cmdutils:158) \
<span class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.929</b> </span>2018-02-26 12:31:00,399 DEBUG \
(MainThread) [root] /sbin/ip -6 route add default via 2001:99::99 dev dummy_aZGko \
(cwd None) (cmdutils:150) <span class="inbox-inbox-timestamp" \
style="box-sizing:border-box"><b style="box-sizing:border-box">00:04:16.930</b> \
</span>2018-02-26 12:31:00,410 DEBUG (MainThread) [root] SUCCESS: <err> = \
''; <rc> = 0 (cmdutils:158) <span class="inbox-inbox-timestamp" \
style="box-sizing:border-box"><b style="box-sizing:border-box">00:04:16.931</b> \
</span>2018-02-26 12:31:00,412 DEBUG (MainThread) [root] /sbin/ip -6 addr flush dev \
dummy_aZGko scope global (cwd None) (cmdutils:150) <span \
class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.932</b> </span>2018-02-26 12:31:00,441 DEBUG \
(MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (cmdutils:158) \
<span class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:16.932</b> </span>2018-02-26 12:31:00,442 DEBUG \
(MainThread) [root] /sbin/ip -6 addr add dev dummy_aZGko 2001:99::1/64 (cwd None) \
(cmdutils:150) <span class="inbox-inbox-timestamp" style="box-sizing:border-box"><b \
style="box-sizing:border-box">00:04:17.049</b> </span>2018-02-26 12:31:00,473 \
DCoverage.py warning: Module \
/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/vdsm was never \
imported. (module-not-imported) <span class="inbox-inbox-timestamp" \
style="box-sizing:border-box"><b style="box-sizing:border-box">00:04:32.618</b> \
</span>EBUG (MainThread) [root] FAILED: <err> = 'RTNETLINK answers: File \
exists\n'; <rc> = 2 (cmdutils:158) <span class="inbox-inbox-timestamp" \
style="box-sizing:border-box"><b style="box-sizing:border-box">00:04:32.619</b> \
</span>2018-02-26 12:31:00,482 DEBUG (MainThread) [root] /sbin/ip link del dev \
dummy_aZGko (cwd None) (cmdutils:150) <span class="inbox-inbox-timestamp" \
style="box-sizing:border-box"><b style="box-sizing:border-box">00:04:32.619</b> \
</span>2018-02-26 12:31:00,549 DEBUG (MainThread) [root] SUCCESS: <err> = \
''; <rc> = 0 (cmdutils:158) <span class="inbox-inbox-timestamp" \
style="box-sizing:border-box"><b style="box-sizing:border-box">00:04:32.620</b> \
</span>--------------------- >> end captured logging << \
--------------------- </pre><br \
class="inbox-inbox-Apple-interchange-newline"></div></div></div></div>
_______________________________________________
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic