[prev in list] [next in list] [prev in thread] [next in thread]
List: e1000-devel
Subject: [E1000-devel] Unicast v. Multicast with e1000e Driver - Dropping
From: Jonathan Haws <Jonathan.Haws () sdl ! usu ! edu>
Date: 2011-05-26 20:00:01
Message-ID: 5ABE8A5E96C0364CAF2B4DC1DEB7CEAC490E90B4 () Mercury ! usurf ! usu ! edu
[Download RAW message or body]
All,
I am not sure if this is the right list for this, but I guess this is a good place to \
start.
I am developing for an embedded system (a 64-bit Core 2 Duo). I am running a vanilla \
kernel (2.6.39). Our application is simply this - receive multicast data from the \
network and dump it to an SSD.
My problem comes in two parts: unicast data works somewhat and multicast data does \
not at all.
Here is my test scenario (everything is UDP - no TCP):
A PC running Win7 blasts a file over the network - this file simple contains a ramp \
of integers (0, 1, 2, 3, 4, 5, ...). On the receiving side I receive the datagrams \
(however large - we are allowing for the IP stack to handle fragmentation) and check \
the first value of the ramp that is in that datagram. When the value of the data in \
the packet is not what I expect, I print an error, set my counter to the value of the \
data, and move on.
When I am unicasting, everything is just peachy until I get to large datagram sizes. \
I can run at high rates (up to 30 MiB/s I have tested) and don't lose anything. I \
have run with varying datagram lengths; from 5000 bytes to 30000 bytes. Once I start \
to get into the 15000 to 20000 byte range, things go south (no offense to anyone from \
the south :) ). My MTU is set to 9000 on the transmit and receive side (and yes, my \
switches handle jumbo frames...).
When I am multicasting, things go bad. Even running at a low rate (say 5 MiB/s) I \
see missing datagrams. The things that really bothers me is that these do not show \
up anywhere. ifconfig does not report anything dropped, and ethtool -S does not \
report any errors whatsoever. But it is clear to me that packets are being missed. \
With a datagram size of 5000 things work okay, but when I step up to 8500 things go \
awry.
It seems like something is up with larger datagrams, but I can't seem to put my \
finger on it. I have tried both of my network interfaces and they both behave in the \
same manner.
I have looked at some sysctl parameters and everything seems to look good. I \
increased net.core.rmem_max and net.core.wmem_max to 64 MB (way overkill I think, \
nevertheless...). net.ipv4.udp_mem is "186048 248064 372096". That is just what it \
is defaulting to, which seems like oodles of pages. Nothing else I could find really \
dealt with my scenario.
I have attached the test code if anyone feels like that may help get to the bottom of \
the issue. I don't know if this is just an exercise in optimizing the network stack, \
or if there really is a bug somewhere. Any way you want to look at it, any help is \
appreciated.
Output follows...
Thanks!
Jonathan
Here is the hardware I have:
ADLGS45 (Advanced Digital Logic PCI/104-Express Core 2 Duo SBC)
eth0 = Intel 82567LM-2 Gigabit Network Controller
eth1 = Intel 82574L Gigabit Network Controller
Here is some output from the system:
# ifconfig
eth0 Link encap:Ethernet HWaddr 00:01:05:0A:4A:92
inet addr:172.31.22.90 Bcast:172.31.22.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:1674366 errors:0 dropped:0 overruns:0 frame:0
TX packets:2620 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:10694569545 (9.9 GiB) TX bytes:459481 (448.7 KiB)
Interrupt:20 Memory:fdfc0000-fdfe0000
# ethtool -i eth0
driver: e1000e
version: 1.3.10-k2
firmware-version: 1.8-4
bus-info: 0000:00:19.0
# ethtool -g eth0
Ring parameters for eth0:
Pre-set maximums:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
Current hardware settings:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
# ethtool -k eth0
Offload parameters for eth0:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: off
# ethtool -S eth0
NIC statistics:
rx_packets: 1674840
tx_packets: 2772
rx_bytes: 10694622001
tx_bytes: 481113
rx_broadcast: 1063
tx_broadcast: 0
rx_multicast: 159412
tx_multicast: 4
rx_errors: 0
tx_errors: 0
tx_dropped: 0
multicast: 159412
collisions: 0
rx_length_errors: 0
rx_over_errors: 0
rx_crc_errors: 0
rx_frame_errors: 0
rx_no_buffer_count: 0
rx_missed_errors: 0
tx_aborted_errors: 0
tx_carrier_errors: 0
tx_fifo_errors: 0
tx_heartbeat_errors: 0
tx_window_errors: 0
tx_abort_late_coll: 0
tx_deferred_ok: 0
tx_single_coll_ok: 0
tx_multi_coll_ok: 0
tx_timeout_count: 0
tx_restart_queue: 0
rx_long_length_errors: 0
rx_short_length_errors: 0
rx_align_errors: 0
tx_tcp_seg_good: 0
tx_tcp_seg_failed: 0
rx_flow_control_xon: 0
rx_flow_control_xoff: 0
tx_flow_control_xon: 0
tx_flow_control_xoff: 0
rx_long_byte_count: 10694622001
rx_csum_offload_good: 106021
rx_csum_offload_errors: 0
rx_header_split: 0
alloc_rx_buff_failed: 0
tx_smbus: 0
rx_smbus: 0
dropped_smbus: 0
rx_dma_failed: 0
tx_dma_failed: 0
Output for eth1 is very similar, but here is ethtool -i eth1
# ethtool -i eth1
driver: e1000e
version: 1.3.10-k2
firmware-version: 15.255-15
bus-info: 0000:02:00.0
Now here is some output from my test program. Packets = XXX --> MB = XXXX is just a \
status thread printing out the current rates.
# benchnetRX
Usage: benchnetRX <ip> <port> <max datagramlen> <recvbuf in KB> <multicast> \
<benchmark>
# benchnetRX 3 19601 50000 16384 0 1
-> Beginning network benchmark ...
-> Creating socket for benchmark on 00000000:19601
-> Packets = 159 --> MB = 1.011
-> Packets = 750 --> MB = 4.768
lost packets = 0
-> Packets = 749 --> MB = 4.762
lost packets = 1
-> Packets = 750 --> MB = 4.768
lost packets = 1
-> Packets = 748 --> MB = 4.756
-> Packets = 749 --> MB = 4.762
lost packets = 2
-> Packets = 746 --> MB = 4.743
lost packets = 5
-> Packets = 739 --> MB = 4.698
-> Packets = 520 --> MB = 3.306
^C -> Exiting network RX benchmark...
# benchnetRX 3 19601 50000 16384 1 1
-> Beginning network benchmark ...
-> Creating socket for benchmark on e0010103:19601
-> Packets = 256 --> MB = 1.628
lost packets = 1
-> Packets = 750 --> MB = 4.768
-> Packets = 749 --> MB = 4.762
lost packets = 3
-> Packets = 750 --> MB = 4.768
lost packets = 0
-> Packets = 751 --> MB = 4.775
lost packets = 0
-> Packets = 749 --> MB = 4.762
-> Packets = 750 --> MB = 4.768
lost packets = 3
-> Packets = 750 --> MB = 4.768
lost packets = 2
-> Packets = 743 --> MB = 4.724
-> Packets = 718 --> MB = 4.565
lost packets = 37
^C -> Exiting network RX benchmark...
--
Jonathan R. Haws
Electrical Engineer
Space Dynamics Laboratory
(435) 713-3489
jhaws@sdl.usu.edu
------------------------------------------------------------------------------
vRanger cuts backup time in half-while increasing security.
With the market-leading solution for virtual backup and recovery,
you get blazing-fast, flexible, and affordable data protection.
Download your free trial now.
http://p.sf.net/sfu/quest-d2dcopy1
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit \
http://communities.intel.com/community/wired
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic