[prev in list] [next in list] [prev in thread] [next in thread]
List: e1000-devel
Subject: Re: [E1000-devel] [EXTERNAL] RE: TCP stack is receiving out of order packer delivery even when conne
From: "Fujinaka, Todd" <todd.fujinaka () intel ! com>
Date: 2015-09-10 15:42:26
Message-ID: 9B4A1B1917080E46B64F07F2989DADD6535EA143 () ORSMSX114 ! amr ! corp ! intel ! com
[Download RAW message or body]
Not at all. You can download the latest specification update and see exactly what the \
problem was. Go to downloadcenter.intel.com (or just intel.com) and search for xl710 \
spec update.
Todd Fujinaka
Software Application Engineer
Networking Division (ND)
Intel Corporation
todd.fujinaka@intel.com
(503) 712-4565
-----Original Message-----
From: Hu, Tan Chang [mailto:tchu@sandia.gov]
Sent: Thursday, September 10, 2015 8:31 AM
To: Fujinaka, Todd; e1000-devel@lists. sourceforge. net \
(e1000-devel@lists.sourceforge.net)
Cc: Naegle, John H
Subject: RE: [E1000-devel] [EXTERNAL] RE: TCP stack is receiving out of order packer \
delivery even when connected back-to-back
A fresh install did not fix the problem of not being able to turn off the flow \
director.
I understand that Intel has put a stop-ship on XL710. Does that have any relevance in \
relation to the issues here?
-----Original Message-----
From: Fujinaka, Todd [mailto:todd.fujinaka@intel.com]
Sent: Wednesday, August 05, 2015 3:12 PM
To: Hu, Tan Chang; e1000-devel@lists. sourceforge. net \
(e1000-devel@lists.sourceforge.net)
Subject: RE: [E1000-devel] [EXTERNAL] RE: TCP stack is receiving out of order packer \
delivery even when connected back-to-back
My mistake.
I think you should start over with a fresh install as I'm having no trouble turning \
off flow director.
Todd Fujinaka
Software Application Engineer
Networking Division (ND)
Intel Corporation
todd.fujinaka@intel.com
(503) 712-4565
-----Original Message-----
From: Hu, Tan Chang [mailto:tchu@sandia.gov]
Sent: Wednesday, August 05, 2015 12:15 PM
To: Fujinaka, Todd
Subject: Re: [E1000-devel] [EXTERNAL] RE: TCP stack is receiving out of order packer \
delivery even when connected back-to-back
XPS is not part of the 2.6.32-504.el6.x86_64 kernel source so you can not turn it \
off. It was not incorporated into the kernel tree until 2.6.38 according to the \
kernel notes. ________________________________________
From: Fujinaka, Todd <todd.fujinaka@intel.com>
Sent: Thursday, July 30, 2015 4:16 PM
To: Hu, Tan Chang
Subject: RE: [E1000-devel] [EXTERNAL] RE: TCP stack is receiving out of order packer \
delivery even when connected back-to-back
I honestly don't know what the difference is in your system. I still haven't gotten \
it to fail.
In any case, I hope you're testing the third fix (turn off XPS in the kernel) and \
aren't getting stuck on this ethtool issue.
[htfujina@htfujina-fc ~]$ uname -a
Linux htfujina-fc.jf.intel.com 2.6.32-573.1.1.el6.x86_64 #1 SMP Tue Jul 14 02:46:51 \
EDT 2015 x86_64 x86_64 x86_64 GNU/Linux [htfujina@htfujina-fc ~]$ sudo ethtool -k \
eth14 Features for eth14:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-unneeded: off [fixed]
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: on
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: on
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: on
tx-tcp6-segmentation: on
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: on
receive-hashing: on
highdma: on
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-udp_tnl-segmentation: on
fcoe-mtu: off [fixed]
loopback: off [fixed]
[htfujina@htfujina-fc ~]$ sudo ethtool -K eth14 ntuple off [htfujina@htfujina-fc ~]$ \
sudo ethtool -k eth14 Features for eth14:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-unneeded: off [fixed]
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: on
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: on
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: on
tx-tcp6-segmentation: on
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: on
highdma: on
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-udp_tnl-segmentation: on
fcoe-mtu: off [fixed]
loopback: off [fixed]
[htfujina@htfujina-fc ~]$
Todd Fujinaka
Software Application Engineer
Networking Division (ND)
Intel Corporation
todd.fujinaka@intel.com
(503) 712-4565
-----Original Message-----
From: Hu, Tan Chang [mailto:tchu@sandia.gov]
Sent: Thursday, July 30, 2015 1:16 PM
To: Fujinaka, Todd
Subject: Re: [E1000-devel] [EXTERNAL] RE: TCP stack is receiving out of order packer \
delivery even when connected back-to-back
We will have to revert the kernel version from 2.6.32-504.30.3.el6.x86_64 and see if \
that works.
________________________________________
From: Fujinaka, Todd <todd.fujinaka@intel.com>
Sent: Thursday, July 30, 2015 2:01 PM
To: Hu, Tan Chang; e1000-devel@lists. sourceforge. net \
(e1000-devel@lists.sourceforge.net)
Subject: RE: [E1000-devel] [EXTERNAL] RE: TCP stack is receiving out of order packer \
delivery even when connected back-to-back
[htfujina@htfujina-fc src]$ uname -a
Linux htfujina-fc.jf.intel.com 2.6.32-431.11.2.el6.x86_64 #1 SMP Mon Mar 3 13:32:45 \
EST 2014 x86_64 x86_64 x86_64 GNU/Linux
Todd Fujinaka
Software Application Engineer
Networking Division (ND)
Intel Corporation
todd.fujinaka@intel.com
(503) 712-4565
-----Original Message-----
From: Hu, Tan Chang [mailto:tchu@sandia.gov]
Sent: Thursday, July 30, 2015 10:58 AM
To: Fujinaka, Todd
Subject: Re: [E1000-devel] [EXTERNAL] RE: TCP stack is receiving out of order packer \
delivery even when connected back-to-back
The NIC firmware is the same revision level as yours. What linux kernel level are you \
at?
[root@r445403 ethtool-4.0]# ethtool -i i40e1
driver: i40e
version: 1.2.48
firmware-version: f4.33.31377 a1.2 n4.42 e1932
bus-info: 0000:41:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
________________________________________
From: Fujinaka, Todd <todd.fujinaka@intel.com>
Sent: Thursday, July 30, 2015 11:45 AM
To: Fujinaka, Todd; Hu, Tan Chang; e1000-devel@lists. sourceforge. net \
(e1000-devel@lists.sourceforge.net)
Subject: RE: [E1000-devel] [EXTERNAL] RE: TCP stack is receiving out of order packer \
delivery even when connected back-to-back
I should mention that you should be carefully checking the version numbers. Is your \
firmware up-to-date? Everything I used is externally available: RHEL6.7 & tools on \
downloadcenter.intel.com.
Todd Fujinaka
Software Application Engineer
Networking Division (ND)
Intel Corporation
todd.fujinaka@intel.com
(503) 712-4565
-----Original Message-----
From: Fujinaka, Todd [mailto:todd.fujinaka@intel.com]
Sent: Thursday, July 30, 2015 10:42 AM
To: Hu, Tan Chang; e1000-devel@lists. sourceforge. net \
(e1000-devel@lists.sourceforge.net)
Subject: Re: [E1000-devel] [EXTERNAL] RE: TCP stack is receiving out of order packer \
delivery even when connected back-to-back
[htfujina@htfujina-fc src]$ sudo ethtool -i eth14
driver: i40e
version: 1.2.48
firmware-version: f4.33.31377 a1.2 n4.42 e1932
bus-info: 0000:04:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
[htfujina@htfujina-fc src]$ sudo ethtool -k eth14 Features for eth14:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
ntuple-filters: on
receive-hashing: on
[htfujina@htfujina-fc src]$ sudo ethtool -K eth14 ntuple off [htfujina@htfujina-fc \
src]$ sudo ethtool -k eth14 Features for eth14:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
ntuple-filters: off
receive-hashing: on
[htfujina@htfujina-fc src]$
Todd Fujinaka
Software Application Engineer
Networking Division (ND)
Intel Corporation
todd.fujinaka@intel.com
(503) 712-4565
-----Original Message-----
From: Hu, Tan Chang [mailto:tchu@sandia.gov]
Sent: Wednesday, July 29, 2015 2:37 PM
To: Fujinaka, Todd; e1000-devel@lists. sourceforge. net \
(e1000-devel@lists.sourceforge.net)
Subject: Re: [EXTERNAL] RE: TCP stack is receiving out of order packer delivery even \
when connected back-to-back
Ethtool version 4.0 returned the same message. Your driver version is 1.3.6-k and not \
version 1.2.48 though.
[root@r445402 ethtool-4.0]# ./ethtool --version ethtool version 4.0
[root@r445402 ethtool-4.0]# ./ethtool -K i40e1 ntuple off Could not change any device \
features [root@r445402 ethtool-4.0]# ./ethtool -k i40e1 Features for i40e1:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-unneeded: off
tx-checksum-ip-generic: off
tx-checksum-ipv6: on
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: on [fixed]
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: on
tx-tcp6-segmentation: on
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: on
.
[root@r445402 ethtool-4.0]# ./ethtool -i i40e1
driver: i40e
version: 1.2.48
firmware-version: f4.33.31377 a1.2 n4.42 e1932
expansion-rom-version:
bus-info: 0000:41:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
________________________________________
From: Fujinaka, Todd <todd.fujinaka@intel.com>
Sent: Wednesday, July 29, 2015 2:23 PM
To: Hu, Tan Chang; e1000-devel@lists. sourceforge. net \
(e1000-devel@lists.sourceforge.net)
Subject: RE: [EXTERNAL] RE: TCP stack is receiving out of order packer delivery even \
when connected back-to-back
So, here's what I see and the versions of the tools I'm running. Perhaps your version \
of ethtool is too old?
[htfujina@htfujina-fc ~]$ sudo ethtool -K p128p1 ntuple off [htfujina@htfujina-fc ~]$ \
sudo ethtool -k p128p1 Features for p128p1:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: on
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: on
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: on
tx-tcp6-segmentation: on
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
...
[htfujina@htfujina-fc ~]$ sudo ethtool -K p128p1 ntuple on [htfujina@htfujina-fc ~]$ \
sudo ethtool -k p128p1 Features for p128p1:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: on
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: on
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: on
tx-tcp6-segmentation: on
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: on
...
[htfujina@htfujina-fc ~]$ sudo ethtool -i p128p1
driver: i40e
version: 1.3.6-k
firmware-version: f4.33.31377 a1.2 n4.42 e1932
bus-info: 0000:04:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
[htfujina@htfujina-fc ~]$ ethtool --version ethtool version 3.18 \
[htfujina@htfujina-fc ~]$
Todd Fujinaka
Software Application Engineer
Networking Division (ND)
Intel Corporation
todd.fujinaka@intel.com
(503) 712-4565
-----Original Message-----
From: Hu, Tan Chang [mailto:tchu@sandia.gov]
Sent: Wednesday, July 29, 2015 7:56 AM
To: Fujinaka, Todd; e1000-devel@lists. sourceforge. net \
(e1000-devel@lists.sourceforge.net)
Subject: RE: [EXTERNAL] RE: TCP stack is receiving out of order packer delivery even \
when connected back-to-back
[root@r445403 src]# ethtool -K i40e1 ntuple off Could not change any device features
[root@r445403 src]# ethtool -k i40e1 | grep ntuple
ntuple-filters: on
-----Original Message-----
From: Fujinaka, Todd [mailto:todd.fujinaka@intel.com]
Sent: Wednesday, July 29, 2015 8:53 AM
To: Hu, Tan Chang; e1000-devel@lists. sourceforge. net \
(e1000-devel@lists.sourceforge.net)
Subject: RE: [EXTERNAL] RE: TCP stack is receiving out of order packer delivery even \
when connected back-to-back
What did you try to disable flow director? This may be a bug.
Todd Fujinaka
Software Application Engineer
Networking Division (ND)
Intel Corporation
todd.fujinaka@intel.com
(503) 712-4565
-----Original Message-----
From: Hu, Tan Chang [mailto:tchu@sandia.gov]
Sent: Wednesday, July 29, 2015 6:55 AM
To: Fujinaka, Todd
Subject: RE: [EXTERNAL] RE: TCP stack is receiving out of order packer delivery even \
when connected back-to-back
The new driver version, 1.2.48, and kernel, 2.6.32-504.30.3.el6.x86_64, doesn't \
allow disabling of the flow director. The combination returns " Could not change any \
device features". We will look at XPS.
-----Original Message-----
From: Fujinaka, Todd [mailto:todd.fujinaka@intel.com]
Sent: Thursday, July 23, 2015 2:45 PM
To: Hu, Tan Chang; e1000-devel@lists.sourceforge.net
Cc: Naegle, John H
Subject: RE: [EXTERNAL] RE: TCP stack is receiving out of order packer delivery even \
when connected back-to-back
You only sent me the last email and, as I stated before, I do NOT have access to the \
customer service database. I can only go by what you tell me in this thread.
I was waiting for you to try the three tests I described in this thread and let me \
know the results. We think this is a known issue.
You've told me that one of the tests invalid since rngd is not running. Please try \
the other two.
Todd Fujinaka
Software Application Engineer
Networking Division (ND)
Intel Corporation
todd.fujinaka@intel.com
(503) 712-4565
-----Original Message-----
From: Hu, Tan Chang [mailto:tchu@sandia.gov]
Sent: Thursday, July 23, 2015 1:41 PM
To: Fujinaka, Todd; e1000-devel@lists.sourceforge.net
Cc: Naegle, John H
Subject: Re: [EXTERNAL] RE: TCP stack is receiving out of order packer delivery even \
when connected back-to-back
Todd,
We already stated in the email that rngd was not running. Seond, if you had access to \
the ticket system you would have seen that we stated the same on a follow up response \
and that email attachment wasn't the last email from customer support.
________________________________________
From: Fujinaka, Todd <todd.fujinaka@intel.com>
Sent: Thursday, July 23, 2015 2:36 PM
To: Hu, Tan Chang; e1000-devel@lists.sourceforge.net
Cc: Naegle, John H
Subject: RE: [EXTERNAL] RE: TCP stack is receiving out of order packer delivery even \
when connected back-to-back
I got the attachments directly, but keep in mind that sourceforge strips the \
attachments and no one else has seen them. It appears you have send me their last \
email and a bunch of ethtool output.
As far as I can see, they asked you to kill rngd and you never responded with whether \
that worked or not. So we're back to them asking you to test and us waiting for those \
results? I gave you two more possible tests as well.
Todd Fujinaka
Software Application Engineer
Networking Division (ND)
Intel Corporation
todd.fujinaka@intel.com
(503) 712-4565
-----Original Message-----
From: Hu, Tan Chang [mailto:tchu@sandia.gov]
Sent: Thursday, July 23, 2015 12:21 PM
To: Fujinaka, Todd; e1000-devel@lists.sourceforge.net
Cc: Naegle, John H
Subject: RE: [EXTERNAL] RE: TCP stack is receiving out of order packer delivery even \
when connected back-to-back
Todd,
Attached is the email we received from customer support and the ethtool \
output pre and post run. Disregard the reference to Mellanox cards as they were not \
in the test setup since June 15, 2015. RGND process was not running on either system. \
There were no other tests we were ask to run except for watching dropped packets. We \
can run the suggested fixes if you can provide the instructions to do so.
Third paragraph, last sentence, is a dangling sentence.
As John mentioned in the last email, we are seeing performance problems with real \
applications and not just iperf. How is Intel setting the NIC for best general \
purpose throughput and what are those numbers?
-----Original Message-----
From: Fujinaka, Todd [mailto:todd.fujinaka@intel.com]
Sent: Thursday, July 23, 2015 11:16 AM
To: Naegle, John H; Hu, Tan Chang; e1000-devel@lists.sourceforge.net
Subject: RE: [EXTERNAL] RE: TCP stack is receiving out of order packer delivery even \
when connected back-to-back
OK, so I went directly to the developers and they told me what they told customer \
service. I'm going to assume you got the whole message.
First, what are the results of the tests that you were asked to try?
The problem was clearly explained to me, again, that the CPU scheduler is kicking us \
off the CPU and flow director takes 20 SKBs to catch up. In that time you can get out \
of order packets. There are things you should have been asked to try to fix this. \
Turn off XPS in the kernel. Turn off Flow Director. Kill the rngd process (we found \
that the rngd process is what usually runs to kick us off the CPU). If you don't know \
how to do that, google is your friend. We have ideas of how to mitigate this without \
doing
Why are you seeing problems? We set up the NIC for the best general purpose \
throughput. When people start running tests on corner cases or artificial tests, we \
see degradation. Turn off flow director, and you'll be happier with your iperf \
numbers on both i40e and ixgbe, but I'm not sure if that's going to help for a \
real-world server.
Some of our competitors don't have flow director, so they won't see this problem.
Todd Fujinaka
Software Application Engineer
Networking Division (ND)
Intel Corporation
todd.fujinaka@intel.com
(503) 712-4565
-----Original Message-----
From: Fujinaka, Todd
Sent: Thursday, July 23, 2015 9:08 AM
To: 'Naegle, John H'; Hu, Tan Chang; e1000-devel@lists.sourceforge.net
Subject: RE: [EXTERNAL] RE: TCP stack is receiving out of order packer delivery even \
when connected back-to-back
If you would like a phone conversation please contact your FAE to arrange this. Right \
now we don't see how this is a widespread or critical issue, but the FAE is the one \
who can determine the urgency.
Todd Fujinaka
Software Application Engineer
Networking Division (ND)
Intel Corporation
todd.fujinaka@intel.com
(503) 712-4565
-----Original Message-----
From: Naegle, John H [mailto:jhnaegl@sandia.gov]
Sent: Thursday, July 23, 2015 8:31 AM
To: Fujinaka, Todd; Hu, Tan Chang; e1000-devel@lists.sourceforge.net
Subject: RE: [EXTERNAL] RE: TCP stack is receiving out of order packer delivery even \
when connected back-to-back
While it is true that iperf is an artificial test, it has been very effective at \
demonstrating TCP/IP issues that also impact our data movement application \
performance for us!
For example, when we discovered this same issue on your 10G NICs, we took a guess and \
turned off LRO. This greatly reduced the number of Out-of-Order events (as reported \
by Wireshark), and our user performance jumped by almost 10X! Users had been \
noticing and complaining about the performance specifically to Intel NICs for some \
time. When we swapped to a Chelsio NIC, the application performance went up even \
more. This was in the WAN where latency dominates performance due to the large round \
trip time.
We have been trading emails about these issues now for several weeks. I'd suggest \
that we have a live phone discussion. I find that it is much easier to come to \
common understanding in complex problems with live discussions where questions, \
ideas, and misunderstandings can be quickly clarified. Richard and I are available \
today after 11:00 MT through 3:00pm MT or tomorrow. We can use my telecon bridge \
number or just a direct call if that is easier. Please let us know if you can \
support such a conversation.
Thanks,
John Naegle
-----Original Message-----
From: Fujinaka, Todd [mailto:todd.fujinaka@intel.com]
Sent: Thursday, July 23, 2015 9:11 AM
To: Hu, Tan Chang; e1000-devel@lists.sourceforge.net
Cc: Naegle, John H
Subject: [EXTERNAL] RE: TCP stack is receiving out of order packer delivery even when \
connected back-to-back
Iperf changes queues in the middle of the test.
I remember this issue (not the ticket - I don't have direct access to this) and some \
of the answers you receive late in the thread were from the development team. I don't \
know if it was edited by customer support (as I said, I don't have access to the \
ticket) and my opinion in hallway conversations about this was that artificial tests \
(such as iperf) generate artificial results. We can tune for good iperf performance \
but the tuning is counterproductive for real traffic. I can't imagine that you're \
buying this hardware just to perform iperf tests.
If there is a real issue, please let us know. The "go-to guy" in our division has \
already responded to customer service on this issue.
Todd Fujinaka
Software Application Engineer
Networking Division (ND)
Intel Corporation
todd.fujinaka@intel.com
(503) 712-4565
-----Original Message-----
From: Hu, Tan Chang [mailto:tchu@sandia.gov]
Sent: Thursday, July 23, 2015 7:58 AM
To: Fujinaka, Todd; e1000-devel@lists.sourceforge.net
Cc: Naegle, John H
Subject: TCP stack is receiving out of order packer delivery even when connected \
back-to-back
Todd,
The history of the ticket 8001197134 gave the problem statement. The Linux \
TCP/IP stack is receiving out of order packets delivery from even when the two host \
with Intel XL710 40 GE are connected directly back to back. The out of order packets \
triggers receiver side TCP/IP DUP ACK and cause the sender to go into fast retransmit \
state and kills throughput performance (especially as the end application of the 40GE \
is data transfer over 100Gbps WAN connections). The Dell R620 servers are connected \
via two Intel E40GQSFPSR transceivers with a Tripp Lite N844-10M-12-P40GBase-SR4 MPO \
cable back-to-back.
Intel customer support doesn't think re-transmits dramatically impact throughput, \
alluding to improper socket buffering, and stated that out of order packets are \
counted in different statistics. Intel customer support did not provide any numbers \
as what "dramatically impact" means nor how they are looking for socket buffering or \
out of order statistics. What does Intel expect in terms of XL710 throughput \
performance, how do you check for Intel NIC socket buffering settings, and what out \
of order statistics?
Comparing "ethtool -S i40e1" before and after the iperf3 run does indicate multiple \
tx changes. Iperf3 was configured to run as a single thread, we expected the Intel \
NIC to handle a single flow in a single queue and not spread around over multiple \
queues. Can you explain the design why scheduling would impact a single flow?
We are also seeing similar behavior from Intel 10GE NIC in production across WAN \
links while other vendor's hardware do not exhibit this Out-Of-Order packet deliver.
-----Original Message-----
From: Fujinaka, Todd [mailto:todd.fujinaka@intel.com]
Sent: Wednesday, July 22, 2015 11:25 AM
To: Hu, Tan Chang; e1000-devel@lists.sourceforge.net
Cc: Naegle, John H
Subject: [EXTERNAL] RE: [E1000-devel] Intel Customer Support Service Request \
Management request # 8001197134
Can you restate your problem so we can start over on this issue? You may want to just \
start with a new subject line as well. From what I've heard from customer service, \
there was no issue and the case was closed but I haven't heard all the details at \
this time.
Another way to get help is to escalate this through your factory contact (whoever \
sold you whatever it is you're having problems with) and they can file a ticket in \
IPS.
My suggestion is to start over here with a problem statement and we can ask you for \
more details as we go. I'd also suggest filing a bug on sourceforge if you have \
attachments to send (inline in email is hard to read), but sourceforge is having \
issues at the moment.
Todd Fujinaka
Software Application Engineer
Networking Division (ND)
Intel Corporation
todd.fujinaka@intel.com
(503) 712-4565
-----Original Message-----
From: Hu, Tan Chang [mailto:tchu@sandia.gov]
Sent: Wednesday, July 22, 2015 8:27 AM
To: e1000-devel@lists.sourceforge.net
Cc: Naegle, John H
Subject: [E1000-devel] Intel Customer Support Service Request Management request # \
8001197134
Desr sir,
As per understanding from the last email from Intel support dated July 22, 2015, the \
ticket 8001197234 has been passed on to the Intel driver development team. Please let \
us know if additional iformation, besides the previously submitted, are needed to \
resolve this issue.
Thanks
------------------------------------------------------------------------------
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit \
http://communities.intel.com/community/wired
------------------------------------------------------------------------------
Monitor Your Dynamic Infrastructure at Any Scale With Datadog!
Get real-time metrics from all of your servers, apps and tools
in one place.
SourceForge users - Click here to start your Free Trial of Datadog now!
http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit \
http://communities.intel.com/community/wired
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic