[prev in list] [next in list] [prev in thread] [next in thread]
List: freebsd-net
Subject: Re: em0 performance subpar
From: Adam Stylinski <kungfujesus06 () gmail ! com>
Date: 2011-04-29 15:12:59
Message-ID: 20110429151259.GA96064 () ossumpossum ! geop ! uc ! edu
[Download RAW message or body]
On Thu, Apr 28, 2011 at 09:41:01PM -0700, Jack Vogel wrote:
> We rarely test 32bit any more, the only time we would is because of a
> problem,
> so 99% is amd64, so that's not the problem.
>
> Running netperf without any special arguments, using TCP_STREAM and
> TCP_MAERTS tests what numbers are you seeing?
>
> Jack
>
>
> On Thu, Apr 28, 2011 at 5:34 PM, Adam Stylinski <kungfujesus06@gmail.com>wrote:
>
> > On Thu, Apr 28, 2011 at 6:47 PM, Adam Stylinski <kungfujesus06@gmail.com>wrote:
> >
> > > On Thu, Apr 28, 2011 at 02:22:29PM -0700, Jack Vogel wrote:
> > > > My validation engineer set things up on an 8.2 REL system, testing the
> > > > equivalent of
> > > > HEAD, and he reports performance is fine. This is without any tweaks
> > > from
> > > > what's
> > > > checked in.
> > > >
> > > > Increasing the descriptors to 4K is way overkill and might actually
> > > cause
> > > > problems,
> > > > go back to default.
> > > >
> > > > He has a Linux test client, what are you transmitting to?
> > > >
> > > > Jack
> > > >
> > > >
> > > > On Thu, Apr 28, 2011 at 11:00 AM, Adam Stylinski <
> > > kungfujesus06@gmail.com>wrote:
> > > >
> > > > > On Thu, Apr 28, 2011 at 09:52:14AM -0700, Jack Vogel wrote:
> > > > > > Adam,
> > > > > >
> > > > > > The TX ring for the legacy driver is small right now compared to em,
> > > try
> > > > > > this experiment,
> > > > > > edit if_lem.c, search for "lem_txd" and change EM_DEFAULT_TXD to
> > > 1024,
> > > > > see
> > > > > > what
> > > > > > that does, then 2048.
> > > > > >
> > > > > > My real strategy with the legacy code was that it should stable,
> > > meaning
> > > > > not
> > > > > > getting
> > > > > > a lot of changes... that really hasn't worked out over time. I
> > > suppose
> > > > > I'll
> > > > > > have to try and
> > > > > > give it some tweaks and let you try it. The problem with this code
> > > is it
> > > > > > technically supports
> > > > > > a huge range of old stuff we don't test any more, things I do might
> > > cause
> > > > > > other regressions :(
> > > > > >
> > > > > > Oh well, let me know if increasing the TX descriptors helps.
> > > > > >
> > > > > > Jack
> > > > > Jack,
> > > > >
> > > > > Is this the same thing as adjusting these values?:
> > > > >
> > > > > hw.em.rxd=4096
> > > > > hw.em.txd=4096
> > > > >
> > > > > If so I've maxed this out and it's not helping. I'll give it a shot
> > > on my
> > > > > 8-STABLE box as it has a kernel I can play with.
> > > > >
> > > > > Setting the MTU to 1500 gave lower throughput.
> > > > >
> > > > > --
> > > > > Adam Stylinski
> > > > > PGP Key: http://pohl.ececs.uc.edu/~adam/publickey.pub
> > > > > Blog: http://technicallyliving.blogspot.com
> > > > >
> > >
> > > I am transmitting to a linux client (kernel 2.6.38, 9000 byte MTU, PCI-Ex
> > > based card). My sysctl's on the Linux client (apart from the default) look
> > > like so:
> > >
> > > net.ipv4.ip_forward = 0
> > > # Enables source route verification
> > > net.ipv4.conf.default.rp_filter = 1
> > > # Enable reverse path
> > > net.ipv4.conf.all.rp_filter = 1
> > > net.core.rmem_max = 16777216
> > > net.core.wmem_max = 16777216
> > > net.ipv4.tcp_rmem = 4096 87380 16777216
> > > net.ipv4.tcp_wmem = 4096 87380 16777216
> > > net.core.wmem_default = 87380
> > > net.core.rmem_default = 87380
> > > net.ipv4.tcp_mem = 98304 131072 196608
> > > net.ipv4.tcp_no_metrics_save = 1
> > > net.ipv4.tcp_window_scaling = 1
> > > dev.rtc.max-user-freq = 1024
> > >
> > > The exact troublesome device (as reported by pciconf):
> > >
> > > em0@pci0:7:5:0: class=0x020000 card=0x13768086 chip=0x107c8086 rev=0x05
> > > hdr=0x00
> > > vendor = 'Intel Corporation'
> > > device = 'Gigabit Ethernet Controller (Copper) rev 5 (82541PI)'
> > > class = network
> > > subclass = ethernet
> > > Apart from bus saturation (which I don't suspect is the problem) I'm not
> > > sure what the issue could be. What should I try next?
> > >
> > > --
> > > Adam Stylinski
> > > PGP Key: http://pohl.ececs.uc.edu/~adam/publickey.pub
> > > Blog: http://technicallyliving.blogspot.com
> > >
> >
> > One detail that I didn't mention which may or may not matter is that the
> > issues I'm having this with are on amd64 distributions. I have the same
> > card in an x86 system and iperf with nearly default settings managed to do
> > 850ish Mbits. Did your lab tests consist of amd64 machines?
> >
TCP_STREAM:
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.0.121 \
(192.168.0.121) port 0 AF_INET Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
87380 87380 87380 10.00 578.77
TCP_MAERTS:
TCP MAERTS TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.0.121 \
(192.168.0.121) port 0 AF_INET Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
87380 87380 87380 10.00 836.91
interesting, so TCP_STREAM specifically has the issue. I suppose I could read \
documentation on what the tests do but what does this tell?
--
Adam Stylinski
PGP Key: http://pohl.ececs.uc.edu/~adam/publickey.pub
Blog: http://technicallyliving.blogspot.com
[Attachment #3 (application/pgp-signature)]
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic