[prev in list] [next in list] [prev in thread] [next in thread] 

List:       freebsd-net
Subject:    Re: Intel 82580 lagg(4) problem
From:       Adrian Chadd <adrian () freebsd ! org>
Date:       2013-09-30 9:51:34
Message-ID: CAJ-Vmo=00SzCNtcKAxnQpWXdDi4CGo5=_u_Wpejt8SA0Dh=M+w () mail ! gmail ! com
[Download RAW message or body]

Hi,

There's a lagg/lacp option to not hash based on the incoming assigned
flowid on the incoming mbuf when transmitting but instead calculate its own
hash for outbound transmission. It's net.link.lagg.0.use_flowid . Set it to
0 and retry.

Thanks,


-adrian



On 30 September 2013 02:10, Andriy Kopystyansky <anri@polynet.lviv.ua>wrote:

> Greetings!
> We've got a strange problem with lagg(4) interfaces built on Intel's 82580
> chipset igb (dual port 1Gb card).
> We cannot send over lagg interface more than ~1Gb/s, yet receiving ~2GB
> over it.
> Looks like something indicating the problem can be seen here:
> 
> anri@host:[8:13]~#ifstat -i lagg0 -i igb1 -i igb3 1
> lagg0                igb1                igb3
> KB/s in  KB/s out   KB/s in  KB/s out   KB/s in  KB/s out
> 9116.50  26515.16   4147.70  28871.43   5004.86  23683.31
> 8423.08  26544.62   3853.22  28980.60   4594.21  23520.97
> 8796.48  26395.28   4248.46  28344.00   4567.42  23978.17
> 
> Note (IN) traffic - everything's ok, about 50% on each igb interfaces and
> total summary on lagg0.
> But (OUT) traffic looks weird - there is more traffic on the single igb1
> than on lagg0!
> Tried with default driver came with the system, also the new one
> igb-2.3.10 from Intel site - no luck.
> 
> Testing lagg interface built on other cards (em, bce) on the same machine
> shows expected normal behavior for both directions.
> ------------------------------**---------
> anri@host:[9:02]~#uname -a
> FreeBSD host 9.1-RELEASE-p4 FreeBSD 9.1-RELEASE-p4 #5: Fri Sep 20 14:52:18
> EEST 2013     anri@host:/usr/obj/usr/src/**sys/NODEONE  amd64
> ------------------------------**--------
> Interfaces:
> lagg0: flags=8843<UP,BROADCAST,**RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
> 1500
> options=401bb<RXCSUM,TXCSUM,**VLAN_MTU,VLAN_HWTAGGING,JUMBO_**
> MTU,VLAN_HWCSUM,TSO4,VLAN_**HWTSO>
> ether 90:e2:ba:36:95:1d
> inet 172.16.0.1 netmask 0xffffffc0 broadcast 172.16.0.63
> media: Ethernet autoselect
> status: active
> laggproto lacp lagghash l2,l3,l4
> laggport: igb3 flags=1c<ACTIVE,COLLECTING,**DISTRIBUTING>
> laggport: igb1 flags=1c<ACTIVE,COLLECTING,**DISTRIBUTING>
> 
> igb1: flags=8843<UP,BROADCAST,**RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
> 1500
> options=401bb<RXCSUM,TXCSUM,**VLAN_MTU,VLAN_HWTAGGING,JUMBO_**
> MTU,VLAN_HWCSUM,TSO4,VLAN_**HWTSO>
> ether 90:e2:ba:36:95:1d
> media: Ethernet autoselect (1000baseT <full-duplex>)
> status: active
> 
> igb3: flags=8843<UP,BROADCAST,**RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
> 1500
> options=401bb<RXCSUM,TXCSUM,**VLAN_MTU,VLAN_HWTAGGING,JUMBO_**
> MTU,VLAN_HWCSUM,TSO4,VLAN_**HWTSO>
> ether 90:e2:ba:36:95:1d
> media: Ethernet autoselect (1000baseT <full-duplex>)
> status: active
> 
> ------------------------------**---------
> Hardware: IBM 3550 with 2x Intel(R) Xeon(R) CPU  E5430  @ 2.66GHz (amd64)
> pciconf -lv:
> igb1@pci0:19:0:1:       class=0x020000 card=0x12b28086 chip=0x15168086
> rev=0x01 hdr=0x00
> vendor     = 'Intel Corporation'
> device     = '82580 Gigabit Network Connection'
> class      = network
> subclass   = ethernet
> 
> ------------------------------**---------
> /boot/loader.conf:
> hw.igb.rxd=4096
> hw.igb.txd=4096
> hw.igb.max_interrupt_rate=**32000
> hw.igb.num_queues=2
> net.isr.maxthreads=8
> net.isr.dispatch=direct
> net.isr.bindthreads=1
> net.link.ifqmaxlen=10240
> 
> /etc/sysctl.conf: (relevant part)
> 
> dev.igb.1.rx_processing_limit=**-1
> dev.igb.3.rx_processing_limit=**-1
> net.inet.ip.intr_queue_maxlen=**512
> net.route.netisr_maxqlen=4096
> net.inet.ip.redirect=0
> net.inet.ip.fastforwarding=1
> kern.maxfiles=204800
> kern.maxfilesperproc=200000
> kern.ipc.nmbclusters=204800
> kern.ipc.maxsockets=204800
> 
> Any tips, advices?
> 
> thanks in advance, with best regards,
> Andriy Kopystyansky
> 
> 
> ______________________________**_________________
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/**mailman/listinfo/freebsd-net<http://lists.freebsd.org/mailman/listinfo/freebsd-net>
>  To unsubscribe, send any mail to \
> "freebsd-net-unsubscribe@**freebsd.org<freebsd-net-unsubscribe@freebsd.org> "
> 
_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"


[prev in list] [next in list] [prev in thread] [next in thread]