[prev in list] [next in list] [prev in thread] [next in thread]
List: freebsd-net
Subject: Re: Netmap - Vale switch - tcp problem
From: Vincenzo Maffione <vmaffione () freebsd ! org>
Date: 2020-05-30 12:29:45
Message-ID: CA+_eA9j9zTM2JwCeMy6FmJQJmorVzXns2ijFgjuXa+rRQH400A () mail ! gmail ! com
[Download RAW message or body]
Il giorno ven 29 mag 2020 alle ore 19:01 Anthony Arnaud <
antho.arnaudisce@gmail.com> ha scritto:
> Hi Vincenzo,
>
> thanks for your hints!
> I rebooted my guest FBSD 12.1 machine, and I have perfomed your steps
>
> #ifconfig vtnet1 -txcsum -rxcsum -tso4 -tso6 up promisc
>
> vtnet1:
> flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC> metric
> 0 mtu 1500
>
> options=6c04b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
> ether 0e:bd:ec:7a:08:06
> media: Ethernet 10Gbase-T <full-duplex>
> status: active
> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
>
> tcpdump is ok.
>
> Does it mean you see both ICMP, UDP and TCP traffic?
> But after:
>
> #vale-ctl
>
> 446.196827 bdg_ctl [149] bridge:0 port:0 vale0:vtnet1
> 446.196855 bdg_ctl [149] bridge:0 port:1 vale0:vtnet1^
>
> tcpdump not work anymore.
>
Do you see ICMP/UDP only and not TCP?
> I don't understand why the tcp traffic disappears.
> In my configuration vtnet1 is a mirror port created by Open vSwitch, but I
> don't think that's the reason.
>
> No, I don't think that's relevant.
In my setup, vtnet0 is a guest interface backed by a host tap device
(attached to a linux bridge), and hypervisor is QEMU/KVM.
Here are the steps I follow in the VM (in this order):
# ifconfig vtnet0 -txcsum -rxcsum -tso4 -tso6 up 192.168.100.2/24
# vale-ctl -h vale0:vtnet0
# nc 192.168.100.1 5555 # connect to listening netcat in the host.
hello
abc
[...]
# tcpdump -ni vtnet0 # This shows the TCP traffic.
I start to see problems when I change the offloads:
# ifconfig vtnet0 -lro
Cheers,
Vincenzo
> Below some info about my configuration:
>
> dev.netmap.iflib_rx_miss_bufs: 0
> dev.netmap.iflib_rx_miss: 0
> dev.netmap.iflib_crcstrip: 1
> dev.netmap.bridge_batch: 1024
> dev.netmap.default_pipes: 0
> dev.netmap.priv_buf_num: 4098
> dev.netmap.priv_buf_size: 2048
> dev.netmap.buf_curr_num: 163840
> dev.netmap.buf_num: 163840
> dev.netmap.buf_curr_size: 2048
> dev.netmap.buf_size: 2048
> dev.netmap.priv_ring_num: 4
> dev.netmap.priv_ring_size: 20480
> dev.netmap.ring_curr_num: 200
> dev.netmap.ring_num: 200
> dev.netmap.ring_curr_size: 36864
> dev.netmap.ring_size: 36864
> dev.netmap.priv_if_num: 2
> dev.netmap.priv_if_size: 1024
> dev.netmap.if_curr_num: 100
> dev.netmap.if_num: 100
> dev.netmap.if_curr_size: 1024
> dev.netmap.if_size: 1024
> dev.netmap.ptnet_vnet_hdr: 1
> dev.netmap.generic_rings: 1
> dev.netmap.generic_ringsize: 1024
> dev.netmap.generic_mit: 100000
> dev.netmap.generic_hwcsum: 0
> dev.netmap.admode: 0
> dev.netmap.fwd: 0
> dev.netmap.txsync_retry: 2
> dev.netmap.no_pendintr: 1
> dev.netmap.no_timestamp: 0
> dev.netmap.verbose: 0
>
>
> dev.vtnet.1.txq0.rescheduled: 0
> dev.vtnet.1.txq0.tso: 0
> dev.vtnet.1.txq0.csum: 0
> dev.vtnet.1.txq0.omcasts: 0
> dev.vtnet.1.txq0.obytes: 0
> dev.vtnet.1.txq0.opackets: 0
> dev.vtnet.1.rxq0.rescheduled: 0
> dev.vtnet.1.rxq0.csum_failed: 0
> dev.vtnet.1.rxq0.csum: 66
> dev.vtnet.1.rxq0.ierrors: 0
> dev.vtnet.1.rxq0.iqdrops: 0
> dev.vtnet.1.rxq0.ibytes: 11904780
> dev.vtnet.1.rxq0.ipackets: 40984
> dev.vtnet.1.tx_task_rescheduled: 0
> dev.vtnet.1.tx_tso_offloaded: 0
> dev.vtnet.1.tx_csum_offloaded: 0
> dev.vtnet.1.tx_defrag_failed: 0
> dev.vtnet.1.tx_defragged: 0
> dev.vtnet.1.tx_tso_not_tcp: 0
> dev.vtnet.1.tx_tso_bad_ethtype: 0
> dev.vtnet.1.tx_csum_bad_ethtype: 0
> dev.vtnet.1.rx_task_rescheduled: 0
> dev.vtnet.1.rx_csum_offloaded: 0
> dev.vtnet.1.rx_csum_failed: 0
> dev.vtnet.1.rx_csum_bad_proto: 0
> dev.vtnet.1.rx_csum_bad_offset: 0
> dev.vtnet.1.rx_csum_bad_ipproto: 0
> dev.vtnet.1.rx_csum_bad_ethtype: 0
> dev.vtnet.1.rx_mergeable_failed: 0
> dev.vtnet.1.rx_enq_replacement_failed: 0
> dev.vtnet.1.rx_frame_too_large: 0
> dev.vtnet.1.mbuf_alloc_failed: 0
> dev.vtnet.1.act_vq_pairs: 1
> dev.vtnet.1.requested_vq_pairs: 0
> dev.vtnet.1.max_vq_pairs: 1
> dev.vtnet.1.%parent: virtio_pci4
> dev.vtnet.1.%pnpinfo:
> dev.vtnet.1.%location:
> dev.vtnet.1.%driver: vtnet
> dev.vtnet.1.%desc: VirtIO Networking Adapter
> dev.vtnet.0.txq0.rescheduled: 0
>
> Cheers,
> Anthony
>
> Il giorno gio 28 mag 2020 alle ore 21:38 Vincenzo Maffione <
> vmaffione@freebsd.org> ha scritto:
>
> > Hi,
> > I was trying to reproduce your problem (same FreeBSD release as yours).
> > It looks like there is some sort of bad interaction with LRO.
> >
> > Starting from a fresh boot, if you keep lro enabled, e.g.
> > # ifconfig vtnet0 -txcsum -rxcsum -tso4 -tso6
> > # vale-ctl ....
> > then I experience no problem (TCP works between vtnet0 and the host,
> > tcpdump on vtnet0 works as expected).
> >
> > As soon as you disable LRO:
> > # ifconfig vtnet0 -lro
> > both TCP and tcpdump stop working.
> > If I enable LRO again, TCP restarts working, but tcpdump doesn't. I need
> > to reboot the machine to fix it.
> >
> > Btw, creating vi0 (persistent VALE port) is not relevant for this test.
> > You may as well use ephemeral VALE ports (e.g. run pkt-gen -i vale0:1 -f
> > rx).
> >
> > I will have a look at the LRO issue asap. In the meantime you could avoid
> > disabling LRO and see if that works for you.
> >
> > Cheers,
> > Vincenzo
> >
> > Il giorno gio 28 mag 2020 alle ore 17:16 Anthony Arnaud <
> > antho.arnaudisce@gmail.com> ha scritto:
> >
> > > I already disabled the checksum, the vtnet config is:
> > >
> > > ifconfig vtnet1 -txcsum -rxcsum -tso4 -tso6 -lro -txcsum6 -rxcsum6
> > > -vlanmtu
> > > -vlanhwtag -vlanhwfilter -vlanhwtso -vlanhwcsum up promisc
> > >
> > > vtnet1:
> > > flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC>
> > > metric
> > > 0 mtu 1500
> > > options=1800a8<VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE,NETMAP>
> > > ether 0e:bd:ec:7a:08:06
> > > media: Ethernet 10Gbase-T <full-duplex>
> > > status: active
> > > nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> > >
> > > Sorry for not having posted vtnet config before.
> > > PS: VLAN_HWCSUM is not switchable off for some reason!
> > > the problem is not that.
> > >
> > > Cheers
> > > Anthony
> > >
> > >
> > >
> > >
> > > Il giorno gio 28 mag 2020 alle ore 16:05 Luigi Rizzo <rizzo@iet.unipi.it
> > > >
> > > ha scritto:
> > >
> > > >
> > > >
> > > > On Thursday, May 28, 2020, Anthony Arnaud <antho.arnaudisce@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi everyone!
> > > > > I would like to create a vale switch with an interface attached with
> > > the
> > > > > host stack and some virtual.
> > > > > My env is a VM with FBSD-12.1 12.1-RELEASE FreeBSD 12.1-RELEASE
> > > r354233
> > > > > GENERIC amd64
> > > > > and VirtIO support.
> > > > >
> > > > > I performed:
> > > > >
> > > > > vale-ctl -h vale0:vtnet1
> > > > > vale-ctl -n vi0
> > > > > vale-ctl -a vale0:vi0
> > > > >
> > > > > 615.925514 bdg_ctl [149] bridge:0 port:0 vale0:vtnet1
> > > > > 615.925559 bdg_ctl [149] bridge:0 port:1 vale0:vtnet1^
> > > > > 615.925572 bdg_ctl [149] bridge:0 port:2 vale0:vi0
> > > > >
> > > > > vtnet1 is configured as mirror port.
> > > > > But if:
> > > > >
> > > > > tcpdump -i vtnet1
> > > > > or
> > > > > tcpdump -i vale0:vi0
> > > > >
> > > > > why can't I see any TCP packets?
> > > > > UDP and ICMP packet are ok.
> > > > >
> > > > > Without vale switch tcpdump show all TCP packets correctly.
> > > >
> > > >
> > > > You have to disable checksum offloading on vtnet1.
> > > >
> > > > Cheers
> > > > Luigi
> > > >
> > > > It is a bug?
> > > > > Thanks to all!
> > > > > <freebsd-net@freebsd.org>
> > > > > _______________________________________________
> > > > > freebsd-net@freebsd.org mailing list
> > > > > https://lists.freebsd.org/mailman/listinfo/freebsd-net
> > > > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org
> > > "
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > -----------------------------------------+-------------------------------
> > > > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing.
> > > dell'Informazione
> > > > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa
> > > > TEL +39-050-2217533 . via Diotisalvi 2
> > > > Mobile +39-338-6809875 . 56122 PISA (Italy)
> > > >
> > > -----------------------------------------+-------------------------------
> > > >
> > > >
> > > _______________________________________________
> > > freebsd-net@freebsd.org mailing list
> > > https://lists.freebsd.org/mailman/listinfo/freebsd-net
> > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
> > >
> >
_______________________________________________
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic