[prev in list] [next in list] [prev in thread] [next in thread] 

List:       postfix-users
Subject:    Re: postfix milter body chunk length
From:       Wietse Venema <wietse () porcupine ! org>
Date:       2019-08-20 11:09:00
Message-ID: 46CSj82RvqzJrNs () spike ! porcupine ! org
[Download RAW message or body]

Matthias Schneider:
> I want to share my research about my milter tcp speed issue:
> 
> 
> I have turned on smtpd -v in master.cf to see the mss in the logfile:
> 
> On my ubuntu 18.04 the loopback device has a mtu of 65536 which results 
> in a "vstream_tweak_tcp: TCP_MAXSEG 21845 "
> This is slow and has the nagle timeout problem.

Why does the kernel return this nonsensical mss value which is
1/3 of the MTU? 40kB of packet the framing overhead for loopback?

> When I set the mtu to 65535 (or 65534) it still gets TCP_MAXSEG 21845 
> (and is still slow).
> 
> 
> Once i do "ip link set dev lo mtu 21845" i will get vstream_tweak_tcp: 
> TCP_MAXSEG 21793 and the performance is great (only 1 second for 100mb 
> body).
> 
> 
> Installing the Milter on another machine (which is available throught 
> the eno1 (1500mtu) interface i will get TCP_MAXSEG 1448 and still bad 
> performance (43secs instead of 1.30m).

Are you on PPPoE or some other tunnel? I would expect mss=1460
for TCP/IP over ethernet.

> Since this is strange i prefer to switch to unixsocket.

Don't bother fighting with a hostile network stack :-)

	Wietse
> 
> I also researched where the delay happened in my milter: its while 
> waiting for the 64k packet 
> https://github.com/mschneider82/milter/blob/b4a6d7c78ac39b8a1d71b53b74bc08d95652d310/session.go#L101
> 
> 
> Best regards
> 
> Matthias Schneider
> 
> 
> 
> Am 19.08.19 um 20:05 schrieb Wietse Venema:
> > Matthias Schneider:
> >> Hi Wietse,
> >>
> >>
> >>> I suspect that the bottleneck is on the receiving side.
> >>>
> >>> - Maybe the loopback stack does not like the 65535 block size. Try
> >>> using an ethernet interface address instead.
> >> I tried switching from interface lo to eno1 - same issue.
> > What's the en01 MTU size? Postfix has code (in vstream_tweak.c
> > which is called by the Milter client and other Postfix code) to
> > automatically increase the VSTREAM buffer from 4096 to the maximal
> > segment size, to avoid Nagle delays, though that would not help if
> > the MTU is > 65535 (the Milter's 'packet' size).
> >
> > I would therefore not expect Nagle delays on en01.
> >
> >> I compared some tcpdumps with postfix as client and my milterclient
> >> (which is fast) as client.
> >>
> >> It seems that we run into nagle timeout (the ACK is always delayed).
> >> Setting TCP_QUICKACK on my milter socket doesnt help, is there a way to
> >> set TCP_NODELAY on the milter connection in postfix?
> > You've got the source.
> >
> > 	Wietse
> 
[prev in list] [next in list] [prev in thread] [next in thread]