From postfix-users Tue Aug 20 07:44:22 2019 From: Matthias Schneider Date: Tue, 20 Aug 2019 07:44:22 +0000 To: postfix-users Subject: Re: postfix milter body chunk length Message-Id: <1b4fbd7d-181b-2e6b-23d2-3bfed7736b88 () rmail ! de> X-MARC-Message: https://marc.info/?l=postfix-users&m=156628712632018 I want to share my research about my milter tcp speed issue: I have turned on smtpd -v in master.cf to see the mss in the logfile: On my ubuntu 18.04 the loopback device has a mtu of 65536 which results in a "vstream_tweak_tcp: TCP_MAXSEG 21845 " This is slow and has the nagle timeout problem. When I set the mtu to 65535 (or 65534) it still gets TCP_MAXSEG 21845 (and is still slow). Once i do "ip link set dev lo mtu 21845" i will get vstream_tweak_tcp: TCP_MAXSEG 21793 and the performance is great (only 1 second for 100mb body). Installing the Milter on another machine (which is available throught the eno1 (1500mtu) interface i will get TCP_MAXSEG 1448 and still bad performance (43secs instead of 1.30m). Since this is strange i prefer to switch to unixsocket. I also researched where the delay happened in my milter: its while waiting for the 64k packet https://github.com/mschneider82/milter/blob/b4a6d7c78ac39b8a1d71b53b74bc08d95652d310/session.go#L101 Best regards Matthias Schneider Am 19.08.19 um 20:05 schrieb Wietse Venema: > Matthias Schneider: >> Hi Wietse, >> >> >>> I suspect that the bottleneck is on the receiving side. >>> >>> - Maybe the loopback stack does not like the 65535 block size. Try >>> using an ethernet interface address instead. >> I tried switching from interface lo to eno1 - same issue. > What's the en01 MTU size? Postfix has code (in vstream_tweak.c > which is called by the Milter client and other Postfix code) to > automatically increase the VSTREAM buffer from 4096 to the maximal > segment size, to avoid Nagle delays, though that would not help if > the MTU is > 65535 (the Milter's 'packet' size). > > I would therefore not expect Nagle delays on en01. > >> I compared some tcpdumps with postfix as client and my milterclient >> (which is fast) as client. >> >> It seems that we run into nagle timeout (the ACK is always delayed). >> Setting TCP_QUICKACK on my milter socket doesnt help, is there a way to >> set TCP_NODELAY on the milter connection in postfix? > You've got the source. > > Wietse