[prev in list] [next in list] [prev in thread] [next in thread] 

List:       freebsd-hackers
Subject:    Re: FreeBSD performing worse than Linux?
From:       Terry Lambert <tlambert2 () mindspring ! com>
Date:       2001-12-31 23:53:18
[Download RAW message or body]

"Louis A. Mamakos" wrote:
> 
> An underlying issue here is why applications decide to set TCP_NODELAY
> options on sockets, rather than just letting Nagle's algorithm do
> the right thing.  I recall some handwaving about this in the X server
> some years ago to make mouse movements "smoother".

OK, I've been making an assumption here that is perhaps incorrect,
but is consistant with the traces posted so far: I've assumed that
the data is lost between the ethernet side of the dongle and the
USB side of the dongle.

If the FreeBSD USB driver is really a dog (I have a hard time
buying that) or the dongle USB hardware is superior to that in
the PC being used (I have a hard time buying that one, either),
then it's just possible that the data losses are FreeBSD's, and
not a result of dongle buffer overrun.  You'd need a breakout
box for USB, and a USB system without the problem (Linux?) to
prove it via dumps, though.

That said:

There is no problem with turning on TCP_NODELAY.  If anything,
the resulting data sent through the dongle would be *less*
bursty (being application paced), and tend to be *less* likely
to fill up any pool in the middle to the point of overflow.

This really has all the earmarks of a pool congestion (not a
link congestion) problem in the dongle, to my mind.


> For the problem at hand, if both the client and server machines didn't
> do TCP_NODLEAY, then there'd only be one packet smaller than the
> TCP MSS in flight between the transmitter and receiver at any one
> time.  I think that poking OpenSSH to not set the TCP_NODELAY option
> "fixed" this problem.

I think it masked it.  If you had a less CPU-intensive test
application, I think you would see that you can still overrun
the buffer.

The masking is a natural effect of decreasing the overall
payload encapsulation overhead.  But increase the amount of
payload, and you should still see the problem rear its ugly
head, since the link rate differential is so large on either
side of the dongle.


A fugly way of handling the problem would be to initiate a
flow control mechanism between the dongle and the FreeBSD
ethernet driver.

This is tantamount to the hardware flow control required for
the MNP modem knock-offs by Microcomm competitors who used
flow control as a means of keeping the pool size manageable,
so that they could really cheap out on putting RAM in the
modem, and undercut Microcomm's prices.

For this to work, you'd have to have cooperative firmware in
the dongles; I think that taking that approach is a lost
cause, unless we are downloading our own firmware.

I think we will have to tackle it another way.  Teaching the
driver about TCP windows, and delaying transmissions until
50% of the receive window is satisfied would probably work,
but is a very unsatisfactory approach (even though we know
that transmit over the link is not going to have the same
problem as receive, due to the speed differential across the
dongle).

-- Terry

To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
[prev in list] [next in list] [prev in thread] [next in thread]