[prev in list] [next in list] [prev in thread] [next in thread] 

List:       netfilter
Subject:    Re: Figuring out odd web traffic from clients
From:       Miles Stevenson <miles () mstevenson ! org>
Date:       2012-03-17 20:28:31
Message-ID: 04AA66A9B9884FF69D3526953026D58E () gmail ! com
[Download RAW message or body]

Jozsef,
> The packet was not dropped but ignored by conntrack.

Thanks for the reply. That explains a lot of my confusion then. It also explains why \
I don't see an accompanying log showing each of those packets were rejected, because \
they probably weren't. They were probably added to the state table and passed through \
successfully.

> It can happen easily: packets seen by the firewall can get lost in either 
> direction or client-server can be out of sync. In your case it's a 
> webserver: probably the client might just try to re-establish a timed out 
> connection attempt: it sent a SYN which was not answered by the server 
> (overloaded, slow to answer, etc) and sent a SYN with new parameters, and 
> both attempts were seen by conntrack. However the webserver was slow to 
> process the first connection attemtp and just sends the SYN/ACK answer to 
> the first SYN attempt, which is detected by conntrack - and the 
> packet ignored.
> 
That sounds plausible and would be consistent with what I'm seeing actually get \
rejected by the filter rules. That is, the errant FIN and RST packets from both \
directions getting rejected, because those sessions are out of sync with the state \
table completely and don't even match a known "tuple" by the state table. I guess \
more tcpdump/wireshark analysis would help to confirm this.
> 
> Maybe the webserver cannot cope with the connection load. Or it's a client 
> side issue. A full tcpdump can help to tell what happens exactly.


This was my first thought. The web server definitely has a hard time during peak \
loads. But this also happens during light loads. I'm still seeing this behavior right \
now, during fairly light traffic. The web server load averages are all low, and \
Apache is seeing an average of about 20 requests/sec, for a total of 0.5MB/sec \
inbound + outbound web traffic total. It's a Xeon multi-CPU / multi-core server, so \
it should definitely be able to cope with this. So I might have a configuration \
problem somewhere or might be experiencing a bug of some sort with Apache or the \
CentOS kernel, not sure. 

Thanks for all the input, I really appreciate it. My troubleshooting will continue, \
but if anyone wants to suggest anything else that I should try or investigate, I'm \
all ears. Thanks everyone. This is a quality mailing list.

-Miles


--
To unsubscribe from this list: send the line "unsubscribe netfilter" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[prev in list] [next in list] [prev in thread] [next in thread]