[prev in list] [next in list] [prev in thread] [next in thread] 

List:       apache-modperl
Subject:    Re: Search Bot
From:       martin langhoff <ml () nzl ! com ! ar>
Date:       2004-01-29 4:49:25
Message-ID: 401890D5.8030004 () nzl ! com ! ar
[Download RAW message or body]

> 
> Shawn wrote:
> 
>>Hi, I have been trying to figure out a way to limit the massive amount
>>of bandwidth that search bots (Googlebot/2.1) consume daily from my 
>>website. My problem is that I am running Apache::ASP and about 90% of 

Doesn't Apache::Throttle, configured in apache to kick as a fixuphandler 
only for Googlebot, address the problem?

I seem to recall a thread where people were using it to throttle clients 
based on a "bad client" IP list that was updated on a cron script.

cheers,





martin

-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html

[prev in list] [next in list] [prev in thread] [next in thread]