[prev in list] [next in list] [prev in thread] [next in thread] 

List:       exim-users
Subject:    Re: [exim] spammers IP ban
From:       "Peter Bowyer" <peeebeee () gmail ! com>
Date:       2006-02-28 11:00:52
Message-ID: 56152ae90602280300t4bf77827v () mail ! gmail ! com
[Download RAW message or body]

On 28/02/06, David Saez Padros <david@ols.es> wrote:
> Hi !!
>
> >>is faster to directly write to mysql instead of parsing the log,
> >>specially if the log file is very big,
> >
> > Exim writing to the log has no incremental overhead at all - unless
> > you've turned off logging for some reason. However fast a MySQL INSERT
> > is, it's slower than doing nothing extra at all.
>
> but then you have to parse the log, which takes some time (our log
> file is really very very big) and then you have to do something with
> the extracted data, in the case Mathieu comments he just ... well
> he just insert that data on a mysql database ... so he adds extra
> lines to the log, then it has to parse the log, extract the data
> and insert in into a mysql database. The only optimization he could
> do is make only one insert with all data but i don't think this will
> be really more faster than just doing delayed inserts from within exim

OK, this horse probably doesn't need any more flogging but....

There's a distinction between elapsed time and potential delays in the
SMTP transaction. I find it prudent to avoid delays in-line with the
SMTP transaction, and also to defend against potential failures there.
Hence no MySQL calls which could block/delay.

The log scraping is in real time - as I said earlier, new log lines
are spotted within a second of being written by Exim, and the database
updated immediately. So, there's no significant elapsed time either.
The only time a large log needs grepping in its entirety is if the log
scrape process should die and be restarted.

(See http://search.cpan.org/dist/File-Tail/)

> > Exim doesn't parse any of these for me - the MySQL database is used to
> > generate a config file for rbldnsd; Exim queries the private rbldnsd
> > zone which is really quick, and you can control Exim's behaviour
> > should the rbldnsd daemon be unavailable for some reason.
>
> if you want speed a local cdb file is faster than drbldns

It is, but in my case the edge MX servers which need to use the
information are distributed geographically with no private network
connectivity, and a DNS-based solution meets my needs well. If
everything were local, I'd agree that cdb might be better.

Peter


--
Peter Bowyer
Email: peter@bowyer.org
Tel: +44 1296 768003
VoIP: sip:peter@bowyer.org
VoIP: *5048707000@sipbroker.com
FWD: **275*5048707000
VoipTalk: **473*5048707000

-- 
## List details at http://www.exim.org/mailman/listinfo/exim-users 
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/

[prev in list] [next in list] [prev in thread] [next in thread]