[prev in list] [next in list] [prev in thread] [next in thread]
List: mimedefang
Subject: Re: [Mimedefang] Sender validation
From: Jonas Eckerman <jonas_lists () frukt ! org>
Date: 2004-06-27 21:47:49
Message-ID: 2004627234749.988246 () ubbe
[Download RAW message or body]
On Thu, 24 Jun 2004 15:39:04 +0200, Jonas Eckerman wrote:
> If this stuff keeps working without problems, I'll
> post again if I actually start rejecting based on it.
Ok. Now the test has been running for a week, and it has hit exactly *no* legit mail. \
I just upgraded it from data collection to actually rejecting based on senders. I'll \
keep montitoring the stuff to see if I should make more excemptions and some more \
result checking. And of course I'll keep an eye on the stats to see if it rejects \
enough to be worth it. I'm also keeping the debug log stuff on for a while so I can \
see that it works as intended.
As Les Mikesell suggested, I'm now using a database (my greylist databse actually, \
but with a different key prefix). Also as he suggested, rejected addresses have a \
shorter cache time initially, but it grows for each reject.
My complete filter is available at:
http://whatever.frukt.org/mimedefang-filter.shtml
Below are some snippets from it. Disclaimer: This stuff is new. It may well have \
bugs. It seems to work, but...
Some settings. I haven't really spent any time finding the best values for the cache \
times below.
--8<--
$sendercheck = 1;
$sc_cache_valid = 7*24*60*60;
$sc_cache_invalid = 60*60;
$sc_cache_unknown = 7*24*60*60;
$sc_cache_invalid_add = 60*60;
$sc_cache_invalid_max = 24*60*60;
--8<--
Checking a mail address. Note that only 550, 551, 553 and 554 are taken as rejections \
(meaning that, for example, the fact that a mailbox is full is not seen as a \
rejection here). Also, I check for some stuff that can indicate that the other \
servers rejects because it doesn't like the sender <>.
--8<--
# Check a mail address against a mail server
sub check_mail_address($$) {
my ($a,$s) = @_;
my ($ok,$msg,$code,$dsn) = \
md_check_against_smtp_server('<>',$a,$MyFilterHostName,$s); my $txt = "$code $dsn \
$msg"; $txt =~ s/\s\s+/ /g;
$txt =~ s/^\s+//;
$txt =~ s/\s+$//;
# Disregard some REJECTs because they aren't really rejecting the recipient address.
return (3,$txt) if ($ok eq 'REJECT' && ($code !~ /^55[0134]$/ || $msg =~ \
/(sender|mail from|return|<>)/i)); return (1,'') if ($ok eq 'CONTINUE');
return (0,$txt) if ($ok eq 'REJECT');
return (2,'');
}
--8<--
Find MX servera for an address and check it against them using the above sub. This \
tests MX servers until it gets a clear reject or ok, wich means that I'll get a clear \
ok from backup servers if the primary server don't give a clear reject even for \
invalid addresses.
--8<--
# Check a mail address against it's MX server(s)
sub check_mail_address_mx_i($) {
my ($a) = @_;
return (4,'') if ($a =~ /^<?>?$/);
my $d = $a;
$d =~ s/^.*@([^@>]*)>?$/$1/;
return (5,'') if (!$d);
my $dns = Net::DNS::Resolver->new;
$dns->defnames(0); # do not search default domain
$dns->persistent_tcp(0);
$dns->tcp_timeout(15);
#$dns->udp_timeout(15);
my $mx = $dns->query($d, 'MX');
return (6,'') if (!$mx);
my %mx;
foreach my $r ($mx->answer) {
$mx{$r->preference} = $r->exchange if ($r->type eq 'MX');
}
return (7,'') if (!%mx);
my @rinfs = ();
foreach my $mp (sort keys %mx) {
my ($ok,$rinf) = check_mail_address($a,$mx{$mp});
return (0,$rinf) if (!$ok);
return (1,$rinf) if ($ok == 1);
push @rinfs, $rinf if ($rinf);
}
return (8,join('; ',@rinfs));
}
--8<--
This stuff uses the above function for checking an address with it's MX servers, \
cacheing the results in the greylist database according to the cache time settings. \
The first three fields in the records are the same as in my greylist implementation \
so that the database cleaner doesn't have to understand different \
records.
--8<--
# Caches the result in the greylist database.
sub check_mail_address_mx($) {
my ($a) = @_;
my $key = "S:".address_strip($a);
my $rc = 1;
my $now = time();
my $created = $now;
my $modified = $now;
my $count = 0;
my ($res,$txt);
my $ores = -1;
if (tie(%GDB,'DB_File','/var/spool/MIMEDefang/.greylistdb',O_RDWR|O_CREAT|O_EXLOCK,0600)) \
{ if (defined($GDB{$key})) {
my $data = $GDB{$key};
($created,$modified,$count,$res,$txt) = split(/;/,$data,5);
$ores = $res;
my $ct;
if (!$res) {
$ct = $sc_cache_invalid + ($sc_cache_invalid_add * ($count - 1));
$ct = $sc_cache_invalid_max if ($ct > $sc_cache_invalid_max);
} elsif ($res == 1) {
$ct = $sc_cache_valid;
} else {
$ct = $sc_cache_unknown;
}
$rc = ($now - $modified > $ct);
debug_log(0,"check_mail_address_mx: found, key=\"$key\" data=\"$data\"");
if (!$rc) {
$count ++;
$data = join(';',$created,$modified,$count,$res,$txt);
debug_log(0,"check_mail_address_mx: count, key=\"$key\" data=\"$data\"");
$GDB{$key} = $data;
} else {
my $a = $now-$modified;
debug_log(0,"check_mail_address_mx: renew, key=\"$key\" cache=$ct age=$a\n");
}
}
untie %GDB;
}
if ($rc) {
$modified = $now;
($res,$txt) = check_mail_address_mx_i($a);
$count = 0 if ($ores != $res);
$count ++;
my $data = join(';',$created,$modified,$count,$res,$txt);
if (tie(%GDB,'DB_File','/var/spool/MIMEDefang/.greylistdb',O_RDWR|O_CREAT|O_EXLOCK,0600)) \
{ $GDB{$key} = $data;
untie %GDB;
debug_log(0,"check_mail_address_mx: saved, key=\"$key\" data=\"$data\"");
}
}
return ($res,$txt);
}
--8<--
And this is from filter_sender. As you can see there's a bunch of checks for \
addresses that I don't validate for different reasons.
--8<--
# Check if sender address is valid. Exempt a bunch of addresses from the check
# in order to be less abusive with regards to big list servers and that sort
# of stuff.
# If from some of our local domains, check locally.
if ($sendercheck && $sender !~ /^<?>?$/
&& $sender !~ /^<?(postmaster|abuse)@/i && $sender !~ \
/^<?(|.*[-_+=])(daemon|gateway)(|[-_+=].*)@/i && $sender !~ \
/@(|[^@]+\.)(bounces?|returns?|lists?|newsletters?)\.[^@\.]+\.[^@\.]+[^@]*$/i && \
($sender !~ /^<?(|.*[-_+=])(anonymous|undisclosed|unspecified|lists?|returns?|users|bounces?|\d+)(|[-_+=].*)@/i \
|| $sender !~ /^<?(|.*[-_+=])$OurDomains(|[-_+=].*)@/i)) {
if ($sender =~ /^.+\@$OurDomains>?$/i && $sender !~ /^.*@(|[^@]\.)frukt.org>?$/i) {
my $hst = '127.0.0.1';
my $dom = address_strip($sender);
$dom =~ s/.*@([^@]+)$/$1/;
$hst = $storingservers{$dom} if ($dom && defined($storingservers{$dom}));
my ($ok,$rinf) = check_mail_address($sender,$hst);
debug_log(0,"filter_sender: $sender = $ok ($hst) [$rinf] {$dom}");
if (!$ok) {
md_syslog('info',"MDLOG,$MsgID,bad_sender,local,$ip,$sender,?,?");
return ('REJECT',"Bad sender address: $sender! Responsible server said: $rinf");
}
} elsif ($sender !~ /^.+\@$OurDomains>?$/i) {
my ($ok,$rinf) = check_mail_address_mx($sender);
debug_log(0,"filter_sender: $sender = $ok (MX) [$rinf]");
if (!$ok) {
md_syslog('info',"MDLOG,$MsgID,bad_sender,mx,$ip,$sender,?,?");
return ('REJECT',"Bad sender address: $sender! Responsible server(s) said: \
$rinf"); }
} else {
debug_log(0,"filter_sender: $sender (unchecked 2)");
}
} else {
debug_log(0,"filter_sender: $sender (unchecked 1)");
}
--8<--
Regards
/Jonas
--
Jonas Eckerman, jonas_lists@frukt.org
http://www.fsdb.org/
_______________________________________________
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic