[prev in list] [next in list] [prev in thread] [next in thread] 

List:       clamav-devel
Subject:    Re: [Clamav-devel] ClamAV effectiveness
From:       "David F. Skoll" <dfs () roaringpenguin ! com>
Date:       2013-10-12 15:18:49
Message-ID: 20131012111849.5c86697d () shishi ! roaringpenguin ! com
[Download RAW message or body]

On Sat, 12 Oct 2013 12:00:02 +0200
clamav-devel-request@lists.clamav.net wrote:

> Date: Fri, 11 Oct 2013 10:41:33 -0400
> From: Nick Johnson <npjohnso@cs.princeton.edu>

> (1) You're measuring effectiveness against your assumption that 99% of
> .exe files in email have malware.  Although I agree with that
> assumption, it should really be validated (perhaps with another AV
> program) before we accept it as truth and declare that clamav has 80%
> false negatives.

It's validated by eye; I look at the message subjects and they are obviously
viruses.

> (2) You are confusing two different metrics.  One is the % of .exe
> files which clamav declares clean.  The other is the % of malware
> which clamav declares clean.  These are different because one malware
> could appear in several .exe files.

It's of academic interest; Clam is leaking like a sieve and our customers
are not particularly interested in the reasons.

> When a new malware appears, there is a brief window during which
> signature-based detection schemes (from ANY vendor) cannot find it.

Absolutely.

> It's entirely possible that there is ONE new malware that appears in
> 137K .exe files sampled in 'a few days'.

Possible.

> In that case, clamav would
> identify all but one malware, yet the statistics look very bad because
> that ONE undetectable malware appeared 137K times.  So, I would ask:
> of these 137K .exe files, are they all identical?  Perhaps you could
> report the number of distinct file sizes or number of distinct
> md5sums.

I will have to run that analysis next week.  I suspect they are not all
identical, but I suspect too that there's a clump of a few or a few dozen
distinct viruses.

> From: Joel Esler <jesler@sourcefire.com>
> It helps the ClamAV tremendously if these files are submitted to the
> ClamAV team for analysis.

Do you have an efficient mechanism for submitting hundreds or thousands
of files?  I can dedupe them and submit, but it has to be something
semi-automated; please reply off-list if you have such a mechanism.

Regards,

David.
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net
[prev in list] [next in list] [prev in thread] [next in thread]