[prev in list] [next in list] [prev in thread] [next in thread] 

List:       oss-security
Subject:    Re: [oss-security] Thousands of vulnerabilities, almost no CVEs: OSS-Fuzz
From:       John Haxby <john.haxby () oracle ! com>
Date:       2019-06-24 17:08:55
Message-ID: 159D0B1B-83D0-4C1F-A91E-DA544F7F4C7E () oracle ! com
[Download RAW message or body]

> On 24 Jun 2019, at 14:01, Dmitry Vyukov <dvyukov@google.com> wrote:
> 
> On Mon, Jun 17, 2019 at 1:32 PM Marcus Meissner <meissner@suse.de> wrote:
> > 
> > Hi,
> > 
> > 
> > On Sat, Jun 15, 2019 at 11:49:03AM -0400, Alex Gaynor wrote:
> > > Hi everyone,
> > > 
> > > OSS-Fuzz is Google's project to provide continious large-scale fuzzing.
> > > Since it launched in 2016, it's found just shy of 3000 things it counts as
> > > security bugs [0][1]. I'm not a developer of OSS-Fuzz (at Google), but I've
> > > helped several projects integrate with it.
> > > 
> > > You can see that it's had some amazing success across a variety of projects
> > > -- I've written previously to this list about the things I thought made it
> > > particularly effective working with ImageMagick and GraphicsMagick [2].
> > > 
> > > Today I'd like to highlight what I see as a tremendous issue: very few of
> > > these security bugs ever has a CVE issued for it. This is probably due to a
> > > few factors, a) the relative difficulty of obtaining a CVE, b) the lack of
> > > a human reporter who is interested in obtaining one for "credit" purposes,
> > > c) the sheer number of bugs that we're talking about.
> > > 
> > > CVEs are not important for their own sake. The true value is in all of the
> > > downstream processing that uses them as input: the Linux distributions that
> > > use them to figure out what fixes to backport, the docker security scanners
> > > that look for vulnerable code on the system, the corporate
> > > threat-intelligence feeds, etc.
> > > 
> > > A test of a random ImageMagick vulnerability against Ubuntu Xenial shows
> > > that it, indeed, continues to reproduce.
> > > 
> > > This is in addition to the >100 security bugs OSS-Fuzz found and publicly
> > > disclosed due to hitting their disclosure deadline, and which still have
> > > not been fixed [3].
> > > 
> > > I haven't analyzed any of these vulnerabilities for exploitability, and I
> > > doubt anyone else has either.
> > > 
> > > I do not have a solution to this problem. I wanted to raise awareness of
> > > it, in the hope that it would start a discussion which might come to a
> > > solution.
> > 
> > So as this was not yet discussed, lets have it closer look at the gaps
> > in the workflow.
> > 
> > (I am not going into the orthogonal approaches, like surface reduction,
> > mitigations, replacement etc.)
> > 
> > "topic" vs "automation state"
> > 
> > 
> > Bugfinding:
> > 
> > - Is manual to fully automated these days, and improving.
> > 
> > The fully automated bugfinding is a significant contributor to amount of bugs.
> > 
> > Bugfixing:
> > 
> > - Largely manual. Some research in automation by DARPA et.al.
> > 
> > 
> > This is a significant gap of the scale issues, automated bugfinding
> > can easily overload opensource projects.
> > 
> > 
> > Security IR Tracking:
> > 
> > CVE Allocation:
> > 
> > - Mostly manual, some tool help at most.
> > 
> > Significant gap here (as you wrote).
> > 
> > This seems to be low hanging fruit... There is nothing stopping to
> > 
> > - allocate big CVE blocks to "automation sub-CNA"s
> > - have a OSS-Fuzz / Syzkaller / whatever CNA doing automated CVE assignments out \
> > of this block
> 
> Hi,
> 
> I see syzkaller come up already. Yes, syzbot (automated continuous
> kernel fuzzing) has the same problem: thousands of crashes, most don't
> have any security assessment (too expensive):
> https://syzkaller.appspot.com/upstream
> Besides the update problem, there is also bug fixing problem: loud
> CVEs attract lots of attention and gets fixed quickly, but require up
> to months of manual labor (per bug). "Just a use-after-free" may not
> get any attention, while being more harmful in the end. Even a WARNING
> (Linux kernel term for a non-fatal assertion) may be a VM info leak in
> the end.
> 
> So what are community thoughts on automatic CVE assignment?
> That would definitely get some attention to these bugs by vendors
> (because that's open CVEs in their products then). And this should be
> implementable because both OSS-Fuzz and syzbot are automated enough
> already. However I afraid that these CVEs may be as automatically
> sorted into a trashcan then :)


Unfortunately there are people who runs scans to see what CVEs are fixed and if all \
the known CVEs aren't fixed then they scream and shout.   It doesn't matter whether \
the CVE represents a viable exploit, it has to be fixed.

Yes, off-by-one errors, overflows and the like are all *potential* security flaws, \
but by allocating a CVE for all those potential issues you're trying to turn CVEs \
into a bug database.

I'm looking at one here that says "allows local users to cause a denial of service \
(NULL pointer dereference) or possibly have unspecified other impact".   That gets a \
CVSS score up in the stratosphere when, in this case, it's a bug that a user would \
have to inflict on themselves provided they have the right hardware.   I know I'm \
over-simplifying this one, but it remains that the "unspecified other impact" is \
highly speculative, not to mention dubious, and the DoS is not the total loss of \
service that the stratospheric score indicates.

On the face of it, even a partial DoS is a security problem, but in this case the \
pre-requisites for exercising this particular bug would pretty much also categorise \
"rm -rf /*" as a security issue.

Automatically creating CVEs from fuzzer results is going to just mean that the real \
problems get passed by: a little thought needs to go into it first.

jch


> 
> 
> > Rating:
> > 
> > - largely manual / partially automated, done by NVD and distributions seperately.
> > 
> > Could be automated by "type" by the fuzzer, similar to above.
> > 
> > 
> > Structured Vulnerability information storing:
> > 
> > - Not really existing right now.
> > 
> > - On top of CVE:
> > - referencing reproducers
> > - affected versions
> > - ratings
> > - referencing patches
> > 
> > These could be supplied / attached by automatisms in a automation CNA.
> > 
> > 
> > Distribution tracking / update preparation / packaging / QA :
> > 
> > - done by distributions, largely manual to semi automatic.
> > 
> > With better structured upstream vulnerability information storage its automation
> > could be improved.
> > 
> > Some thoughts are going betweenm distributions on sharing information / load, but \
> > as this is a competition issue this might be hard.
> > 
> > So main gaps I personally see:
> > 
> > - bugfixing automation or help at least
> > 
> > - (better) structured storage in a global database, either CVE or something \
> > entirely new. 
> > Ciao, Marcus


["signature.asc" (signature.asc)]

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iHUEAREIAB0WIQT+pxvb11CFWUkNSOVFC7t+lC+jyAUCXREDpwAKCRBFC7t+lC+j
yD07AP4pb521/otUrTAJFh3bvvt5mq2p53xzYdTcXkNQFM2LpAD/bxnbjkeC9V/C
u5X90C4PBMlpa8jt2Cr2v5UWR2KyCRo=
=P55B
-----END PGP SIGNATURE-----


[prev in list] [next in list] [prev in thread] [next in thread]