[prev in list] [next in list] [prev in thread] [next in thread] 

List:       websecurity
Subject:    Re: [WEB SECURITY] Web Application Testing (Black Box)
From:       mhellman () taxandfinance ! com
Date:       2010-01-26 15:19:08
Message-ID: 6fdb661fc08dc26d03e1bf4bc8903b62.squirrel () email ! taxandfinance ! com
[Download RAW message or body]

+1. disclaimer, i do internal assessments, no consulting. Unless it's a
very tiny/simple application, 40 hrs = pushing the shiny red button and
validating those results (including the 3 iterations of scans you'll do
trying to get said tool to work). This isn't a "pen test".  We can't do
full on pen tests of everything though, so there is value in doing this.

I usually throw out 2-3 weeks per app vuln. assessment, although being
internal I may have more flexibility to be inaccurate in my estimates:-) I
also am not 100% dedicated.  I'm not doing pen tests either, if I can show
that there isn't proper output encoding...it's a finding and I move on
(usually, sometimes exploit examples are required...I know when I'm
dealing with these folks).

That seems to be the sweet spot for me, but again the size of the
application really matters. I spend a fair amount of time just trying to
understand the application so I can find pressure points, something you
really have to do to just to know that the "shiny red button" scan did
even a modest job.

> Ridiculous, Raf.
>
> Consulting Pen-Test engagements vary wildly in my experience, and even
> more
> so in today's economy I imagine. But your scanner example is very low IRL.
>
> + I used to estimate and book 2-3 weeks per "application" when I scoped
> consulting.
>
> This would reflect a 15k to 20k retail project-price per site (time
> reflected). This included writing up the findings.
>
> However over the years I had to tune to all sorts of engagement realities,
> 1
> day, 1 weekend, etc. nine months, etc. On average over quite a few years
> doing app assessments, we billed about 2 to 2.5 weeks of hours in project
> pricing. I have heard many folks say 40 hours but I believe these are
> folks
> selling low-quality, high-volume assessments to large organizations.
>
> For example I had n00b project managers scope piss-poor quality projects
> --
> things like "scan 150 websites in 4 weeks" using the 2-5 hours of
> validation
> per-scan/per-site model. Using tools like Webinspect, which can generate
> literally thousands of false positives per site, this just isn't realistic
> at all. Especially if it takes you a week to tune the scanner alone.
>
> Now if by tuning you mean "turn off all the checks that are too noisy" -
> then I agree. This is what most people do. Almost everyone running
> scanners
> that I run into turns off checks like "Blind SQL" and "Possible SQL" etc.
> etc. since it's not worth wading through 2600 false positives to find the
> one valid sapphire in the mud. However, this puts you squarely in
> false-negative land. Which means you will raise your cost-per-site testing
> these sites by hand (Black Box) which goes back to my 2-3 weeks.
>
> At the end of the day everyone doing consulting pricing for anything
> length
> of time knows that you often wind up tuning to the customer's budget.
> Calculators or not - this often comes down to a mix of trust and price:
>
> Trust -- "I know and trust your team's work" e.g.-if you tell me 2 weeks
> minimum, I believe that is required to be "good enough"
> Price -- I have 240k an need to test 24 webapps this year, and re-verify
> all
> found vulns after mitigation/remediation
>
> You almost always have to balance between those.
>
> Balance = Security Risk Religion Zeal: Deep vs. Wide
>
> Where you balance usually is driven by focusing on "critical apps" for the
> Security Risk Religion types that believe in "Going Deep".
>
> e.g.-"I have 100 apps. What should I do?"....a: Well, start by "deep" pen
> testing the 24 most "critical" apps this year. (2-3 weeks per)
>
> I tended to shy away from offering testing to the "Go Wide" Security Risk
> Religious types, but there is money to be made there. Offering fast, easy
> unauthenticated scans with cursory (or no) human validation is a cheap way
> to provide this level of insight.
>
> Personally when I ran into Go Wide religious types -- I encouraged them to
> use me to help them prioritize efforts and assets, define a "Wide"
> baseline,
> and select automation to measure it....instead of some shallow, wide
> testing
> services that will miss tons of vulns they will blame me for later. (or
> create a false sense of security)
>
> --
>
> Raf, sorry for picking on you. Your 40 hour suggestion is not that
> ridiculous for scanner-jockey work today...but I hadn't taken a shot at
> you
> this year so figured I was overdue. Cheers!
>
> (Since the subject was "Black Box" I am deliberately leaving out all
> discussions of WB, Threat Modeling, Architectural reviews, etc. that can
> be
> added/combined, not to mention the need to switch focus back and forth
> between BB/WB depending on application, organizational goals, etc. etc.)
>
> I talk to a lot of folks that still do BB auditing and pen-testing, and
> most
> tell me their hours are getting squeezed, so my number examples above are
> so
> 2005 and probably too aggressive for today's budget landscape. But, no
> need
> to race to the bottom of pricing and quality just because "everyone else
> is
> doing it".
>
> </china>
>
>
> Arian Evans
> ---
> capitalist marksman. eats animals.
>
>
> On Mon, Jan 25, 2010 at 9:36 AM, Rafal Los <Rafal@ishackingyou.com> wrote:
>
>> Matt,
>>        I think 80hrs is quite ambitious ...and may actually get most of
>> the
>> companies (I guess this also depends on whether you're doing the work
>> internally or for customers, wink-wink) to shy away.
>>
>>        [For a site of average size/complexity] I think 40hrs is pretty
>> good
>> given that a well-configured, purposed "automated black-box scanner"
>> tool
>> should take you no more than 24hrs (remember tools don't need to sleep)
>> and
>> then another 2-5hrs to validate results.  You're then looking at another
>> 1.5
>> days work to manually re-rest or investigate deeper into the application
>> and
>> a half-day to write up the report (or pretty-up a pre-made report).
>>
>> Just my $0.0199.
>>
>> Rafal "Raf" Los
>> InfoSec Specialist & Blogger
>> Twitter: RafalLos
>> Blog: http://preachsecurity.blogspot.com
>>
>> -----Original Message-----
>> From: Matt Parsons [mailto:mparsons1980@gmail.com]
>> Sent: Monday, January 25, 2010 10:11 AM
>> To: 'Nitchi DaMon'; websecurity@webappsec.org
>> Subject: RE: [WEB SECURITY] Web Application Testing (Black Box)
>>
>> I do both static code analysis and web penetration testing.   For my web
>> penetration engagements I have 80 hours to complete the manual and
>> automated
>> review of the website.
>>
>> I use tools like web inspect and app scan to get an intial site
>> assessment
>> of the application then I manually "poke" around with tools like tamper
>> data
>> and burp suite professional.
>>
>> This usually just gets the low hanging fruit vulnerabilities and is
>> rather
>> challenging without the source code.   I would prefer to have the source
>> code on these engagements but I do not.
>>
>> I suppose the black box test is better than nothing but some of the
>> false
>> positives and false negatives found and not found by the tools have to
>> be
>> considered.
>>
>> I used the OWASP Web testing guide for my engagements and created my own
>> small check list.
>>
>> In this field all we can do is the best we can.
>>
>> Hope this helps.
>>
>> All the best.
>> Matt
>>
>>
>> Matt Parsons, MSM, CISSP
>> 315-559-3588 Blackberry
>> 817-294-3789 Home office
>> mailto:mparsons1980@gmail.com
>> http://www.parsonsisconsulting.com
>> http://www.o2-ounceopen.com/o2-power-users/
>> http://www.linkedin.com/in/parsonsconsulting
>> http://parsonsisconsulting.blogspot.com/
>> http://www.vimeo.com/8939668
>>
>>
>>
>>
>>
>>
>>
>>
>> -----Original Message-----
>> From: Nitchi DaMon [mailto:nitchimon@yahoo.com]
>> Sent: Monday, January 25, 2010 9:40 AM
>> To: websecurity@webappsec.org
>> Subject: [WEB SECURITY] Web Application Testing (Black Box)
>>
>>
>> Greetings all.
>>
>> Tools being tools both manual and automatic, each tool utilized within
>> the
>> scop of "block box testing" of an application takes a period of time to
>> complete the process.
>>
>> The process being, testing, auditing and reporting.
>>
>>
>> On the average, what woudl you consider to be the "average time
>> required"
>> to
>> perform this multi-part task ?
>>
>> I know thaqt there are so many variations here, but I am looking to see
>> if
>> there are any time standards to look at for black box testing of
>> applications.
>>
>> As we all know, there is less and less time given to accurately run a
>> complete black box test.  Yes, Whitebox testing IS the right way to go,
>> yet
>> with all of the trainig and static analysys testing, there are STILL
>> vulnerabilities being created.
>>
>>
>> thanks in advance
>>
>>
>> nitch
>>
>>
>>
>>
>>
>>
>> ----------------------------------------------------------------------------
>> Join us on IRC: irc.freenode.net #webappsec
>>
>> Have a question? Search The Web Security Mailing List Archives:
>> http://www.webappsec.org/lists/websecurity/archive/
>>
>> Subscribe via RSS:
>> http://www.webappsec.org/rss/websecurity.rss [RSS Feed]
>>
>> Join WASC on LinkedIn
>> http://www.linkedin.com/e/gis/83336/4B20E4374DBA
>>
>>
>>
>>
>> ----------------------------------------------------------------------------
>> Join us on IRC: irc.freenode.net #webappsec
>>
>> Have a question? Search The Web Security Mailing List Archives:
>> http://www.webappsec.org/lists/websecurity/archive/
>>
>> Subscribe via RSS:
>> http://www.webappsec.org/rss/websecurity.rss [RSS Feed]
>>
>> Join WASC on LinkedIn
>> http://www.linkedin.com/e/gis/83336/4B20E4374DBA
>>
>>
>>
>>
>> ----------------------------------------------------------------------------
>> Join us on IRC: irc.freenode.net #webappsec
>>
>> Have a question? Search The Web Security Mailing List Archives:
>> http://www.webappsec.org/lists/websecurity/archive/
>>
>> Subscribe via RSS:
>> http://www.webappsec.org/rss/websecurity.rss [RSS Feed]
>>
>> Join WASC on LinkedIn
>> http://www.linkedin.com/e/gis/83336/4B20E4374DBA
>>
>>
>
> --
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.
>
>



-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


----------------------------------------------------------------------------
Join us on IRC: irc.freenode.net #webappsec

Have a question? Search The Web Security Mailing List Archives: 
http://www.webappsec.org/lists/websecurity/archive/

Subscribe via RSS: 
http://www.webappsec.org/rss/websecurity.rss [RSS Feed]

Join WASC on LinkedIn
http://www.linkedin.com/e/gis/83336/4B20E4374DBA

[prev in list] [next in list] [prev in thread] [next in thread]