[WEB SECURITY] In depth security scanning versus breadth based

robert at webappsec.org robert at webappsec.org
Wed Jul 8 13:17:58 EDT 2009


> Current issues our team has with running our scanning tool:
> 
> 1. Integration of our scanning tool and QA's testing software. They
> currently do not play with each other well, thereby, making it harder to
> leverage QA's test scripts.
> 2. Test Data. Keeping up with test data needs for the new pieces of
> functionality being introduced into every release has been tough.

Yes integration and keeping the urls/sample data/user accounts up to date seems to be the most challenging aspect
and not the technical capabilities of the tool (assuming you fully understand what to expect
from the tool). 


> 3. Time. Weeding out the false positives is just time consuming.

An approach that is probably better, is for infosec to identify the specific checks
that work well with few false positives against your environment, and use a small policy. For example many tools
are decent at finding xss, cookie flags, sqli, open url redirectors, OS Commanding, comments in html/js, etc... I find
that scanning your specific applications and only whitelisting a small series of checks
to be a better approach. Of course you'll still need someone to perform a system wide scan of all checks
but this is probably best suited for an infosec person to do as QA isn't going to understand how to check
for these types of things.  
 
Regards,
- Robert
http://www.cgisecurity.com/
http://www.qasec.com/

> 
> -Wil
> 
> On Tue, Jul 7, 2009 at 9:09 PM, <robert at webappsec.org> wrote:
> 
> > > For simple web applications with no "wizard-like" logic, this might be
> > > achieved... but when the application starts to get more complex, IMHO
> > > you've got two ways to go:
> > >
> > > - Throw away the scanner, and perform manual testing, which is the
> > > only real way in which you'll be sure that all the web application
> > > vulnerabilities are being found. With this, I'm not saying that
> >
> > You make the general statement assuming that manpower is available for
> > repen testing
> > every parameter on every product release. The reality is most people will
> > never be able
> > to afford this. Not to mention hiring penetration testers doesn't scale,
> > and QA is only capable
> > of so much.
> >
> > Scanners are good at finding *certain issues* (each app/site is different),
> > and as I said in my
> > previous email I don't want to get into that debate/discussion. I am asking
> > the list if anyone has tried
> > implementing this and had good/bad luck and to share their experiences.
> >
> > > scanners are useless! Damn... I write one in my spare time! What I'm
> > > saying is that when the web application gets really, really complex...
> > > the only way to test it properly is by hand.
> >
> > Nobody is debating the need for deep dive pen tests, but if you wish to
> > test on every release
> > this is simply impossible for 99.99999999% of companies/organizations.
> >
> > Regards,
> > - Robert
> > http://www.cgisecurity.com/
> >
> >
> >
> > ----------------------------------------------------------------------------
> > Join us on IRC: irc.freenode.net #webappsec
> >
> > Have a question? Search The Web Security Mailing List Archives:
> > http://www.webappsec.org/lists/websecurity/archive/
> >
> > Subscribe via RSS:
> > http://www.webappsec.org/rss/websecurity.rss [RSS Feed]
> >
> > Join WASC on LinkedIn
> > http://www.linkedin.com/e/gis/83336/4B20E4374DBA
> >
> >
> 
> --0015174c141854151b046e2cd2dc
> Content-Type: text/html; charset=ISO-8859-1
> Content-Transfer-Encoding: quoted-printable
> 
> Current issues our team has with running our scanning tool:<br><br>1. Integ=
> ration of our scanning tool and QA's testing software. They currently d=
> o not play with each other well, thereby, making it harder to leverage QA&#=
> 39;s test scripts.<br>
> 2. Test Data. Keeping up with test data needs for the new pieces of functio=
> nality being introduced into every release has been tough.<br>3. Time. Weed=
> ing out the false positives is just time consuming. <br><br>-Wil<br><br>
> <div class=3D"gmail_quote">On Tue, Jul 7, 2009 at 9:09 PM,  <span dir=3D"lt=
> r"><<a href=3D"mailto:robert at webappsec.org">robert at webappsec.org</a>>=
> </span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"border-left: 1=
> px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"=
> >
> <div class=3D"im">> For simple web applications with no "wizard-lik=
> e" logic, this might be<br>
> > achieved... but when the application starts to get more complex, IMHO<=
> br>
> > you've got two ways to go:<br>
> ><br>
> </div><div class=3D"im">> - Throw away the scanner, and perform manual t=
> esting, which is the<br>
> > only real way in which you'll be sure that all the web application=
> <br>
> > vulnerabilities are being found. With this, I'm not saying that<br=
> >
> <br>
> </div>You make the general statement assuming that manpower is available fo=
> r repen testing<br>
> every parameter on every product release. The reality is most people will n=
> ever be able<br>
> to afford this. Not to mention hiring penetration testers doesn't scale=
> , and QA is only capable<br>
> of so much.<br>
> <br>
> Scanners are good at finding *certain issues* (each app/site is different),=
>  and as I said in my<br>
> previous email I don't want to get into that debate/discussion. I am as=
> king the list if anyone has tried<br>
> implementing this and had good/bad luck and to share their experiences.<br>
> <div class=3D"im"><br>
> > scanners are useless! Damn... I write one in my spare time! What I&#39=
> ;m<br>
> > saying is that when the web application gets really, really complex...=
> <br>
> > the only way to test it properly is by hand.<br>
> <br>
> </div>Nobody is debating the need for deep dive pen tests, but if you wish =
> to test on every release<br>
> this is simply impossible for 99.99999999% of companies/organizations.<br>
> <br>
> Regards,<br>
> - Robert<br>
> <a href=3D"http://www.cgisecurity.com/" target=3D"_blank">http://www.cgisec=
> urity.com/</a><br>
> <div><div></div><div class=3D"h5"><br>
> <br>
> ---------------------------------------------------------------------------=
> -<br>
> Join us on IRC: <a href=3D"http://irc.freenode.net" target=3D"_blank">irc.f=
> reenode.net</a> #webappsec<br>
> <br>
> Have a question? Search The Web Security Mailing List Archives:<br>
> <a href=3D"http://www.webappsec.org/lists/websecurity/archive/" target=3D"_=
> blank">http://www.webappsec.org/lists/websecurity/archive/</a><br>
> <br>
> Subscribe via RSS:<br>
> <a href=3D"http://www.webappsec.org/rss/websecurity.rss" target=3D"_blank">=
> http://www.webappsec.org/rss/websecurity.rss</a> [RSS Feed]<br>
> <br>
> Join WASC on LinkedIn<br>
> <a href=3D"http://www.linkedin.com/e/gis/83336/4B20E4374DBA" target=3D"_bla=
> nk">http://www.linkedin.com/e/gis/83336/4B20E4374DBA</a><br>
> <br>
> </div></div></blockquote></div><br>
> 
> --0015174c141854151b046e2cd2dc--
> 


----------------------------------------------------------------------------
Join us on IRC: irc.freenode.net #webappsec

Have a question? Search The Web Security Mailing List Archives: 
http://www.webappsec.org/lists/websecurity/archive/

Subscribe via RSS: 
http://www.webappsec.org/rss/websecurity.rss [RSS Feed]

Join WASC on LinkedIn
http://www.linkedin.com/e/gis/83336/4B20E4374DBA



More information about the websecurity mailing list