[WEB SECURITY] Need a vulnerable XML Web Service

robert at webappsec.org robert at webappsec.org
Thu Jun 10 13:42:55 EDT 2010


> Narkolayev,
> 
> *3) The **greatest **problem: Current scanners have very few attack
> signatures that really can bypass filters and WAFs.*
> *
> *
> This is FAR from the greatest problem.  Using XSS as an example (one of MAN=
> Y
> areas), every scanner I've used (including Burp) completely sucks at
> handling AJAX or properly traversing the DOM when there is any dynamic
> content.  Unless you are spoon feeding these products the information they
> need, they are rather useless and you can achieve the same damn thing using
> an intercepting proxy and having a solid approach for testing an
> application.  If you want an exercise in "useless", point <insert tool here=
> >
> at your Burp proxy, and watch the "attack" patterns that pass through your
> proxy.  You'll see nothing but generic, unintelligent attempts at detecting
> common issues.  They do a terrible job at detecting context, and an equally
> bad job (ie- don't even attempt to) figure out what works and doesn't work.
>  Imagine banging your head against a wall over and over again, and you get
> the idea.....
> 
>  I don't think more "Attack Signatures" are needed.  Rather, a
> re-engineering of the common approach to blackbox testing in an automated
> fashion.  Andre was 100% correct with his statement.  I don't have pretty
> metrics and charts to back this up, just a lot of real-world experience.


I agree. I wrote an article back in 2006 explaining some of the issues with blackbox
scanners.

Challenges faced by automated web application security assessment tools 
http://www.cgisecurity.com/scannerchallenges.html

I just started to write part two of this article which should be out in a few months (as time permits).

Regards,
- Robert Auger
http://www.webappsec.org/
http://www.cgisecurity.com/
http://www.qasec.com/


> 
> -Jack
> 
> On Thu, Jun 10, 2010 at 3:42 AM, Shlomi Narkolayev <shlominar at gmail.com>wro=
> te:
> 
> > Hey,
> >
> >
> > I tend to agree with you, but think of it this way: Because of these
> > reasons there is always room for the little and the quicker ones in the
> > security field.
> >
> > I think companies that developing scanning tools (NT OBJECTives, IBMs'
> > Appscan, Acunetix, White Hats' Sentinel, HP's Web Inspect, Cenzics=92* **
> > Hailstorm*, Etc) need to invest (Money, Time, Research!) in the Engine as
> > they invest in the functionality of the product.
> >
> > *
> > *
> >
> > *In today's scanners, I see three main problems:*
> >
> > 1) Very high FP (More then 50%).
> >
> > 2) Lack of serious exploitation functionality (of course, this module cou=
> ld
> > significantly improve the first problem).
> >
> > 3) The *greatest *problem: Current scanners have very few attack
> > signatures that really can bypass filters and WAFs.
> >
> > Just for comparison, attacks like: XSS, SQLi, RFI, OSi, Directory
> > Traversal, and many others can have more than 1000 attack signatures for
> > each type of attack I described above, and current scanners have only abo=
> ut
> > 4-20 attack signatures for each type of attack.
> >
> >
> >
> > This situation is good for us (the researches), we need continue our
> > researches and make a "better place=94 for the security technology - It m=
> eans Enlarge
> > ours attack signatures lists ;-)
> >
> > Kind Regards,
> > Narkolayev Shlomi.
> >
> > Visit my blog: http://Narkolayev-Shlomi.blogspot.com
> >
> >
> >
> > On Thu, Jun 10, 2010 at 2:08 AM, Andre Gironda <andreg at gmail.com> wrote:
> >
> >> On Wed, Jun 9, 2010 at 2:24 PM, Jim Manico <jim at manico.net> wrote:
> >> > Do you think raw scanning engine technology has hit a plateau? Or coul=
> d
> >> it be that the cost to innovate scanning engines beyond today's state of=
>  the
> >> art is getting prohibitive?
> >>
> >> Jim,
> >>
> >> Do you think investors of research projects for commercial app
> >> scanners should continue their current strategy, or should it be
> >> changed?
> >>
> >> If you were responsible for scan engine technology for an app scanner
> >> at one of the following companies, would you invest in the current
> >> model -- or would you try something completely different such as
> >> Fortify's approach with PTA and RTA?
> >>
> >> HP Software
> >> IBM Watchfire
> >> Acunetix
> >> NTOBJECTives
> >> Cenzic
> >> Qualys
> >> WhiteHat Security
> >>
> >> These vendors, IMO, have all failed to deliver quality products that
> >> constantly push the envelope. Instead, their "scan engine technology"
> >> is all but replaced by cheap ($200-$3000/year/person, unlimited use)
> >> products such as Burp Suite Professional and Mavituna Security
> >> Netsparker Pro.
> >>
> >> Look at the Wivet (wivet.googlecode.com) results. Look at the
> >> SQLiBENCH (sqlibench.googlecode.com) results. Run all of those
> >> scanners in crawl-only mode through Casaba x5s and see how many HTML
> >> injections you get from one scanner to the next. Guess who comes out
> >> on top? The cheapest commercial products, especially Burp Suite Pro
> >> and Netsparker Pro. To add to that fire, Burp Suite Free Edition and
> >> Netsparker Community Edition also blow the other commercial products
> >> away.
> >>
> >> What else is left to say? The free scanning engines (that are tied to
> >> the cheapest products in the appsec space) are better than the
> >> expensive, commercial scanning engines. The above 7 vendors can't and
> >> won't get their acts together. They have had their time. That time is
> >> now over.
> >>
> >> Thoughts?
> >>
> >> Andre
> >>
> >>
> >> ------------------------------------------------------------------------=
> ----
> >> Join us on IRC: irc.freenode.net #webappsec
> >>
> >> Have a question? Search The Web Security Mailing List Archives:
> >> http://www.webappsec.org/lists/websecurity/archive/
> >>
> >> Subscribe via RSS:
> >> http://www.webappsec.org/rss/websecurity.rss [RSS Feed]
> >>
> >> To unsubscribe email websecurity-unsubscribe at webappsec.org and reply to
> >> the confirmation email
> >>
> >> Join WASC on LinkedIn
> >> http://www.linkedin.com/e/gis/83336/4B20E4374DBA
> >>
> >>
> >
> 
> --0016e6db2d7bf46ca20488af7ebd
> Content-Type: text/html; charset=windows-1252
> Content-Transfer-Encoding: quoted-printable
> 
> Narkolayev,<div><br></div><div><i>3) The=A0</i><u><b><i>greatest</i></b><i>=
> =A0</i></u><i>problem: Current scanners have very few attack signatures tha=
> t really can bypass filters and WAFs.</i></div><div><i><br></i></div><div>T=
> his is FAR from the greatest problem. =A0Using XSS as an example (one of MA=
> NY areas), every scanner I've used (including Burp) completely sucks at=
>  handling AJAX or properly traversing the DOM when there is any dynamic con=
> tent. =A0Unless you are spoon feeding these products the information they n=
> eed, they are rather useless and you can achieve the same damn thing using =
> an intercepting proxy and having a solid approach for testing an applicatio=
> n. =A0If you want an exercise in "useless", point <insert tool=
>  here> at your Burp proxy, and watch the "attack" patterns tha=
> t pass through your proxy. =A0You'll see nothing but generic, unintelli=
> gent attempts at detecting common issues. =A0They do a terrible job at dete=
> cting context, and an equally bad job (ie- don't even attempt to) figur=
> e out what works and doesn't work. =A0Imagine banging your head against=
>  a wall over and over again, and you get the idea.....</div>
> <div><br></div><div>=A0I don't think more "Attack Signatures"=
>  are needed. =A0Rather, a re-engineering of the common approach to blackbox=
>  testing in an automated fashion. =A0Andre was 100% correct with his statem=
> ent. =A0I don't have pretty metrics and charts to back this up, just a =
> lot of real-world experience.</div>
> <div><br></div><div>-Jack</div><div><br><div class=3D"gmail_quote">On Thu, =
> Jun 10, 2010 at 3:42 AM, Shlomi Narkolayev <span dir=3D"ltr"><<a href=3D=
> "mailto:shlominar at gmail.com">shlominar at gmail.com</a>></span> wrote:<br><=
> blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px=
>  #ccc solid;padding-left:1ex;">
> <div dir=3D"ltr"><div style=3D"text-align:right"><div style=3D"text-align:l=
> eft">
> 
> <p class=3D"MsoNormal">Hey,</p><p class=3D"MsoNormal"><br></p><p class=3D"M=
> soNormal">I tend to agree with you, but think of it this way: Because
> of these reasons there is always room for the little and the quicker ones i=
> n
> the security field.</p>
> 
> <p class=3D"MsoNormal">I think companies that developing scanning tools (NT
> OBJECTives, IBMs' Appscan, Acunetix, White Hats' Sentinel, HP's=
>  Web Inspect, Cenzics=92<em><span> </span></em><em><span style=3D"font-styl=
> e:normal">Hailstorm</span></em>, Etc)
> need to invest (Money, Time, Research!) in the Engine as they invest in the
> functionality of the product.</p>
> 
> <p class=3D"MsoNormal"><u><br></u></p><p class=3D"MsoNormal"><u>In today&#3=
> 9;s scanners, I see three main problems:</u></p>
> 
> <p class=3D"MsoNormal">1) Very high FP (More then 50%).</p>
> 
> <p class=3D"MsoNormal"></p>
> 
> <p class=3D"MsoNormal">2) Lack of serious exploitation functionality (of co=
> urse,
> this module could significantly improve the first problem).</p>
> 
> <p class=3D"MsoNormal"></p>
> 
> <p class=3D"MsoNormal">3) The <u><b>greatest</b> </u>problem: Current scann=
> ers have very
> few attack signatures that really can bypass filters and WAFs.</p>
> 
> <p class=3D"MsoNormal">Just for comparison, attacks like: XSS, SQLi, RFI, O=
> Si,
> Directory Traversal, and many others can have more than 1000 attack signatu=
> res
> for each type of attack I described above, and current scanners have only a=
> bout
> 4-20 attack signatures for each type of attack.</p>
> 
> <p class=3D"MsoNormal">=A0</p>
> 
> <p class=3D"MsoNormal">This situation is good for us (the researches), we n=
> eed continue
> our researches and make a "better place=94 for the security technology=
>  - It means <span><span style=3D"background-color:rgb(230, 236, 249);color:=
> rgb(0, 0, 0)" title=3D""></span></span>Enlarge ours attack signatures lists=
>  ;-)<br>
> </p>
> 
> </div><br clear=3D"all"></div>Kind Regards,<br>Narkolayev Shlomi.<br><br>Vi=
> sit my blog: <a href=3D"http://Narkolayev-Shlomi.blogspot.com" target=3D"_b=
> lank">http://Narkolayev-Shlomi.blogspot.com</a><div><div></div><div class=
> =3D"h5">
> <br>
> <br><br><div class=3D"gmail_quote">On Thu, Jun 10, 2010 at 2:08 AM, Andre G=
> ironda <span dir=3D"ltr"><<a href=3D"mailto:andreg at gmail.com" target=3D"=
> _blank">andreg at gmail.com</a>></span> wrote:<br><blockquote class=3D"gmai=
> l_quote" style=3D"margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 2=
> 04, 204);padding-left:1ex">
> 
> 
> <div>On Wed, Jun 9, 2010 at 2:24 PM, Jim Manico <<a href=3D"mailto:jim at m=
> anico.net" target=3D"_blank">jim at manico.net</a>> wrote:<br>
> > Do you think raw scanning engine technology has hit a plateau? Or coul=
> d it be that the cost to innovate scanning engines beyond today's state=
>  of the art is getting prohibitive?<br>
> <br>
> </div>Jim,<br>
> <br>
> Do you think investors of research projects for commercial app<br>
> scanners should continue their current strategy, or should it be<br>
> changed?<br>
> <br>
> If you were responsible for scan engine technology for an app scanner<br>
> at one of the following companies, would you invest in the current<br>
> model -- or would you try something completely different such as<br>
> Fortify's approach with PTA and RTA?<br>
> <br>
> HP Software<br>
> IBM Watchfire<br>
> Acunetix<br>
> NTOBJECTives<br>
> Cenzic<br>
> Qualys<br>
> WhiteHat Security<br>
> <br>
> These vendors, IMO, have all failed to deliver quality products that<br>
> constantly push the envelope. Instead, their "scan engine technology&q=
> uot;<br>
> is all but replaced by cheap ($200-$3000/year/person, unlimited use)<br>
> products such as Burp Suite Professional and Mavituna Security<br>
> Netsparker Pro.<br>
> <br>
> Look at the Wivet (<a href=3D"http://wivet.googlecode.com" target=3D"_blank=
> ">wivet.googlecode.com</a>) results. Look at the<br>
> SQLiBENCH (<a href=3D"http://sqlibench.googlecode.com" target=3D"_blank">sq=
> libench.googlecode.com</a>) results. Run all of those<br>
> scanners in crawl-only mode through Casaba x5s and see how many HTML<br>
> injections you get from one scanner to the next. Guess who comes out<br>
> on top? The cheapest commercial products, especially Burp Suite Pro<br>
> and Netsparker Pro. To add to that fire, Burp Suite Free Edition and<br>
> Netsparker Community Edition also blow the other commercial products<br>
> away.<br>
> <br>
> What else is left to say? The free scanning engines (that are tied to<br>
> the cheapest products in the appsec space) are better than the<br>
> expensive, commercial scanning engines. The above 7 vendors can't and<b=
> r>
> won't get their acts together. They have had their time. That time is<b=
> r>
> now over.<br>
> <br>
> Thoughts?<br>
> <div><div></div><div><br>
> Andre<br>
> <br>
> ---------------------------------------------------------------------------=
> -<br>
> Join us on IRC: <a href=3D"http://irc.freenode.net" target=3D"_blank">irc.f=
> reenode.net</a> #webappsec<br>
> <br>
> Have a question? Search The Web Security Mailing List Archives:<br>
> <a href=3D"http://www.webappsec.org/lists/websecurity/archive/" target=3D"_=
> blank">http://www.webappsec.org/lists/websecurity/archive/</a><br>
> <br>
> Subscribe via RSS:<br>
> <a href=3D"http://www.webappsec.org/rss/websecurity.rss" target=3D"_blank">=
> http://www.webappsec.org/rss/websecurity.rss</a> [RSS Feed]<br>
> <br>
> To unsubscribe email <a href=3D"mailto:websecurity-unsubscribe at webappsec.or=
> g" target=3D"_blank">websecurity-unsubscribe at webappsec.org</a> and reply to=
> <br>
> the confirmation email<br>
> <br>
> Join WASC on LinkedIn<br>
> <a href=3D"http://www.linkedin.com/e/gis/83336/4B20E4374DBA" target=3D"_bla=
> nk">http://www.linkedin.com/e/gis/83336/4B20E4374DBA</a><br>
> <br>
> </div></div></blockquote></div><br></div></div></div>
> </blockquote></div><br></div>
> 
> --0016e6db2d7bf46ca20488af7ebd--
> 


----------------------------------------------------------------------------
Join us on IRC: irc.freenode.net #webappsec

Have a question? Search The Web Security Mailing List Archives: 
http://www.webappsec.org/lists/websecurity/archive/

Subscribe via RSS: 
http://www.webappsec.org/rss/websecurity.rss [RSS Feed]

To unsubscribe email websecurity-unsubscribe at webappsec.org and reply to 
the confirmation email

Join WASC on LinkedIn
http://www.linkedin.com/e/gis/83336/4B20E4374DBA



More information about the websecurity mailing list