websecurity@lists.webappsec.org

The Web Security Mailing List

View all threads

open source tools is not as good as imaged

孙松柏
Sat, Jun 4, 2011 3:18 AM

hello everyone

i recently do some pentest. i used several tools both open source and
commerical  tools !

for the commerical ones ,i use appscan & acunetix

for the open source ones skipfish &arachni &w3af

for the free one  netsparker community edition

obviously , the open source tools is not stable(w3af), and the  three of
them can not scan some fatal vulnerable(such as sql injection) as appscan
can easily discovery.

netsparker is good in both speed and result . but it's community edition has
a lot restriction.

so anyone has a project to help the opensource tools upgrade.

--
FIT1-213
Department of Computer Science
Tsinghua University, Beijing, 100084
http://about.me/anakin/bio

hello everyone i recently do some pentest. i used several tools both open source and commerical tools ! for the commerical ones ,i use appscan & acunetix for the open source ones skipfish &arachni &w3af for the free one netsparker community edition obviously , the open source tools is not stable(w3af), and the three of them can not scan some fatal vulnerable(such as sql injection) as appscan can easily discovery. netsparker is good in both speed and result . but it's community edition has a lot restriction. so anyone has a project to help the opensource tools upgrade. -- FIT1-213 Department of Computer Science Tsinghua University, Beijing, 100084 http://about.me/anakin/bio
AG
Andre Gironda
Sun, Jun 5, 2011 12:24 AM

On Fri, Jun 3, 2011 at 8:18 PM, 孙松柏 lukesun629@gmail.com wrote:

hello everyone
i recently do some pentest. i used several tools both open source and
commerical  tools !
for the commerical ones ,i use appscan & acunetix
for the open source ones skipfish &arachni &w3af

wavsep.googlecode.com shows that open-source & free tools such as
Wapiti, Grabber, and Sandcat Free actually do better than the
commercial tools for SQL injection, XSS, and similar. In other words,
they have lower false positives and less false negatives because of
their injection and analysis techniques. I have not yet seen the
wavsep numbers for Acunetix or Appscan, but I have seen the numbers
for most other commercial scanners. It is possible that Appscan and
Acunetix perform better than Skipfish, Arachni, and W3AF -- but I
would assume that they do much worse than Wapiti, Grabber, and Sandcat
Free when pitted against the categories where those tools did well.

Open-source & free tools do not benchmark as well with regards to
crawling as shown in the wivet.googlecode.com results. This may cause
the open-source tools to miss hostnames, IP addresses, virtual hosts,
URI's, parameters, forms, custom headers (including cookies), and
links that can be extracted from the various places a crawler will
look for them. What's interesting is that not only that none of the
crawlers performed perfectly with regards to link extraction (possibly
because of the programming problems related to parsing malformed HTML
-- in addition to extensive problems with Ajax and Flash), but also
that they are generally incapable of providing the correct context to
submit an HTML/Ajax/Flash form without error or without the issuing
request reaching the appropriate server-side context as it would in a
normal use case. This sort of activity would require human
interaction, or at the very least, a Microformat (or similar
technology).

Many application penetration-testers prefer to use a browser and read
the HTML/Ajax/Flash content in order to accurately extract links and
provide context to forms. They will run all of their activities
through a web proxy such as Burp Suite Free Edition, Burp Suite
Professional, Fiddler, W3AF spiderMan discovery plugin, Watobo, and
others. The IBurpExtender interface for Java in Burp Suite
Professional is top notch, which is why it is the #1 tool of choice
for web application security professionals since late 2006 when it
became available. However, the other web proxies I mentioned are
utilized by some professionals for a variety of reasons: Fiddler also
has a rich extension capability, seen through its plugins (often .NET)
-- and W3AF and Watobo are open-source projects with tons of
extensibility and rich feature-sets.

Thus, the best tools are custom-built using scripting language support
for the IBurpExtender interface, found in Burp Python and Buby, and
custom lists that can be leaned on for fault-injection such as
SVNDigger and fuzzdb. This is unlikely to change without 5 years of
investment like has been done with the IBurpExtender interface,
however it is possible that a commercial scanner solution could find a
way to branch into this testing model.

Cheers,
Andre

On Fri, Jun 3, 2011 at 8:18 PM, 孙松柏 <lukesun629@gmail.com> wrote: > hello everyone > i recently do some pentest. i used several tools both open source and > commerical  tools ! > for the commerical ones ,i use appscan & acunetix > for the open source ones skipfish &arachni &w3af wavsep.googlecode.com shows that open-source & free tools such as Wapiti, Grabber, and Sandcat Free actually do better than the commercial tools for SQL injection, XSS, and similar. In other words, they have lower false positives and less false negatives because of their injection and analysis techniques. I have not yet seen the wavsep numbers for Acunetix or Appscan, but I have seen the numbers for most other commercial scanners. It is possible that Appscan and Acunetix perform better than Skipfish, Arachni, and W3AF -- but I would assume that they do much worse than Wapiti, Grabber, and Sandcat Free when pitted against the categories where those tools did well. Open-source & free tools do not benchmark as well with regards to crawling as shown in the wivet.googlecode.com results. This may cause the open-source tools to miss hostnames, IP addresses, virtual hosts, URI's, parameters, forms, custom headers (including cookies), and links that can be extracted from the various places a crawler will look for them. What's interesting is that not only that none of the crawlers performed perfectly with regards to link extraction (possibly because of the programming problems related to parsing malformed HTML -- in addition to extensive problems with Ajax and Flash), but also that they are generally incapable of providing the correct context to submit an HTML/Ajax/Flash form without error or without the issuing request reaching the appropriate server-side context as it would in a normal use case. This sort of activity would require human interaction, or at the very least, a Microformat (or similar technology). Many application penetration-testers prefer to use a browser and read the HTML/Ajax/Flash content in order to accurately extract links and provide context to forms. They will run all of their activities through a web proxy such as Burp Suite Free Edition, Burp Suite Professional, Fiddler, W3AF spiderMan discovery plugin, Watobo, and others. The IBurpExtender interface for Java in Burp Suite Professional is top notch, which is why it is the #1 tool of choice for web application security professionals since late 2006 when it became available. However, the other web proxies I mentioned are utilized by some professionals for a variety of reasons: Fiddler also has a rich extension capability, seen through its plugins (often .NET) -- and W3AF and Watobo are open-source projects with tons of extensibility and rich feature-sets. Thus, the best tools are custom-built using scripting language support for the IBurpExtender interface, found in Burp Python and Buby, and custom lists that can be leaned on for fault-injection such as SVNDigger and fuzzdb. This is unlikely to change without 5 years of investment like has been done with the IBurpExtender interface, however it is possible that a commercial scanner solution could find a way to branch into this testing model. Cheers, Andre
AR
Andres Riancho
Sun, Jun 5, 2011 2:13 AM

孙松柏,

On Sat, Jun 4, 2011 at 12:18 AM, 孙松柏 lukesun629@gmail.com wrote:

hello everyone
i recently do some pentest. i used several tools both open source and
commerical  tools !
for the commerical ones ,i use appscan & acunetix
for the open source ones skipfish &arachni &w3af
for the free one  netsparker community edition
obviously , the open source tools is not stable(w3af),

Have you tested the latest version? We've REALLY improved the

stability of the project in our latest release. If it still crashes in
some way for you, lets work together to make it work as expected.

and the  three of
them can not scan some fatal vulnerable(such as sql injection) as appscan
can easily discovery.

This is not uncommon, but you can find the revert case also.

netsparker is good in both speed and result . but it's community edition has
a lot restriction.
so anyone has a project to help the opensource tools upgrade.

FIT1-213
Department of Computer Science
Tsinghua University, Beijing, 100084
http://about.me/anakin/bio


The Web Security Mailing List

WebSecurity RSS Feed
http://www.webappsec.org/rss/websecurity.rss

Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA

WASC on Twitter
http://twitter.com/wascupdates

websecurity@lists.webappsec.org
http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org

--
Andrés Riancho
Director of Web Security at Rapid7 LLC
Founder at Bonsai Information Security
Project Leader at w3af

孙松柏, On Sat, Jun 4, 2011 at 12:18 AM, 孙松柏 <lukesun629@gmail.com> wrote: > hello everyone > i recently do some pentest. i used several tools both open source and > commerical  tools ! > for the commerical ones ,i use appscan & acunetix > for the open source ones skipfish &arachni &w3af > for the free one  netsparker community edition > obviously , the open source tools is not stable(w3af), Have you tested the latest version? We've REALLY improved the stability of the project in our latest release. If it still crashes in some way for you, lets work together to make it work as expected. > and the  three of > them can not scan some fatal vulnerable(such as sql injection) as appscan > can easily discovery. This is not uncommon, but you can find the revert case also. > netsparker is good in both speed and result . but it's community edition has > a lot restriction. > so anyone has a project to help the opensource tools upgrade. > -- > FIT1-213 > Department of Computer Science > Tsinghua University, Beijing, 100084 > http://about.me/anakin/bio > > _______________________________________________ > The Web Security Mailing List > > WebSecurity RSS Feed > http://www.webappsec.org/rss/websecurity.rss > > Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA > > WASC on Twitter > http://twitter.com/wascupdates > > websecurity@lists.webappsec.org > http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org > > -- Andrés Riancho Director of Web Security at Rapid7 LLC Founder at Bonsai Information Security Project Leader at w3af
P
psiinon
Sun, Jun 5, 2011 3:31 PM

Have you tried the OWASP Zed Attack Proxy -
https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project?
It is open source and completely free (there is no paid for 'pro' version).
Its also intended to be a community project - so we encourage involvement.
If you submit good quality code then you'll get commit access :)

Psiinon - OWASP ZAP Project Lead.

On Sat, Jun 4, 2011 at 4:18 AM, 孙松柏 lukesun629@gmail.com wrote:

hello everyone

i recently do some pentest. i used several tools both open source and
commerical  tools !

for the commerical ones ,i use appscan & acunetix

for the open source ones skipfish &arachni &w3af

for the free one  netsparker community edition

obviously , the open source tools is not stable(w3af), and the  three of
them can not scan some fatal vulnerable(such as sql injection) as appscan
can easily discovery.

netsparker is good in both speed and result . but it's community edition
has a lot restriction.

so anyone has a project to help the opensource tools upgrade.

--
FIT1-213
Department of Computer Science
Tsinghua University, Beijing, 100084
http://about.me/anakin/bio


The Web Security Mailing List

WebSecurity RSS Feed
http://www.webappsec.org/rss/websecurity.rss

Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA

WASC on Twitter
http://twitter.com/wascupdates

websecurity@lists.webappsec.org
http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org

Have you tried the OWASP Zed Attack Proxy - https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project? It is open source and completely free (there is no paid for 'pro' version). Its also intended to be a community project - so we encourage involvement. If you submit good quality code then you'll get commit access :) Psiinon - OWASP ZAP Project Lead. On Sat, Jun 4, 2011 at 4:18 AM, 孙松柏 <lukesun629@gmail.com> wrote: > hello everyone > > i recently do some pentest. i used several tools both open source and > commerical tools ! > > for the commerical ones ,i use appscan & acunetix > > for the open source ones skipfish &arachni &w3af > > for the free one netsparker community edition > > obviously , the open source tools is not stable(w3af), and the three of > them can not scan some fatal vulnerable(such as sql injection) as appscan > can easily discovery. > > netsparker is good in both speed and result . but it's community edition > has a lot restriction. > > so anyone has a project to help the opensource tools upgrade. > > -- > FIT1-213 > Department of Computer Science > Tsinghua University, Beijing, 100084 > http://about.me/anakin/bio > > _______________________________________________ > The Web Security Mailing List > > WebSecurity RSS Feed > http://www.webappsec.org/rss/websecurity.rss > > Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA > > WASC on Twitter > http://twitter.com/wascupdates > > websecurity@lists.webappsec.org > http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org > >
AG
Andre Gironda
Sun, Jun 5, 2011 10:42 PM

On Sun, Jun 5, 2011 at 8:31 AM, psiinon psiinon@gmail.com wrote:

Have you tried the OWASP Zed Attack Proxy -
https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project?
It is open source and completely free (there is no paid for 'pro' version).
Its also intended to be a community project - so we encourage involvement.
If you submit good quality code then you'll get commit access :)
Psiinon - OWASP ZAP Project Lead.

Any intent to improve the wavsep.googlecode.com or
wivet.googlecode.com results from ZAP?

ZAP scores worse than both Andiparos and Paros on SQLi categories, and
worse than most tools in other categories when run against WAVSEP.
It's also one of the worst crawlers, as seen in its WIVET results.

Many tools such as W3AF can export their findings as XML (and their
request data as HTML,Ajax,Ruby,Python), which can be imported into The
Dradis Framework (which outputs its own XML, or to HTML, Word, or
Mediawiki). Burp Pro Scanner can export its data as XML and HTML, and
so does the "analyse target" tool -- plus you can save
request/response data in Repeater and store session files that contain
this data. Fiddler can save a SAZ file full of request/response data
and export as a variety of Microsoft Internet Explorer and Visual
Studio XML formats. Do you have any plans to make ZAP more extensible
in these ways?

On Sun, Jun 5, 2011 at 8:31 AM, psiinon <psiinon@gmail.com> wrote: > Have you tried the OWASP Zed Attack Proxy - > https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project? > It is open source and completely free (there is no paid for 'pro' version). > Its also intended to be a community project - so we encourage involvement. > If you submit good quality code then you'll get commit access :) > Psiinon - OWASP ZAP Project Lead. Any intent to improve the wavsep.googlecode.com or wivet.googlecode.com results from ZAP? ZAP scores worse than both Andiparos and Paros on SQLi categories, and worse than most tools in other categories when run against WAVSEP. It's also one of the worst crawlers, as seen in its WIVET results. Many tools such as W3AF can export their findings as XML (and their request data as HTML,Ajax,Ruby,Python), which can be imported into The Dradis Framework (which outputs its own XML, or to HTML, Word, or Mediawiki). Burp Pro Scanner can export its data as XML and HTML, and so does the "analyse target" tool -- plus you can save request/response data in Repeater and store session files that contain this data. Fiddler can save a SAZ file full of request/response data and export as a variety of Microsoft Internet Explorer and Visual Studio XML formats. Do you have any plans to make ZAP more extensible in these ways?
P
psiinon
Mon, Jun 6, 2011 6:38 AM

Hi Andre,

Yes, we intend to improve ZAP in all these areas :)
Regarding exporting data, the next version (which will be released very
soon) will provide an API that supports JSON, XML and HTML, and we'll do our
best to ensure that ZAP plays well with other applications with more changes
in future releases.

Psiinon

On Sun, Jun 5, 2011 at 11:42 PM, Andre Gironda andreg@gmail.com wrote:

On Sun, Jun 5, 2011 at 8:31 AM, psiinon psiinon@gmail.com wrote:

Have you tried the OWASP Zed Attack Proxy -
https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project?
It is open source and completely free (there is no paid for 'pro'

version).

Its also intended to be a community project - so we encourage

involvement.

If you submit good quality code then you'll get commit access :)
Psiinon - OWASP ZAP Project Lead.

Any intent to improve the wavsep.googlecode.com or
wivet.googlecode.com results from ZAP?

ZAP scores worse than both Andiparos and Paros on SQLi categories, and
worse than most tools in other categories when run against WAVSEP.
It's also one of the worst crawlers, as seen in its WIVET results.

Many tools such as W3AF can export their findings as XML (and their
request data as HTML,Ajax,Ruby,Python), which can be imported into The
Dradis Framework (which outputs its own XML, or to HTML, Word, or
Mediawiki). Burp Pro Scanner can export its data as XML and HTML, and
so does the "analyse target" tool -- plus you can save
request/response data in Repeater and store session files that contain
this data. Fiddler can save a SAZ file full of request/response data
and export as a variety of Microsoft Internet Explorer and Visual
Studio XML formats. Do you have any plans to make ZAP more extensible
in these ways?

Hi Andre, Yes, we intend to improve ZAP in all these areas :) Regarding exporting data, the next version (which will be released very soon) will provide an API that supports JSON, XML and HTML, and we'll do our best to ensure that ZAP plays well with other applications with more changes in future releases. Psiinon On Sun, Jun 5, 2011 at 11:42 PM, Andre Gironda <andreg@gmail.com> wrote: > On Sun, Jun 5, 2011 at 8:31 AM, psiinon <psiinon@gmail.com> wrote: > > Have you tried the OWASP Zed Attack Proxy - > > https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project? > > It is open source and completely free (there is no paid for 'pro' > version). > > Its also intended to be a community project - so we encourage > involvement. > > If you submit good quality code then you'll get commit access :) > > Psiinon - OWASP ZAP Project Lead. > > Any intent to improve the wavsep.googlecode.com or > wivet.googlecode.com results from ZAP? > > ZAP scores worse than both Andiparos and Paros on SQLi categories, and > worse than most tools in other categories when run against WAVSEP. > It's also one of the worst crawlers, as seen in its WIVET results. > > Many tools such as W3AF can export their findings as XML (and their > request data as HTML,Ajax,Ruby,Python), which can be imported into The > Dradis Framework (which outputs its own XML, or to HTML, Word, or > Mediawiki). Burp Pro Scanner can export its data as XML and HTML, and > so does the "analyse target" tool -- plus you can save > request/response data in Repeater and store session files that contain > this data. Fiddler can save a SAZ file full of request/response data > and export as a variety of Microsoft Internet Explorer and Visual > Studio XML formats. Do you have any plans to make ZAP more extensible > in these ways? >