[WEB SECURITY] Crawl Form based authenticated website

ruby glow rubyglow.prideindia at gmail.com
Mon Jul 23 03:05:18 EDT 2012

Hi all,

I am trying to create a small framework which automates the general web
security assessment. this is a web based framework.

I have used wget to perform spidering. it was working fine. but now i am
struck up at crawling a website which is authenticated. The tester has the
login credentials but how do I pass these credentials to the wget is the
issue here. I have seen that wget has a way to creating cookies and using
them for further spidering, but thats not working for all kinds of web
applications. Sometimes its able to store the cookie values, but sometimes
it fails. Most of the time, I am seeing that the application is failed when
it is webservices based.

Even I tried to crawl my own gmail account by providing gmail username and
password to wget. It fails to generate cookies.

Could someone please suggest any other good open source command line tool
which can crawl form based authenticated websites.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webappsec.org/pipermail/websecurity_lists.webappsec.org/attachments/20120723/ed53d9ff/attachment-0003.html>

More information about the websecurity mailing list