[WEB SECURITY] Looking for Addressing some Questions

[Randal L. Schwartz]

>>>>> "mohammad" == mohammad zoroufi <m_zoroufi at comp.iust.ac.ir> writes:

mohammad> 1)Are there any difference in functionality of spiders and crawlers?

Of course.

mohammad> 2)Looking for a spider/crawler that plays an importance role for me, I
mohammad> want 
mohammad> to navigate(scan) all the navigations existing in any web page containing
mohammad> all hyper links and submission buttons( I want to crawl whole site using 
mohammad> links between any resources between html page, server page and etc.).
mohammad> Whose 
mohammad> crawler has the above capability?

You *really* don't want to do that.  A crawler that submits "POST" links as
well as "GET" links will very likely change the state of the server, which is
no longer a benign act.  Semantically, "GET" should be idempotent (repeatable
because it doesn't change the server state), and is safe, which is how
crawlers can keep from damaging things.  (Stories have been told about
badly-designed interfaces like wikis that deleted content on a GET... oops!)

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn at stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

----------------------------------------------------------------------------
The Web Security Mailing List: 
http://www.webappsec.org/lists/websecurity/

The Web Security Mailing List Archives: 
http://www.webappsec.org/lists/websecurity/archive/
http://www.webappsec.org/rss/websecurity.rss [RSS Feed]