[WEB SECURITY] Looking for Addressing some Questions

Mon Sep 25 11:25:15 EDT 2006

Randal is right, but I would like to add a couple caveats and expand on
what he said a bit.

The difference between GET and POST are in submission encoding, only.
That said, the answer to this discussion is rooted in deciphering the
intent of the HTTP and HTML specs.

The HTML 4.0 spec says...

"If the processing of a form is idempotent (i.e. it has no lasting
observable effect on the state of the world), then the form method
should be GET. Many database searches have no visible side-effects and
make ideal applications of query forms."

...

"If the service associated with the processing of a form has side
effects (for example, modification of a database or subscription to a
service), the method should be POST."

"Should be" sounds an awful lot like a suggestion to me. Stronger
language like "must be" is not used. In practice, both can be (and are)
used interchangeably. Some apps use all POST in order to hide the
querystring from a non-proxied browser user entirely. Others use GET for
everything. It's against the intent of the spec, but there's a lot of
this stuff out there getting used every day in the wild.

The HTTP 1.1 spec bears this out:

"Naturally, it is not possible to ensure that the server does not
generate side-effects as a result of performing a GET request; in fact,
some dynamic resources consider that a feature. The important
distinction here is that the user did not request the side-effects, so
therefore cannot be held accountable for them."

I believe that the context being discussed in the spec would be an
example like an order form, where hitting the back page and reloading
automagically (as in the case of a GET) would mean that you would
inadvertantly order another widget.

That said, while testing web apps I've occasionally run into some
bizarre code. On one occasion I was testing a bank interface web
interface and it responded to both GET and POST. Submitted as a POST,
how the app worked from the web interface, there was proper
authorization for each request. If you were to use a proxy to reformat
the POST into a GET... that all went out the window. You could touch
*and modify* the accounts of other users via a GET, which should have
been idempotent.

What this says to me is that automated spiders are ineffective at
mapping an entire site unless it's 100% HTML and follows the RFC
conventions entirely. Not only because of the GET/POST issues we've been
discussing, but also because of dynamic screen elements like AJAX, Flash
and ActiveX and other client-side technologies. For "modern" sites, some
manual intervention is usually necessary to completely spider your
target. There is no substitute for a browser and HTTP proxy in competent
hands.

Best regards
Adam

On Mon, 2006-09-25 at 13:18 +0300, Ory Segal wrote:

> Hi Randal,
> 
> While I agree that following links (or forms using GET) is less
> intrusive than sending POST requests that might change things on the
> server side, you have to remember that most applications include links
> that are behind forms that use the POST method. The best example would
> be login forms, which usually use POST.
> 
> -Ory Segal
> 
>  
> 
> -----Original Message-----
> From: Randal L. Schwartz [mailto:merlyn at stonehenge.com] 
> Sent: Saturday, September 23, 2006 4:34 PM
> To: mohammad zoroufi
> Cc: websecurity at webappsec.org
> Subject: Re: [WEB SECURITY] Looking for Addressing some Questions
> 
> >>>>> "mohammad" == mohammad zoroufi <m_zoroufi at comp.iust.ac.ir> writes:
> 
> mohammad> 1)Are there any difference in functionality of spiders and
> crawlers?
> 
> Of course.
> 
> mohammad> 2)Looking for a spider/crawler that plays an importance role 
> mohammad> for me, I want to navigate(scan) all the navigations existing 
> mohammad> in any web page containing all hyper links and submission 
> mohammad> buttons( I want to crawl whole site using links between any 
> mohammad> resources between html page, server page and etc.).
> mohammad> Whose
> mohammad> crawler has the above capability?
> 
> You *really* don't want to do that.  A crawler that submits "POST" links
> as well as "GET" links will very likely change the state of the server,
> which is no longer a benign act.  Semantically, "GET" should be
> idempotent (repeatable because it doesn't change the server state), and
> is safe, which is how crawlers can keep from damaging things.  (Stories
> have been told about badly-designed interfaces like wikis that deleted
> content on a GET... oops!)
> 
> --
> Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777
> 0095 <merlyn at stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
> Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
> See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl
> training!
> 
> ------------------------------------------------------------------------
> ----
> The Web Security Mailing List: 
> http://www.webappsec.org/lists/websecurity/
> 
> The Web Security Mailing List Archives: 
> http://www.webappsec.org/lists/websecurity/archive/
> http://www.webappsec.org/rss/websecurity.rss [RSS Feed]
> 
> 
> 
> ----------------------------------------------------------------------------
> The Web Security Mailing List: 
> http://www.webappsec.org/lists/websecurity/
> 
> The Web Security Mailing List Archives: 
> http://www.webappsec.org/lists/websecurity/archive/
> http://www.webappsec.org/rss/websecurity.rss [RSS Feed]
> 
> 

     Adam Muntner, CISSP | c: 602-793-5969 
                 Partner | f: 866-272-8194
         QuietMove, Inc. | w: http://www.quietmove.com
Securing the Nexus Between People, Technology, and Information.
                       ((Q))

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webappsec.org/pipermail/websecurity_lists.webappsec.org/attachments/20060925/9cbf390c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 191 bytes
Desc: This is a digitally signed message part
URL: <http://lists.webappsec.org/pipermail/websecurity_lists.webappsec.org/attachments/20060925/9cbf390c/attachment.asc>