[WEB SECURITY] Repository of site URL structures?

Chris Weber chris at casabasecurity.com
Tue Jun 21 14:36:47 EDT 2011

> There have been multiple situations where I've needed example of ! and ;
> as URL delimeters (which I've seen before but lack urls for), or @ within
> a URL (not in the context of user at domain.com auth). Or urls using comma's
> such as http://site/foo?12,12,12 .
> I am just looking for a central repository that I can point people to.
> > https://github.com/cweb/iri-tests/blob/master/tests.xml
> >
> > Webkit also has a testing suite at
> > http://trac.webkit.org/browser/trunk/LayoutTests/fast/url/ Note: I'm in
> > process of incorporating all of these tests into my test.xml above.
> Cool this is helpful thanks.

Those test cases are included in both of these repositories.  Webkit's is at
least stable but spread out across as arrays in a bunch of js files.  The
test suite itself is only concerned with the DOM parsing.  It's nice and
portable so you can easily run it in any browser.

My tests.xml will be changing a lot over the next few weeks to include as
many of Webkit's tests as I can plus others.  Contributions are welcome!
Some goals of mine are to include all of the tests in one XML file with a
unique id per test, plus an expected result.  The reason you see a domain
name like iris.test.ing is because I use a custom DNS zone in my test

> > Everyone is definitely not following the RFC guidelines consistently.  I
> > built a test harness that correlates the DOM parsing of these URIs with
> > HTTP request and the DNS queries.  The differences are dramatic in some
> > cases.
> So how come we haven't seen more advisories/bugs from you? Surely there
> are tons to be found :)

I expect the same :) But there's a lot of work to do still.  Right now it
seems like interop bugs mostly, and the exploit scenarios seem more
distributed, like depending how apps or security WAFs/filters/what-have-you
handle the strings.

Consider the following test cases, can you think of any 'general-purpose'

Test Case: http://0028.iris.test.ing;g

The DOM parsing is different in each FF, IE, and Opera - while both Safari
and Chrome error.  FF drops the ";g", Opera uses it in the path, and IE uses
it in the hostname...

Scheme    Hostname                 Path    Browser
http:          0028.iris.test.ing       /         Firefox/4.0.1
http:          0028.iris.test.ing       /;g      Opera/9.80
http:          0028.iris.test.ing;g               MSIE 7.0

But the raw HTTP request is interesting because Firefox does it differently
than its DOM parsing.  Neither Chrome, Safari, or IE even make an HTTP

Path    Browser
/;g       Firefox/4.0.1
/;g       Opera/9.80

Test Case: http://0029.iris.test.ing;./g

In this slightly different case Firefox has changed its handling of the ";"
trailing the hostname, and treats it instead as part of the path.

Scheme       Hostname                Path    Browser
http:            0029.iris.test.ing      /;./g    Firefox/4.0.1
http:            0029.iris.test.ing      /;./g    Opera/9.80
http:            0029.iris.test.ing;.    g          MSIE 7.0

Similar results as above for the HTTP request.

Path    Browser
/;./g    Firefox/4.0.1
/;./g    Opera/9.80

There's many more edge cases.   Check out the DOM parsing results of the
following test case:


Path                     Browser
/foo%7Cbar/    Chrome/12
foo%7Cbar/      MSIE 7.0
/foo|bar/          Opera/9.80
/foo|bar/          Safari/5.0.5
/foo|bar/          Firefox/4.0.1

But the more interesting thing here is that the raw HTTP request doesn't
match for Safari:

Path                     Browser
/foo%7Cbar/    Chrome/12
/foo%7Cbar/    MSIE 7.0
/foo|bar/          Opera/9.80
/foo%7Cbar/    Safari/5.0.5
/foo|bar/          Firefox/4.0.1

In this case Safari's DOM 'path' property is different than the raw HTTP
request 'path' it generates to fetch the resource.


More information about the websecurity mailing list