On Fri, 6 Jan 2012, Hill, Brad wrote:

(disclaimer: I'm late to this discussion and haven't seen all the posts.)

The industry does this all the time, for example by accounting for
non-standard browser behavior with XSS defenses (as reflected in the XSS
cheat sheet, HTML5 Security Cheatsheet, etc).  Not that this has been
wildly successful given all the XSS variants that keep cropping up that
only apply to 1 or 2 browsers, but it seems like a reasonable approach
short of normalizing inputs/outputs in a predictable fashion
(non-standard), or modifying the standards to design out these kinds of
problems in the first place (unrealistic, infeasible, and maybe
impossible).  Anti-virus products, IDS, mail scanners, and other products
also have to wrestle with the same issue of accounting for
popular-but-non-standard behaviors in order to work properly.  Should they
be responsible for this kind of protection?  Philosophically speaking, no;
but operationally speaking, they have no choice.

• Steve

Hi Steve,

I appreciate what you're saying, and yes, many de facto hacks exist to
protect people.  The more I think about it, the more I think that
nearly all web frameworks probably do this header namespace squashing.
The squashing clearly creates security problems, as we have discussed.

Let us balance that then against the security down sides of making a
change to web frameworks.  If today, "X^MyHeader" and "X*MyHeader" are
treated as equivalent, what is the harm in not treating them as
equivalent in the future?  The only header an application can search
for is "X_MYHEADER".  If we make such a change, they're still going to
be looking up "X_MYHEADER".  Sure, applications sending that header
would need to be more careful about what format of the header they are
sending (such as sending "X_MyHeader" in all cases) , but that's not a
security issue, just a backward compatibility one.  How would the
application be harmed in a security sense?

Brad's suggested approach would be to start restricting the types of
headers that can be sent to begin with.  So if XMLHttpRequest simply
blocked the less common variants of these headers (essentially
rejecting "Accept[!#$%&'*+-._^`|~]Encoding", and so on), then great,
we've prevented these headers from working their way in to these
requests at the web layer.  But as I've pointed out, there are other
situations where this approach would need to be taken.  Essentially,
to enforce it, all servers would need to start rejecting the uncommon
characters.  This amounts to changing the HTTP spec.  From a security
perspective, this clearly prevents a lot of problems, but changing
HTTP header names from a permissive set of characters to a restrictive
set is difficult.  Any legacy servers will continue to be "vulnerable"
in a sense, basically forever.

Both changes are very difficult from an implementation perspective,
but if there are no security downsides to fixing web frameworks
(allowing most/all HTTP headers), then that seems like the better
approach to me, not to mention easier.

tim

Looks like this is a quite old issue, dating to 2007 with Flash:

http://kuza55.blogspot.com/2007/07/exploiting-reflected-xss.html

Adobe patched, it but the W3C editors for XHR won't:

http://lists.w3.org/Archives/Public/public-webapps/2012JanMar/0024.html

In my testing, Apache did not handle all "bypassed" headers the same
way. For example, it was not possible to submit a modified
"Transfer-Encoding" header (eg. Transfer.Encoding; Transfer*Encoding;
Transfer Encoding), and have Apache handle the request with the
specified encoding.

The environment variable HTTP_TRANSFER_ENCODING is still set, but the
actual request isn't treated with the provided encoding.

If you could get the browser to send a Transfer*Encoding Header, and
if Apache treats it like normal, then you could possibly abuse it for
Request Splitting/Smuggling.
http://www.mindedsecurity.com/MSA01240108.html

On Thu, Jan 5, 2012 at 4:13 AM, Kousuke Ebihara kousuke@co3k.org wrote:

Ack.  I am learning PHP today to write some test cases and I find that PHP also uses the "-" to "_" conversion before exposing HTTP headers, so this server vulnerability is much more widespread than just ancient CGIs.  :(

Yeah, after our conversation a while back, I looked into a bit more
and realized that just about every web framework does this foolish
transliteration of characters to _.  Almost every special character is
converted like this.  The CGI spec only requires - be transliterated
to _ and doesn't require anything else be.

Of course those other frameworks don't have to adhere to CGI, but even
if their developers wanted to claim "oh, we're just being backward
compatible with CGI", they don't have a sound argument.

tim

In your investigation did you find an instance of a framework converting single header names
such as _Referer or _Host?

Regards,

• Robert

On Thu, 2 Feb 2012, Tim wrote: