[WEB SECURITY] 2009 Top 25 Programming Errors

Arian J. Evans arian.evans at anachronic.com
Wed Jan 14 19:13:14 EST 2009


I'm not going to let you drag this down into a pointless semantic
debate. That's what the SC-L list is for.

My initial point is that this your new standard, whether your realize
it or not. SANS and CWE will see to that.

If you like it; cool. I do not.

My initial post also states what bothers me.

I said I would like to see something better from OWASP or WASC. I am
not interested in a line-by-line comparison of the verbiage with any
other legacy descriptions. I have always said that both the OWASP Top
10 and the TC 1.0 need some work. I am hoping TC 2.0 fixes TC 1.0
issues.

The things that suck are about the Top 25 are:

1.1 Negative language and classification
1.2 Implication that someone did something wrong or bad
1.3 Amateur language
1.4 Subjective and misleading details (buckets like "Remediation
Cost", testing recommendations, implementation & actionable-fix
details)

Questions that you should be asking:

2) Top 25 of what?
3) Top 25 to whom?
4) Top 25 for what use?
5) For what audience is this document intended?
6) What is the intended outcome of releasing this document as written?


People have been telling me that this is not a new standard. Even
Steven M naively wrote that.

SANS steps in and I'm sure CIS will be right behind. It's a
"standard". I didn't even need to see the quote to know what Allen
Paller would say.

I would like to see a different "standard" for testing software, or
measuring software, and also for building software. Now we have a
one-size fits all thingy extracted from CWE and championed by SANS,
and I am not a fan of it.

I did not say the whole thing is bad. I did not say any of the people
involved were bad people. I did not claim that anyone involved has
been bad, in the Santa-Claus-list sense of the word. Though I'm sure a
few have been. And I clearly said I expect Steven and the Mitre folks
to improve it over time.

I said what I said specifically, and now I have no more to say.

-- 
-- 
Arian Evans

Anti-Gun/UN people: you should weep for
Mumbai. Your actions leave defenseless dead.

"Among the many misdeeds of the British
rule in India, history will look upon the Act
depriving a whole nation of arms, as the
blackest." -- Mahatma Gandhi







On Wed, Jan 14, 2009 at 2:57 PM, Chris Eng <ceng at veracode.com> wrote:
> Hi Arian,
>
>
>
> Like you, I've dealt with lots of customers while delivering various
> security projects over the years.  I can't picture one of my current or
> former customers being insulted or offended by the Top 25 list, nor do I
> believe it would reinforce this stereotype of "amateur security
> professionals" that you believe is so pervasive.  Is it really just that
> opening "Discussion" paragraph for each Top 25 issue that's bothering you?
> I can sort of see where you're coming from in that it doesn't read like a
> polished deliverable, but that doesn't make it offensive.  I certainly don't
> intend to reproduce the content verbatim in my own reporting, but then
> again, that's not usually what one does with these types of lists.
>
>
>
> For the sake of comparison, here's the CWE description of XSS, linked from
> the Top-25 page:
>
>
>
> The software does not sufficiently validate, filter, escape, and encode
> user-controllable input before it is placed in output that is used as a web
> page that is served to other users.
>
>
>
> Cross-site scripting (XSS) vulnerabilities occur when:
>
>
>
> 1. Untrusted data enters a web application, typically from a web request.
>
> 2. The web application dynamically generates a web page that contains this
> untrusted data.
>
> 3. During page generation, the application does not prevent the data from
> containing content that is executable by a web browser, such as JavaScript,
> HTML tags, HTML attributes, mouse events, Flash, ActiveX, etc.
>
> 4. A victim visits the generated web page through a web browser, which
> contains malicious script that was injected using the untrusted data.
>
> 5. Since the script comes from a web page that was sent by the web server,
> the web browser executes the malicious script in the context of the web
> server's domain.
>
> 6. This effectively violates the intention of the web browser's same-origin
> policy, which states that scripts in one domain should not be able to access
> resources or run code in a different domain.
>
>
>
> Here is the WASC TC description of XSS:
>
>
>
> Cross-site Scripting (XSS) is an attack technique that forces a web
>
> site to echo attacker-supplied executable code, which loads in a
>
> user's browser. The code itself is usually written in HTML/JavaScript,
>
> but may also extend to VBScript, ActiveX, Java, Flash, or any other
>
> browser-supported technology.
>
>
>
> When an attacker gets a user's browser to execute his code, the
>
> code will run within the security context (or zone) of the hosting web
>
> site. With this level of privilege, the code has the ability to read,
>
> modify and transmit any sensitive data accessible by the browser. A
>
> Cross-site Scripted user could have his account hijacked (cookie
>
> theft), their browser redirected to another location, or possibly shown
>
> fraudulent content delivered by the web site they are visiting. Crosssite
>
> Scripting attacks essentially compromise the trust relationship
>
> between a user and the web site.
>
>
>
> Both go on to discuss reflected vs. stored, example code, etc. but I've
> omitted those for brevity.  I think both descriptions fairly describe the
> flaw, with a slight edge to CWE for the step-by-step commentary.  Both would
> benefit from some rewording (e.g. "A Cross-site Scripted user"?)
>
>
>
> -chris
>
>
>
>
>
>
>
>
>
>> -----Original Message-----
>
>> From: arian.evans at gmail.com [mailto:arian.evans at gmail.com] On Behalf Of
>
>> Arian J. Evans
>
>> Sent: Wednesday, January 14, 2009 5:19 PM
>
>> To: Chris Eng; websecurity at webappsec.org
>
>> Subject: Re: [WEB SECURITY] 2009 Top 25 Programming Errors
>
>>
>
>> Chris -- good point. I too have found that
>
>> insulting and confusing people is effective
>
>> in getting their attention.
>
>>
>
>> Professionally I have found this approach
>
>> to be less productive than others.
>
>>
>
>> The content I quoted is the primary
>
>> "Discussion" and the first content the
>
>> reader encounters under each "issue".
>
>>
>
>> If our goal is to persuade developers
>
>> and business owners, we can be far
>
>> more effective than this.
>
>>
>
>> If you like the doc and the negative
>
>> wording, cool, use it all you want.
>
>>
>
>> For those of you that do not like this
>
>> approach -- this was meant as a head's
>
>> up that this is how SANS is going to have
>
>> you doing business going forward.
>
>>
>
>>
>
>> --
>
>> --
>
>> Arian Evans
>
>>
>
>>
>
>> Anti-Gun/UN people: you should weep for
>
>> Mumbai. Your actions leave defenseless dead.
>
>>
>
>> "Among the many misdeeds of the British
>
>> rule in India, history will look upon the Act
>
>> depriving a whole nation of arms, as the
>
>> blackest." -- Mahatma Gandhi
>
>>
>
>>
>
>>
>
>> On Wed, Jan 14, 2009 at 12:51 PM, Chris Eng <ceng at veracode.com> wrote:
>
>> > Arian,
>
>> >
>
>> > You realize that the XSS description you quoted -- the one written in
>
>> > 2nd person with a casual tone -- is not the primary content?  If you
>
>> > click on the CWE-79 link (http://cwe.mitre.org/data/definitions/79.html)
>
>> > you get the technical, professional version that you probably expected
>
>> > to see, complete with examples, etc.  I think the paragraph you quoted
>
>> > is intended more to catch the developer's attention and get them to read
>
>> > further.  The individual CWE pages are generally pretty well written,
>
>> > and I don't think anyone expected that the one paragraph would be
>
>> > sufficient to cover XSS.
>
>> >
>
>> > -chris
>
>> >
>
>> >
>
>> >
>
>> >> -----Original Message-----
>
>> >> From: arian.evans at gmail.com [mailto:arian.evans at gmail.com] On Behalf
>
>> > Of
>
>> >> Arian J. Evans
>
>> >> Sent: Wednesday, January 14, 2009 3:36 PM
>
>> >> To: Andy Steingruebl
>
>> >> Cc: websecurity at webappsec.org
>
>> >> Subject: Re: [WEB SECURITY] 2009 Top 25 Programming Errors
>
>> >>
>
>> >> Andy,
>
>> >>
>
>> >> I am not in any way attacking or denigrating CWE
>
>> >> or Mitre. CWE is a fine effort by the first rate minds
>
>> >> over at Mitre.
>
>> >>
>
>> >> My comments are about this "Top 25" list.
>
>> >>
>
>> >> These are "standards" because they are being used as
>
>> >> a "standards" banner for products, contracts, RFPs, etc.
>
>> >> already and will continue to be like other Top SANS lists.
>
>> >>
>
>> >> As I've said to several folks: It is an amateur piece of
>
>> >> work that will simply reinforce the amateur nature of most
>
>> >> security professionals to business owners and seasoned
>
>> >> software developers. Apology to anyone whose kid I'm
>
>> >> calling ugly here, but your kid is ugly.
>
>> >>
>
>> >> The WASC TC is a better guideline of issues with web
>
>> >> software and in most ares more effective for communication.
>
>> >>
>
>> >> I know there are those that dislike these "Top N" lists
>
>> >> but the spirit of this Top 25 is fine in my book.
>
>> >>
>
>> >> I would like to see a more professional list published and
>
>> >> maintained by OWASP or WASC.
>
>> >>
>
>> >> My main criticism of this Top 25 list is with the content
>
>> >> and the descriptions. Fixable? Sure.
>
>> >>
>
>> >> The fact that this was released is such an unprofessional
>
>> >> state solidifies in my mind that Mitre and SANS are not
>
>> >> the folks to be maintaining this list.
>
>> >>
>
>> >> I do not want to deal with this document and the inevitable
>
>> >> reality that will stem from it. I do not want to respond to
>
>> >> RFPs that ask if you test for "Failure to Protect Web Pages"
>
>> >> and I do not want to have to build reports that report on
>
>> >> this list with this garbage content and I do not want to
>
>> >> have to explain why the remediation language, costs, and
>
>> >> confusing suggestions are inaccurate or misguided.
>
>> >>
>
>> >> Take the "Remediation Cost:" bucket. I used to use these
>
>> >> in my reports all the time, and agree with the spirit of intent
>
>> >> on including this in a "Top 25".
>
>> >>
>
>> >> However -- the implementation in the Top 25 is misguided
>
>> >> and useless as a result and should be removed.
>
>> >>
>
>> >> Half of the list is marked "Low" and there are even items
>
>> >> marked "Low to High". (? why bother)
>
>> >>
>
>> >> There is no explanation or justification for any of it other than
>
>> >> letting people know "this stuff is easy to fix".
>
>> >>
>
>> >> The reality of the cost of remediation is contextual to software
>
>> >> and a business capacity and situation. Any effort to communicate
>
>> >> cost should clearly communicate these realities and give the
>
>> >> reader some notion of context, what they need to consider
>
>> >> when evaluating cost, or simply be removed. Statistically
>
>> >> speaking, the more likely you are to have massive XSS issues
>
>> >> the more likely you are to have a hard time fixing it (because
>
>> >> you've got a mess of ASP classic spaghetti code that got
>
>> >> you where you are today). But I digress.
>
>> >>
>
>> >> I could go on criticizing but I won't. I will recommend that
>
>> >> WASC or OWASP make a new Top 10 or Top 25 list and
>
>> >> that we tackle the same spirit of intent, understanding that
>
>> >> it will instantly become a "standard".
>
>> >>
>
>> >> Barring that we are going to live with this document for
>
>> >> a while, and I am voicing my discontent.
>
>> >>
>
>> >> Steven @ Mitre is a smart guy and I bet this will improve
>
>> >> over time if he starts filtering whom he listens too and
>
>> >> who is allowed to contribute content. Or provides a guideline
>
>> >> for content copy. Given who will use this and how it will
>
>> >> be used this should be a very professional business document,
>
>> >> not security-nerd speak with Star Treck naming conventions.
>
>> >>
>
>> >> Folks who name their variables or servers R2D2 and C3PO
>
>> >> are probably not gonna agree with me here.
>
>> >>
>
>> >> If I had seen this mess up for review before it was
>
>> >> published I would have commented or contributed
>
>> >> much earlier.
>
>> >>
>
>> >>
>
>> >> ...
>
>> >>
>
>> >> Feel free to note that I am too unmotivated to start my
>
>> >> own project to make a new Top N list.
>
>> >>
>
>> >>
>
>> >> --
>
>> >> --
>
>> >> Arian Evans
>
>> >>
>
>> >> Anti-Gun/UN people: you should weep for
>
>> >> Mumbai. Your actions leave defenseless dead.
>
>> >>
>
>> >> "Among the many misdeeds of the British
>
>> >> rule in India, history will look upon the Act
>
>> >> depriving a whole nation of arms, as the
>
>> >> blackest." -- Mahatma Gandhi
>
>> >>
>
>> >>
>
>> >>
>
>> >>
>
>> >>
>
>> >> On Wed, Jan 14, 2009 at 11:17 AM, Andy Steingruebl
>
>> > <steingra at gmail.com>
>
>> >> wrote:
>
>> >> > On Wed, Jan 14, 2009 at 10:20 AM, Arian J. Evans
>
>> >> > <arian.evans at anachronic.com> wrote:
>
>> >> >>
>
>> >> >> For all of you ignoring this -- this is going to replace the OWASP
>
>> > Top
>
>> >> >> 10 and WASC TC 1.0 or 2.0 etc. That is the goal/agenda of SANS &
>
>> > CWE.
>
>> >> >> Begin press releases, beat the marketing campaign drums.
>
>> >> >
>
>> >> > How are any of these "Standards" anyway?  I'm not sure I understand
>
>> >> > what you're getting at here.  Are you saying there ought to be some
>
>> >> > standard that tells people what they have to check for and that
>
>> > ought
>
>> >> > to be the TC2.0?  If so, I think that is misguided as well.
>
>> >> >
>
>> >> > In either case though people are looking for requirements to stick
>
>> >> > into software purchase contracts to describe minimum security levels
>
>> >> > for something.  None of these lists is particularly well suited to
>
>> >> > that task, but at the same time the CC (Common Criteria) wasn't
>
>> >> > exactly fitting that need either.  Maybe you'd like to write a
>
>> >> > protection profile that your COTS software must meet?  Not me.
>
>> >> >
>
>> >> > So, while the language of the list of items might not be perfect, I
>
>> > do
>
>> >> > have a lot of respect for the CWE itself, as it does a pretty good
>
>> > job
>
>> >> > as a taxonomy.
>
>> >> >
>
>> >> > What are you looking to have produced to counter this?  How does the
>
>> >> > TC fit into that at all?
>
>> >> >
>
>> >> > --
>
>> >> > Andy Steingruebl
>
>> >> > steingra at gmail.com
>
>> >> >
>
>> >>

----------------------------------------------------------------------------
Join us on IRC: irc.freenode.net #webappsec

Have a question? Search The Web Security Mailing List Archives: 
http://www.webappsec.org/lists/websecurity/archive/

Subscribe via RSS: 
http://www.webappsec.org/rss/websecurity.rss [RSS Feed]

Join WASC on LinkedIn
http://www.linkedin.com/e/gis/83336/4B20E4374DBA



More information about the websecurity mailing list