[WEB SECURITY] 2009 Top 25 Programming Errors

Arian J. Evans arian.evans at anachronic.com
Thu Jan 15 15:22:08 EST 2009

Glad you dropped by this dialogue tree Steve. I appreciate your efforts.

I'll add one last reply to my original points. I'm trying to keep my
objection focused and not let this drag down into a larger debate
despite my use of loaded rhetoric (which, I am aware, is the thing I
am decrying in your document :).

I am not a fan of strong language or cartoon antics in literature for
consumption by non-security folks. I think it alienates people and
makes us look like the proverbial security sandbags.

I deal directly with end consumers of software security services, and
have for decade. Today I deal directly with more consumers, from CSOs
to design architects to implementation developers, to third party
consumers of web services, than in my entire previous career. I would
go so far as to say I probably deal with more of such folks than the
majority of this community; certainly far more than most folks I see
commenting on SC-L.

I resisted posting to SC-L since I know the effect and responses I am
going to get from most of the folks in that community.

People today in the business world and in the software world ARE
actively aware of these issues. My own mother, an admitted
technophobe, even understands that software sucks, and that many of
her issues with her Windows + IE combination are the result of
security defects. She understands the web is not a safe place, and
that software isn't always safe. She knows there are bad people poking
around the Internet looking to make a buck off her any way they can,
not much from folks lingering in a Walmart parking lot in shady part
of town, except they have college degrees.

I vehemently believe that we do not need to parade around in Batman
suits shouting and waving our arms to get attention to this
problem-domain. I think it makes us look stupid and immature. It
ultimately makes us, and this problem domain, be taken less seriously.
"Yeah, yeah, you guys again. The sky is falling. Whatever."

I believe the language and the misguided "remediation cost" sections
of the Top 25 do just that.

I don't know who you got your input from, but I will be happy to try
and steer some business owners and software developers your way. Many
of them do not like security people, and especially ones who play
"Security Hero". That game should stay in your PS3.

Your current efforts with CWE and this "Top 25" are different by kind,
and not degree, from previous "Top" lists. I helped author the early
versions of the SANS Top 10 and later Top 20 so I well remember what
those constituted of. There was an attempt to base the original Top-N
lists off of metrics. Both attack data (what was being actively
targeted on the Internet) and flaw data (what the most common
configuration or missing patch issues were that we could find
prevalent on the Internet). I do not see this current Top 25 document
being data-centric at all. What data was used in the creation process?
Or was it simply the biased sample of a Mitre-mailing-list democracy?
(not criticizing you here, just asking)

The unique kind of thing you are addressing with this new standard is
that it this effort is dabbling in business case. Previous SANS lists
did not. This is a different kind of issue than server configs and IIS

I made many mistakes early on as a consultant, and many of the lessons
I learned were on how to communicate to effect change, to persuade, to
positively enable folks to get the right thing done. Telling people
struggling to build software that works and feeds their families, that
they have "Failed" is not the most productive approach. There is too
much finger-wagging going on in the software security community, and
not enough math and statistics telling people what the bad people are
doing, how it impacts them, what the odds are of it impacting them,
and what the best approaches are to avoid being a target or to
mitigate the impact.

The verbiage in the Top 25 smacks of "Security for Security's Sake" I
strongly dislike "security for security's sake". I had to deal with
those kind of people on the receiving end before I sold out and joined
the security community and I do not remember my experiences fondly.

My motive on this list was to push a "competing" standard. Via WASC or
OWASP. CWE for better or for worse draws a very academic crowd and I
want something simple and effective that is controlled by people in
the actual hands-on web security community. I have nothing against CWE
and if I had more hours in the day I'd actively contribute. Heck -- I
proposed something very similar to CWE years ago at NIST. I was very
frustrated by lack of taxonomy and hierarchy confusion between uses of
Risk, Threat, Attack, Weakness, and Vulnerability, and your efforts
here are long overdue.

We still run into pockets of folks that "don't get it" or "aren't
aware" but it is very different today than five years ago. It doesn't
take long for them to "get it". They can Google subject matter, talk
to buddies, read the news, and ramp up quickly.

You have written and released a standard. I personally think the value
of "awareness" is not as great or as needed today. Maybe two years
ago, for sure five years ago, but this effort is a bit late to the
game. I think that this thing needs to be approached and treated as a
minimum standard of due care (in testing, building, measuring) because
that is how it's going to be used.

I think there is a substantial difference between a report filled with
"Failures" and unjustified remediation cost implications and a
positive report describing weaknesses, the attacks that leverage those
weaknesses, and degrees of remediation options with some guidelines
how to evaluate cost. Security people shoot themselves in the foot by
telling business owners and developers that they have failed, and that
it won't cost much to fix. (They usually don't have any idea what it
will cost to fix). Sometimes they don't need to fix their software.
Sometimes it isn't cost effective to fix their software. There are
various kinds and degrees of mitigation and they all have a place.

The bottom line is we are not improving software to shore up security
implications. That notion is ridiculous. We are changing our software
programming patterns to mitigate attacks or potential attacks that
have security implications....that may lead to business implications.
(Or, in the case of government agencies, security implications in and
of themselves may be the issue, but this is the not the normal
business case.)

I think telling people that they have "Failed to validate input" is
also a silly idea. Especially given the world we live in today and the
direction modern software is taking on the Internet. It is hard to
write software and not know that it is a good idea to strongly type
your data. Pointing out that a given data typing, or no data typing,
is a weakness that leads to specific exposures is an effective way to
communicate the issue. But I find that **rarely** do people,
especially business owners, act on the notion that they have "Failed
to validate input". Or even care. They will act on specific examples
of attacks that their software is susceptible to, by enhancing the
weakness(es) that lead to that exposure.

Maybe that is a different document. Maybe it is a revised version of
what you have created.

I do not want to use your document and language for communicating with
my clients, but the current initiative guarantees that we all will be
using it as a standard.

I got a lot of offline commentary from peers to this exact point. I do
not think I am alone here.

As always, thanks for your reasonable considerations and responses
Steve, I'm not always so fair myself,

Arian Evans

Anti-Gun/UN people: you should weep for
Mumbai. Your actions leave defenseless dead.

"Among the many misdeeds of the British
rule in India, history will look upon the Act
depriving a whole nation of arms, as the
blackest." -- Mahatma Gandhi

On Thu, Jan 15, 2009 at 10:21 AM, Steven M. Christey
<coley at linus.mitre.org> wrote:
> Sorry to step in late, I wasn't subscribed to this list.
> Arian, your passionate arguments and concerns are understood and well
> appreciated, and (un)fortunately most of them anticipated.  I think we're
> all striving toward the same goal of more secure software.
> Backdrop: I'm the primary author of the Top 25 document as posted on the
> CWE web site.  We worked closely with SANS, but we were the authors of the
> document. SANS helped bring in other contributors, and without them, there
> would be no USA Today, BBC, etc. attention paid to this.  While MITRE is a
> not-for-profit company and SANS is for-profit, we have worked together on
> these kinds of efforts for almost 10 years now.
> I'm exceedingly pleased with all the attention this has gotten.  It
> appears that a lot of consumers don't even know that programmer errors are
> the root cause of so many incidents out there.
> As Chris Eng pointed out, the descriptions were informal in order to
> gather attention of developers, who were a primary target audience (and
> based on delicious bookmarks and others, looks like they're hearing it).
> This was not a rash decision; I was genuinely concerned whether this was
> the right approach and asked the Top 25 group for their feedback on this,
> and nearly all of the people who expressed an opinion were in favor.
> There's obviously some backlash, but with such a diverse audience, that's
> going to happen.
> As Chris also pointed out, the CWE web site has more technical content
> that may be more appropriate for programmers.  The linkage to "pure" CWE
> is not sufficient on the current top 25 page, so we're working on that
> piece.
> I do think of the Top 25 as an awareness tool and as a huge stick for
> customers to use to start asking for more secure software.  It looks like
> developers are hearing about this.  It is also prompting them to speak out
> about how the push to release overrides security - OK so it's only in the
> blogosphere but there you go.
> Maybe the message is too simplistic, but I don't think it's a coincidence
> that Paul Kurtz, an author of the US National Strategy to Secure
> Cyberspace, asked "What took you so long [to come up with this list]?"
> He, and many others, probably view this kind of concrete effort as long
> overdue from the security community.
> Gary McGraw has also posted on the dangers of the Top 25.  And I think
> that's perfectly fine.  The Top 25 gets our foot in the door for the REAL
> change in language for procurement.
> If you look at the text of the New York State procurement language at
> http://www.sans.org/appseccontract/ you will quickly see that the Top 25
> is one bullet.
> By the way, the OWASP efforts for contract language are also very strong.
> I suggest promoting these efforts *NOW* while people are still paying
> attention.  (We should probably link to them from the Top 25 page,
> actually...)
> Maybe I'm wrong in taking such a long and optimistic view, but there you
> go.
> - Steve

Join us on IRC: irc.freenode.net #webappsec

Have a question? Search The Web Security Mailing List Archives: 

Subscribe via RSS: 
http://www.webappsec.org/rss/websecurity.rss [RSS Feed]

Join WASC on LinkedIn

More information about the websecurity mailing list