[WEB SECURITY] WASC Threat Classification vs. OWASP ASVS

Jeff Williams planetlevel at gmail.com
Wed Jul 15 22:57:15 EDT 2009


> Weaknesses are the flip-side of the coin to controls, so organizing
> around weaknesses is testing controls (if I understand your definition
> of controls, which those Aspect Security guys have been helping me to
> grok). I think we are in agreement here.

Agree. I put weaknesses into these buckets: missing controls, broken
controls, unused controls, and misused controls.

> ASVS has places where the terms are more control-based than
> attack-based, hence my recommendation that you could plug WASC/TC into
> ASVS if you wanted to add more of an attack-centric "checklist" or
> guideline go follow for BB. That's all.

Agree. When you're verifying a control isn't broken, making sure it doesn't
have any of the common weaknesses is a pretty good start.  Would be
interesting to see how much coverage the TC weaknesses have compared to the
ASVS requirements.

> If you think there is a better way to do BB from experience -- I am
> all ears. And I'm not saying that as a flippant challenge; I'm
> genuinely curious if you find an entirely non-attack centric approach
> to BB effective.

Since reviews are always time-limited, prioritization is critical. So
selecting the BB tests (and WB tests) you are going to perform based on the
security controls you are trying to verify makes sense to me. Too many
testers wander around trying attack after attack without any real roadmap of
what's important to the business. I think using controls to focus the BB
tests performed helps ensure completeness. Could just be my background.

This is part of the transformation from an industry focused on proving that
appsec is a problem to one that is genuinely focused on helping customers
improve. Historically, BB testing hasn't worried about completeness, but
it's time for that to change. I think this is what Brian Chess was trying to
get across in his pentesting rant from last year.

> I would say that security BB testing today is predominantly (syntax)
> attack-based, and secondarily control or weakness based, most of which
> falls under what we call semantic or "business logic" testing, which
> are still verified by "attacks". As syntax attacks go away, assuming
> they will start fading away, these ratios might change in the future,
> but for now, the reflect the rations of types of exploitable issues in
> applications.

If you're suggesting that because BB is more like attacks that it is
therefore more effective at finding today's exploitable issues, I disagree
with your logic.

--Jeff


> 
> --
> Arian Evans
> 
> 
> 
> 
> 
> On Wed, Jul 15, 2009 at 6:22 PM, Jeff Williams<planetlevel at gmail.com>
> wrote:
> > Glad to see this discussion.  Sorry I’m late.
> >
> >
> >
> > ASVS is not a process guide at all.  ASVS defines the set
> requirements for
> > security controls that need to be verified in an appsec scan/code
> > review/pentest/architecture review.  The process you use to do this
> > verification is up to you.  To allow organizations to compare the
> coverage
> > of different application security services in the market, ASVS has 4
> levels:
> > 1. automated, 2. manual, 3. architecture, and 4. internal.  Very few
> > commercial applications are reviewed beyond level 2.
> >
> >
> >
> > Philosophically, ASVS (and OWASP for that matter) doesn’t really care
> > whether you use a black-box or white-box technique to verify the
> > requirements. I suggest you use the cheapest approach on a
> > requirement-by-requirement basis depending on the particular
> application to
> > be verified.  Any other approach will waste money by definition.
> >
> >
> >
> > Organizing security reviews around security controls makes sense (to
> me
> > anyway). It’s just easier to understand the coverage. It’s *possible*
> to
> > organize these efforts around attacks and weaknesses, but there are
> so many
> > of them that completeness becomes very difficult.  Is there a plan to
> cover
> > the 700+ CWE in the TC?
> >
> >
> >
> > --Jeff
> >
> >
> >
> >
> >
> > From: arian.evans at gmail.com [mailto:arian.evans at gmail.com] On Behalf
> Of
> > Arian J. Evans
> > Sent: Monday, July 13, 2009 3:37 PM
> > To: Roger Munk; websecurity at webappsec.org
> > Subject: Re: [WEB SECURITY] WASC Threat Classification vs. OWASP ASVS
> >
> >
> >
> > The WASC "threat classification" is a mix of both attack-nodes and
> > weakness-nodes, largely from a blackbox perspective.
> >
> > ASVS is primarily Whitebox and a process and "requirements" document.
> The
> > WASC 24 covers none of that.
> >
> > You could use both. ASVS as your process guide, but cover all the of
> the
> > WASC/24 as part of your BB checklist of "things to test for".
> >
> > OWASP tends to be a whiteboxed focused organization, and WASC tends
> to be a
> > black-box focused organization, if that helps clarify for you.
> >
> > The ASVS has "requirements" which WASC does not. Many are good, but
> some are
> > arbitrary requirements, like "verify validation is done with
> whitelists".
> >
> > You have to take these types of absolutes provided without any
> business
> > context with a grain of salt. Sometimes whitelists work, and
> sometimes they
> > are just not feasible, and sometimes blacklists actually work better
> (not
> > often, but sometimes). At the end of the day the business context is
> > everything. Aside from that stuff, ASVS looks like a great "process
> guide"
> > to start with if you want to do more than blackbox test.
> >
> > Where the WASC being attack-node centric does not have any "you must
> design
> > your system like this". It is mostly a list of "you are weak to
> Attack Type
> > X". Even systemic weaknesses (like a weak authorization system) are
> usually
> > described in some form of a parameter-tampering attack.
> >
> > Does that answer?
> >
> > --
> > Arian Evans
> >
> >
> >
> > On Mon, Jul 13, 2009 at 11:09 AM, Roger Munk <roger.munk at gmail.com>
> wrote:
> >
> > I'm putting together a requirements list for black box web pen
> testing
> > and want to include a standards requirement. I've looked intothe WASC
> > Threat Classification and OWASP's ASVS. The former seems to focus on
> > high level threats, while the latter on testing controls present in
> > the app. With the release of version two of the threat
> classification,
> > which standard is more appropriate to use for web app pen testing and
> > why?
> >
> > Thanks,
> >  Roger
> >
> > ---------------------------------------------------------------------
> -------
> > Join us on IRC: irc.freenode.net #webappsec
> >
> > Have a question? Search The Web Security Mailing List Archives:
> > http://www.webappsec.org/lists/websecurity/archive/
> >
> > Subscribe via RSS:
> > http://www.webappsec.org/rss/websecurity.rss [RSS Feed]
> >
> > Join WASC on LinkedIn
> > http://www.linkedin.com/e/gis/83336/4B20E4374DBA
> >
> >


----------------------------------------------------------------------------
Join us on IRC: irc.freenode.net #webappsec

Have a question? Search The Web Security Mailing List Archives: 
http://www.webappsec.org/lists/websecurity/archive/

Subscribe via RSS: 
http://www.webappsec.org/rss/websecurity.rss [RSS Feed]

Join WASC on LinkedIn
http://www.linkedin.com/e/gis/83336/4B20E4374DBA



More information about the websecurity mailing list