[WEB SECURITY] WASC Threat Classification vs. OWASP ASVS
Arian J. Evans
arian.evans at anachronic.com
Wed Jul 15 21:59:38 EDT 2009
You can always show up late to the dance, Jeff.
I disagree about ASVS: I complimented ASVS because it is a process of
sorts, call it a process framework or a program guide (or whatever):
it gives people some tactical guidance on what to do that is not part
of existing classification systems like TC or CWE. So that is good.
Weaknesses are the flip-side of the coin to controls, so organizing
around weaknesses is testing controls (if I understand your definition
of controls, which those Aspect Security guys have been helping me to
grok). I think we are in agreement here.
BB and WB testing have different goals. Hence the different terms and
different ways to test.
Attack-based testing is useful, highly useful, in BB, for finding
omissions (of controls or defensive coding against a specific attack),
for whatever the reason you have omissions. BB can find "commissions
in code" in some cases -- e.g.- a broken control, but BB is not very
helpful at explicitly identifying the *reason* (the commission). It
only helps you identify that there is an omission (the code is not
defending itself). Whether or not that is the result of a commission
(broken code), or an omission (no control spec, or flawed design), is
best identified by WB.
WB is exceptionally effective for finding explicit commissions in
code, both in validation of control implementation, and through
validation of routines.
ASVS has places where the terms are more control-based than
attack-based, hence my recommendation that you could plug WASC/TC into
ASVS if you wanted to add more of an attack-centric "checklist" or
guideline go follow for BB. That's all.
If you think there is a better way to do BB from experience -- I am
all ears. And I'm not saying that as a flippant challenge; I'm
genuinely curious if you find an entirely non-attack centric approach
to BB effective.
I would say that security BB testing today is predominantly (syntax)
attack-based, and secondarily control or weakness based, most of which
falls under what we call semantic or "business logic" testing, which
are still verified by "attacks". As syntax attacks go away, assuming
they will start fading away, these ratios might change in the future,
but for now, the reflect the rations of types of exploitable issues in
The 700 CWE largely do not map to the common, exploitable weaknesses
that TC aims to capture. Many of the implementation-level weaknesses
do not apply to web apps, or do not create exploitable conditions.
However, I am not in charge of TCv2, so I should not really speak for
it. (sorry, Robert, your floor)
On Wed, Jul 15, 2009 at 6:22 PM, Jeff Williams<planetlevel at gmail.com> wrote:
> Glad to see this discussion. Sorry I’m late.
> ASVS is not a process guide at all. ASVS defines the set requirements for
> security controls that need to be verified in an appsec scan/code
> review/pentest/architecture review. The process you use to do this
> verification is up to you. To allow organizations to compare the coverage
> of different application security services in the market, ASVS has 4 levels:
> 1. automated, 2. manual, 3. architecture, and 4. internal. Very few
> commercial applications are reviewed beyond level 2.
> Philosophically, ASVS (and OWASP for that matter) doesn’t really care
> whether you use a black-box or white-box technique to verify the
> requirements. I suggest you use the cheapest approach on a
> requirement-by-requirement basis depending on the particular application to
> be verified. Any other approach will waste money by definition.
> Organizing security reviews around security controls makes sense (to me
> anyway). It’s just easier to understand the coverage. It’s *possible* to
> organize these efforts around attacks and weaknesses, but there are so many
> of them that completeness becomes very difficult. Is there a plan to cover
> the 700+ CWE in the TC?
> From: arian.evans at gmail.com [mailto:arian.evans at gmail.com] On Behalf Of
> Arian J. Evans
> Sent: Monday, July 13, 2009 3:37 PM
> To: Roger Munk; websecurity at webappsec.org
> Subject: Re: [WEB SECURITY] WASC Threat Classification vs. OWASP ASVS
> The WASC "threat classification" is a mix of both attack-nodes and
> weakness-nodes, largely from a blackbox perspective.
> ASVS is primarily Whitebox and a process and "requirements" document. The
> WASC 24 covers none of that.
> You could use both. ASVS as your process guide, but cover all the of the
> WASC/24 as part of your BB checklist of "things to test for".
> OWASP tends to be a whiteboxed focused organization, and WASC tends to be a
> black-box focused organization, if that helps clarify for you.
> The ASVS has "requirements" which WASC does not. Many are good, but some are
> arbitrary requirements, like "verify validation is done with whitelists".
> You have to take these types of absolutes provided without any business
> context with a grain of salt. Sometimes whitelists work, and sometimes they
> are just not feasible, and sometimes blacklists actually work better (not
> often, but sometimes). At the end of the day the business context is
> everything. Aside from that stuff, ASVS looks like a great "process guide"
> to start with if you want to do more than blackbox test.
> Where the WASC being attack-node centric does not have any "you must design
> your system like this". It is mostly a list of "you are weak to Attack Type
> X". Even systemic weaknesses (like a weak authorization system) are usually
> described in some form of a parameter-tampering attack.
> Does that answer?
> Arian Evans
> On Mon, Jul 13, 2009 at 11:09 AM, Roger Munk <roger.munk at gmail.com> wrote:
> I'm putting together a requirements list for black box web pen testing
> and want to include a standards requirement. I've looked intothe WASC
> Threat Classification and OWASP's ASVS. The former seems to focus on
> high level threats, while the latter on testing controls present in
> the app. With the release of version two of the threat classification,
> which standard is more appropriate to use for web app pen testing and
> Join us on IRC: irc.freenode.net #webappsec
> Have a question? Search The Web Security Mailing List Archives:
> Subscribe via RSS:
> http://www.webappsec.org/rss/websecurity.rss [RSS Feed]
> Join WASC on LinkedIn
Join us on IRC: irc.freenode.net #webappsec
Have a question? Search The Web Security Mailing List Archives:
Subscribe via RSS:
http://www.webappsec.org/rss/websecurity.rss [RSS Feed]
Join WASC on LinkedIn
More information about the websecurity