[WASC-WAFEC] What should we change in WAFEC 2.0?

Ryan Barnett rcbarnett at gmail.com
Thu Jun 7 09:18:31 EDT 2012

Comments inline below.

From:  Ofer Shezaf <ofer at shezaf.com>
Date:  Wed, 6 Jun 2012 14:39:29 +0300
To:  <wasc-wafec at lists.webappsec.org>
Subject:  [WASC-WAFEC] What should we change in WAFEC 2.0?

> Since you are all very quiet, I understand that WAFEC 2 will solve my pain and
> needs only J
> To that end, let me start with summarizing issues raised in the previous
> discussions on the mailing list (which I actually went and readŠ).
> No specific order intended. This is what you wrote, though I must say I think
> it captures well the issues I am aware of and that generally speaking I agree
> with most.
> 1.       Remove non WAF related criteria for example around application
> delivery. 
> ·         While integrating a WAF with other solutions is compelling to the
> client, it is not directly about WAFs and is also unbounded. This does present
> the challenge deciding what is relevant to a WAF in border cases such as an
> SSO functionality

I recommend that we consider using a "Levels" approach similar to what OWASP
ASVS uses - http://code.google.com/p/owasp-asvs/wiki/ASVS.  This way, we can
group items and the user can be clear which items are considered "core" WAF
features and which ones provide added value.

> 2.       Update the list of threats covered

I agree that we should map to the WASC TC.  The OWASP German Chapter did
something similar with the WAF Best Practices document listing coverage for
the OWASP Top 10 2007 -
plication_Firewalls.  This will help to show which attack/vuln categories
WAFs excel at mitigating (injection flaws, info leakages) and which ones
they don't (authorization, logic flaws).

> 3.       Focus on customer use cases rather than how a WAF operates
> ·         I think there was some hidden controversy here as I read opinions to
> focus on ³technical² which I take to be opposite. I personally very much agree
> with this comment.

OK, I guess here is as good a place as any to state my two biggest issues I
think we need to address with WAFEC -

1) Minimum Requirements for WAF ­ whether we like it or not, most WAFEC
users tried to use it as  a "minimum requirements" document.  I can't tell
you how many times I was sent the speadsheet by a WAF prospect and asked how
our WAF "conformed" to itŠ  So, I do think that we need to re-organize the
data so that there is a clear distinction between what features a product
must have to be considered a WAF.  This is an important issue as there are
many other vendors in the FW/IPS space that tout that they can do the same
things as WAFs and that is just marketing BS.  For instance ­ in Larry
Suto's WAF Eval doc
b_Application_Firewalls.pdf) one of the general takeaways was that an IPS
configured with virtual patches from a DAST scanner would be just as
effective as a WAF.  I disagree as there were other categories of vulns that
weren't tested and also I don't believe that the testing scenario used any
simulated user traffic to allow the WAFs auto-learning/profiling systems
create profilesŠ  

Another example to consider is PCI DSS supplemental document for Requirement
6.6 - 
f.  The most relevant section is the "Recommended Capabilities" section
where it states that it should be able to enforce both negative and positive
security models.  I believe that this item alone will disqualify network
FW/IPS as they only do negative security/blacklist filtering (as far as I

2) WAF Testing ­ people use WAFEC during WAF bake-offs when they are
evaluating multiple WAFs.  Unfortunately, the actual "testing" or evaluating
of WAFs against each other is an exercise left to the userŠ  I had to laugh
when one prospect said that they chose one WAF over ours because they ran a
scanner agains the website and the competitor's WAF produced more alerts
than ours so it had to be better.  :(  I explained to the prospect that this
was a flawed testing design as our product consolidates multiple individual
alerts into an overall summary alert for the transaction as a whole.  This
was used to help with event management in production.  Needless to say, they
ended up with our product after trying to use the competitors product and
were flooded with alerts that they could not manage.

For this reason, I believe that we should attempt to make some type of
agreed upon testing methodology for users to follow.  How should users
actually test WAFs to ensure that they are working as advertised?

> 4.       Not just a laundry list ­
> ·         Classify the importance of requirements. I believe that a minimal
> approach specifying several levels, for example: ³mandatory², ³important²,
> ³nice to have² and ³site specific².
> ·         Another complementing idea is to classify requirements as
> ³security²/²functionality²/²performance² etc. letting the user determine if he
> prefers security over functionality etc.
> ·         This would also provide the minimum requirements for a solution to
> be a WAF ­ the ³mandatory² requirements.
> ·         Regarding site specific requirements, it should be easy to the user
> to determine his own requirements, for example using a decision tree.

 I touched on some of these in earlier comments.  WE need to clearly specify
minimum requirements for WAF.

> 5.       The ³ethical² questions:
> ·         How to address alternative solutions such as fixing the code?

This can indirectly be addressed by having a mapping to WASC TC and
providing mitigation coverage for each issue.  This will highlight
attack/vuln types that can be better addressed within the code.

> 6.       Outreach ­ beyond the document
> ·         Approaching NSS, ICSA and the likes to use WAFEC

This is important for the "WAF Testing" issue I raised above.  Users need
guidance on how to run WAF testing during bake-offs.

> ·         Release process, PR etc.
> ·         Managing a list of public references to WAFEC
> ·         Promote actual evaluations data sharing - No more spreadsheets.
> 7.       Specific notes on V1, I have collected for further work.
> A major question raised with opinions on both sides was a re-write vs. an
> update. I do think that understanding the requirements should direct that.
> Some issues raised which directly relate to that are:
> 1.       Is the order of sections correct?

This brings up the issue of WAF Deployment Model.  This should probably be
one of the first items listed as the choice of deployment model directly
impacts many other capabilities.

> 2.       Incorporating the German OWASP chapter work on the same subject:
> http://www.owasp.org/index.php/Best_Practices:_Web_Application_Firewalls

In my view, WAFEC would function as a sub-chapter within this document.  The
OWASP WAF Best Practices doc starts off discussing the rationale for even
considering a WAF.  Once they make this decision, they would use WAFEC to
evaluate products.  Once they choose a WAF, then they go back to the OWASP
WAF Best Practices document for deciding how to integrate it and manage it.

> ~ Ofer
> From: Ofer Shezaf [mailto:ofer at shezaf.com]
> Sent: Thursday, May 31, 2012 1:45 PM
> To: wasc-wafec at lists.webappsec.org
> Subject: WAFEC 2.0 phase 1: exploratory discussion (deadline: June 14th)
> Thanks to all who volunteered to contribute to this project going forward (and
> those who didn¹t ­ you still can!)
> I would like to boot up the project with a short exploratory phase identifying
> why we need a new release and therefore what we need in it.
> To guide the discussion, I think that the reasons we need v2 fall into two
> categories:
> 1.      Things that have changed - new (or obsolete) deployment modes,
> techniques, attacks, or even something new altogether.
> 2.      Issues we discovered in WAFEC over the years. Some issues I
> encountered are identifying specific requirements and sorting out what¹s
> important and what¹s not.
> From this discussion I hope to derive a mission statement, a tasks list and
> therefore a schedule for the V2 project. All those will be the next phase.
> I would give this phase two weeks (until June 14th), however I am on vacation
> from the 9th, so would accept input but not join the discussion on the last
> few days.
> I would also want to thank Thorsten and Mirko for leading the project until
> now. I do hope that I will get from you all more cooperation than they did! I
> would also want to extend a personal apology to Thorsten and Mirko as the
> leader switch was not well coordinated. Thorsten and I discussed this over the
> last week and he gracefully agreed to let me give a try to leading this
> project forward.
> Thank you all!
> ~ Ofer
> Ofer Shezaf
> [+972-54-4431119; ofer at shezaf.com, www.shezaf.com <http://www.shezaf.com> ]
> _______________________________________________ wasc-wafec mailing list
> wasc-wafec at lists.webappsec.org
> http://lists.webappsec.org/mailman/listinfo/wasc-wafec_lists.webappsec.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webappsec.org/pipermail/wasc-wafec_lists.webappsec.org/attachments/20120607/ca2da6af/attachment-0003.html>

More information about the wasc-wafec mailing list