Since you are all very quiet, I understand that WAFEC 2 will solve my pain
and needs only J

To that end, let me start with summarizing issues raised in the previous
discussions on the mailing list (which I actually went and read.).

No specific order intended. This is what you wrote, though I must say I
think it captures well the issues I am aware of and that generally speaking
I agree with most.

1.   Remove non WAF related criteria for example around application
   

delivery.

.        While integrating a WAF with other solutions is compelling to the
client, it is not directly about WAFs and is also unbounded. This does
present the challenge deciding what is relevant to a WAF in border cases
such as an SSO functionality

2.   Update the list of threats covered
   

3.   Focus on customer use cases rather than how a WAF operates
   

.        I think there was some hidden controversy here as I read opinions
to focus on "technical" which I take to be opposite. I personally very much
agree with this comment.

4.   Not just a laundry list - 
   

.        Classify the importance of requirements. I believe that a minimal
approach specifying several levels, for example: "mandatory", "important",
"nice to have" and "site specific".

.        Another complementing idea is to classify requirements as
"security"/"functionality"/"performance" etc. letting the user determine if
he prefers security over functionality etc.

.        This would also provide the minimum requirements for a solution to
be a WAF - the "mandatory" requirements.

.        Regarding site specific requirements, it should be easy to the
user to determine his own requirements, for example using a decision tree.

5.   The "ethical" questions:
   

.        How to address alternative solutions such as fixing the code?

6.   Outreach - beyond the document
   

.        Approaching NSS, ICSA and the likes to use WAFEC

.        Release process, PR etc.

.        Managing a list of public references to WAFEC

.        Promote actual evaluations data sharing - No more spreadsheets.

7.   Specific notes on V1, I have collected for further work.
   

A major question raised with opinions on both sides was a re-write vs. an
update. I do think that understanding the requirements should direct that.
Some issues raised which directly relate to that are:

1.   Is the order of sections correct?
   

2.   Incorporating the German OWASP chapter work on the same subject:
   

http://www.owasp.org/index.php/Best_Practices:_Web_Application_Firewalls

~ Ofer

From: Ofer Shezaf [mailto:ofer@shezaf.com]
Sent: Thursday, May 31, 2012 1:45 PM
To: wasc-wafec@lists.webappsec.org
Subject: WAFEC 2.0 phase 1: exploratory discussion (deadline: June 14th)

Thanks to all who volunteered to contribute to this project going forward
(and those who didn't - you still can!)

I would like to boot up the project with a short exploratory phase
identifying why we need a new release and therefore what we need in it.

To guide the discussion, I think that the reasons we need v2 fall into two
categories:

1.  Things that have changed - new (or obsolete) deployment modes,
   

techniques, attacks, or even something new altogether.

2.  Issues we discovered in WAFEC over the years. Some issues I
   

encountered are identifying specific requirements and sorting out what's
important and what's not.

therefore a schedule for the V2 project. All those will be the next phase.

I would give this phase two weeks (until June 14th), however I am on
vacation from the 9th, so would accept input but not join the discussion on
the last few days.

I would also want to thank Thorsten and Mirko for leading the project until
now. I do hope that I will get from you all more cooperation than they did!
I would also want to extend a personal apology to Thorsten and Mirko as the
leader switch was not well coordinated. Thorsten and I discussed this over
the last week and he gracefully agreed to let me give a try to leading this
project forward.

Thank you all!

~ Ofer

Ofer Shezaf

[+972-54-4431119; ofer@shezaf.com, www.shezaf.com]

Hi Ofer,
I tend to agree with all but the first one. I think that the WAFEC should help customers to choose the right WAF for their environment. If we neglect options like integration with other network devices/ security solutions or capabilities that effect the performance of a WAF or the application behind a WAF then we will damage the original goal.

Over the recent years, WAFs pretty much stopped being another point solution in the network and most toady offer additional services and features that contribute to their adoption.

Let me give you an example that we see all the time: Security team asks for a WAF, business owners and the network team say that a WAF will hurt the customer experience as it will add latency, at that point, if the security team claims that the WAF has a built in fast cache that actually offloads work from the web server and contributes to the user experience then they may be able to persuade the network team and business owners and get a WAF installed.

By hiding these "non-WAF" related features from WAFEC we make WAFEC less attractive as it will require much more additional research to find the right solution to a specific environment/ problem.

Cheers,
Ido

From: wasc-wafec-bounces@lists.webappsec.org [mailto:wasc-wafec-bounces@lists.webappsec.org] On Behalf Of Ofer Shezaf
Sent: Wednesday, June 06, 2012 2:39 PM
To: wasc-wafec@lists.webappsec.org
Subject: [WASC-WAFEC] What should we change in WAFEC 2.0?

Since you are all very quiet, I understand that WAFEC 2 will solve my pain and needs only :)

To that end, let me start with summarizing issues raised in the previous discussions on the mailing list (which I actually went and read...).

No specific order intended. This is what you wrote, though I must say I think it captures well the issues I am aware of and that generally speaking I agree with most.

1.   Remove non WAF related criteria for example around application delivery.
   

•     While integrating a WAF with other solutions is compelling to the client, it is not directly about WAFs and is also unbounded. This does present the challenge deciding what is relevant to a WAF in border cases such as an SSO functionality
  

2.   Update the list of threats covered
   

3.   Focus on customer use cases rather than how a WAF operates
   

•     I think there was some hidden controversy here as I read opinions to focus on "technical" which I take to be opposite. I personally very much agree with this comment.
  

4.   Not just a laundry list -
   

•     Classify the importance of requirements. I believe that a minimal approach specifying several levels, for example: "mandatory", "important", "nice to have" and "site specific".
  

•     Another complementing idea is to classify requirements as "security"/"functionality"/"performance" etc. letting the user determine if he prefers security over functionality etc.
  

•     This would also provide the minimum requirements for a solution to be a WAF - the "mandatory" requirements.
  

•     Regarding site specific requirements, it should be easy to the user to determine his own requirements, for example using a decision tree.
  

5.   The "ethical" questions:
   

•     How to address alternative solutions such as fixing the code?
  

6.   Outreach - beyond the document
   

•     Approaching NSS, ICSA and the likes to use WAFEC
  

•     Release process, PR etc.
  

•     Managing a list of public references to WAFEC
  

•     Promote actual evaluations data sharing - No more spreadsheets.
  

7.   Specific notes on V1, I have collected for further work.
   

A major question raised with opinions on both sides was a re-write vs. an update. I do think that understanding the requirements should direct that. Some issues raised which directly relate to that are:

1.   Is the order of sections correct?
   

2.   Incorporating the German OWASP chapter work on the same subject: http://www.owasp.org/index.php/Best_Practices:_Web_Application_Firewalls
   

~ Ofer

From: Ofer Shezaf [mailto:ofer@shezaf.com]mailto:[mailto:ofer@shezaf.com]
Sent: Thursday, May 31, 2012 1:45 PM
To: wasc-wafec@lists.webappsec.orgmailto:wasc-wafec@lists.webappsec.org
Subject: WAFEC 2.0 phase 1: exploratory discussion (deadline: June 14th)

Thanks to all who volunteered to contribute to this project going forward (and those who didn't - you still can!)

I would like to boot up the project with a short exploratory phase identifying why we need a new release and therefore what we need in it.

To guide the discussion, I think that the reasons we need v2 fall into two categories:

1.  Things that have changed - new (or obsolete) deployment modes, techniques, attacks, or even something new altogether.
   

2.  Issues we discovered in WAFEC over the years. Some issues I encountered are identifying specific requirements and sorting out what's important and what's not.
   

I would give this phase two weeks (until June 14th), however I am on vacation from the 9th, so would accept input but not join the discussion on the last few days.

I would also want to thank Thorsten and Mirko for leading the project until now. I do hope that I will get from you all more cooperation than they did! I would also want to extend a personal apology to Thorsten and Mirko as the leader switch was not well coordinated. Thorsten and I discussed this over the last week and he gracefully agreed to let me give a try to leading this project forward.

Thank you all!
~ Ofer

Ofer Shezaf
[+972-54-4431119; ofer@shezaf.commailto:ofer@shezaf.com, www.shezaf.comhttp://www.shezaf.com]

My thoughts are:

1 - I agree with Ofer to remove the non WAF related criteria.  I realize this will come down to the camp of those that focus on those things (e.g., Ido&F5) and those that don't (e.g., me&Imperva).  Perhaps a compromise would be to have a section for "related capabilities" outside of the main flow.

2 - we need to have some sort of update on threats.  This normally turns into a complicated discussion of the ontology of threat classification.  Is there a way to avoid that?

3 - I think the issue that came up here was that the usage of the document and the content of it were at odds.  In particular, when used as a template evaluation tool without any processing (which it often is), it results in conflicting "requirements" to evaluate against, especially with regard to deployment mode. I'd suggest taking a middle ground and keeping the deployment modes section, but changing the nature of the content to better explain and lend itself to a non-conflicting evaluation, but also to include customer goals / use cases as a section.

I also think that the use cases section could help us solve #2.

4 - my opinion would be to keep a flat list, but to provide a tool that let's the customer adjust importance based on their needs.

5 - I would leave this out of the criteria (since WAFs don't fix code it doesn't make sense to have fixing code as an evaluation element).  IMO, this is a better topic for a different kind of forum...this tool is supposed to be a tool to evaluate WAFs.

6 - I think really orienting the criteria to be useable framework for an actual evaluation will make this simpler.

From: wasc-wafec-bounces@lists.webappsec.org [mailto:wasc-wafec-bounces@lists.webappsec.org] On Behalf Of Ofer Shezaf
Sent: Wednesday, June 06, 2012 4:39 AM
To: wasc-wafec@lists.webappsec.org
Subject: [WASC-WAFEC] What should we change in WAFEC 2.0?

Since you are all very quiet, I understand that WAFEC 2 will solve my pain and needs only :)

To that end, let me start with summarizing issues raised in the previous discussions on the mailing list (which I actually went and read...).

No specific order intended. This is what you wrote, though I must say I think it captures well the issues I am aware of and that generally speaking I agree with most.

1.   Remove non WAF related criteria for example around application delivery.
   

•     While integrating a WAF with other solutions is compelling to the client, it is not directly about WAFs and is also unbounded. This does present the challenge deciding what is relevant to a WAF in border cases such as an SSO functionality
  

2.   Update the list of threats covered
   

3.   Focus on customer use cases rather than how a WAF operates
   

•     I think there was some hidden controversy here as I read opinions to focus on "technical" which I take to be opposite. I personally very much agree with this comment.
  

4.   Not just a laundry list -
   

•     Classify the importance of requirements. I believe that a minimal approach specifying several levels, for example: "mandatory", "important", "nice to have" and "site specific".
  

•     Another complementing idea is to classify requirements as "security"/"functionality"/"performance" etc. letting the user determine if he prefers security over functionality etc.
  

•     This would also provide the minimum requirements for a solution to be a WAF - the "mandatory" requirements.
  

•     Regarding site specific requirements, it should be easy to the user to determine his own requirements, for example using a decision tree.
  

5.   The "ethical" questions:
   

•     How to address alternative solutions such as fixing the code?
  

6.   Outreach - beyond the document
   

•     Approaching NSS, ICSA and the likes to use WAFEC
  

•     Release process, PR etc.
  

•     Managing a list of public references to WAFEC
  

•     Promote actual evaluations data sharing - No more spreadsheets.
  

7.   Specific notes on V1, I have collected for further work.
   

A major question raised with opinions on both sides was a re-write vs. an update. I do think that understanding the requirements should direct that. Some issues raised which directly relate to that are:

1.   Is the order of sections correct?
   

2.   Incorporating the German OWASP chapter work on the same subject: http://www.owasp.org/index.php/Best_Practices:_Web_Application_Firewalls
   

~ Ofer

From: Ofer Shezaf [mailto:ofer@shezaf.com]
Sent: Thursday, May 31, 2012 1:45 PM
To: wasc-wafec@lists.webappsec.orgmailto:wasc-wafec@lists.webappsec.org
Subject: WAFEC 2.0 phase 1: exploratory discussion (deadline: June 14th)

Thanks to all who volunteered to contribute to this project going forward (and those who didn't - you still can!)

I would like to boot up the project with a short exploratory phase identifying why we need a new release and therefore what we need in it.

To guide the discussion, I think that the reasons we need v2 fall into two categories:

1.  Things that have changed - new (or obsolete) deployment modes, techniques, attacks, or even something new altogether.
   

2.  Issues we discovered in WAFEC over the years. Some issues I encountered are identifying specific requirements and sorting out what's important and what's not.
   

I would give this phase two weeks (until June 14th), however I am on vacation from the 9th, so would accept input but not join the discussion on the last few days.

I would also want to thank Thorsten and Mirko for leading the project until now. I do hope that I will get from you all more cooperation than they did! I would also want to extend a personal apology to Thorsten and Mirko as the leader switch was not well coordinated. Thorsten and I discussed this over the last week and he gracefully agreed to let me give a try to leading this project forward.

Thank you all!
~ Ofer

Ofer Shezaf
[+972-54-4431119; ofer@shezaf.commailto:ofer@shezaf.com, www.shezaf.comhttp://www.shezaf.com]

1.   speaking on behalf of Citrix, I actually agree with Mark.  I'd like our WAF to be evaluated on it's own merits.  Yes, it integrates into the NetScaler ADC, but that shouldn't in the WAFEC.   The WAFEC should be criteria to choose a WAF, not an entire ecosystem, as most WAF customers already have an ADC infrastructure of some sort.
   

2.    Yes, cross site request forgery, etc (and many other threats) would be helpful.
   

3.   From a use case perspective, it would be helpful to distinguish different products - proxying vs SPAN port vs transparent, and the level of blocking - RSTs, buffering and proxying the request, or just logging/alerting, etc.  (all valid deployment models going towards different goals)
   

4.   I'd like to see the list categorized into security features, logging/alerting, etc.  This would help users of the list identify what is important and not important (which will vary quite a bit between use cases)
   

5.   Agreed.  It's up to the consumer to decide whether or not to fix code, and not the job of the WAFEC to decide that.
   

6.   Agreed with Mark.  The more we can make this a list that consumers can use to decide which solution to use, the more it will be used.
   

--
Kit Wetzler
Sr. Manager, Sales Engineering - West Region
Citrix Systems, Inc
Networking and Cloud Product Group (NetScaler, Branch Repeater and Access Gateway)
408.892.1424
kit.wetzler@citrix.commailto:kit.wetzler@citrix.com

From: wasc-wafec-bounces@lists.webappsec.org [mailto:wasc-wafec-bounces@lists.webappsec.org] On Behalf Of Mark Kraynak
Sent: Wednesday, June 06, 2012 1:54 PM
To: Ofer Shezaf; wasc-wafec@lists.webappsec.org
Subject: Re: [WASC-WAFEC] What should we change in WAFEC 2.0?

My thoughts are:

1 - I agree with Ofer to remove the non WAF related criteria.  I realize this will come down to the camp of those that focus on those things (e.g., Ido&F5) and those that don't (e.g., me&Imperva).  Perhaps a compromise would be to have a section for "related capabilities" outside of the main flow.

2 - we need to have some sort of update on threats.  This normally turns into a complicated discussion of the ontology of threat classification.  Is there a way to avoid that?

3 - I think the issue that came up here was that the usage of the document and the content of it were at odds.  In particular, when used as a template evaluation tool without any processing (which it often is), it results in conflicting "requirements" to evaluate against, especially with regard to deployment mode. I'd suggest taking a middle ground and keeping the deployment modes section, but changing the nature of the content to better explain and lend itself to a non-conflicting evaluation, but also to include customer goals / use cases as a section.

I also think that the use cases section could help us solve #2.

4 - my opinion would be to keep a flat list, but to provide a tool that let's the customer adjust importance based on their needs.

5 - I would leave this out of the criteria (since WAFs don't fix code it doesn't make sense to have fixing code as an evaluation element).  IMO, this is a better topic for a different kind of forum...this tool is supposed to be a tool to evaluate WAFs.

6 - I think really orienting the criteria to be useable framework for an actual evaluation will make this simpler.

From: wasc-wafec-bounces@lists.webappsec.orgmailto:wasc-wafec-bounces@lists.webappsec.org [mailto:wasc-wafec-bounces@lists.webappsec.org]mailto:[mailto:wasc-wafec-bounces@lists.webappsec.org] On Behalf Of Ofer Shezaf
Sent: Wednesday, June 06, 2012 4:39 AM
To: wasc-wafec@lists.webappsec.orgmailto:wasc-wafec@lists.webappsec.org
Subject: [WASC-WAFEC] What should we change in WAFEC 2.0?

Since you are all very quiet, I understand that WAFEC 2 will solve my pain and needs only :)

To that end, let me start with summarizing issues raised in the previous discussions on the mailing list (which I actually went and read...).

No specific order intended. This is what you wrote, though I must say I think it captures well the issues I am aware of and that generally speaking I agree with most.

1.   Remove non WAF related criteria for example around application delivery.
   

•     While integrating a WAF with other solutions is compelling to the client, it is not directly about WAFs and is also unbounded. This does present the challenge deciding what is relevant to a WAF in border cases such as an SSO functionality
  

2.   Update the list of threats covered
   

3.   Focus on customer use cases rather than how a WAF operates
   

•     I think there was some hidden controversy here as I read opinions to focus on "technical" which I take to be opposite. I personally very much agree with this comment.
  

4.   Not just a laundry list -
   

•     Classify the importance of requirements. I believe that a minimal approach specifying several levels, for example: "mandatory", "important", "nice to have" and "site specific".
  

•     Another complementing idea is to classify requirements as "security"/"functionality"/"performance" etc. letting the user determine if he prefers security over functionality etc.
  

•     This would also provide the minimum requirements for a solution to be a WAF - the "mandatory" requirements.
  

•     Regarding site specific requirements, it should be easy to the user to determine his own requirements, for example using a decision tree.
  

5.   The "ethical" questions:
   

•     How to address alternative solutions such as fixing the code?
  

6.   Outreach - beyond the document
   

•     Approaching NSS, ICSA and the likes to use WAFEC
  

•     Release process, PR etc.
  

•     Managing a list of public references to WAFEC
  

•     Promote actual evaluations data sharing - No more spreadsheets.
  

7.   Specific notes on V1, I have collected for further work.
   

A major question raised with opinions on both sides was a re-write vs. an update. I do think that understanding the requirements should direct that. Some issues raised which directly relate to that are:

1.   Is the order of sections correct?
   

2.   Incorporating the German OWASP chapter work on the same subject: http://www.owasp.org/index.php/Best_Practices:_Web_Application_Firewalls
   

~ Ofer

From: Ofer Shezaf [mailto:ofer@shezaf.com]
Sent: Thursday, May 31, 2012 1:45 PM
To: wasc-wafec@lists.webappsec.orgmailto:wasc-wafec@lists.webappsec.org
Subject: WAFEC 2.0 phase 1: exploratory discussion (deadline: June 14th)

Thanks to all who volunteered to contribute to this project going forward (and those who didn't - you still can!)

I would like to boot up the project with a short exploratory phase identifying why we need a new release and therefore what we need in it.

To guide the discussion, I think that the reasons we need v2 fall into two categories:

1.  Things that have changed - new (or obsolete) deployment modes, techniques, attacks, or even something new altogether.
   

2.  Issues we discovered in WAFEC over the years. Some issues I encountered are identifying specific requirements and sorting out what's important and what's not.
   

I would give this phase two weeks (until June 14th), however I am on vacation from the 9th, so would accept input but not join the discussion on the last few days.

I would also want to thank Thorsten and Mirko for leading the project until now. I do hope that I will get from you all more cooperation than they did! I would also want to extend a personal apology to Thorsten and Mirko as the leader switch was not well coordinated. Thorsten and I discussed this over the last week and he gracefully agreed to let me give a try to leading this project forward.

Thank you all!
~ Ofer

Ofer Shezaf
[+972-54-4431119; ofer@shezaf.commailto:ofer@shezaf.com, www.shezaf.comhttp://www.shezaf.com]

Ofer,

On Wed, Jun 6, 2012 at 9:39 PM, Ofer Shezaf ofer@shezaf.com wrote:

I would recommend that if this change is accepted that the content
capture in V1 be transitioned to an Annexure  so that it is not lost
in the next release?

On Wed, Jun 6, 2012 at 9:39 PM, Ofer Shezaf ofer@shezaf.com wrote:

Can this be correlated to
http://projects.webappsec.org/w/page/13246978/Threat%20Classification
which might also help educate the consumer on the limitations and
benefits of WAF?

On Wed, Jun 6, 2012 at 9:39 PM, Ofer Shezaf ofer@shezaf.com wrote:

For this to benefit the end user the use cases would need to be
defined and then correlated to the relevant sections of WAFEC?

On Wed, Jun 6, 2012 at 9:39 PM, Ofer Shezaf ofer@shezaf.com wrote:

This would add creditability to WAFEC as we have identified and
addressed the conflict of interest of vendors.

On Wed, Jun 6, 2012 at 9:39 PM, Ofer Shezaf ofer@shezaf.com wrote:

This should be addressed once the next release of WAFEC is released to
the public - I would recommend escalating this to
http://webappsec.org/officers.shtml for guidance?

On Wed, Jun 6, 2012 at 9:39 PM, Ofer Shezaf ofer@shezaf.com wrote:

I believe the easiest roadmap would be:

1. Update the existing WAFEC v1 sections
2. Correlate to
   http://www.owasp.org/index.php/Best_Practices:_Web_Application_Firewalls
3. Consider modifying the order of the sections of WAFEC V1
4. Publish as a major release i.e. v2

--
Regards,
Christian Heinrich

http://cmlh.id.au/contact

Thinking a bit more on the deployment modes section, would it make sense to elaborate on which features may not available per deployment mode?

From: wasc-wafec-bounces@lists.webappsec.org [mailto:wasc-wafec-bounces@lists.webappsec.org] On Behalf Of Ofer Shezaf
Sent: Wednesday, June 06, 2012 2:39 PM
To: wasc-wafec@lists.webappsec.org
Subject: [WASC-WAFEC] What should we change in WAFEC 2.0?

Since you are all very quiet, I understand that WAFEC 2 will solve my pain and needs only :)

To that end, let me start with summarizing issues raised in the previous discussions on the mailing list (which I actually went and read...).

No specific order intended. This is what you wrote, though I must say I think it captures well the issues I am aware of and that generally speaking I agree with most.

1.   Remove non WAF related criteria for example around application delivery.
   

•     While integrating a WAF with other solutions is compelling to the client, it is not directly about WAFs and is also unbounded. This does present the challenge deciding what is relevant to a WAF in border cases such as an SSO functionality
  

2.   Update the list of threats covered
   

3.   Focus on customer use cases rather than how a WAF operates
   

•     I think there was some hidden controversy here as I read opinions to focus on "technical" which I take to be opposite. I personally very much agree with this comment.
  

4.   Not just a laundry list -
   

•     Classify the importance of requirements. I believe that a minimal approach specifying several levels, for example: "mandatory", "important", "nice to have" and "site specific".
  

•     Another complementing idea is to classify requirements as "security"/"functionality"/"performance" etc. letting the user determine if he prefers security over functionality etc.
  

•     This would also provide the minimum requirements for a solution to be a WAF - the "mandatory" requirements.
  

•     Regarding site specific requirements, it should be easy to the user to determine his own requirements, for example using a decision tree.
  

5.   The "ethical" questions:
   

•     How to address alternative solutions such as fixing the code?
  

6.   Outreach - beyond the document
   

•     Approaching NSS, ICSA and the likes to use WAFEC
  

•     Release process, PR etc.
  

•     Managing a list of public references to WAFEC
  

•     Promote actual evaluations data sharing - No more spreadsheets.
  

7.   Specific notes on V1, I have collected for further work.
   

A major question raised with opinions on both sides was a re-write vs. an update. I do think that understanding the requirements should direct that. Some issues raised which directly relate to that are:

1.   Is the order of sections correct?
   

2.   Incorporating the German OWASP chapter work on the same subject: http://www.owasp.org/index.php/Best_Practices:_Web_Application_Firewalls
   

~ Ofer

From: Ofer Shezaf [mailto:ofer@shezaf.com]mailto:[mailto:ofer@shezaf.com]
Sent: Thursday, May 31, 2012 1:45 PM
To: wasc-wafec@lists.webappsec.orgmailto:wasc-wafec@lists.webappsec.org
Subject: WAFEC 2.0 phase 1: exploratory discussion (deadline: June 14th)

Thanks to all who volunteered to contribute to this project going forward (and those who didn't - you still can!)

I would like to boot up the project with a short exploratory phase identifying why we need a new release and therefore what we need in it.

To guide the discussion, I think that the reasons we need v2 fall into two categories:

1.  Things that have changed - new (or obsolete) deployment modes, techniques, attacks, or even something new altogether.
   

2.  Issues we discovered in WAFEC over the years. Some issues I encountered are identifying specific requirements and sorting out what's important and what's not.
   

I would give this phase two weeks (until June 14th), however I am on vacation from the 9th, so would accept input but not join the discussion on the last few days.

I would also want to thank Thorsten and Mirko for leading the project until now. I do hope that I will get from you all more cooperation than they did! I would also want to extend a personal apology to Thorsten and Mirko as the leader switch was not well coordinated. Thorsten and I discussed this over the last week and he gracefully agreed to let me give a try to leading this project forward.

Thank you all!
~ Ofer

Ofer Shezaf
[+972-54-4431119; ofer@shezaf.commailto:ofer@shezaf.com, www.shezaf.comhttp://www.shezaf.com]

Ivan Ristic had previously created a "Deployment Method Capabilities Matrix"
that is on gooogle docs -
https://docs.google.com/document/d/1_wnWjug9cAuvb1uw4OPPHU1oks0D5EAw-tcl31Ot
a28/edit?pli=1

You could start with that.

-Ryan

From:  Ido Breger I.Breger@F5.com
Date:  Thu, 7 Jun 2012 11:35:08 +0000
To:  Ofer Shezaf ofer@shezaf.com, "wasc-wafec@lists.webappsec.org"
wasc-wafec@lists.webappsec.org
Subject:  Re: [WASC-WAFEC] What should we change in WAFEC 2.0?

Hi Ofer,
My thoughts:

1. I agree, WAF is a web application security filter and as such the focus
   should be on its functionality.
2. I'm not sure this is what you meant in this section but my concern is
   that security requirements (that are related to "how waf operates") will
   have proper description.
   e.g. - if WAF should protect from CSRF attack we need to have good
   description that will differentiate between those that protect by inspect
   referer header and those how rewrite data and add tokens.

Thanks.

Or Katz

On Wed, Jun 6, 2012 at 2:39 PM, Ofer Shezaf ofer@shezaf.com wrote:

Comments inline below.

From:  Ofer Shezaf ofer@shezaf.com
Date:  Wed, 6 Jun 2012 14:39:29 +0300
To:  wasc-wafec@lists.webappsec.org
Subject:  [WASC-WAFEC] What should we change in WAFEC 2.0?

I recommend that we consider using a "Levels" approach similar to what OWASP
ASVS uses - http://code.google.com/p/owasp-asvs/wiki/ASVS.  This way, we can
group items and the user can be clear which items are considered "core" WAF
features and which ones provide added value.

I agree that we should map to the WASC TC.  The OWASP German Chapter did
something similar with the WAF Best Practices document listing coverage for
the OWASP Top 10 2007 -
https://www.owasp.org/index.php/Category:OWASP_Best_Practices:_Use_of_Web_Ap
plication_Firewalls.  This will help to show which attack/vuln categories
WAFs excel at mitigating (injection flaws, info leakages) and which ones
they don't (authorization, logic flaws).

OK, I guess here is as good a place as any to state my two biggest issues I
think we need to address with WAFEC -

1. Minimum Requirements for WAF ­ whether we like it or not, most WAFEC
   users tried to use it as  a "minimum requirements" document.  I can't tell
   you how many times I was sent the speadsheet by a WAF prospect and asked how
   our WAF "conformed" to itŠ  So, I do think that we need to re-organize the
   data so that there is a clear distinction between what features a product
   must have to be considered a WAF.  This is an important issue as there are
   many other vendors in the FW/IPS space that tout that they can do the same
   things as WAFs and that is just marketing BS.  For instance ­ in Larry
   Suto's WAF Eval doc
   (http://www.manvswebapp.com/wp-content/uploads/Analyzing_Effectiveness_of_We
   b_Application_Firewalls.pdf) one of the general takeaways was that an IPS
   configured with virtual patches from a DAST scanner would be just as
   effective as a WAF.  I disagree as there were other categories of vulns that
   weren't tested and also I don't believe that the testing scenario used any
   simulated user traffic to allow the WAFs auto-learning/profiling systems
   create profilesŠ

Another example to consider is PCI DSS supplemental document for Requirement
6.6 -
https://www.pcisecuritystandards.org/documents/information_supplement_6.6.pd
f.  The most relevant section is the "Recommended Capabilities" section
where it states that it should be able to enforce both negative and positive
security models.  I believe that this item alone will disqualify network
FW/IPS as they only do negative security/blacklist filtering (as far as I
understand).

2. WAF Testing ­ people use WAFEC during WAF bake-offs when they are
   evaluating multiple WAFs.  Unfortunately, the actual "testing" or evaluating
   of WAFs against each other is an exercise left to the userŠ  I had to laugh
   when one prospect said that they chose one WAF over ours because they ran a
   scanner agains the website and the competitor's WAF produced more alerts
   than ours so it had to be better.  :(  I explained to the prospect that this
   was a flawed testing design as our product consolidates multiple individual
   alerts into an overall summary alert for the transaction as a whole.  This
   was used to help with event management in production.  Needless to say, they
   ended up with our product after trying to use the competitors product and
   were flooded with alerts that they could not manage.

For this reason, I believe that we should attempt to make some type of
agreed upon testing methodology for users to follow.  How should users
actually test WAFs to ensure that they are working as advertised?

I touched on some of these in earlier comments.  WE need to clearly specify
minimum requirements for WAF.

This can indirectly be addressed by having a mapping to WASC TC and
providing mitigation coverage for each issue.  This will highlight
attack/vuln types that can be better addressed within the code.

This is important for the "WAF Testing" issue I raised above.  Users need
guidance on how to run WAF testing during bake-offs.

This brings up the issue of WAF Deployment Model.  This should probably be
one of the first items listed as the choice of deployment model directly
impacts many other capabilities.

In my view, WAFEC would function as a sub-chapter within this document.  The
OWASP WAF Best Practices doc starts off discussing the rationale for even
considering a WAF.  Once they make this decision, they would use WAFEC to
evaluate products.  Once they choose a WAF, then they go back to the OWASP
WAF Best Practices document for deciding how to integrate it and manage it.

On #1:

I think it is beneficial to at least make the ease with which a WAF does integrate with other solutions or a list of solutions that the WAF has documented integration with a criterion for a WAF.  No one is ever going to buy a WAF and deploy it by itself without anything else around it.

As a customer, I probably already have a number of other solutions and it would be extremely valuable for me to know if any given WAF innately integrates with those other solutions or has an easy, documented ability to do so.  I might also be looking at other solutions at the same time I am deploying a WAF and knowing what all my option are is also extremely valuable.

So ... we don't have to compare WAFs by saying "oh, we have SSL VPN and yours doesn't" as that is not a specific WAF criterion, but it is very relevant and important to a customer who is evaluating these devices to know if the solution will integrate with their existing VPN or has options available to integrate with one down the road. (and I just picked SSL VPN off the top of my head, it could be SSO or whatever).

KJ (Ken) Salchow, Jr. | Program Manager, Technical Certification

D 651.423.1133

M 612.868.1258

P 206.272.5555

F 206.272.5555

www.f5.comhttp://www.f5.com/

[Description: Description: F5_Logo-TechCert_TM_042312sm]

From: wasc-wafec-bounces@lists.webappsec.org [mailto:wasc-wafec-bounces@lists.webappsec.org] On Behalf Of Kit Wetzler
Sent: Wednesday, June 06, 2012 4:10 PM
To: wasc-wafec@lists.webappsec.org
Subject: Re: [WASC-WAFEC] What should we change in WAFEC 2.0?

1.  speaking on behalf of Citrix, I actually agree with Mark.  I'd like our WAF to be evaluated on it's own merits.  Yes, it integrates into the NetScaler ADC, but that shouldn't in the WAFEC.   The WAFEC should be criteria to choose a WAF, not an entire ecosystem, as most WAF customers already have an ADC infrastructure of some sort.
   

2.   Yes, cross site request forgery, etc (and many other threats) would be helpful.
   

3.  From a use case perspective, it would be helpful to distinguish different products - proxying vs SPAN port vs transparent, and the level of blocking - RSTs, buffering and proxying the request, or just logging/alerting, etc.  (all valid deployment models going towards different goals)
   

4.  I'd like to see the list categorized into security features, logging/alerting, etc.  This would help users of the list identify what is important and not important (which will vary quite a bit between use cases)
   

5.  Agreed.  It's up to the consumer to decide whether or not to fix code, and not the job of the WAFEC to decide that.
   

6.  Agreed with Mark.  The more we can make this a list that consumers can use to decide which solution to use, the more it will be used.
   

--
Kit Wetzler
Sr. Manager, Sales Engineering - West Region
Citrix Systems, Inc
Networking and Cloud Product Group (NetScaler, Branch Repeater and Access Gateway)
408.892.1424
kit.wetzler@citrix.commailto:kit.wetzler@citrix.com

From: wasc-wafec-bounces@lists.webappsec.orgmailto:wasc-wafec-bounces@lists.webappsec.org [mailto:wasc-wafec-bounces@lists.webappsec.org]mailto:[mailto:wasc-wafec-bounces@lists.webappsec.org] On Behalf Of Mark Kraynak
Sent: Wednesday, June 06, 2012 1:54 PM
To: Ofer Shezaf; wasc-wafec@lists.webappsec.orgmailto:wasc-wafec@lists.webappsec.org
Subject: Re: [WASC-WAFEC] What should we change in WAFEC 2.0?

My thoughts are:

1 - I agree with Ofer to remove the non WAF related criteria.  I realize this will come down to the camp of those that focus on those things (e.g., Ido&F5) and those that don't (e.g., me&Imperva).  Perhaps a compromise would be to have a section for "related capabilities" outside of the main flow.

2 - we need to have some sort of update on threats.  This normally turns into a complicated discussion of the ontology of threat classification.  Is there a way to avoid that?

3 - I think the issue that came up here was that the usage of the document and the content of it were at odds.  In particular, when used as a template evaluation tool without any processing (which it often is), it results in conflicting "requirements" to evaluate against, especially with regard to deployment mode. I'd suggest taking a middle ground and keeping the deployment modes section, but changing the nature of the content to better explain and lend itself to a non-conflicting evaluation, but also to include customer goals / use cases as a section.

I also think that the use cases section could help us solve #2.

4 - my opinion would be to keep a flat list, but to provide a tool that let's the customer adjust importance based on their needs.

5 - I would leave this out of the criteria (since WAFs don't fix code it doesn't make sense to have fixing code as an evaluation element).  IMO, this is a better topic for a different kind of forum...this tool is supposed to be a tool to evaluate WAFs.

6 - I think really orienting the criteria to be useable framework for an actual evaluation will make this simpler.

From: wasc-wafec-bounces@lists.webappsec.orgmailto:wasc-wafec-bounces@lists.webappsec.org [mailto:wasc-wafec-bounces@lists.webappsec.org]mailto:[mailto:wasc-wafec-bounces@lists.webappsec.org] On Behalf Of Ofer Shezaf
Sent: Wednesday, June 06, 2012 4:39 AM
To: wasc-wafec@lists.webappsec.orgmailto:wasc-wafec@lists.webappsec.org
Subject: [WASC-WAFEC] What should we change in WAFEC 2.0?

Since you are all very quiet, I understand that WAFEC 2 will solve my pain and needs only :)

To that end, let me start with summarizing issues raised in the previous discussions on the mailing list (which I actually went and read...).

No specific order intended. This is what you wrote, though I must say I think it captures well the issues I am aware of and that generally speaking I agree with most.

1.  Remove non WAF related criteria for example around application delivery.
   

•    While integrating a WAF with other solutions is compelling to the client, it is not directly about WAFs and is also unbounded. This does present the challenge deciding what is relevant to a WAF in border cases such as an SSO functionality
  

2.  Update the list of threats covered
   

3.  Focus on customer use cases rather than how a WAF operates
   

•    I think there was some hidden controversy here as I read opinions to focus on "technical" which I take to be opposite. I personally very much agree with this comment.
  

4.  Not just a laundry list -
   

•    Classify the importance of requirements. I believe that a minimal approach specifying several levels, for example: "mandatory", "important", "nice to have" and "site specific".
  

•    Another complementing idea is to classify requirements as "security"/"functionality"/"performance" etc. letting the user determine if he prefers security over functionality etc.
  

•    This would also provide the minimum requirements for a solution to be a WAF - the "mandatory" requirements.
  

•    Regarding site specific requirements, it should be easy to the user to determine his own requirements, for example using a decision tree.
  

5.  The "ethical" questions:
   

•    How to address alternative solutions such as fixing the code?
  

6.  Outreach - beyond the document
   

•    Approaching NSS, ICSA and the likes to use WAFEC
  

•    Release process, PR etc.
  

•    Managing a list of public references to WAFEC
  

•    Promote actual evaluations data sharing - No more spreadsheets.
  

7.  Specific notes on V1, I have collected for further work.
   

A major question raised with opinions on both sides was a re-write vs. an update. I do think that understanding the requirements should direct that. Some issues raised which directly relate to that are:

1.  Is the order of sections correct?
   

2.  Incorporating the German OWASP chapter work on the same subject: http://www.owasp.org/index.php/Best_Practices:_Web_Application_Firewalls
   

~ Ofer

From: Ofer Shezaf [mailto:ofer@shezaf.com]
Sent: Thursday, May 31, 2012 1:45 PM
To: wasc-wafec@lists.webappsec.orgmailto:wasc-wafec@lists.webappsec.org
Subject: WAFEC 2.0 phase 1: exploratory discussion (deadline: June 14th)

Thanks to all who volunteered to contribute to this project going forward (and those who didn't - you still can!)

I would like to boot up the project with a short exploratory phase identifying why we need a new release and therefore what we need in it.

To guide the discussion, I think that the reasons we need v2 fall into two categories:

1. Things that have changed - new (or obsolete) deployment modes, techniques, attacks, or even something new altogether.
   

2. Issues we discovered in WAFEC over the years. Some issues I encountered are identifying specific requirements and sorting out what's important and what's not.
   

I would give this phase two weeks (until June 14th), however I am on vacation from the 9th, so would accept input but not join the discussion on the last few days.

I would also want to thank Thorsten and Mirko for leading the project until now. I do hope that I will get from you all more cooperation than they did! I would also want to extend a personal apology to Thorsten and Mirko as the leader switch was not well coordinated. Thorsten and I discussed this over the last week and he gracefully agreed to let me give a try to leading this project forward.

Thank you all!
~ Ofer

Ofer Shezaf
[+972-54-4431119; ofer@shezaf.commailto:ofer@shezaf.com, www.shezaf.comhttp://www.shezaf.com]