Empathy List Archives

SK

Sherif Koussa

Thu, Aug 18, 2011 3:53 PM

Hi All,

Ok, so the work on the categories is done. Now we move to the
sub-categories. Since we have a relatively large number of sub-categories,
please check the sub-categories
herehttp://projects.webappsec.org/w/page/42093482/Static%20Analysis%20Tool%20Evaluation%20Criteria%20Working.
Please send me only the ones that you think should be edited or removed, and
I will assume that the ones that are not included as keepers. So here is an
example:

1.2 Skills required to perform initial installation - REMOVE
3.1 Languages supported by the tool - EDIT: Languages supported by the tool
in a single license (or whatever)
7.1 Frequency of signature update - EDIT: Frequency of rules update
...etc

We will keep the voting open until Friday August 26th. Looking forward to
hear from you all.

Regards,
Sherif

Hi All, Ok, so the work on the categories is done. Now we move to the sub-categories. Since we have a relatively large number of sub-categories, please check the sub-categories here<http://projects.webappsec.org/w/page/42093482/Static%20Analysis%20Tool%20Evaluation%20Criteria%20Working>. Please send me only the ones that you think should be edited or removed, and I will assume that the ones that are not included as keepers. So here is an example: 1.2 Skills required to perform initial installation - REMOVE 3.1 Languages supported by the tool - EDIT: Languages supported by the tool in a single license (or whatever) 7.1 Frequency of signature update - EDIT: Frequency of rules update ...etc We will keep the voting open until *Friday August 26th*. Looking forward to hear from you all. Regards, Sherif

AS

Alec Shcherbakov

Wed, Aug 24, 2011 1:40 AM

Here's what I have to add so far.

Tool Setup and Installation
1.6 3d party components required to run the tool (web/application/database servers, web services, containers, API libraries etc.) - ADD
Tool Coverage:
3.3 Support for Syntactic Analysis - REMOVE: in my opinion syntactic analysis does not directly unveil vulnerabilities, but rather serves as a preliminary step: a part of the parsing process when the source code is parsed to some sort of a program model. Then the model is used for all other kinds of analysis (semantic, data flow, etc.)
3.6 Ability of the tool to trace taint through complex data flows (when taint goes through collection objects, database, config. files) - ADD
3.7 Support for Program Structure Analysis - ADD
3.8 Support for Control/Data Flow Analysis - ADD
3.9 Support for Configuration Analysis - ADD
Detection Accuracy
4.1 Number of false positives - EDIT: Better use %% of false positives to the total # of findings in the category. The absolute number may not be as useful for the analysis and comparison purposes as it depends on many factors.
4.2 Number of true negatives - EDIT: Should it not be "false negatives" instead?
4.3 Accuracy % - EDIT: not sure what this means, can you elaborate?
Triage and Remediation Process
5.9 Ability to customize marking classification (e.g. Exploitable, False positive, Bad practice etc.) - ADD
5.10 Ability to add reviewer comments for each reviewed finding - ADD

Alec

From: wasc-satec-bounces@lists.webappsec.org [mailto:wasc-satec-bounces@lists.webappsec.org] On Behalf Of Sherif Koussa
Sent: Thursday, August 18, 2011 8:53 AM
To: wasc-satec@lists.webappsec.org
Subject: [WASC-SATEC] Sub-Categories Voting Now Open

Hi All,

Ok, so the work on the categories is done. Now we move to the sub-categories. Since we have a relatively large number of sub-categories, please check the sub-categories herehttp://projects.webappsec.org/w/page/42093482/Static%20Analysis%20Tool%20Evaluation%20Criteria%20Working. Please send me only the ones that you think should be edited or removed, and I will assume that the ones that are not included as keepers. So here is an example:

1.2 Skills required to perform initial installation - REMOVE
3.1 Languages supported by the tool - EDIT: Languages supported by the tool in a single license (or whatever)
7.1 Frequency of signature update - EDIT: Frequency of rules update
...etc

We will keep the voting open until Friday August 26th. Looking forward to hear from you all.

Regards,
Sherif

Here's what I have to add so far. 1. Tool Setup and Installation 1.6 3d party components required to run the tool (web/application/database servers, web services, containers, API libraries etc.) - ADD 3. Tool Coverage: 3.3 Support for Syntactic Analysis - REMOVE: in my opinion syntactic analysis does not directly unveil vulnerabilities, but rather serves as a preliminary step: a part of the parsing process when the source code is parsed to some sort of a program model. Then the model is used for all other kinds of analysis (semantic, data flow, etc.) 3.6 Ability of the tool to trace taint through complex data flows (when taint goes through collection objects, database, config. files) - ADD 3.7 Support for Program Structure Analysis - ADD 3.8 Support for Control/Data Flow Analysis - ADD 3.9 Support for Configuration Analysis - ADD 4. Detection Accuracy 4.1 Number of false positives - EDIT: Better use %% of false positives to the total # of findings in the category. The absolute number may not be as useful for the analysis and comparison purposes as it depends on many factors. 4.2 Number of true negatives - EDIT: Should it not be "false negatives" instead? 4.3 Accuracy % - EDIT: not sure what this means, can you elaborate? 5. Triage and Remediation Process 5.9 Ability to customize marking classification (e.g. Exploitable, False positive, Bad practice etc.) - ADD 5.10 Ability to add reviewer comments for each reviewed finding - ADD - Alec From: wasc-satec-bounces@lists.webappsec.org [mailto:wasc-satec-bounces@lists.webappsec.org] On Behalf Of Sherif Koussa Sent: Thursday, August 18, 2011 8:53 AM To: wasc-satec@lists.webappsec.org Subject: [WASC-SATEC] Sub-Categories Voting Now Open Hi All, Ok, so the work on the categories is done. Now we move to the sub-categories. Since we have a relatively large number of sub-categories, please check the sub-categories here<http://projects.webappsec.org/w/page/42093482/Static%20Analysis%20Tool%20Evaluation%20Criteria%20Working>. Please send me only the ones that you think should be edited or removed, and I will assume that the ones that are not included as keepers. So here is an example: 1.2 Skills required to perform initial installation - REMOVE 3.1 Languages supported by the tool - EDIT: Languages supported by the tool in a single license (or whatever) 7.1 Frequency of signature update - EDIT: Frequency of rules update ...etc We will keep the voting open until Friday August 26th. Looking forward to hear from you all. Regards, Sherif

HS

Herman Stevens

Wed, Aug 24, 2011 7:00 AM

I Would like to suggest for the reporting category:

 Output format supported (e.g. XML, PDF, HTML, ..)

```
 Ability to generate 'diff' reports
```

 Ability to map findings to CVE, CWE, CAPAC

Herman

From: wasc-satec-bounces@lists.webappsec.org mailto:wasc-satec-bounces@lists.webappsec.org [mailto:wasc-satec-bounces@lists.webappsec.org]mailto:[mailto:wasc-satec-bounces@lists.webappsec.org] On Behalf Of Sherif Koussa
Sent: Thursday, August 18, 2011 8:53 AM
To: wasc-satec@lists.webappsec.org mailto:wasc-satec@lists.webappsec.org
Subject: [WASC-SATEC] Sub-Categories Voting Now Open

Hi All,

Ok, so the work on the categories is done. Now we move to the sub-categories. Since we have a relatively large number of sub-categories, please check the sub-categories herehttp://projects.webappsec.org/w/page/42093482/Static%20Analysis%20Tool%20Evaluation%20Criteria%20Working. Please send me only the ones that you think should be edited or removed, and I will assume that the ones that are not included as keepers. So here is an example:

1.2 Skills required to perform initial installation - REMOVE
3.1 Languages supported by the tool - EDIT: Languages supported by the tool in a single license (or whatever)
7.1 Frequency of signature update - EDIT: Frequency of rules update
...etc

We will keep the voting open until Friday August 26th. Looking forward to hear from you all.

Regards,
Sherif

I Would like to suggest for the reporting category: 1) Output format supported (e.g. XML, PDF, HTML, ..) 2) Ability to generate 'diff' reports 3) Ability to map findings to CVE, CWE, CAPAC Herman From: wasc-satec-bounces@lists.webappsec.org<mailto:wasc-satec-bounces@lists.webappsec.org> [mailto:wasc-satec-bounces@lists.webappsec.org]<mailto:[mailto:wasc-satec-bounces@lists.webappsec.org]> On Behalf Of Sherif Koussa Sent: Thursday, August 18, 2011 8:53 AM To: wasc-satec@lists.webappsec.org<mailto:wasc-satec@lists.webappsec.org> Subject: [WASC-SATEC] Sub-Categories Voting Now Open Hi All, Ok, so the work on the categories is done. Now we move to the sub-categories. Since we have a relatively large number of sub-categories, please check the sub-categories here<http://projects.webappsec.org/w/page/42093482/Static%20Analysis%20Tool%20Evaluation%20Criteria%20Working>. Please send me only the ones that you think should be edited or removed, and I will assume that the ones that are not included as keepers. So here is an example: 1.2 Skills required to perform initial installation - REMOVE 3.1 Languages supported by the tool - EDIT: Languages supported by the tool in a single license (or whatever) 7.1 Frequency of signature update - EDIT: Frequency of rules update ...etc We will keep the voting open until Friday August 26th. Looking forward to hear from you all. Regards, Sherif