wasc-satec@lists.webappsec.org

WASC Static Analysis Tool Evaluation Criteria

View all threads

My Vote

MA
Mushtaq Ahmed
Fri, Aug 12, 2011 8:14 PM

My vote with my two cents,

  1. Tool Setup and Installation  - KEEP

  2. Configuration and Project Setup  - KEEP

  3. Scan Coverage and Accuracy - KEEP (Falls positives and falls negatives
    need to be detailed out, Further this should be scanning the security issues
    based on the OWASP top 10, SANS 20 and well known industrial standards,
    Where would PCI requirements fall ? This section is the crux i feel, having
    sub divisions would help)

  4. Triage and Remediation Process - KEEP

  5. UI Simplicity and Intuitiveness - KEEP

  6. Product Update Quality - KEEP

  7. Product Maturity and Scalability - KEEP

  8. Enterprise Offerings - KEEP

  9. Reporting Capabilities - KEEP

  10. Tool Customization and Automation - KEEP

- Support needs to be considered as well, the solution might be a open
source, freeware or might be an commercial product. In any case support
would make a lot of sense to organizations whose primary business is not
development and where security is still a concern.
- Should the solution be language specific .NET, Java, etc .. what
happens to the legacy technology such as CICS, Tandem etc ..

Regards,
Mushtaq

My vote with my two cents, > > 1. Tool Setup and Installation - KEEP > > 2. Configuration and Project Setup - KEEP > > 3. Scan Coverage and Accuracy - KEEP (Falls positives and falls negatives > need to be detailed out, Further this should be scanning the security issues > based on the OWASP top 10, SANS 20 and well known industrial standards, > Where would PCI requirements fall ? This section is the crux i feel, having > sub divisions would help) > > > 4. Triage and Remediation Process - KEEP > > 5. UI Simplicity and Intuitiveness - KEEP > > 6. Product Update Quality - KEEP > > 7. Product Maturity and Scalability - KEEP > > 8. Enterprise Offerings - KEEP > > > 9. Reporting Capabilities - KEEP > > 10. Tool Customization and Automation - KEEP > > > - Support needs to be considered as well, the solution might be a open > source, freeware or might be an commercial product. In any case support > would make a lot of sense to organizations whose primary business is not > development and where security is still a concern. > - Should the solution be language specific .NET, Java, etc .. what > happens to the legacy technology such as CICS, Tandem etc .. > > > Regards, > Mushtaq >
RA
Robert A.
Fri, Aug 12, 2011 8:31 PM

Regarding 'support', I would consider support a service and not an
attribute of the tool itself. Given that this criteria is focused on the
tool side it would seem out of place.

my 0.02.

  • Robert

On Sat, 13 Aug 2011, Mushtaq Ahmed wrote:

My vote with my two cents,

  1. Tool Setup and Installation  - KEEP

  2. Configuration and Project Setup  - KEEP

  3. Scan Coverage and Accuracy - KEEP (Falls positives and falls negatives
    need to be detailed out, Further this should be scanning the security issues
    based on the OWASP top 10, SANS 20 and well known industrial standards,
    Where would PCI requirements fall ? This section is the crux i feel, having
    sub divisions would help)

  4. Triage and Remediation Process - KEEP

  5. UI Simplicity and Intuitiveness - KEEP

  6. Product Update Quality - KEEP

  7. Product Maturity and Scalability - KEEP

  8. Enterprise Offerings - KEEP

  9. Reporting Capabilities - KEEP

  10. Tool Customization and Automation - KEEP

- Support needs to be considered as well, the solution might be a open
source, freeware or might be an commercial product. In any case support
would make a lot of sense to organizations whose primary business is not
development and where security is still a concern.
- Should the solution be language specific .NET, Java, etc .. what
happens to the legacy technology such as CICS, Tandem etc ..

Regards,
Mushtaq

Regarding 'support', I would consider support a service and not an attribute of the tool itself. Given that this criteria is focused on the tool side it would seem out of place. my 0.02. - Robert On Sat, 13 Aug 2011, Mushtaq Ahmed wrote: > My vote with my two cents, >> >> 1. Tool Setup and Installation - KEEP >> >> 2. Configuration and Project Setup - KEEP >> >> 3. Scan Coverage and Accuracy - KEEP (Falls positives and falls negatives >> need to be detailed out, Further this should be scanning the security issues >> based on the OWASP top 10, SANS 20 and well known industrial standards, >> Where would PCI requirements fall ? This section is the crux i feel, having >> sub divisions would help) >> >> >> 4. Triage and Remediation Process - KEEP >> >> 5. UI Simplicity and Intuitiveness - KEEP >> >> 6. Product Update Quality - KEEP >> >> 7. Product Maturity and Scalability - KEEP >> >> 8. Enterprise Offerings - KEEP >> >> >> 9. Reporting Capabilities - KEEP >> >> 10. Tool Customization and Automation - KEEP >> >> >> - Support needs to be considered as well, the solution might be a open >> source, freeware or might be an commercial product. In any case support >> would make a lot of sense to organizations whose primary business is not >> development and where security is still a concern. >> - Should the solution be language specific .NET, Java, etc .. what >> happens to the legacy technology such as CICS, Tandem etc .. >> >> >> Regards, >> Mushtaq >> >
M
Mushtaq
Fri, Aug 12, 2011 10:15 PM

Well Robert I get your point how ever my thought process was as below

If I were from an organization who is planning to implement a source code scanner and i noticed that the scan solution is not updated since 2006 and no company owns this free scanner then would i ask the management to implement this solution even though it does the job for now. How would we convince the management that if we run into issues post implementation then how do we still solved the glitches. All this assumption are based on the fact that the company planning to implement the sca's primary business is not development.

Support's availability and un-availability can be a big turn off in any implementation.

E.g why are we moving to the latest windows even though the legacy windows does the job we require. One of the reasons being Microsoft stops supporting the legacy releases. Point to think about. But this is just my thoughts.

Regards,
Mushtaq

On Aug 13, 2011, at 12:31 AM, "Robert A." robert@webappsec.org wrote:

Regarding 'support', I would consider support a service and not an attribute of the tool itself. Given that this criteria is focused on the tool side it would seem out of place.

my 0.02.

  • Robert

On Sat, 13 Aug 2011, Mushtaq Ahmed wrote:

My vote with my two cents,

  1. Tool Setup and Installation  - KEEP

  2. Configuration and Project Setup  - KEEP

  3. Scan Coverage and Accuracy - KEEP (Falls positives and falls negatives
    need to be detailed out, Further this should be scanning the security issues
    based on the OWASP top 10, SANS 20 and well known industrial standards,
    Where would PCI requirements fall ? This section is the crux i feel, having
    sub divisions would help)

  4. Triage and Remediation Process - KEEP

  5. UI Simplicity and Intuitiveness - KEEP

  6. Product Update Quality - KEEP

  7. Product Maturity and Scalability - KEEP

  8. Enterprise Offerings - KEEP

  9. Reporting Capabilities - KEEP

  10. Tool Customization and Automation - KEEP

  • Support needs to be considered as well, the solution might be a open
    source, freeware or might be an commercial product. In any case support
    would make a lot of sense to organizations whose primary business is not
    development and where security is still a concern.
  • Should the solution be language specific .NET, Java, etc .. what
    happens to the legacy technology such as CICS, Tandem etc ..

Regards,
Mushtaq

Well Robert I get your point how ever my thought process was as below If I were from an organization who is planning to implement a source code scanner and i noticed that the scan solution is not updated since 2006 and no company owns this free scanner then would i ask the management to implement this solution even though it does the job for now. How would we convince the management that if we run into issues post implementation then how do we still solved the glitches. All this assumption are based on the fact that the company planning to implement the sca's primary business is not development. Support's availability and un-availability can be a big turn off in any implementation. E.g why are we moving to the latest windows even though the legacy windows does the job we require. One of the reasons being Microsoft stops supporting the legacy releases. Point to think about. But this is just my thoughts. Regards, Mushtaq On Aug 13, 2011, at 12:31 AM, "Robert A." <robert@webappsec.org> wrote: > Regarding 'support', I would consider support a service and not an attribute of the tool itself. Given that this criteria is focused on the tool side it would seem out of place. > > my 0.02. > > - Robert > > > > On Sat, 13 Aug 2011, Mushtaq Ahmed wrote: > >> My vote with my two cents, >>> >>> 1. Tool Setup and Installation - KEEP >>> >>> 2. Configuration and Project Setup - KEEP >>> >>> 3. Scan Coverage and Accuracy - KEEP (Falls positives and falls negatives >>> need to be detailed out, Further this should be scanning the security issues >>> based on the OWASP top 10, SANS 20 and well known industrial standards, >>> Where would PCI requirements fall ? This section is the crux i feel, having >>> sub divisions would help) >>> >>> >>> 4. Triage and Remediation Process - KEEP >>> >>> 5. UI Simplicity and Intuitiveness - KEEP >>> >>> 6. Product Update Quality - KEEP >>> >>> 7. Product Maturity and Scalability - KEEP >>> >>> 8. Enterprise Offerings - KEEP >>> >>> >>> 9. Reporting Capabilities - KEEP >>> >>> 10. Tool Customization and Automation - KEEP >>> >>> >>> - Support needs to be considered as well, the solution might be a open >>> source, freeware or might be an commercial product. In any case support >>> would make a lot of sense to organizations whose primary business is not >>> development and where security is still a concern. >>> - Should the solution be language specific .NET, Java, etc .. what >>> happens to the legacy technology such as CICS, Tandem etc .. >>> >>> >>> Regards, >>> Mushtaq >>> >>