My vote with my two cents, 1. Tool Setup and Installation - KEEP 2. Configuration and Project Setup - KEEP 3. Scan Coverage and Accuracy - KEEP (Falls positives and falls negatives need to be detailed out, Further this should be scanning the security issues based on the OWASP top 10, SANS 20 and well known industrial standards, Where would PCI requirements fall ? This section is the crux i feel, having sub divisions would help) 4. Triage and Remediation Process - KEEP 5. UI Simplicity and Intuitiveness - KEEP 6. Product Update Quality - KEEP 7. Product Maturity and Scalability - KEEP 8. Enterprise Offerings - KEEP 9. Reporting Capabilities - KEEP 10. Tool Customization and Automation - KEEP - Support needs to be considered as well, the solution might be a open source, freeware or might be an commercial product. In any case support would make a lot of sense to organizations whose primary business is not development and where security is still a concern. - Should the solution be language specific .NET, Java, etc .. what happens to the legacy technology such as CICS, Tandem etc .. Regards, Mushtaq