websecurity@lists.webappsec.org

The Web Security Mailing List

View all threads

Security test case automation

VS
vedantam sekhar
Thu, Jan 23, 2014 4:30 AM

Hi group,

Need your help here. as part of QA team, we will be writing security test
cases and also executing them manually using OWASP standard. However, i
have been given task to see the possibility to automate these test cases.
are there any tools/frameworks available for us to achieve this?

Thanks and Regards,

sekhar

Hi group, Need your help here. as part of QA team, we will be writing security test cases and also executing them manually using OWASP standard. However, i have been given task to see the possibility to automate these test cases. are there any tools/frameworks available for us to achieve this? Thanks and Regards, sekhar
SD
Stephen de Vries
Thu, Jan 23, 2014 6:14 AM

On 23 Jan 2014, at 05:30, vedantam sekhar vedantamsekhar@gmail.com wrote:

Need your help here. as part of QA team, we will be writing security test cases and also executing them manually using OWASP standard. However, i have been given task to see the possibility to automate these test cases. are there any tools/frameworks available for us to achieve this?

Hi Sekhar,

You could take a look at the BDD-Security framework http://www.continuumsecurity.net/bdd-intro.html  (I’m the author), it was designed to do exactly this using Selenium WebDriver, OWASP ZAP, JBehave and optionally TestNG.
One of the core principals is to separate the security test cases from the navigation of the application under test, so it comes bundles with a number of pre-written generic security tests that can be applied to most web applications without modification.  The focus is on writing the tests in JBehave which allows them to be written in a natural language, English by default but it can be changed.  If however you prefer to work in pure Java, the same tests are also provided in TestNG, so they can be run directly from an IDE.
One of the advantages of using test cases over pure scanning is that you can do more in-depth and intelligent testing, for example, automated access control tests between users and between roles: http://www.continuumsecurity.net/2013/12/07/Automated-Access-Control-Tests.html

The documentation is far from complete, but there are some useful bits captured in the blog posts.  Code is open source:  https://github.com/continuumsecurity/bdd-security
Feel free to get in touch off-list if you run into any issues.

regards,
Stephen

On 23 Jan 2014, at 05:30, vedantam sekhar <vedantamsekhar@gmail.com> wrote: > Need your help here. as part of QA team, we will be writing security test cases and also executing them manually using OWASP standard. However, i have been given task to see the possibility to automate these test cases. are there any tools/frameworks available for us to achieve this? Hi Sekhar, You could take a look at the BDD-Security framework http://www.continuumsecurity.net/bdd-intro.html (I’m the author), it was designed to do exactly this using Selenium WebDriver, OWASP ZAP, JBehave and optionally TestNG. One of the core principals is to separate the security test cases from the navigation of the application under test, so it comes bundles with a number of pre-written generic security tests that can be applied to most web applications without modification. The focus is on writing the tests in JBehave which allows them to be written in a natural language, English by default but it can be changed. If however you prefer to work in pure Java, the same tests are also provided in TestNG, so they can be run directly from an IDE. One of the advantages of using test cases over pure scanning is that you can do more in-depth and intelligent testing, for example, automated access control tests between users and between roles: http://www.continuumsecurity.net/2013/12/07/Automated-Access-Control-Tests.html The documentation is far from complete, but there are some useful bits captured in the blog posts. Code is open source: https://github.com/continuumsecurity/bdd-security Feel free to get in touch off-list if you run into any issues. regards, Stephen
WA
Ward, Allan
Thu, Jan 23, 2014 7:05 AM

Veracode provide a service that you can run dynamic testing  and static  code analysis to test for OWASP vulnerabilities. There are other solutions that can also be used.
Do you have any spend or you looking for open source?

W use App scan, and Parasoft, but Fortify also have very good tools in this space.

Regards

Allan

Allan Ward, Internal Controls & Compliance Specialist (SOX), Global Security and Risk, D&B, Marlow International, Parkway, Marlow, SL7 1AJ, * (44) (0)1628 492709, * warda@dnb.commailto:warda@dnb.com

From: websecurity [mailto:websecurity-bounces@lists.webappsec.org] On Behalf Of Stephen de Vries
Sent: 23 January 2014 06:15
To: vedantam sekhar
Cc: Webappsec Group
Subject: Re: [WEB SECURITY] Security test case automation

On 23 Jan 2014, at 05:30, vedantam sekhar <vedantamsekhar@gmail.commailto:vedantamsekhar@gmail.com> wrote:

Need your help here. as part of QA team, we will be writing security test cases and also executing them manually using OWASP standard. However, i have been given task to see the possibility to automate these test cases. are there any tools/frameworks available for us to achieve this?

Hi Sekhar,

You could take a look at the BDD-Security framework http://www.continuumsecurity.net/bdd-intro.html  (I'm the author), it was designed to do exactly this using Selenium WebDriver, OWASP ZAP, JBehave and optionally TestNG.
One of the core principals is to separate the security test cases from the navigation of the application under test, so it comes bundles with a number of pre-written generic security tests that can be applied to most web applications without modification.  The focus is on writing the tests in JBehave which allows them to be written in a natural language, English by default but it can be changed.  If however you prefer to work in pure Java, the same tests are also provided in TestNG, so they can be run directly from an IDE.
One of the advantages of using test cases over pure scanning is that you can do more in-depth and intelligent testing, for example, automated access control tests between users and between roles: http://www.continuumsecurity.net/2013/12/07/Automated-Access-Control-Tests.html

The documentation is far from complete, but there are some useful bits captured in the blog posts.  Code is open source:  https://github.com/continuumsecurity/bdd-security
Feel free to get in touch off-list if you run into any issues.

regards,
Stephen

Veracode provide a service that you can run dynamic testing and static code analysis to test for OWASP vulnerabilities. There are other solutions that can also be used. Do you have any spend or you looking for open source? W use App scan, and Parasoft, but Fortify also have very good tools in this space. Regards Allan Allan Ward, Internal Controls & Compliance Specialist (SOX), Global Security and Risk, D&B, Marlow International, Parkway, Marlow, SL7 1AJ, * (44) (0)1628 492709, * warda@dnb.com<mailto:warda@dnb.com> From: websecurity [mailto:websecurity-bounces@lists.webappsec.org] On Behalf Of Stephen de Vries Sent: 23 January 2014 06:15 To: vedantam sekhar Cc: Webappsec Group Subject: Re: [WEB SECURITY] Security test case automation On 23 Jan 2014, at 05:30, vedantam sekhar <vedantamsekhar@gmail.com<mailto:vedantamsekhar@gmail.com>> wrote: Need your help here. as part of QA team, we will be writing security test cases and also executing them manually using OWASP standard. However, i have been given task to see the possibility to automate these test cases. are there any tools/frameworks available for us to achieve this? Hi Sekhar, You could take a look at the BDD-Security framework http://www.continuumsecurity.net/bdd-intro.html (I'm the author), it was designed to do exactly this using Selenium WebDriver, OWASP ZAP, JBehave and optionally TestNG. One of the core principals is to separate the security test cases from the navigation of the application under test, so it comes bundles with a number of pre-written generic security tests that can be applied to most web applications without modification. The focus is on writing the tests in JBehave which allows them to be written in a natural language, English by default but it can be changed. If however you prefer to work in pure Java, the same tests are also provided in TestNG, so they can be run directly from an IDE. One of the advantages of using test cases over pure scanning is that you can do more in-depth and intelligent testing, for example, automated access control tests between users and between roles: http://www.continuumsecurity.net/2013/12/07/Automated-Access-Control-Tests.html The documentation is far from complete, but there are some useful bits captured in the blog posts. Code is open source: https://github.com/continuumsecurity/bdd-security Feel free to get in touch off-list if you run into any issues. regards, Stephen
PJ
Paul Johnston
Thu, Jan 23, 2014 9:44 AM

Hi,

You can automate some security testing. You are best using a dedicated
security testing tool, rather than putting security test cases into a
general testing tool. There are two main approaches: DAST which scans a
running web app, and SAST which analyses source code. They have
different strengths and weaknesses, so you get the most benefit from
running both and combining the results.

What you cannot automate is the mindset of a hacker. Security is not
just about checking for a known set of issues. It is about using
creativity and intuition to think up new ways of attacking a particular
application. So while doing your own QA using DAST/SAST is good, you
should also include some manual penetration testing in your security
programme.

Paul

On 23/01/2014 04:30, vedantam sekhar wrote:

Hi group,

Need your help here. as part of QA team, we will be writing security
test cases and also executing them manually using OWASP standard.
However, i have been given task to see the possibility to automate
these test cases. are there any tools/frameworks available for us to
achieve this?

Thanks and Regards,

sekhar


The Web Security Mailing List

WebSecurity RSS Feed
http://www.webappsec.org/rss/websecurity.rss

Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA

WASC on Twitter
http://twitter.com/wascupdates

websecurity@lists.webappsec.org
http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org

--

Pentest - The Application Security Specialists

Paul Johnston - IT Security Consultant / Tiger SST
Office: +44 (0) 161 233 0100
Mobile: +44 (0) 7817 219 072

We're exhibiting at Infosecurity Europe!
Stand K97, Earl's Court London - 29th April - 1st May

Infosecurity Europe 2014

Email policy: http://www.pentest.co.uk/legal.shtml#emailpolicy
Registered Number: 4217114 England & Wales
Registered Office: 26a The Downs, Altrincham, Cheshire, WA14 2PU, UK
Accreditations: ISO 9001 (44/100/107029) / ISO 27001 (IS 558982) / Tiger
Scheme

Hi, You can automate some security testing. You are best using a dedicated security testing tool, rather than putting security test cases into a general testing tool. There are two main approaches: DAST which scans a running web app, and SAST which analyses source code. They have different strengths and weaknesses, so you get the most benefit from running both and combining the results. What you cannot automate is the mindset of a hacker. Security is not just about checking for a known set of issues. It is about using creativity and intuition to think up new ways of attacking a particular application. So while doing your own QA using DAST/SAST is good, you should also include some manual penetration testing in your security programme. Paul On 23/01/2014 04:30, vedantam sekhar wrote: > Hi group, > > Need your help here. as part of QA team, we will be writing security > test cases and also executing them manually using OWASP standard. > However, i have been given task to see the possibility to automate > these test cases. are there any tools/frameworks available for us to > achieve this? > > Thanks and Regards, > > sekhar > > > > _______________________________________________ > The Web Security Mailing List > > WebSecurity RSS Feed > http://www.webappsec.org/rss/websecurity.rss > > Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA > > WASC on Twitter > http://twitter.com/wascupdates > > websecurity@lists.webappsec.org > http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org -- Pentest - The Application Security Specialists Paul Johnston - IT Security Consultant / Tiger SST Office: +44 (0) 161 233 0100 Mobile: +44 (0) 7817 219 072 *We're exhibiting at Infosecurity Europe! Stand K97, Earl's Court London - 29th April - 1st May* Infosecurity Europe 2014 Email policy: _http://www.pentest.co.uk/legal.shtml#emailpolicy_ Registered Number: 4217114 England & Wales Registered Office: 26a The Downs, Altrincham, Cheshire, WA14 2PU, UK Accreditations: ISO 9001 (44/100/107029) / ISO 27001 (IS 558982) / Tiger Scheme
SD
Stephen de Vries
Thu, Jan 23, 2014 9:49 AM

On 23 Jan 2014, at 10:44, Paul Johnston paul.johnston@pentest.co.uk wrote:

What you cannot automate is the mindset of a hacker. Security is not just about checking for a known set of issues. It is about using creativity and intuition to think up new ways of attacking a particular application. So while doing your own QA using DAST/SAST is good, you should also include some manual penetration testing in your security programme.

…and once you’ve found vulnerabilities through a manual test you can record and automate those findings with a testing framework.  Then you can re-run those same tests on your application periodically or even continuously to ensure that code changes to the app don’t introduce security regressions.

Stephen

On 23/01/2014 04:30, vedantam sekhar wrote:

Hi group,

Need your help here. as part of QA team, we will be writing security test cases and also executing them manually using OWASP standard. However, i have been given task to see the possibility to automate these test cases. are there any tools/frameworks available for us to achieve this?

Thanks and Regards,

sekhar


The Web Security Mailing List

WebSecurity RSS Feed
http://www.webappsec.org/rss/websecurity.rss

Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA

WASC on Twitter
http://twitter.com/wascupdates

websecurity@lists.webappsec.org
http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org

--
Pentest - The Application Security Specialists

Paul Johnston - IT Security Consultant / Tiger SST
Office: +44 (0) 161 233 0100
Mobile: +44 (0) 7817 219 072

We're exhibiting at Infosecurity Europe!
Stand K97, Earl's Court London - 29th April - 1st May
<logos-dl-infosec-withoutdates.png>

Email policy: http://www.pentest.co.uk/legal.shtml#emailpolicy
Registered Number: 4217114 England & Wales
Registered Office: 26a The Downs, Altrincham, Cheshire, WA14 2PU, UK
Accreditations: ISO 9001 (44/100/107029) / ISO 27001 (IS 558982) / Tiger Scheme


The Web Security Mailing List

WebSecurity RSS Feed
http://www.webappsec.org/rss/websecurity.rss

Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA

WASC on Twitter
http://twitter.com/wascupdates

websecurity@lists.webappsec.org
http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org

On 23 Jan 2014, at 10:44, Paul Johnston <paul.johnston@pentest.co.uk> wrote: > > What you cannot automate is the mindset of a hacker. Security is not just about checking for a known set of issues. It is about using creativity and intuition to think up new ways of attacking a particular application. So while doing your own QA using DAST/SAST is good, you should also include some manual penetration testing in your security programme. …and once you’ve found vulnerabilities through a manual test you can record and automate those findings with a testing framework. Then you can re-run those same tests on your application periodically or even continuously to ensure that code changes to the app don’t introduce security regressions. Stephen > > > On 23/01/2014 04:30, vedantam sekhar wrote: >> Hi group, >> >> Need your help here. as part of QA team, we will be writing security test cases and also executing them manually using OWASP standard. However, i have been given task to see the possibility to automate these test cases. are there any tools/frameworks available for us to achieve this? >> >> Thanks and Regards, >> >> sekhar >> >> >> >> _______________________________________________ >> The Web Security Mailing List >> >> WebSecurity RSS Feed >> http://www.webappsec.org/rss/websecurity.rss >> >> Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA >> >> WASC on Twitter >> http://twitter.com/wascupdates >> >> websecurity@lists.webappsec.org >> http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org > > -- > Pentest - The Application Security Specialists > > Paul Johnston - IT Security Consultant / Tiger SST > Office: +44 (0) 161 233 0100 > Mobile: +44 (0) 7817 219 072 > > We're exhibiting at Infosecurity Europe! > Stand K97, Earl's Court London - 29th April - 1st May > <logos-dl-infosec-withoutdates.png> > > Email policy: http://www.pentest.co.uk/legal.shtml#emailpolicy > Registered Number: 4217114 England & Wales > Registered Office: 26a The Downs, Altrincham, Cheshire, WA14 2PU, UK > Accreditations: ISO 9001 (44/100/107029) / ISO 27001 (IS 558982) / Tiger Scheme > > _______________________________________________ > The Web Security Mailing List > > WebSecurity RSS Feed > http://www.webappsec.org/rss/websecurity.rss > > Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA > > WASC on Twitter > http://twitter.com/wascupdates > > websecurity@lists.webappsec.org > http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org
WA
Ward, Allan
Thu, Jan 23, 2014 10:04 AM

SAST is best built into the code build process as an automated process, so all code submitted to the library gets static code analysis (SAST).

SAST scans code a little like a word spell checker, looking for coding errors that create vulnerabilities.
DAST runs on the running solution, can also be automated.

Manual Pen testing should be included, but if your not doing anything now, then starting with SAST is a good start, but you really do need to do all 3 activities to establish a  good SSDLC process.
SAST tests for vulnerabilities in the early lifecycle.

If you want to go one stage further, you could deploy Cigital's secure assist, or similar tool onto the developers / programmers workstation. This will then highlight exposures as they create code - Very early in the lifecycle.

Remember, the earlier you build security into your lifecycle the lower the cost to remediate, so secure requirements, and threat / attack modelling should be done before coding starts to identify the exposures, software flaws etc.
Take a look at the BSIMM for more information on building a Secure SDLC process, including testing or the OWASP CLASP model.

Allan

Allan Ward, Internal Controls & Compliance Specialist (SOX), Global Security and Risk, D&B, Marlow International, Parkway, Marlow, SL7 1AJ, * (44) (0)1628 492709, * warda@dnb.commailto:warda@dnb.com

From: websecurity [mailto:websecurity-bounces@lists.webappsec.org] On Behalf Of Stephen de Vries
Sent: 23 January 2014 09:50
To: Paul Johnston
Cc: websecurity@lists.webappsec.org
Subject: Re: [WEB SECURITY] Security test case automation

On 23 Jan 2014, at 10:44, Paul Johnston <paul.johnston@pentest.co.ukmailto:paul.johnston@pentest.co.uk> wrote:

What you cannot automate is the mindset of a hacker. Security is not just about checking for a known set of issues. It is about using creativity and intuition to think up new ways of attacking a particular application. So while doing your own QA using DAST/SAST is good, you should also include some manual penetration testing in your security programme.

...and once you've found vulnerabilities through a manual test you can record and automate those findings with a testing framework.  Then you can re-run those same tests on your application periodically or even continuously to ensure that code changes to the app don't introduce security regressions.

Stephen

On 23/01/2014 04:30, vedantam sekhar wrote:
Hi group,
Need your help here. as part of QA team, we will be writing security test cases and also executing them manually using OWASP standard. However, i have been given task to see the possibility to automate these test cases. are there any tools/frameworks available for us to achieve this?
Thanks and Regards,

sekhar


The Web Security Mailing List

WebSecurity RSS Feed

http://www.webappsec.org/rss/websecurity.rss

Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA

WASC on Twitter

http://twitter.com/wascupdates

websecurity@lists.webappsec.orgmailto:websecurity@lists.webappsec.org

http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org

--

Pentest - The Application Security Specialists

Paul Johnston - IT Security Consultant / Tiger SST
Office: +44 (0) 161 233 0100
Mobile: +44 (0) 7817 219 072

We're exhibiting at Infosecurity Europe!
Stand K97, Earl's Court London - 29th April - 1st May
<logos-dl-infosec-withoutdates.png>

Email policy: http://www.pentest.co.uk/legal.shtml#emailpolicy
Registered Number: 4217114 England & Wales
Registered Office: 26a The Downs, Altrincham, Cheshire, WA14 2PU, UK
Accreditations: ISO 9001 (44/100/107029) / ISO 27001 (IS 558982) / Tiger Scheme


The Web Security Mailing List

WebSecurity RSS Feed
http://www.webappsec.org/rss/websecurity.rss

Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA

WASC on Twitter
http://twitter.com/wascupdates

websecurity@lists.webappsec.orgmailto:websecurity@lists.webappsec.org
http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org

SAST is best built into the code build process as an automated process, so all code submitted to the library gets static code analysis (SAST). SAST scans code a little like a word spell checker, looking for coding errors that create vulnerabilities. DAST runs on the running solution, can also be automated. Manual Pen testing should be included, but if your not doing anything now, then starting with SAST is a good start, but you really do need to do all 3 activities to establish a good SSDLC process. SAST tests for vulnerabilities in the early lifecycle. If you want to go one stage further, you could deploy Cigital's secure assist, or similar tool onto the developers / programmers workstation. This will then highlight exposures as they create code - Very early in the lifecycle. Remember, the earlier you build security into your lifecycle the lower the cost to remediate, so secure requirements, and threat / attack modelling should be done before coding starts to identify the exposures, software flaws etc. Take a look at the BSIMM for more information on building a Secure SDLC process, including testing or the OWASP CLASP model. Allan Allan Ward, Internal Controls & Compliance Specialist (SOX), Global Security and Risk, D&B, Marlow International, Parkway, Marlow, SL7 1AJ, * (44) (0)1628 492709, * warda@dnb.com<mailto:warda@dnb.com> From: websecurity [mailto:websecurity-bounces@lists.webappsec.org] On Behalf Of Stephen de Vries Sent: 23 January 2014 09:50 To: Paul Johnston Cc: websecurity@lists.webappsec.org Subject: Re: [WEB SECURITY] Security test case automation On 23 Jan 2014, at 10:44, Paul Johnston <paul.johnston@pentest.co.uk<mailto:paul.johnston@pentest.co.uk>> wrote: What you cannot automate is the mindset of a hacker. Security is not just about checking for a known set of issues. It is about using creativity and intuition to think up new ways of attacking a particular application. So while doing your own QA using DAST/SAST is good, you should also include some manual penetration testing in your security programme. ...and once you've found vulnerabilities through a manual test you can record and automate those findings with a testing framework. Then you can re-run those same tests on your application periodically or even continuously to ensure that code changes to the app don't introduce security regressions. Stephen On 23/01/2014 04:30, vedantam sekhar wrote: Hi group, Need your help here. as part of QA team, we will be writing security test cases and also executing them manually using OWASP standard. However, i have been given task to see the possibility to automate these test cases. are there any tools/frameworks available for us to achieve this? Thanks and Regards, sekhar _______________________________________________ The Web Security Mailing List WebSecurity RSS Feed http://www.webappsec.org/rss/websecurity.rss Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA WASC on Twitter http://twitter.com/wascupdates websecurity@lists.webappsec.org<mailto:websecurity@lists.webappsec.org> http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org -- Pentest - The Application Security Specialists Paul Johnston - IT Security Consultant / Tiger SST Office: +44 (0) 161 233 0100 Mobile: +44 (0) 7817 219 072 We're exhibiting at Infosecurity Europe! Stand K97, Earl's Court London - 29th April - 1st May <logos-dl-infosec-withoutdates.png> Email policy: http://www.pentest.co.uk/legal.shtml#emailpolicy Registered Number: 4217114 England & Wales Registered Office: 26a The Downs, Altrincham, Cheshire, WA14 2PU, UK Accreditations: ISO 9001 (44/100/107029) / ISO 27001 (IS 558982) / Tiger Scheme _______________________________________________ The Web Security Mailing List WebSecurity RSS Feed http://www.webappsec.org/rss/websecurity.rss Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA WASC on Twitter http://twitter.com/wascupdates websecurity@lists.webappsec.org<mailto:websecurity@lists.webappsec.org> http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org
SD
Stephen de Vries
Thu, Jan 23, 2014 10:26 AM

On 23 Jan 2014, at 11:04, Ward, Allan WardA@DNB.com wrote:

Remember, the earlier you build security into your lifecycle the lower the cost to remediate, so secure requirements, and threat / attack modelling should be done before coding starts to identify the exposures, software flaws etc.

This is where the power of automated tests and BDD shine through (OK I’m a little biased ;) ).  If the developers are using agile methods like Test Driven Development or Behaviour Driven Development, then those tests are written up front before the code is written.  So the power of including security tests there is that developers know what they’re meant to be building before they start writing code.  The tests effectively serve as the security requirements, and those requirements including both functional and non-functional security requirements.

The magic of BDD (hey I already said I was biased) is that those requirements can be written in English so that everyone understands them (Business, Security, Dev and QA) and not just the developers.  E.g.:

Scenario: Authentication credentials should be transmitted over SSL

Given a browser configured to use an intercepting proxy
And the proxy logs are cleared
And the default user logs in with credentials from: users.table
And the HTTP request-response containing the default credentials is inspected
Then the protocol should be HTTPS

The blindspot for both SAST and DAST scanners is that they only find security bugs, not architectural flaws or functional security flaws.  For those you need humans, but with a human-only approach you lose scalability and repeatability (and it costs a fortune).  Humans + automation of the human’s findings gives you a tasty sweet spot of coverage and repeatability.
Depending on how test orientated the development team is, I would even suggest wrapping the SAST and DAST tools with tests so that developers only deal with one kind of “stuff”: Tests.

Q: What security features do I need to build into the app?
A: Check the tests
Q: Have I implemented all the required security features?
A: Check the tests
Q: Has my code introduced security bugs?
A: Check the tests
Q: Has a new change I made to old code introduced new security bugs, or broken a previously working security feature?
A: Check the tests

E.g. BDD-Security wraps OWASP ZAP scanning like so:

Scenario: The application should not contain SQL injection vulnerabilities

Given a fresh scanner with all policies disabled
And the scannable methods of the application are navigated through the proxy
And the SQL-Injection policy is enabled
And the MySQL-SQL-Injection policy is enabled
And the Hypersonic-SQL-Injection policy is enabled
And the Oracle-SQL-Injection policy is enabled
And the PostgreSQL-SQL-Injection policy is enabled
When the scanner is run
And false positives described in: tables/false_positives.table are removed
Then no Medium or higher risk vulnerabilities should be present

From: websecurity [mailto:websecurity-bounces@lists.webappsec.org] On Behalf Of Stephen de Vries
Sent: 23 January 2014 09:50
To: Paul Johnston
Cc: websecurity@lists.webappsec.org
Subject: Re: [WEB SECURITY] Security test case automation

On 23 Jan 2014, at 10:44, Paul Johnston paul.johnston@pentest.co.uk wrote:

What you cannot automate is the mindset of a hacker. Security is not just about checking for a known set of issues. It is about using creativity and intuition to think up new ways of attacking a particular application. So while doing your own QA using DAST/SAST is good, you should also include some manual penetration testing in your security programme.

…and once you’ve found vulnerabilities through a manual test you can record and automate those findings with a testing framework.  Then you can re-run those same tests on your application periodically or even continuously to ensure that code changes to the app don’t introduce security regressions.

Stephen

On 23/01/2014 04:30, vedantam sekhar wrote:
Hi group,

Need your help here. as part of QA team, we will be writing security test cases and also executing them manually using OWASP standard. However, i have been given task to see the possibility to automate these test cases. are there any tools/frameworks available for us to achieve this?

Thanks and Regards,

sekhar


The Web Security Mailing List

WebSecurity RSS Feed
http://www.webappsec.org/rss/websecurity.rss

Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA

WASC on Twitter
http://twitter.com/wascupdates

websecurity@lists.webappsec.org
http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org

--
Pentest - The Application Security Specialists

Paul Johnston - IT Security Consultant / Tiger SST
Office: +44 (0) 161 233 0100
Mobile: +44 (0) 7817 219 072

We're exhibiting at Infosecurity Europe!
Stand K97, Earl's Court London - 29th April - 1st May
<logos-dl-infosec-withoutdates.png>

Email policy: http://www.pentest.co.uk/legal.shtml#emailpolicy
Registered Number: 4217114 England & Wales
Registered Office: 26a The Downs, Altrincham, Cheshire, WA14 2PU, UK
Accreditations: ISO 9001 (44/100/107029) / ISO 27001 (IS 558982) / Tiger Scheme


The Web Security Mailing List

WebSecurity RSS Feed
http://www.webappsec.org/rss/websecurity.rss

Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA

WASC on Twitter
http://twitter.com/wascupdates

websecurity@lists.webappsec.org
http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org

On 23 Jan 2014, at 11:04, Ward, Allan <WardA@DNB.com> wrote: > > Remember, the earlier you build security into your lifecycle the lower the cost to remediate, so secure requirements, and threat / attack modelling should be done before coding starts to identify the exposures, software flaws etc. This is where the power of automated tests and BDD shine through (OK I’m a little biased ;) ). If the developers are using agile methods like Test Driven Development or Behaviour Driven Development, then those tests are written _up front_ before the code is written. So the power of including security tests there is that developers know what they’re meant to be building before they start writing code. The tests effectively serve as the security requirements, and those requirements including both functional and non-functional security requirements. The magic of BDD (hey I already said I was biased) is that those requirements can be written in English so that everyone understands them (Business, Security, Dev and QA) and not just the developers. E.g.: Scenario: Authentication credentials should be transmitted over SSL Given a browser configured to use an intercepting proxy And the proxy logs are cleared And the default user logs in with credentials from: users.table And the HTTP request-response containing the default credentials is inspected Then the protocol should be HTTPS The blindspot for both SAST and DAST scanners is that they only find security bugs, not architectural flaws or functional security flaws. For those you need humans, but with a human-only approach you lose scalability and repeatability (and it costs a fortune). Humans + automation of the human’s findings gives you a tasty sweet spot of coverage and repeatability. Depending on how test orientated the development team is, I would even suggest wrapping the SAST and DAST tools with tests so that developers only deal with one kind of “stuff”: Tests. Q: What security features do I need to build into the app? A: Check the tests Q: Have I implemented all the required security features? A: Check the tests Q: Has my code introduced security bugs? A: Check the tests Q: Has a new change I made to old code introduced new security bugs, or broken a previously working security feature? A: Check the tests E.g. BDD-Security wraps OWASP ZAP scanning like so: Scenario: The application should not contain SQL injection vulnerabilities Given a fresh scanner with all policies disabled And the scannable methods of the application are navigated through the proxy And the SQL-Injection policy is enabled And the MySQL-SQL-Injection policy is enabled And the Hypersonic-SQL-Injection policy is enabled And the Oracle-SQL-Injection policy is enabled And the PostgreSQL-SQL-Injection policy is enabled When the scanner is run And false positives described in: tables/false_positives.table are removed Then no Medium or higher risk vulnerabilities should be present > > From: websecurity [mailto:websecurity-bounces@lists.webappsec.org] On Behalf Of Stephen de Vries > Sent: 23 January 2014 09:50 > To: Paul Johnston > Cc: websecurity@lists.webappsec.org > Subject: Re: [WEB SECURITY] Security test case automation > > > On 23 Jan 2014, at 10:44, Paul Johnston <paul.johnston@pentest.co.uk> wrote: > > What you cannot automate is the mindset of a hacker. Security is not just about checking for a known set of issues. It is about using creativity and intuition to think up new ways of attacking a particular application. So while doing your own QA using DAST/SAST is good, you should also include some manual penetration testing in your security programme. > > …and once you’ve found vulnerabilities through a manual test you can record and automate those findings with a testing framework. Then you can re-run those same tests on your application periodically or even continuously to ensure that code changes to the app don’t introduce security regressions. > > Stephen > > > > > > > > On 23/01/2014 04:30, vedantam sekhar wrote: > Hi group, > > Need your help here. as part of QA team, we will be writing security test cases and also executing them manually using OWASP standard. However, i have been given task to see the possibility to automate these test cases. are there any tools/frameworks available for us to achieve this? > > Thanks and Regards, > > sekhar > > > > > _______________________________________________ > The Web Security Mailing List > > WebSecurity RSS Feed > http://www.webappsec.org/rss/websecurity.rss > > Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA > > WASC on Twitter > http://twitter.com/wascupdates > > websecurity@lists.webappsec.org > http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org > > -- > Pentest - The Application Security Specialists > > Paul Johnston - IT Security Consultant / Tiger SST > Office: +44 (0) 161 233 0100 > Mobile: +44 (0) 7817 219 072 > > We're exhibiting at Infosecurity Europe! > Stand K97, Earl's Court London - 29th April - 1st May > <logos-dl-infosec-withoutdates.png> > > Email policy: http://www.pentest.co.uk/legal.shtml#emailpolicy > Registered Number: 4217114 England & Wales > Registered Office: 26a The Downs, Altrincham, Cheshire, WA14 2PU, UK > Accreditations: ISO 9001 (44/100/107029) / ISO 27001 (IS 558982) / Tiger Scheme > > _______________________________________________ > The Web Security Mailing List > > WebSecurity RSS Feed > http://www.webappsec.org/rss/websecurity.rss > > Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA > > WASC on Twitter > http://twitter.com/wascupdates > > websecurity@lists.webappsec.org > http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org
WJ
Will Jefferies
Thu, Jan 23, 2014 2:43 PM

I second Static code analysis tools.  I use Checkmarx suite and scan each of our codebases weekly (we have a lot).  It is very good at finding OWasp and SANS 25 vulns, albeit with a lot of false positives, but once you mark those as "not exploitable", the engine ignores them on future scans unless that particular code changes.  I've used the other big name STAT also, but find checkmarx to be a good balance of speed and accuracy.

From: websecurity [mailto:websecurity-bounces@lists.webappsec.org] On Behalf Of vedantam sekhar
Sent: Wednesday, January 22, 2014 10:33 PM
To: websecurity@webappsec.org
Subject: [WEB SECURITY] Security test case automation

Hi group,
Need your help here. as part of QA team, we will be writing security test cases and also executing them manually using OWASP standard. However, i have been given task to see the possibility to automate these test cases. are there any tools/frameworks available for us to achieve this?
Thanks and Regards,

sekhar

Confidentiality Notice: This message is for the sole use of the intended recipient(s).  It may contain confidential or proprietary information and may be subject to the attorney-client privilege or other confidentiality protections. If this message was misdirected, neither FNC Holding Company, Inc. nor any of its subsidiaries waive any confidentiality, privilege, or trade secrets. If you are not a designated recipient, you may not review, print, copy, retransmit, disseminate, or otherwise use this message. If you have received this message in error, please notify the sender by reply e-mail and delete this message.

I second Static code analysis tools. I use Checkmarx suite and scan each of our codebases weekly (we have a lot). It is very good at finding OWasp and SANS 25 vulns, albeit with a lot of false positives, but once you mark those as "not exploitable", the engine ignores them on future scans unless that particular code changes. I've used the other big name STAT also, but find checkmarx to be a good balance of speed and accuracy. From: websecurity [mailto:websecurity-bounces@lists.webappsec.org] On Behalf Of vedantam sekhar Sent: Wednesday, January 22, 2014 10:33 PM To: websecurity@webappsec.org Subject: [WEB SECURITY] Security test case automation Hi group, Need your help here. as part of QA team, we will be writing security test cases and also executing them manually using OWASP standard. However, i have been given task to see the possibility to automate these test cases. are there any tools/frameworks available for us to achieve this? Thanks and Regards, sekhar Confidentiality Notice: This message is for the sole use of the intended recipient(s). It may contain confidential or proprietary information and may be subject to the attorney-client privilege or other confidentiality protections. If this message was misdirected, neither FNC Holding Company, Inc. nor any of its subsidiaries waive any confidentiality, privilege, or trade secrets. If you are not a designated recipient, you may not review, print, copy, retransmit, disseminate, or otherwise use this message. If you have received this message in error, please notify the sender by reply e-mail and delete this message.
DC
Dinis Cruz
Fri, Jan 24, 2014 1:09 PM

Hi, a great place to start is on your current QA automation and testing.

How are you currently QAing the app? I'm specifically thinking of
Integration Unit Tests that walk-through the multiple parts of the app

On that topic, what kind of app is it? Website, Web Services, .Net, Java,
RoR?
On 23 Jan 2014 04:32, "vedantam sekhar" vedantamsekhar@gmail.com wrote:

Hi group,

Need your help here. as part of QA team, we will be writing security test
cases and also executing them manually using OWASP standard. However, i
have been given task to see the possibility to automate these test cases.
are there any tools/frameworks available for us to achieve this?

Thanks and Regards,

sekhar


The Web Security Mailing List

WebSecurity RSS Feed
http://www.webappsec.org/rss/websecurity.rss

Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA

WASC on Twitter
http://twitter.com/wascupdates

websecurity@lists.webappsec.org
http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org

Hi, a great place to start is on your current QA automation and testing. How are you currently QAing the app? I'm specifically thinking of Integration Unit Tests that walk-through the multiple parts of the app On that topic, what kind of app is it? Website, Web Services, .Net, Java, RoR? On 23 Jan 2014 04:32, "vedantam sekhar" <vedantamsekhar@gmail.com> wrote: > Hi group, > > Need your help here. as part of QA team, we will be writing security test > cases and also executing them manually using OWASP standard. However, i > have been given task to see the possibility to automate these test cases. > are there any tools/frameworks available for us to achieve this? > > Thanks and Regards, > > sekhar > > > _______________________________________________ > The Web Security Mailing List > > WebSecurity RSS Feed > http://www.webappsec.org/rss/websecurity.rss > > Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA > > WASC on Twitter > http://twitter.com/wascupdates > > websecurity@lists.webappsec.org > http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org > >