[WEB SECURITY] Experience with using HTTP proxy tools for QA testers
rklists at gmail.com
Mon Feb 28 18:38:57 EST 2011
Thanks Andre - interesting perspective. With respect to QA automation, you
might be right about the correlation between security and QA hygiene. Some
of the clients are actually quite mature with respect to application
security process (when compared to others in their respective industries);
generally their QA processes are less consistent between groups.
Regarding pen testing in early iterations, this would work well but the
problem is often scalability. One reason dynamic testing tools are so
popular is because many organizations are unable/unwilling to have enough
penetration testers to get involved with every application at every
iteration (or early on in testing in waterfall shops).
I think there are some cases of low hanging fruit. For example session
fixation, authentication throttling / lockout, forcible browsing, limited
CSRF testing (assuming they can leverage domain knowledge), picking up on
verbose error messages, etc. Even some basic parameter manipulation &
horizontal privilege escalation may be possible with clear guidance on an
HTTP proxy tool. I agree that most of these could be picked-up by
penetration testers more efficiently and accurately. I'm currently working
with some QA shops now, so I'll let you know effective it ends up being.
Seasoned testers can certainly teach us a lot. I'm learning a lot from them
right now and I suspect many other security people could as well. Any ideas
on ways to improve the knowledge sharing from QA to security more generally?
On Mon, Feb 28, 2011 at 5:59 PM, Andre Gironda <andreg at gmail.com> wrote:
> On Mon, Feb 28, 2011 at 2:00 PM, Rohit Sethi <rklists at gmail.com> wrote:
> > I think relying soly on QA for security testing is a slippery slope.
> I don't think I've ever seen anyone propose that, but I'd fight it ;>
> > heard interest from our clients about reducing the number of
> > that get caught in pen testing, rather than using pen testing as the only
> > testing method. Rather than making QA comprehensive, we're trying to see
> > which low-hanging fruit can be caught with minimal effort - both by those
> > who use automated run-time assessment scanners and those who do not.
> More on this below in psiinon's comments.
> > Interestingly, we're not seeing many of our clients make use of QA
> > automation tools. While developers may make extensive integration testing
> > suites in JUnit/NUnit we've found less evidence of QTP / Selnium being
> > in enterprise QA. What have your experiences been? Clearly automation is
> > important but if it requires training in a new tool then it's less likely
> > be adopted.
> I think it's more common to see testing automation in place where
> there is also a focus on application lifecycle management. If a CxO or
> R&D director/manager has focused on hygiene to solve some systemic
> project issues, then you'll probably find testing automation.
> Because many development projects, including Enterprise, involve only
> one developer, you will only find JUnit/NUnit where the app developer
> has done their own work (usually because they were given the time by
> management to focus on these efforts), and usually they have had past
> experience with unit testing frameworks. QTP/Selenium style testing
> automation, as you suggest, is even rarer to find -- possibly because
> app developers do not have the experience or exposure to these at all.
> I haven't seen training budget for test automation frameworks and we
> do not see these classes being offered -- especially not at the same
> pace as app development or even app penetration-testing.
> When I see application developers get to a major iteration for a
> project/product that is absolutely going to be released -- and they
> know that the app is going to be around for 5+ years or decades...
> then I always question them: Why haven't you put more rigor into unit
> and component test automation when you know that somebody (even if
> it's not you) is going to be refactoring this project/product?
> An even more important question would be about keeping the test
> automation up-to-date. I have seen many projects implement JUnit with
> either HtmlUnit or Selenium to later abandon the test automation,
> leaving it several revisions out of date compared to the latest
> iteration of the system build. When performing continuous integration,
> particularly nightly builds -- all build code and test code should
> remain in working order on the build server(s).
> Rohit -- one interesting point here is that you and your company may
> have had experience with many development shops that lack hygiene in
> both test automation and appsec because the two concepts are
> intrinsically linked. It's a lot less likely that you'll walk into a
> dev shop that recently had an appsec-related incident (or proactively
> hires consultants because they are worried) to find that the
> documentation and test automation is world-class.
> On Mon, Feb 28, 2011 at 2:30 PM, psiinon <psiinon at gmail.com> wrote:
> > As I try to make clear in the preso here:
> > http://www.owasp.org/images/c/c8/Conference_Style_slides_for_ZAP.ppt I
> > definitely dont think we should just rely on QA for security testing.
> > There is no silver bullet, and so this isnt it ;)
> > However professional pentesting is expensive, and often gets done late
> > in the product cycle.
> Why is penetration-testing performed late in the product cycle? Why
> not put pen-testers and app developers together during a
> project/product's first iteration? If there is a "system" (anything
> past the point of wireframes to include deployable instances or even
> mocks) to test, I say test it when it's considered a system, not when
> it's considered "ready" by some other standards. An afternoon spent at
> an early iteration would easily find and fix more issues than a more
> formal pen-test right before an app is flagged for push into
> production. Better, using continuous deployments could allow
> pen-testers and app developers to work together during blue
> deployments (where the code push/release is active for some) instead
> of green deployments (active for everyone) if a project/product is
> already in production and this new iteration is actually a new code
> > If you can catch even a few security vulnerabilities earlier on then
> > it will save time, effort and therefore money.
> > Its not cost effective to pay pen testers to find the simple issues -
> > pick those up in QA and then get value for money from the
> > professionals by clearing the way for them to look for the hard ones
> I don't agree with this. Training dev-testers or quality testers to
> understand HTTP/TLS requests/responses or other nuances (for appsec!)
> is too difficult. It takes time, research, dedication, and focus to
> become fluent with web application vulnerability
> testing/exploitation/risk-analysis. I would rather that quality
> testers understand the architectural appsec issues facing
> in-development apps: e.g. they should know that Classic ASP and PHP
> are ripe for problems as compared to a standardized ESAPI Java -- and
> grab an appsec expert to discuss the ramifications coming down the
> pipeline. In other words, QA should be trained as whistleblowers and
> get the experts involved. They can easily be rewarded as such -- just
> give bonuses to any QA teams or individuals that report risky
> applications that include sensitive data to the InfoSec team.
> Another thing here: There are no easy problems. You don't see
> eradication of SQLi or XSS anywhere. QA is not going to add any value,
> even when armed with the "right" tools (note: I don't believe any
> commercial appsec tools today work in the ways that they are
> marketed). You can take a QA person and turn him or her into an app
> penetration-tester, but again -- this takes time, research,
> dedication, and focus on appsec and only appsec. Bring this person
> onto an appsec team instead of leaving them behind in QA. Best of all
> -- they probably already know the app developers and have a better
> working relationship with them than the appsec team does. Now put them
> in between QA and development -- they should be on the development
> team as a dev-tester, not at a quality/security gate.
> In many ways, forcing or helping QA to add in some low-hanging fruit
> vulnerability-finding techniques only placates the notion that appsec
> is not being taken seriously.
> On a completely separate topic point, I'd say that appsec teams and
> app penetration-testers can learn more from test automation experts
> and quality testers rather than vice-versa. I have never seen or heard
> of an app pen-tester who utilizes equivalence classification wisely,
> nor combinatorial/pairwise testing, exploratory testing, or
> domain-specific knowledge. At the very least -- they do not call these
> concepts out directly/correctly, or identify their own "ad-hoc"
> techniques (even if similar, and even if formalized). If app
> pen-testers aren't using the correct language, then who are they to
> say that they know anything about testing software at all?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the websecurity