[WEB SECURITY] C# test suite for testing static code analyzers

romain r at fuckthespam.com
Sat Jan 24 20:29:25 EST 2009


Michael,
It really depends on what kind of test suite you want to have.
If it's about how tools handle specific API/code construct etc. I think 
you can do a test suite like the one you can find on the SAMATE project:
   http://samate.nist.gov/SRD

Otherwise, if you want something more realistic (meaning, actually using 
that to compare tools), this is a totally different subject.
I think the best would be to actually run the tools on your code (maybe 
just some part of it) and then compare...
But to have meaningful comparison, you should know that most of tools 
(commercials) are customizable and this is a very important feature for 
security testing.

--Romain
http://rgaucher.info


Michael Williams wrote:
> Romain,
>  
> Yes you are correct I am not looking for a tool, I am looking for a 
> suite of C# programs that have security vulnerabilities in them like 
> poor input validation, buffer overflow problems, SQL injection problems, 
> etc that I can use as a test suite to test the quality of static code 
> analyzing tools in their ability to find and report on the problems 
> contained in the C# programs.
>  
> There seems to be lots test suites like this for C,C++ and Java but 
> almost nothing for C#. So I'm starting to think that it would be a 
> pretty nice project for me to write a dozen or so small C# programs 
> which contain the standard list of application security programming 
> errors and make it freely available for people who are trying to decide 
> which static code analyzer is best at picking out these vulnerabilities.
> 
> 
>  
> 
> 
>  > Date: Sat, 24 Jan 2009 12:40:10 -0500
>  > From: r at fuckthespam.com
>  > To: mostafa.siraj at gmail.com
>  > CC: sjensen1207 at hotmail.com; mw7301 at hotmail.com; 
> websecurity at webappsec.org
>  > Subject: Re: [WEB SECURITY] C# test suite for testing static code 
> analyzers
>  >
>  > Mostafa:
>  > He is looking for a test suite, not a tool... but you're right, CAT.NET
>  > seems to be a nice tool (glorified LAPSE for .NET? :))
>  >
>  > Michael:
>  > I am not aware of any test suite for C# and this is a shame, it would be
>  > interesting to create a "securibench" for C#...
>  >
>  > --Romain
>  > http://rgaucher.info
>  >
>  > Mostafa Siraj wrote:
>  > > CAT.NET <http://CAT.NET> is a nice free tool that integrates with 
> Visual
>  > > Studio
>  > >
>  > > On Fri, Jan 23, 2009 at 10:55 PM, steve jensen 
> <sjensen1207 at hotmail.com
>  > > <mailto:sjensen1207 at hotmail.com>> wrote:
>  > >
>  > > There are several on the market. Just google for .NET source code
>  > > analysis.
>  > >
>  > >
>  > > 
> ------------------------------------------------------------------------
>  > >
>  > > From: mw7301 at hotmail.com <mailto:mw7301 at hotmail.com>
>  > > To: websecurity at webappsec.org <mailto:websecurity at webappsec.org>
>  > > Date: Fri, 23 Jan 2009 20:00:11 +0000
>  > > Subject: [WEB SECURITY] C# test suite for testing static code analyzers
>  > >
>  > >
>  > > Do any of you know of a suite of C# programs that could be used to
>  > > test static code analyzers for their ability to find distinct
>  > > security vulnerabilities? There is lots of this kind of code
>  > > available for C, C++ and Java but I haven't been able to find a
>  > > similar thing for C#.
>  > >
>  > >
>  > >
>  > >
>  > >
>  > >
>  > >
>  > >
>  > >
>  > >
>  > >
>  > >
>  > >
>  > > 
> ------------------------------------------------------------------------
>  > >
>  > > Windows Live™: E-mail. Chat. Share. Get more ways to connect. See
>  > > how it works.
>  > > 
> <http://windowslive.com/howitworks?ocid=TXT_TAGLM_WL_t2_allup_howitworks_012009>
>  > >
>  > > 
> ------------------------------------------------------------------------
>  > > Hotmail® goes where you go. On a PC, on the Web, on your phone. See
>  > > how.
>  > > 
> <http://www.windowslive-hotmail.com/learnmore/versatility.aspx#mobile?ocid=TXT_TAGHM_WL_HM_versatility_121208>
>  > >
>  > >
>  > >
>  > >
>  > > --
>  > > "Our deepest fear is not that we are inadequate. Our deepest fear is
>  > > that we are powerful beyond measure. It is our light, not our 
> darkness,
>  > > that most frightens us. We ask ourselves, who am I to be brilliant,
>  > > gorgeous, talented, and fabulous?Actually, who are you not to be? You
>  > > are a child of God. Your playing small doesn't serve the world. 
> There's
>  > > nothing enlightened about shrinking so that other people won't feel
>  > > insecure around you. We are all meant to shine, as children do. We are
>  > > born to make manifest the glory of God that is within us. It's not 
> just
>  > > in some of us, it's in everyone. And as we let our own light shine, we
>  > > unconsciously give other people permission to do the same. As we are
>  > > liberated from our own fear, our presence automatically liberates
>  > > others." --Nelson Mandela--
>  >
>  > 
> ----------------------------------------------------------------------------
>  > Join us on IRC: irc.freenode.net #webappsec
>  >
>  > Have a question? Search The Web Security Mailing List Archives:
>  > http://www.webappsec.org/lists/websecurity/archive/
>  >
>  > Subscribe via RSS:
>  > http://www.webappsec.org/rss/websecurity.rss [RSS Feed]
>  >
>  > Join WASC on LinkedIn
>  > http://www.linkedin.com/e/gis/83336/4B20E4374DBA
>  >
> 
> 
> ------------------------------------------------------------------------
> Windows Live™ Hotmail®:…more than just e-mail. Check it out. 
> <http://windowslive.com/explore?ocid=TXT_TAGLM_WL_t2_hm_justgotbetter_explore_012009>

----------------------------------------------------------------------------
Join us on IRC: irc.freenode.net #webappsec

Have a question? Search The Web Security Mailing List Archives: 
http://www.webappsec.org/lists/websecurity/archive/

Subscribe via RSS: 
http://www.webappsec.org/rss/websecurity.rss [RSS Feed]

Join WASC on LinkedIn
http://www.linkedin.com/e/gis/83336/4B20E4374DBA



More information about the websecurity mailing list