[WEB SECURITY] C# test suite for testing static code analyzers

romain r at fuckthespam.com
Sun Jan 25 12:03:30 EST 2009


That's a good idea, especially if you share that on the NIST website.
I just wouldn't recommend using OWASP top 10 for such a thing, using a 
weaknesses classification (like CWE) would be way better for source code.

--Romain
http://rgaucher.info

Michael Williams wrote:
> Romain,
>  
> I was thinking of doing a similar thing that the SAMATE project is doing 
> for Java for examples but tie all my C# examples as closely as possible 
> to the OWASP top 10. That way a person would have a standard benchmark 
> to measure against. 
> 
>  
>  
>  
>  
>  
> 
> 
> 
> 
> 
> 
> 
>  > Date: Sat, 24 Jan 2009 20:29:25 -0500
>  > From: r at fuckthespam.com
>  > To: mw7301 at hotmail.com
>  > CC: websecurity at webappsec.org
>  > Subject: Re: [WEB SECURITY] C# test suite for testing static code 
> analyzers
>  >
>  > Michael,
>  > It really depends on what kind of test suite you want to have.
>  > If it's about how tools handle specific API/code construct etc. I think
>  > you can do a test suite like the one you can find on the SAMATE project:
>  > http://samate.nist.gov/SRD
>  >
>  > Otherwise, if you want something more realistic (meaning, actually using
>  > that to compare tools), this is a totally different subject.
>  > I think the best would be to actually run the tools on your code (maybe
>  > just some part of it) and then compare...
>  > But to have meaningful comparison, you should know that most of tools
>  > (commercials) are customizable and this is a very important feature for
>  > security testing.
>  >
>  > --Romain
>  > http://rgaucher.info
>  >
>  >
>  > Michael Williams wrote:
>  > > Romain,
>  > >
>  > > Yes you are correct I am not looking for a tool, I am looking for a
>  > > suite of C# programs that have security vulnerabilities in them like
>  > > poor input validation, buffer overflow problems, SQL injection 
> problems,
>  > > etc that I can use as a test suite to test the quality of static code
>  > > analyzing tools in their ability to find and report on the problems
>  > > contained in the C# programs.
>  > >
>  > > There seems to be lots test suites like this for C,C++ and Java but
>  > > almost nothing for C#. So I'm starting to think that it would be a
>  > > pretty nice project for me to write a dozen or so small C# programs
>  > > which contain the standard list of application security programming
>  > > errors and make it freely available for people who are trying to 
> decide
>  > > which static code analyzer is best at picking out these 
> vulnerabilities.
>  > >
>  > >
>  > >
>  > >
>  > >
>  > > > Date: Sat, 24 Jan 2009 12:40:10 -0500
>  > > > From: r at fuckthespam.com
>  > > > To: mostafa.siraj at gmail.com
>  > > > CC: sjensen1207 at hotmail.com; mw7301 at hotmail.com;
>  > > websecurity at webappsec.org
>  > > > Subject: Re: [WEB SECURITY] C# test suite for testing static code
>  > > analyzers
>  > > >
>  > > > Mostafa:
>  > > > He is looking for a test suite, not a tool... but you're right, 
> CAT.NET
>  > > > seems to be a nice tool (glorified LAPSE for .NET? :))
>  > > >
>  > > > Michael:
>  > > > I am not aware of any test suite for C# and this is a shame, it 
> would be
>  > > > interesting to create a "securibench" for C#...
>  > > >
>  > > > --Romain
>  > > > http://rgaucher.info
>  > > >
>  > > > Mostafa Siraj wrote:
>  > > > > CAT.NET <http://CAT.NET> is a nice free tool that integrates with
>  > > Visual
>  > > > > Studio
>  > > > >
>  > > > > On Fri, Jan 23, 2009 at 10:55 PM, steve jensen
>  > > <sjensen1207 at hotmail.com
>  > > > > <mailto:sjensen1207 at hotmail.com>> wrote:
>  > > > >
>  > > > > There are several on the market. Just google for .NET source code
>  > > > > analysis.
>  > > > >
>  > > > >
>  > > > >
>  > > 
> ------------------------------------------------------------------------
>  > > > >
>  > > > > From: mw7301 at hotmail.com <mailto:mw7301 at hotmail.com>
>  > > > > To: websecurity at webappsec.org <mailto:websecurity at webappsec.org>
>  > > > > Date: Fri, 23 Jan 2009 20:00:11 +0000
>  > > > > Subject: [WEB SECURITY] C# test suite for testing static code 
> analyzers
>  > > > >
>  > > > >
>  > > > > Do any of you know of a suite of C# programs that could be used to
>  > > > > test static code analyzers for their ability to find distinct
>  > > > > security vulnerabilities? There is lots of this kind of code
>  > > > > available for C, C++ and Java but I haven't been able to find a
>  > > > > similar thing for C#.
>  > > > >
>  > > > >
>  > > > >
>  > > > >
>  > > > >
>  > > > >
>  > > > >
>  > > > >
>  > > > >
>  > > > >
>  > > > >
>  > > > >
>  > > > >
>  > > > >
>  > > 
> ------------------------------------------------------------------------
>  > > > >
>  > > > > Windows Live™: E-mail. Chat. Share. Get more ways to connect. See
>  > > > > how it works.
>  > > > >
>  > > 
> <http://windowslive.com/howitworks?ocid=TXT_TAGLM_WL_t2_allup_howitworks_012009>
>  > > > >
>  > > > >
>  > > 
> ------------------------------------------------------------------------
>  > > > > Hotmail® goes where you go. On a PC, on the Web, on your phone. See
>  > > > > how.
>  > > > >
>  > > 
> <http://www.windowslive-hotmail.com/learnmore/versatility.aspx#mobile?ocid=TXT_TAGHM_WL_HM_versatility_121208>
>  > > > >
>  > > > >
>  > > > >
>  > > > >
>  > > > > --
>  > > > > "Our deepest fear is not that we are inadequate. Our deepest 
> fear is
>  > > > > that we are powerful beyond measure. It is our light, not our
>  > > darkness,
>  > > > > that most frightens us. We ask ourselves, who am I to be brilliant,
>  > > > > gorgeous, talented, and fabulous?Actually, who are you not to 
> be? You
>  > > > > are a child of God. Your playing small doesn't serve the world.
>  > > There's
>  > > > > nothing enlightened about shrinking so that other people won't feel
>  > > > > insecure around you. We are all meant to shine, as children do. 
> We are
>  > > > > born to make manifest the glory of God that is within us. It's not
>  > > just
>  > > > > in some of us, it's in everyone. And as we let our own light 
> shine, we
>  > > > > unconsciously give other people permission to do the same. As 
> we are
>  > > > > liberated from our own fear, our presence automatically liberates
>  > > > > others." --Nelson Mandela--
>  > > >
>  > > >
>  > > 
> ----------------------------------------------------------------------------
>  > > > Join us on IRC: irc.freenode.net #webappsec
>  > > >
>  > > > Have a question? Search The Web Security Mailing List Archives:
>  > > > http://www.webappsec.org/lists/websecurity/archive/
>  > > >
>  > > > Subscribe via RSS:
>  > > > http://www.webappsec.org/rss/websecurity.rss [RSS Feed]
>  > > >
>  > > > Join WASC on LinkedIn
>  > > > http://www.linkedin.com/e/gis/83336/4B20E4374DBA
>  > > >
>  > >
>  > >
>  > > 
> ------------------------------------------------------------------------
>  > > Windows Live™ Hotmail®:…more than just e-mail. Check it out.
>  > > 
> <http://windowslive.com/explore?ocid=TXT_TAGLM_WL_t2_hm_justgotbetter_explore_012009>
>  >
>  > 
> ----------------------------------------------------------------------------
>  > Join us on IRC: irc.freenode.net #webappsec
>  >
>  > Have a question? Search The Web Security Mailing List Archives:
>  > http://www.webappsec.org/lists/websecurity/archive/
>  >
>  > Subscribe via RSS:
>  > http://www.webappsec.org/rss/websecurity.rss [RSS Feed]
>  >
>  > Join WASC on LinkedIn
>  > http://www.linkedin.com/e/gis/83336/4B20E4374DBA
>  >
> 
> 
> ------------------------------------------------------------------------
> Windows Live™ Hotmail®…more than just e-mail. See how it works. 
> <http://windowslive.com/howitworks?ocid=TXT_TAGLM_WL_t2_hm_justgotbetter_howitworks_012009>

----------------------------------------------------------------------------
Join us on IRC: irc.freenode.net #webappsec

Have a question? Search The Web Security Mailing List Archives: 
http://www.webappsec.org/lists/websecurity/archive/

Subscribe via RSS: 
http://www.webappsec.org/rss/websecurity.rss [RSS Feed]

Join WASC on LinkedIn
http://www.linkedin.com/e/gis/83336/4B20E4374DBA



More information about the websecurity mailing list