[WEB SECURITY] How to perform Antivirus Security Testing

Josh More guppie at starmind.org
Wed Apr 20 17:35:40 EDT 2011


Wayne makes a very good point.  I was thinking of the common desktop use
case and completely ignoring API issues.

One thing to add... if you are using this in a cloud environment and
planning to tie into VShield, be aware that almost all of the vendors will
be crippled.  This technology allows you to schedule file-based scans and be
extremely effective in your use of RAM.  However, behavioural profiling,
HIPS and stuff will not work.

You can also shift the game entirely and look at application whitelisting.

-Josh More

On Wed, Apr 20, 2011 at 4:32 PM, Wayne Huang <wayne at armorize.com> wrote:

> (We're not an antivirus vendor)
>
> We have had to select antivirus vendors to work with, and incorporate their
> scanning engines. Because we leverage their engines to do large-scale
> scanning, licensing fees are very expensive and therefore, we had to make
> sure we make the right selection.
>
> (Of course we have our own technologies as well and don't just rely on AV
> engines)
>
> So although the best over-all score may go to the bigger players that Josh
> mentioned below, I was aware of some differences during my past tests:
>
> A. For most AV vendors, detection rates of their desktop versions differ
> greatly with their API offerings (which is what we, as armorize, need). For
> desktop versions some AV vendors hook into the browser, and this allows them
> to see exactly what the browser is doing, what the javascript engine is
> doing, and what the browser plugins (eg flash) are doing. So when they hit a
> malware, even if it is heavily obfuscated and therefore their signatures
> fail, they can still rely on behavior.
>
> However, very few have the same implementation for their API versions
> because API versions run stand-alone without user environments, and often
> under linux, and therefore, behavior capabilities are limited. Virus Total
> results are based on API versions and not desktop versions. So if you are
> looking at the API versions (like us) then Virus Total is a good reference;
> if you're looking at the desktop versions then Virus Total current cannot
> fully reflect capability differences.
>
> B. What are the objectives? If you can deal with false positives but cannot
> accept false negatives, then another set of vendors, for example Avira comes
> out top, especially when it comes to Web malware. If you're doing mass
> scanning and cloud costs (servers) is a big issue then you'd have to test
> out performance, and sometimes, vendors that excel at desktop-based
> detection, have very slow and ineffective API implementations. Some vendors
> don't have good signatures but have very good behavior and therefore for
> desktop versions they actually do very very well, while their performance is
> bad on Virus Total. At the same time, some vendors only focus on their API
> versions and therefore do very well with it.
>
> This is our talk at blackhat / defcon focused on Web malware (script-based)
> but not that much on PE malware:
> http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detection
>
>
> <http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detection>In
> it you'll find a few tables comparing the AV vendors against drivesploit, an
> open source drive-by download pack.
>
> --
> Wayne
> Armorize Technologies
> http://www.armorize.com
>
>
> On Thu, Apr 21, 2011 at 1:17 AM, Josh More <guppie at starmind.org> wrote:
>
>> Don't bother.
>>
>> Seriously, the top players are:  Symantec, McAfee, Trend Micro, Kaspersky
>> and Sophos.  Read the "independent" reviews and these five are always at the
>> top.  Look at the scores from places like http://www.virusbtn.com/ and
>> these five are always there.  Odds are that one of them will work for you
>> just fine.  (I usually pick Sophos for my clients.)
>>
>> Then look at the extra features.  Learn why each one is necessary (note:
>> they all exist to supplement flaws in the legacy signature-based system).
>> Figure out which features you need and throw out the vendors that don't
>> provide them.
>>
>> Then look at the UI's.  If it will be difficult to use one of the systems
>> in operations, throw it out.  Find out if any of the admins are biased
>> against a system (Symantec is a popular one for admins to hate.)  You get
>> more problems with malware from admins who resist caring for the system than
>> you get from systems failing to catch stuff.
>>
>> Then look at the licensing.  If you can't understand it or if they're
>> nickel-and-diming you on price, throw them out.  It's not worth the pain
>> otherwise.
>>
>> If this process doesn't get you down to a single vendor, look at how they
>> handle 24/7 support and make test support calls.  If their support is poor,
>> throw them out.  If they don't offer 24/7, throw them out (malware doesn't
>> wait for sun-up).  If they force their people to work more than an eight
>> hour shift, throw them out.
>>
>> This process will get you a solution that meets real world needs.  If you
>> try to test from a technical perspective, you're just going to be selecting
>> the system that best protects against attackers that think just like you
>> do... which you've already protected against through system hardening and
>> network design.
>>
>> -Josh More
>>
>> On Tue, Apr 19, 2011 at 11:28 PM, prashant Kar <kar.prashant at gmail.com>wrote:
>>
>>> Dear All,
>>>
>>> Kindly guide me on how to do antivirus application security testing.
>>>
>>> Any tools/methodology/approach/checklist that will help, please suggest.
>>>
>>> Best Regards,
>>> Prashant
>>>
>>> --
>>> Technical Skill is the mastery of complexity,
>>> while Creativity is the master of simplicity.....
>>>
>>> The Future Belongs To Those Who Believe in The Beauty of Their Dreams.
>>> Keep up the spirit!!!!
>>>
>>> Prashant Kar
>>>
>>> _______________________________________________
>>> The Web Security Mailing List
>>>
>>> WebSecurity RSS Feed
>>> http://www.webappsec.org/rss/websecurity.rss
>>>
>>> Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA
>>>
>>> WASC on Twitter
>>> http://twitter.com/wascupdates
>>>
>>> websecurity at lists.webappsec.org
>>>
>>> http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org
>>>
>>>
>>
>> _______________________________________________
>> The Web Security Mailing List
>>
>> WebSecurity RSS Feed
>> http://www.webappsec.org/rss/websecurity.rss
>>
>> Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA
>>
>> WASC on Twitter
>> http://twitter.com/wascupdates
>>
>> websecurity at lists.webappsec.org
>>
>> http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webappsec.org/pipermail/websecurity_lists.webappsec.org/attachments/20110420/85fca3f3/attachment-0003.html>


More information about the websecurity mailing list