[WEB SECURITY] How to perform Antivirus Security Testing

Wayne Huang wayne at armorize.com
Wed Apr 20 18:22:36 EDT 2011


whitelisting++

secret weapon of some AV vendors these days

On Thu, Apr 21, 2011 at 5:35 AM, Josh More <guppie at starmind.org> wrote:

> Wayne makes a very good point.  I was thinking of the common desktop use
> case and completely ignoring API issues.
>
> One thing to add... if you are using this in a cloud environment and
> planning to tie into VShield, be aware that almost all of the vendors will
> be crippled.  This technology allows you to schedule file-based scans and be
> extremely effective in your use of RAM.  However, behavioural profiling,
> HIPS and stuff will not work.
>
> You can also shift the game entirely and look at application whitelisting.
>
> -Josh More
>
>
> On Wed, Apr 20, 2011 at 4:32 PM, Wayne Huang <wayne at armorize.com> wrote:
>
>> (We're not an antivirus vendor)
>>
>> We have had to select antivirus vendors to work with, and incorporate
>> their scanning engines. Because we leverage their engines to do large-scale
>> scanning, licensing fees are very expensive and therefore, we had to make
>> sure we make the right selection.
>>
>> (Of course we have our own technologies as well and don't just rely on AV
>> engines)
>>
>> So although the best over-all score may go to the bigger players that Josh
>> mentioned below, I was aware of some differences during my past tests:
>>
>> A. For most AV vendors, detection rates of their desktop versions differ
>> greatly with their API offerings (which is what we, as armorize, need). For
>> desktop versions some AV vendors hook into the browser, and this allows them
>> to see exactly what the browser is doing, what the javascript engine is
>> doing, and what the browser plugins (eg flash) are doing. So when they hit a
>> malware, even if it is heavily obfuscated and therefore their signatures
>> fail, they can still rely on behavior.
>>
>> However, very few have the same implementation for their API versions
>> because API versions run stand-alone without user environments, and often
>> under linux, and therefore, behavior capabilities are limited. Virus Total
>> results are based on API versions and not desktop versions. So if you are
>> looking at the API versions (like us) then Virus Total is a good reference;
>> if you're looking at the desktop versions then Virus Total current cannot
>> fully reflect capability differences.
>>
>> B. What are the objectives? If you can deal with false positives but
>> cannot accept false negatives, then another set of vendors, for example
>> Avira comes out top, especially when it comes to Web malware. If you're
>> doing mass scanning and cloud costs (servers) is a big issue then you'd have
>> to test out performance, and sometimes, vendors that excel at desktop-based
>> detection, have very slow and ineffective API implementations. Some vendors
>> don't have good signatures but have very good behavior and therefore for
>> desktop versions they actually do very very well, while their performance is
>> bad on Virus Total. At the same time, some vendors only focus on their API
>> versions and therefore do very well with it.
>>
>> This is our talk at blackhat / defcon focused on Web malware
>> (script-based) but not that much on PE malware:
>> http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detection
>>
>>
>> <http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detection>In
>> it you'll find a few tables comparing the AV vendors against drivesploit, an
>> open source drive-by download pack.
>>
>> --
>> Wayne
>> Armorize Technologies
>> http://www.armorize.com
>>
>>
>> On Thu, Apr 21, 2011 at 1:17 AM, Josh More <guppie at starmind.org> wrote:
>>
>>> Don't bother.
>>>
>>> Seriously, the top players are:  Symantec, McAfee, Trend Micro, Kaspersky
>>> and Sophos.  Read the "independent" reviews and these five are always at the
>>> top.  Look at the scores from places like http://www.virusbtn.com/ and
>>> these five are always there.  Odds are that one of them will work for you
>>> just fine.  (I usually pick Sophos for my clients.)
>>>
>>> Then look at the extra features.  Learn why each one is necessary (note:
>>> they all exist to supplement flaws in the legacy signature-based system).
>>> Figure out which features you need and throw out the vendors that don't
>>> provide them.
>>>
>>> Then look at the UI's.  If it will be difficult to use one of the systems
>>> in operations, throw it out.  Find out if any of the admins are biased
>>> against a system (Symantec is a popular one for admins to hate.)  You get
>>> more problems with malware from admins who resist caring for the system than
>>> you get from systems failing to catch stuff.
>>>
>>> Then look at the licensing.  If you can't understand it or if they're
>>> nickel-and-diming you on price, throw them out.  It's not worth the pain
>>> otherwise.
>>>
>>> If this process doesn't get you down to a single vendor, look at how they
>>> handle 24/7 support and make test support calls.  If their support is poor,
>>> throw them out.  If they don't offer 24/7, throw them out (malware doesn't
>>> wait for sun-up).  If they force their people to work more than an eight
>>> hour shift, throw them out.
>>>
>>> This process will get you a solution that meets real world needs.  If you
>>> try to test from a technical perspective, you're just going to be selecting
>>> the system that best protects against attackers that think just like you
>>> do... which you've already protected against through system hardening and
>>> network design.
>>>
>>> -Josh More
>>>
>>> On Tue, Apr 19, 2011 at 11:28 PM, prashant Kar <kar.prashant at gmail.com>wrote:
>>>
>>>> Dear All,
>>>>
>>>> Kindly guide me on how to do antivirus application security testing.
>>>>
>>>> Any tools/methodology/approach/checklist that will help, please suggest.
>>>>
>>>> Best Regards,
>>>> Prashant
>>>>
>>>> --
>>>> Technical Skill is the mastery of complexity,
>>>> while Creativity is the master of simplicity.....
>>>>
>>>> The Future Belongs To Those Who Believe in The Beauty of Their Dreams.
>>>> Keep up the spirit!!!!
>>>>
>>>> Prashant Kar
>>>>
>>>> _______________________________________________
>>>> The Web Security Mailing List
>>>>
>>>> WebSecurity RSS Feed
>>>> http://www.webappsec.org/rss/websecurity.rss
>>>>
>>>> Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA
>>>>
>>>> WASC on Twitter
>>>> http://twitter.com/wascupdates
>>>>
>>>> websecurity at lists.webappsec.org
>>>>
>>>> http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org
>>>>
>>>>
>>>
>>> _______________________________________________
>>> The Web Security Mailing List
>>>
>>> WebSecurity RSS Feed
>>> http://www.webappsec.org/rss/websecurity.rss
>>>
>>> Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA
>>>
>>> WASC on Twitter
>>> http://twitter.com/wascupdates
>>>
>>> websecurity at lists.webappsec.org
>>>
>>> http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org
>>>
>>>
>>
>>
>


-- 
Wayne
Co-Founder, President & CTO
Armorize Technologies
http://www.armorize.com
+1-408-216-7893 ext 102
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webappsec.org/pipermail/websecurity_lists.webappsec.org/attachments/20110421/0a26209e/attachment-0003.html>


More information about the websecurity mailing list