[WEB SECURITY] How to perform Antivirus Security Testing
wayne at armorize.com
Wed Apr 20 18:22:36 EDT 2011
secret weapon of some AV vendors these days
On Thu, Apr 21, 2011 at 5:35 AM, Josh More <guppie at starmind.org> wrote:
> Wayne makes a very good point. I was thinking of the common desktop use
> case and completely ignoring API issues.
> One thing to add... if you are using this in a cloud environment and
> planning to tie into VShield, be aware that almost all of the vendors will
> be crippled. This technology allows you to schedule file-based scans and be
> extremely effective in your use of RAM. However, behavioural profiling,
> HIPS and stuff will not work.
> You can also shift the game entirely and look at application whitelisting.
> -Josh More
> On Wed, Apr 20, 2011 at 4:32 PM, Wayne Huang <wayne at armorize.com> wrote:
>> (We're not an antivirus vendor)
>> We have had to select antivirus vendors to work with, and incorporate
>> their scanning engines. Because we leverage their engines to do large-scale
>> scanning, licensing fees are very expensive and therefore, we had to make
>> sure we make the right selection.
>> (Of course we have our own technologies as well and don't just rely on AV
>> So although the best over-all score may go to the bigger players that Josh
>> mentioned below, I was aware of some differences during my past tests:
>> A. For most AV vendors, detection rates of their desktop versions differ
>> greatly with their API offerings (which is what we, as armorize, need). For
>> desktop versions some AV vendors hook into the browser, and this allows them
>> doing, and what the browser plugins (eg flash) are doing. So when they hit a
>> malware, even if it is heavily obfuscated and therefore their signatures
>> fail, they can still rely on behavior.
>> However, very few have the same implementation for their API versions
>> because API versions run stand-alone without user environments, and often
>> under linux, and therefore, behavior capabilities are limited. Virus Total
>> results are based on API versions and not desktop versions. So if you are
>> looking at the API versions (like us) then Virus Total is a good reference;
>> if you're looking at the desktop versions then Virus Total current cannot
>> fully reflect capability differences.
>> B. What are the objectives? If you can deal with false positives but
>> cannot accept false negatives, then another set of vendors, for example
>> Avira comes out top, especially when it comes to Web malware. If you're
>> doing mass scanning and cloud costs (servers) is a big issue then you'd have
>> to test out performance, and sometimes, vendors that excel at desktop-based
>> detection, have very slow and ineffective API implementations. Some vendors
>> don't have good signatures but have very good behavior and therefore for
>> desktop versions they actually do very very well, while their performance is
>> bad on Virus Total. At the same time, some vendors only focus on their API
>> versions and therefore do very well with it.
>> This is our talk at blackhat / defcon focused on Web malware
>> (script-based) but not that much on PE malware:
>> it you'll find a few tables comparing the AV vendors against drivesploit, an
>> open source drive-by download pack.
>> Armorize Technologies
>> On Thu, Apr 21, 2011 at 1:17 AM, Josh More <guppie at starmind.org> wrote:
>>> Don't bother.
>>> Seriously, the top players are: Symantec, McAfee, Trend Micro, Kaspersky
>>> and Sophos. Read the "independent" reviews and these five are always at the
>>> top. Look at the scores from places like http://www.virusbtn.com/ and
>>> these five are always there. Odds are that one of them will work for you
>>> just fine. (I usually pick Sophos for my clients.)
>>> Then look at the extra features. Learn why each one is necessary (note:
>>> they all exist to supplement flaws in the legacy signature-based system).
>>> Figure out which features you need and throw out the vendors that don't
>>> provide them.
>>> Then look at the UI's. If it will be difficult to use one of the systems
>>> in operations, throw it out. Find out if any of the admins are biased
>>> against a system (Symantec is a popular one for admins to hate.) You get
>>> more problems with malware from admins who resist caring for the system than
>>> you get from systems failing to catch stuff.
>>> Then look at the licensing. If you can't understand it or if they're
>>> nickel-and-diming you on price, throw them out. It's not worth the pain
>>> If this process doesn't get you down to a single vendor, look at how they
>>> handle 24/7 support and make test support calls. If their support is poor,
>>> throw them out. If they don't offer 24/7, throw them out (malware doesn't
>>> wait for sun-up). If they force their people to work more than an eight
>>> hour shift, throw them out.
>>> This process will get you a solution that meets real world needs. If you
>>> try to test from a technical perspective, you're just going to be selecting
>>> the system that best protects against attackers that think just like you
>>> do... which you've already protected against through system hardening and
>>> network design.
>>> -Josh More
>>> On Tue, Apr 19, 2011 at 11:28 PM, prashant Kar <kar.prashant at gmail.com>wrote:
>>>> Dear All,
>>>> Kindly guide me on how to do antivirus application security testing.
>>>> Any tools/methodology/approach/checklist that will help, please suggest.
>>>> Best Regards,
>>>> Technical Skill is the mastery of complexity,
>>>> while Creativity is the master of simplicity.....
>>>> The Future Belongs To Those Who Believe in The Beauty of Their Dreams.
>>>> Keep up the spirit!!!!
>>>> Prashant Kar
>>>> The Web Security Mailing List
>>>> WebSecurity RSS Feed
>>>> Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA
>>>> WASC on Twitter
>>>> websecurity at lists.webappsec.org
>>> The Web Security Mailing List
>>> WebSecurity RSS Feed
>>> Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA
>>> WASC on Twitter
>>> websecurity at lists.webappsec.org
Co-Founder, President & CTO
+1-408-216-7893 ext 102
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the websecurity