websecurity@lists.webappsec.org

The Web Security Mailing List

View all threads

How to perform Antivirus Security Testing

PK
prashant Kar
Wed, Apr 20, 2011 4:28 AM

Dear All,

Kindly guide me on how to do antivirus application security testing.

Any tools/methodology/approach/checklist that will help, please suggest.

Best Regards,
Prashant

--
Technical Skill is the mastery of complexity,
while Creativity is the master of simplicity.....

The Future Belongs To Those Who Believe in The Beauty of Their Dreams.
Keep up the spirit!!!!

Prashant Kar

Dear All, Kindly guide me on how to do antivirus application security testing. Any tools/methodology/approach/checklist that will help, please suggest. Best Regards, Prashant -- Technical Skill is the mastery of complexity, while Creativity is the master of simplicity..... The Future Belongs To Those Who Believe in The Beauty of Their Dreams. Keep up the spirit!!!! Prashant Kar
JM
Josh More
Wed, Apr 20, 2011 5:17 PM

Don't bother.

Seriously, the top players are:  Symantec, McAfee, Trend Micro, Kaspersky
and Sophos.  Read the "independent" reviews and these five are always at the
top.  Look at the scores from places like http://www.virusbtn.com/ and these
five are always there.  Odds are that one of them will work for you just
fine.  (I usually pick Sophos for my clients.)

Then look at the extra features.  Learn why each one is necessary (note:
they all exist to supplement flaws in the legacy signature-based system).
Figure out which features you need and throw out the vendors that don't
provide them.

Then look at the UI's.  If it will be difficult to use one of the systems in
operations, throw it out.  Find out if any of the admins are biased against
a system (Symantec is a popular one for admins to hate.)  You get more
problems with malware from admins who resist caring for the system than you
get from systems failing to catch stuff.

Then look at the licensing.  If you can't understand it or if they're
nickel-and-diming you on price, throw them out.  It's not worth the pain
otherwise.

If this process doesn't get you down to a single vendor, look at how they
handle 24/7 support and make test support calls.  If their support is poor,
throw them out.  If they don't offer 24/7, throw them out (malware doesn't
wait for sun-up).  If they force their people to work more than an eight
hour shift, throw them out.

This process will get you a solution that meets real world needs.  If you
try to test from a technical perspective, you're just going to be selecting
the system that best protects against attackers that think just like you
do... which you've already protected against through system hardening and
network design.

-Josh More

On Tue, Apr 19, 2011 at 11:28 PM, prashant Kar kar.prashant@gmail.comwrote:

Dear All,

Kindly guide me on how to do antivirus application security testing.

Any tools/methodology/approach/checklist that will help, please suggest.

Best Regards,
Prashant

--
Technical Skill is the mastery of complexity,
while Creativity is the master of simplicity.....

The Future Belongs To Those Who Believe in The Beauty of Their Dreams.
Keep up the spirit!!!!

Prashant Kar


The Web Security Mailing List

WebSecurity RSS Feed
http://www.webappsec.org/rss/websecurity.rss

Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA

WASC on Twitter
http://twitter.com/wascupdates

websecurity@lists.webappsec.org
http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org

Don't bother. Seriously, the top players are: Symantec, McAfee, Trend Micro, Kaspersky and Sophos. Read the "independent" reviews and these five are always at the top. Look at the scores from places like http://www.virusbtn.com/ and these five are always there. Odds are that one of them will work for you just fine. (I usually pick Sophos for my clients.) Then look at the extra features. Learn why each one is necessary (note: they all exist to supplement flaws in the legacy signature-based system). Figure out which features you need and throw out the vendors that don't provide them. Then look at the UI's. If it will be difficult to use one of the systems in operations, throw it out. Find out if any of the admins are biased against a system (Symantec is a popular one for admins to hate.) You get more problems with malware from admins who resist caring for the system than you get from systems failing to catch stuff. Then look at the licensing. If you can't understand it or if they're nickel-and-diming you on price, throw them out. It's not worth the pain otherwise. If this process doesn't get you down to a single vendor, look at how they handle 24/7 support and make test support calls. If their support is poor, throw them out. If they don't offer 24/7, throw them out (malware doesn't wait for sun-up). If they force their people to work more than an eight hour shift, throw them out. This process will get you a solution that meets real world needs. If you try to test from a technical perspective, you're just going to be selecting the system that best protects against attackers that think just like you do... which you've already protected against through system hardening and network design. -Josh More On Tue, Apr 19, 2011 at 11:28 PM, prashant Kar <kar.prashant@gmail.com>wrote: > Dear All, > > Kindly guide me on how to do antivirus application security testing. > > Any tools/methodology/approach/checklist that will help, please suggest. > > Best Regards, > Prashant > > -- > Technical Skill is the mastery of complexity, > while Creativity is the master of simplicity..... > > The Future Belongs To Those Who Believe in The Beauty of Their Dreams. > Keep up the spirit!!!! > > Prashant Kar > > _______________________________________________ > The Web Security Mailing List > > WebSecurity RSS Feed > http://www.webappsec.org/rss/websecurity.rss > > Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA > > WASC on Twitter > http://twitter.com/wascupdates > > websecurity@lists.webappsec.org > http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org > >
M
MaXe
Wed, Apr 20, 2011 6:05 PM

Hi Prashant,

One way is to test how easy it is to bypass the scanning engine, with e.g. Assembly and then of course encoding the binary files this way or obfuscating them which tricks heuristic engines.

I have written a paper called: Bypassing Anti-Virus Scanners, which you can download from Exploit-DB in the papers section.

It may be named Bypassing AV Scanners on that site, just so you know.

Best regards,
MaXe
Founder of InterN0T
Blogger at Exploit-DB
Offensive Security Certified Expert (OSCE)
----- Original meddelelse -----

Dear All,

Kindly guide me on how to do antivirus application security testing.

Any tools/methodology/approach/checklist that will help, please suggest.

Best Regards,
Prashant

--
Technical Skill is the mastery of complexity,
while Creativity is the master of simplicity.....

The Future Belongs To Those Who Believe in The Beauty of Their Dreams.
Keep up the spirit!!!!

Prashant Kar

Hi Prashant, One way is to test how easy it is to bypass the scanning engine, with e.g. Assembly and then of course encoding the binary files this way or obfuscating them which tricks heuristic engines. I have written a paper called: Bypassing Anti-Virus Scanners, which you can download from Exploit-DB in the papers section. It may be named Bypassing AV Scanners on that site, just so you know. Best regards, MaXe Founder of InterN0T Blogger at Exploit-DB Offensive Security Certified Expert (OSCE) ----- Original meddelelse ----- > Dear All, > > Kindly guide me on how to do antivirus application security testing. > > Any tools/methodology/approach/checklist that will help, please suggest. > > Best Regards, > Prashant > > -- > Technical Skill is the mastery of complexity, > while Creativity is the master of simplicity..... > > The Future Belongs To Those Who Believe in The Beauty of Their Dreams. > Keep up the spirit!!!! > > Prashant Kar
WH
Wayne Huang
Wed, Apr 20, 2011 9:12 PM

http://www.drivesploit.org/

Open source tool I wrote to test out antivirus capabilities in terms of
drive-by download (Web malware) detection

Mimics various techniques frequently used in the wild

Wayne

On Wed, Apr 20, 2011 at 12:28 PM, prashant Kar kar.prashant@gmail.comwrote:

Dear All,

Kindly guide me on how to do antivirus application security testing.

Any tools/methodology/approach/checklist that will help, please suggest.

Best Regards,
Prashant

--
Technical Skill is the mastery of complexity,
while Creativity is the master of simplicity.....

The Future Belongs To Those Who Believe in The Beauty of Their Dreams.
Keep up the spirit!!!!

Prashant Kar


The Web Security Mailing List

WebSecurity RSS Feed
http://www.webappsec.org/rss/websecurity.rss

Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA

WASC on Twitter
http://twitter.com/wascupdates

websecurity@lists.webappsec.org
http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org

--
Wayne
Co-Founder, President & CTO
Armorize Technologies
http://www.armorize.com
+1-408-216-7893 ext 102

http://www.drivesploit.org/ Open source tool I wrote to test out antivirus capabilities in terms of drive-by download (Web malware) detection Mimics various techniques frequently used in the wild Wayne On Wed, Apr 20, 2011 at 12:28 PM, prashant Kar <kar.prashant@gmail.com>wrote: > Dear All, > > Kindly guide me on how to do antivirus application security testing. > > Any tools/methodology/approach/checklist that will help, please suggest. > > Best Regards, > Prashant > > -- > Technical Skill is the mastery of complexity, > while Creativity is the master of simplicity..... > > The Future Belongs To Those Who Believe in The Beauty of Their Dreams. > Keep up the spirit!!!! > > Prashant Kar > > _______________________________________________ > The Web Security Mailing List > > WebSecurity RSS Feed > http://www.webappsec.org/rss/websecurity.rss > > Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA > > WASC on Twitter > http://twitter.com/wascupdates > > websecurity@lists.webappsec.org > http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org > > -- Wayne Co-Founder, President & CTO Armorize Technologies http://www.armorize.com +1-408-216-7893 ext 102
WH
Wayne Huang
Wed, Apr 20, 2011 9:32 PM

(We're not an antivirus vendor)

We have had to select antivirus vendors to work with, and incorporate their
scanning engines. Because we leverage their engines to do large-scale
scanning, licensing fees are very expensive and therefore, we had to make
sure we make the right selection.

(Of course we have our own technologies as well and don't just rely on AV
engines)

So although the best over-all score may go to the bigger players that Josh
mentioned below, I was aware of some differences during my past tests:

A. For most AV vendors, detection rates of their desktop versions differ
greatly with their API offerings (which is what we, as armorize, need). For
desktop versions some AV vendors hook into the browser, and this allows them
to see exactly what the browser is doing, what the javascript engine is
doing, and what the browser plugins (eg flash) are doing. So when they hit a
malware, even if it is heavily obfuscated and therefore their signatures
fail, they can still rely on behavior.

However, very few have the same implementation for their API versions
because API versions run stand-alone without user environments, and often
under linux, and therefore, behavior capabilities are limited. Virus Total
results are based on API versions and not desktop versions. So if you are
looking at the API versions (like us) then Virus Total is a good reference;
if you're looking at the desktop versions then Virus Total current cannot
fully reflect capability differences.

B. What are the objectives? If you can deal with false positives but cannot
accept false negatives, then another set of vendors, for example Avira comes
out top, especially when it comes to Web malware. If you're doing mass
scanning and cloud costs (servers) is a big issue then you'd have to test
out performance, and sometimes, vendors that excel at desktop-based
detection, have very slow and ineffective API implementations. Some vendors
don't have good signatures but have very good behavior and therefore for
desktop versions they actually do very very well, while their performance is
bad on Virus Total. At the same time, some vendors only focus on their API
versions and therefore do very well with it.

This is our talk at blackhat / defcon focused on Web malware (script-based)
but not that much on PE malware:
http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detection

http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detectionIn
it you'll find a few tables comparing the AV vendors against drivesploit, an
open source drive-by download pack.

--
Wayne
Armorize Technologies
http://www.armorize.com

On Thu, Apr 21, 2011 at 1:17 AM, Josh More guppie@starmind.org wrote:

Don't bother.

Seriously, the top players are:  Symantec, McAfee, Trend Micro, Kaspersky
and Sophos.  Read the "independent" reviews and these five are always at the
top.  Look at the scores from places like http://www.virusbtn.com/ and
these five are always there.  Odds are that one of them will work for you
just fine.  (I usually pick Sophos for my clients.)

Then look at the extra features.  Learn why each one is necessary (note:
they all exist to supplement flaws in the legacy signature-based system).
Figure out which features you need and throw out the vendors that don't
provide them.

Then look at the UI's.  If it will be difficult to use one of the systems
in operations, throw it out.  Find out if any of the admins are biased
against a system (Symantec is a popular one for admins to hate.)  You get
more problems with malware from admins who resist caring for the system than
you get from systems failing to catch stuff.

Then look at the licensing.  If you can't understand it or if they're
nickel-and-diming you on price, throw them out.  It's not worth the pain
otherwise.

If this process doesn't get you down to a single vendor, look at how they
handle 24/7 support and make test support calls.  If their support is poor,
throw them out.  If they don't offer 24/7, throw them out (malware doesn't
wait for sun-up).  If they force their people to work more than an eight
hour shift, throw them out.

This process will get you a solution that meets real world needs.  If you
try to test from a technical perspective, you're just going to be selecting
the system that best protects against attackers that think just like you
do... which you've already protected against through system hardening and
network design.

-Josh More

On Tue, Apr 19, 2011 at 11:28 PM, prashant Kar kar.prashant@gmail.comwrote:

Dear All,

Kindly guide me on how to do antivirus application security testing.

Any tools/methodology/approach/checklist that will help, please suggest.

Best Regards,
Prashant

--
Technical Skill is the mastery of complexity,
while Creativity is the master of simplicity.....

The Future Belongs To Those Who Believe in The Beauty of Their Dreams.
Keep up the spirit!!!!

Prashant Kar


The Web Security Mailing List

WebSecurity RSS Feed
http://www.webappsec.org/rss/websecurity.rss

Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA

WASC on Twitter
http://twitter.com/wascupdates

websecurity@lists.webappsec.org

http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org

(We're not an antivirus vendor) We have had to select antivirus vendors to work with, and incorporate their scanning engines. Because we leverage their engines to do large-scale scanning, licensing fees are very expensive and therefore, we had to make sure we make the right selection. (Of course we have our own technologies as well and don't just rely on AV engines) So although the best over-all score may go to the bigger players that Josh mentioned below, I was aware of some differences during my past tests: A. For most AV vendors, detection rates of their desktop versions differ greatly with their API offerings (which is what we, as armorize, need). For desktop versions some AV vendors hook into the browser, and this allows them to see exactly what the browser is doing, what the javascript engine is doing, and what the browser plugins (eg flash) are doing. So when they hit a malware, even if it is heavily obfuscated and therefore their signatures fail, they can still rely on behavior. However, very few have the same implementation for their API versions because API versions run stand-alone without user environments, and often under linux, and therefore, behavior capabilities are limited. Virus Total results are based on API versions and not desktop versions. So if you are looking at the API versions (like us) then Virus Total is a good reference; if you're looking at the desktop versions then Virus Total current cannot fully reflect capability differences. B. What are the objectives? If you can deal with false positives but cannot accept false negatives, then another set of vendors, for example Avira comes out top, especially when it comes to Web malware. If you're doing mass scanning and cloud costs (servers) is a big issue then you'd have to test out performance, and sometimes, vendors that excel at desktop-based detection, have very slow and ineffective API implementations. Some vendors don't have good signatures but have very good behavior and therefore for desktop versions they actually do very very well, while their performance is bad on Virus Total. At the same time, some vendors only focus on their API versions and therefore do very well with it. This is our talk at blackhat / defcon focused on Web malware (script-based) but not that much on PE malware: http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detection <http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detection>In it you'll find a few tables comparing the AV vendors against drivesploit, an open source drive-by download pack. -- Wayne Armorize Technologies http://www.armorize.com On Thu, Apr 21, 2011 at 1:17 AM, Josh More <guppie@starmind.org> wrote: > Don't bother. > > Seriously, the top players are: Symantec, McAfee, Trend Micro, Kaspersky > and Sophos. Read the "independent" reviews and these five are always at the > top. Look at the scores from places like http://www.virusbtn.com/ and > these five are always there. Odds are that one of them will work for you > just fine. (I usually pick Sophos for my clients.) > > Then look at the extra features. Learn why each one is necessary (note: > they all exist to supplement flaws in the legacy signature-based system). > Figure out which features you need and throw out the vendors that don't > provide them. > > Then look at the UI's. If it will be difficult to use one of the systems > in operations, throw it out. Find out if any of the admins are biased > against a system (Symantec is a popular one for admins to hate.) You get > more problems with malware from admins who resist caring for the system than > you get from systems failing to catch stuff. > > Then look at the licensing. If you can't understand it or if they're > nickel-and-diming you on price, throw them out. It's not worth the pain > otherwise. > > If this process doesn't get you down to a single vendor, look at how they > handle 24/7 support and make test support calls. If their support is poor, > throw them out. If they don't offer 24/7, throw them out (malware doesn't > wait for sun-up). If they force their people to work more than an eight > hour shift, throw them out. > > This process will get you a solution that meets real world needs. If you > try to test from a technical perspective, you're just going to be selecting > the system that best protects against attackers that think just like you > do... which you've already protected against through system hardening and > network design. > > -Josh More > > On Tue, Apr 19, 2011 at 11:28 PM, prashant Kar <kar.prashant@gmail.com>wrote: > >> Dear All, >> >> Kindly guide me on how to do antivirus application security testing. >> >> Any tools/methodology/approach/checklist that will help, please suggest. >> >> Best Regards, >> Prashant >> >> -- >> Technical Skill is the mastery of complexity, >> while Creativity is the master of simplicity..... >> >> The Future Belongs To Those Who Believe in The Beauty of Their Dreams. >> Keep up the spirit!!!! >> >> Prashant Kar >> >> _______________________________________________ >> The Web Security Mailing List >> >> WebSecurity RSS Feed >> http://www.webappsec.org/rss/websecurity.rss >> >> Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA >> >> WASC on Twitter >> http://twitter.com/wascupdates >> >> websecurity@lists.webappsec.org >> >> http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org >> >> > > _______________________________________________ > The Web Security Mailing List > > WebSecurity RSS Feed > http://www.webappsec.org/rss/websecurity.rss > > Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA > > WASC on Twitter > http://twitter.com/wascupdates > > websecurity@lists.webappsec.org > http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org > >
JM
Josh More
Wed, Apr 20, 2011 9:35 PM

Wayne makes a very good point.  I was thinking of the common desktop use
case and completely ignoring API issues.

One thing to add... if you are using this in a cloud environment and
planning to tie into VShield, be aware that almost all of the vendors will
be crippled.  This technology allows you to schedule file-based scans and be
extremely effective in your use of RAM.  However, behavioural profiling,
HIPS and stuff will not work.

You can also shift the game entirely and look at application whitelisting.

-Josh More

On Wed, Apr 20, 2011 at 4:32 PM, Wayne Huang wayne@armorize.com wrote:

(We're not an antivirus vendor)

We have had to select antivirus vendors to work with, and incorporate their
scanning engines. Because we leverage their engines to do large-scale
scanning, licensing fees are very expensive and therefore, we had to make
sure we make the right selection.

(Of course we have our own technologies as well and don't just rely on AV
engines)

So although the best over-all score may go to the bigger players that Josh
mentioned below, I was aware of some differences during my past tests:

A. For most AV vendors, detection rates of their desktop versions differ
greatly with their API offerings (which is what we, as armorize, need). For
desktop versions some AV vendors hook into the browser, and this allows them
to see exactly what the browser is doing, what the javascript engine is
doing, and what the browser plugins (eg flash) are doing. So when they hit a
malware, even if it is heavily obfuscated and therefore their signatures
fail, they can still rely on behavior.

However, very few have the same implementation for their API versions
because API versions run stand-alone without user environments, and often
under linux, and therefore, behavior capabilities are limited. Virus Total
results are based on API versions and not desktop versions. So if you are
looking at the API versions (like us) then Virus Total is a good reference;
if you're looking at the desktop versions then Virus Total current cannot
fully reflect capability differences.

B. What are the objectives? If you can deal with false positives but cannot
accept false negatives, then another set of vendors, for example Avira comes
out top, especially when it comes to Web malware. If you're doing mass
scanning and cloud costs (servers) is a big issue then you'd have to test
out performance, and sometimes, vendors that excel at desktop-based
detection, have very slow and ineffective API implementations. Some vendors
don't have good signatures but have very good behavior and therefore for
desktop versions they actually do very very well, while their performance is
bad on Virus Total. At the same time, some vendors only focus on their API
versions and therefore do very well with it.

This is our talk at blackhat / defcon focused on Web malware (script-based)
but not that much on PE malware:
http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detection

http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detectionIn
it you'll find a few tables comparing the AV vendors against drivesploit, an
open source drive-by download pack.

--
Wayne
Armorize Technologies
http://www.armorize.com

On Thu, Apr 21, 2011 at 1:17 AM, Josh More guppie@starmind.org wrote:

Don't bother.

Seriously, the top players are:  Symantec, McAfee, Trend Micro, Kaspersky
and Sophos.  Read the "independent" reviews and these five are always at the
top.  Look at the scores from places like http://www.virusbtn.com/ and
these five are always there.  Odds are that one of them will work for you
just fine.  (I usually pick Sophos for my clients.)

Then look at the extra features.  Learn why each one is necessary (note:
they all exist to supplement flaws in the legacy signature-based system).
Figure out which features you need and throw out the vendors that don't
provide them.

Then look at the UI's.  If it will be difficult to use one of the systems
in operations, throw it out.  Find out if any of the admins are biased
against a system (Symantec is a popular one for admins to hate.)  You get
more problems with malware from admins who resist caring for the system than
you get from systems failing to catch stuff.

Then look at the licensing.  If you can't understand it or if they're
nickel-and-diming you on price, throw them out.  It's not worth the pain
otherwise.

If this process doesn't get you down to a single vendor, look at how they
handle 24/7 support and make test support calls.  If their support is poor,
throw them out.  If they don't offer 24/7, throw them out (malware doesn't
wait for sun-up).  If they force their people to work more than an eight
hour shift, throw them out.

This process will get you a solution that meets real world needs.  If you
try to test from a technical perspective, you're just going to be selecting
the system that best protects against attackers that think just like you
do... which you've already protected against through system hardening and
network design.

-Josh More

On Tue, Apr 19, 2011 at 11:28 PM, prashant Kar kar.prashant@gmail.comwrote:

Dear All,

Kindly guide me on how to do antivirus application security testing.

Any tools/methodology/approach/checklist that will help, please suggest.

Best Regards,
Prashant

--
Technical Skill is the mastery of complexity,
while Creativity is the master of simplicity.....

The Future Belongs To Those Who Believe in The Beauty of Their Dreams.
Keep up the spirit!!!!

Prashant Kar


The Web Security Mailing List

WebSecurity RSS Feed
http://www.webappsec.org/rss/websecurity.rss

Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA

WASC on Twitter
http://twitter.com/wascupdates

websecurity@lists.webappsec.org

http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org

Wayne makes a very good point. I was thinking of the common desktop use case and completely ignoring API issues. One thing to add... if you are using this in a cloud environment and planning to tie into VShield, be aware that almost all of the vendors will be crippled. This technology allows you to schedule file-based scans and be extremely effective in your use of RAM. However, behavioural profiling, HIPS and stuff will not work. You can also shift the game entirely and look at application whitelisting. -Josh More On Wed, Apr 20, 2011 at 4:32 PM, Wayne Huang <wayne@armorize.com> wrote: > (We're not an antivirus vendor) > > We have had to select antivirus vendors to work with, and incorporate their > scanning engines. Because we leverage their engines to do large-scale > scanning, licensing fees are very expensive and therefore, we had to make > sure we make the right selection. > > (Of course we have our own technologies as well and don't just rely on AV > engines) > > So although the best over-all score may go to the bigger players that Josh > mentioned below, I was aware of some differences during my past tests: > > A. For most AV vendors, detection rates of their desktop versions differ > greatly with their API offerings (which is what we, as armorize, need). For > desktop versions some AV vendors hook into the browser, and this allows them > to see exactly what the browser is doing, what the javascript engine is > doing, and what the browser plugins (eg flash) are doing. So when they hit a > malware, even if it is heavily obfuscated and therefore their signatures > fail, they can still rely on behavior. > > However, very few have the same implementation for their API versions > because API versions run stand-alone without user environments, and often > under linux, and therefore, behavior capabilities are limited. Virus Total > results are based on API versions and not desktop versions. So if you are > looking at the API versions (like us) then Virus Total is a good reference; > if you're looking at the desktop versions then Virus Total current cannot > fully reflect capability differences. > > B. What are the objectives? If you can deal with false positives but cannot > accept false negatives, then another set of vendors, for example Avira comes > out top, especially when it comes to Web malware. If you're doing mass > scanning and cloud costs (servers) is a big issue then you'd have to test > out performance, and sometimes, vendors that excel at desktop-based > detection, have very slow and ineffective API implementations. Some vendors > don't have good signatures but have very good behavior and therefore for > desktop versions they actually do very very well, while their performance is > bad on Virus Total. At the same time, some vendors only focus on their API > versions and therefore do very well with it. > > This is our talk at blackhat / defcon focused on Web malware (script-based) > but not that much on PE malware: > http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detection > > > <http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detection>In > it you'll find a few tables comparing the AV vendors against drivesploit, an > open source drive-by download pack. > > -- > Wayne > Armorize Technologies > http://www.armorize.com > > > On Thu, Apr 21, 2011 at 1:17 AM, Josh More <guppie@starmind.org> wrote: > >> Don't bother. >> >> Seriously, the top players are: Symantec, McAfee, Trend Micro, Kaspersky >> and Sophos. Read the "independent" reviews and these five are always at the >> top. Look at the scores from places like http://www.virusbtn.com/ and >> these five are always there. Odds are that one of them will work for you >> just fine. (I usually pick Sophos for my clients.) >> >> Then look at the extra features. Learn why each one is necessary (note: >> they all exist to supplement flaws in the legacy signature-based system). >> Figure out which features you need and throw out the vendors that don't >> provide them. >> >> Then look at the UI's. If it will be difficult to use one of the systems >> in operations, throw it out. Find out if any of the admins are biased >> against a system (Symantec is a popular one for admins to hate.) You get >> more problems with malware from admins who resist caring for the system than >> you get from systems failing to catch stuff. >> >> Then look at the licensing. If you can't understand it or if they're >> nickel-and-diming you on price, throw them out. It's not worth the pain >> otherwise. >> >> If this process doesn't get you down to a single vendor, look at how they >> handle 24/7 support and make test support calls. If their support is poor, >> throw them out. If they don't offer 24/7, throw them out (malware doesn't >> wait for sun-up). If they force their people to work more than an eight >> hour shift, throw them out. >> >> This process will get you a solution that meets real world needs. If you >> try to test from a technical perspective, you're just going to be selecting >> the system that best protects against attackers that think just like you >> do... which you've already protected against through system hardening and >> network design. >> >> -Josh More >> >> On Tue, Apr 19, 2011 at 11:28 PM, prashant Kar <kar.prashant@gmail.com>wrote: >> >>> Dear All, >>> >>> Kindly guide me on how to do antivirus application security testing. >>> >>> Any tools/methodology/approach/checklist that will help, please suggest. >>> >>> Best Regards, >>> Prashant >>> >>> -- >>> Technical Skill is the mastery of complexity, >>> while Creativity is the master of simplicity..... >>> >>> The Future Belongs To Those Who Believe in The Beauty of Their Dreams. >>> Keep up the spirit!!!! >>> >>> Prashant Kar >>> >>> _______________________________________________ >>> The Web Security Mailing List >>> >>> WebSecurity RSS Feed >>> http://www.webappsec.org/rss/websecurity.rss >>> >>> Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA >>> >>> WASC on Twitter >>> http://twitter.com/wascupdates >>> >>> websecurity@lists.webappsec.org >>> >>> http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org >>> >>> >> >> _______________________________________________ >> The Web Security Mailing List >> >> WebSecurity RSS Feed >> http://www.webappsec.org/rss/websecurity.rss >> >> Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA >> >> WASC on Twitter >> http://twitter.com/wascupdates >> >> websecurity@lists.webappsec.org >> >> http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org >> >> > >
WH
Wayne Huang
Wed, Apr 20, 2011 10:22 PM

whitelisting++

secret weapon of some AV vendors these days

On Thu, Apr 21, 2011 at 5:35 AM, Josh More guppie@starmind.org wrote:

Wayne makes a very good point.  I was thinking of the common desktop use
case and completely ignoring API issues.

One thing to add... if you are using this in a cloud environment and
planning to tie into VShield, be aware that almost all of the vendors will
be crippled.  This technology allows you to schedule file-based scans and be
extremely effective in your use of RAM.  However, behavioural profiling,
HIPS and stuff will not work.

You can also shift the game entirely and look at application whitelisting.

-Josh More

On Wed, Apr 20, 2011 at 4:32 PM, Wayne Huang wayne@armorize.com wrote:

(We're not an antivirus vendor)

We have had to select antivirus vendors to work with, and incorporate
their scanning engines. Because we leverage their engines to do large-scale
scanning, licensing fees are very expensive and therefore, we had to make
sure we make the right selection.

(Of course we have our own technologies as well and don't just rely on AV
engines)

So although the best over-all score may go to the bigger players that Josh
mentioned below, I was aware of some differences during my past tests:

A. For most AV vendors, detection rates of their desktop versions differ
greatly with their API offerings (which is what we, as armorize, need). For
desktop versions some AV vendors hook into the browser, and this allows them
to see exactly what the browser is doing, what the javascript engine is
doing, and what the browser plugins (eg flash) are doing. So when they hit a
malware, even if it is heavily obfuscated and therefore their signatures
fail, they can still rely on behavior.

However, very few have the same implementation for their API versions
because API versions run stand-alone without user environments, and often
under linux, and therefore, behavior capabilities are limited. Virus Total
results are based on API versions and not desktop versions. So if you are
looking at the API versions (like us) then Virus Total is a good reference;
if you're looking at the desktop versions then Virus Total current cannot
fully reflect capability differences.

B. What are the objectives? If you can deal with false positives but
cannot accept false negatives, then another set of vendors, for example
Avira comes out top, especially when it comes to Web malware. If you're
doing mass scanning and cloud costs (servers) is a big issue then you'd have
to test out performance, and sometimes, vendors that excel at desktop-based
detection, have very slow and ineffective API implementations. Some vendors
don't have good signatures but have very good behavior and therefore for
desktop versions they actually do very very well, while their performance is
bad on Virus Total. At the same time, some vendors only focus on their API
versions and therefore do very well with it.

This is our talk at blackhat / defcon focused on Web malware
(script-based) but not that much on PE malware:
http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detection

http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detectionIn
it you'll find a few tables comparing the AV vendors against drivesploit, an
open source drive-by download pack.

--
Wayne
Armorize Technologies
http://www.armorize.com

On Thu, Apr 21, 2011 at 1:17 AM, Josh More guppie@starmind.org wrote:

Don't bother.

Seriously, the top players are:  Symantec, McAfee, Trend Micro, Kaspersky
and Sophos.  Read the "independent" reviews and these five are always at the
top.  Look at the scores from places like http://www.virusbtn.com/ and
these five are always there.  Odds are that one of them will work for you
just fine.  (I usually pick Sophos for my clients.)

Then look at the extra features.  Learn why each one is necessary (note:
they all exist to supplement flaws in the legacy signature-based system).
Figure out which features you need and throw out the vendors that don't
provide them.

Then look at the UI's.  If it will be difficult to use one of the systems
in operations, throw it out.  Find out if any of the admins are biased
against a system (Symantec is a popular one for admins to hate.)  You get
more problems with malware from admins who resist caring for the system than
you get from systems failing to catch stuff.

Then look at the licensing.  If you can't understand it or if they're
nickel-and-diming you on price, throw them out.  It's not worth the pain
otherwise.

If this process doesn't get you down to a single vendor, look at how they
handle 24/7 support and make test support calls.  If their support is poor,
throw them out.  If they don't offer 24/7, throw them out (malware doesn't
wait for sun-up).  If they force their people to work more than an eight
hour shift, throw them out.

This process will get you a solution that meets real world needs.  If you
try to test from a technical perspective, you're just going to be selecting
the system that best protects against attackers that think just like you
do... which you've already protected against through system hardening and
network design.

-Josh More

On Tue, Apr 19, 2011 at 11:28 PM, prashant Kar kar.prashant@gmail.comwrote:

Dear All,

Kindly guide me on how to do antivirus application security testing.

Any tools/methodology/approach/checklist that will help, please suggest.

Best Regards,
Prashant

--
Technical Skill is the mastery of complexity,
while Creativity is the master of simplicity.....

The Future Belongs To Those Who Believe in The Beauty of Their Dreams.
Keep up the spirit!!!!

Prashant Kar


The Web Security Mailing List

WebSecurity RSS Feed
http://www.webappsec.org/rss/websecurity.rss

Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA

WASC on Twitter
http://twitter.com/wascupdates

websecurity@lists.webappsec.org

http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org

--
Wayne
Co-Founder, President & CTO
Armorize Technologies
http://www.armorize.com
+1-408-216-7893 ext 102

whitelisting++ secret weapon of some AV vendors these days On Thu, Apr 21, 2011 at 5:35 AM, Josh More <guppie@starmind.org> wrote: > Wayne makes a very good point. I was thinking of the common desktop use > case and completely ignoring API issues. > > One thing to add... if you are using this in a cloud environment and > planning to tie into VShield, be aware that almost all of the vendors will > be crippled. This technology allows you to schedule file-based scans and be > extremely effective in your use of RAM. However, behavioural profiling, > HIPS and stuff will not work. > > You can also shift the game entirely and look at application whitelisting. > > -Josh More > > > On Wed, Apr 20, 2011 at 4:32 PM, Wayne Huang <wayne@armorize.com> wrote: > >> (We're not an antivirus vendor) >> >> We have had to select antivirus vendors to work with, and incorporate >> their scanning engines. Because we leverage their engines to do large-scale >> scanning, licensing fees are very expensive and therefore, we had to make >> sure we make the right selection. >> >> (Of course we have our own technologies as well and don't just rely on AV >> engines) >> >> So although the best over-all score may go to the bigger players that Josh >> mentioned below, I was aware of some differences during my past tests: >> >> A. For most AV vendors, detection rates of their desktop versions differ >> greatly with their API offerings (which is what we, as armorize, need). For >> desktop versions some AV vendors hook into the browser, and this allows them >> to see exactly what the browser is doing, what the javascript engine is >> doing, and what the browser plugins (eg flash) are doing. So when they hit a >> malware, even if it is heavily obfuscated and therefore their signatures >> fail, they can still rely on behavior. >> >> However, very few have the same implementation for their API versions >> because API versions run stand-alone without user environments, and often >> under linux, and therefore, behavior capabilities are limited. Virus Total >> results are based on API versions and not desktop versions. So if you are >> looking at the API versions (like us) then Virus Total is a good reference; >> if you're looking at the desktop versions then Virus Total current cannot >> fully reflect capability differences. >> >> B. What are the objectives? If you can deal with false positives but >> cannot accept false negatives, then another set of vendors, for example >> Avira comes out top, especially when it comes to Web malware. If you're >> doing mass scanning and cloud costs (servers) is a big issue then you'd have >> to test out performance, and sometimes, vendors that excel at desktop-based >> detection, have very slow and ineffective API implementations. Some vendors >> don't have good signatures but have very good behavior and therefore for >> desktop versions they actually do very very well, while their performance is >> bad on Virus Total. At the same time, some vendors only focus on their API >> versions and therefore do very well with it. >> >> This is our talk at blackhat / defcon focused on Web malware >> (script-based) but not that much on PE malware: >> http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detection >> >> >> <http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detection>In >> it you'll find a few tables comparing the AV vendors against drivesploit, an >> open source drive-by download pack. >> >> -- >> Wayne >> Armorize Technologies >> http://www.armorize.com >> >> >> On Thu, Apr 21, 2011 at 1:17 AM, Josh More <guppie@starmind.org> wrote: >> >>> Don't bother. >>> >>> Seriously, the top players are: Symantec, McAfee, Trend Micro, Kaspersky >>> and Sophos. Read the "independent" reviews and these five are always at the >>> top. Look at the scores from places like http://www.virusbtn.com/ and >>> these five are always there. Odds are that one of them will work for you >>> just fine. (I usually pick Sophos for my clients.) >>> >>> Then look at the extra features. Learn why each one is necessary (note: >>> they all exist to supplement flaws in the legacy signature-based system). >>> Figure out which features you need and throw out the vendors that don't >>> provide them. >>> >>> Then look at the UI's. If it will be difficult to use one of the systems >>> in operations, throw it out. Find out if any of the admins are biased >>> against a system (Symantec is a popular one for admins to hate.) You get >>> more problems with malware from admins who resist caring for the system than >>> you get from systems failing to catch stuff. >>> >>> Then look at the licensing. If you can't understand it or if they're >>> nickel-and-diming you on price, throw them out. It's not worth the pain >>> otherwise. >>> >>> If this process doesn't get you down to a single vendor, look at how they >>> handle 24/7 support and make test support calls. If their support is poor, >>> throw them out. If they don't offer 24/7, throw them out (malware doesn't >>> wait for sun-up). If they force their people to work more than an eight >>> hour shift, throw them out. >>> >>> This process will get you a solution that meets real world needs. If you >>> try to test from a technical perspective, you're just going to be selecting >>> the system that best protects against attackers that think just like you >>> do... which you've already protected against through system hardening and >>> network design. >>> >>> -Josh More >>> >>> On Tue, Apr 19, 2011 at 11:28 PM, prashant Kar <kar.prashant@gmail.com>wrote: >>> >>>> Dear All, >>>> >>>> Kindly guide me on how to do antivirus application security testing. >>>> >>>> Any tools/methodology/approach/checklist that will help, please suggest. >>>> >>>> Best Regards, >>>> Prashant >>>> >>>> -- >>>> Technical Skill is the mastery of complexity, >>>> while Creativity is the master of simplicity..... >>>> >>>> The Future Belongs To Those Who Believe in The Beauty of Their Dreams. >>>> Keep up the spirit!!!! >>>> >>>> Prashant Kar >>>> >>>> _______________________________________________ >>>> The Web Security Mailing List >>>> >>>> WebSecurity RSS Feed >>>> http://www.webappsec.org/rss/websecurity.rss >>>> >>>> Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA >>>> >>>> WASC on Twitter >>>> http://twitter.com/wascupdates >>>> >>>> websecurity@lists.webappsec.org >>>> >>>> http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org >>>> >>>> >>> >>> _______________________________________________ >>> The Web Security Mailing List >>> >>> WebSecurity RSS Feed >>> http://www.webappsec.org/rss/websecurity.rss >>> >>> Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA >>> >>> WASC on Twitter >>> http://twitter.com/wascupdates >>> >>> websecurity@lists.webappsec.org >>> >>> http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org >>> >>> >> >> > -- Wayne Co-Founder, President & CTO Armorize Technologies http://www.armorize.com +1-408-216-7893 ext 102
SN
Shlomi Narkolayev
Tue, Apr 26, 2011 12:24 PM

Hi Prashant Kar,

If you like to bypass signature based Antivirus, you can easily do so using
the "Splitting file method", it worked for me for +90% of the time.
You can also try different and new packers.

Have fun...!

Kind Regards,
Narkolayev Shlomi.

Visit my blog: http://Narkolayev-Shlomi.blogspot.com

On Thu, Apr 21, 2011 at 1:22 AM, Wayne Huang wayne@armorize.com wrote:

whitelisting++

secret weapon of some AV vendors these days

On Thu, Apr 21, 2011 at 5:35 AM, Josh More guppie@starmind.org wrote:

Wayne makes a very good point.  I was thinking of the common desktop use
case and completely ignoring API issues.

One thing to add... if you are using this in a cloud environment and
planning to tie into VShield, be aware that almost all of the vendors will
be crippled.  This technology allows you to schedule file-based scans and be
extremely effective in your use of RAM.  However, behavioural profiling,
HIPS and stuff will not work.

You can also shift the game entirely and look at application whitelisting.

-Josh More

On Wed, Apr 20, 2011 at 4:32 PM, Wayne Huang wayne@armorize.com wrote:

(We're not an antivirus vendor)

We have had to select antivirus vendors to work with, and incorporate
their scanning engines. Because we leverage their engines to do large-scale
scanning, licensing fees are very expensive and therefore, we had to make
sure we make the right selection.

(Of course we have our own technologies as well and don't just rely on AV
engines)

So although the best over-all score may go to the bigger players that
Josh mentioned below, I was aware of some differences during my past tests:

A. For most AV vendors, detection rates of their desktop versions differ
greatly with their API offerings (which is what we, as armorize, need). For
desktop versions some AV vendors hook into the browser, and this allows them
to see exactly what the browser is doing, what the javascript engine is
doing, and what the browser plugins (eg flash) are doing. So when they hit a
malware, even if it is heavily obfuscated and therefore their signatures
fail, they can still rely on behavior.

However, very few have the same implementation for their API versions
because API versions run stand-alone without user environments, and often
under linux, and therefore, behavior capabilities are limited. Virus Total
results are based on API versions and not desktop versions. So if you are
looking at the API versions (like us) then Virus Total is a good reference;
if you're looking at the desktop versions then Virus Total current cannot
fully reflect capability differences.

B. What are the objectives? If you can deal with false positives but
cannot accept false negatives, then another set of vendors, for example
Avira comes out top, especially when it comes to Web malware. If you're
doing mass scanning and cloud costs (servers) is a big issue then you'd have
to test out performance, and sometimes, vendors that excel at desktop-based
detection, have very slow and ineffective API implementations. Some vendors
don't have good signatures but have very good behavior and therefore for
desktop versions they actually do very very well, while their performance is
bad on Virus Total. At the same time, some vendors only focus on their API
versions and therefore do very well with it.

This is our talk at blackhat / defcon focused on Web malware
(script-based) but not that much on PE malware:
http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detection

http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detectionIn
it you'll find a few tables comparing the AV vendors against drivesploit, an
open source drive-by download pack.

--
Wayne
Armorize Technologies
http://www.armorize.com

On Thu, Apr 21, 2011 at 1:17 AM, Josh More guppie@starmind.org wrote:

Don't bother.

Seriously, the top players are:  Symantec, McAfee, Trend Micro,
Kaspersky and Sophos.  Read the "independent" reviews and these five are
always at the top.  Look at the scores from places like
http://www.virusbtn.com/ and these five are always there.  Odds are
that one of them will work for you just fine.  (I usually pick Sophos for my
clients.)

Then look at the extra features.  Learn why each one is necessary (note:
they all exist to supplement flaws in the legacy signature-based system).
Figure out which features you need and throw out the vendors that don't
provide them.

Then look at the UI's.  If it will be difficult to use one of the
systems in operations, throw it out.  Find out if any of the admins are
biased against a system (Symantec is a popular one for admins to hate.)  You
get more problems with malware from admins who resist caring for the system
than you get from systems failing to catch stuff.

Then look at the licensing.  If you can't understand it or if they're
nickel-and-diming you on price, throw them out.  It's not worth the pain
otherwise.

If this process doesn't get you down to a single vendor, look at how
they handle 24/7 support and make test support calls.  If their support is
poor, throw them out.  If they don't offer 24/7, throw them out (malware
doesn't wait for sun-up).  If they force their people to work more than an
eight hour shift, throw them out.

This process will get you a solution that meets real world needs.  If
you try to test from a technical perspective, you're just going to be
selecting the system that best protects against attackers that think just
like you do... which you've already protected against through system
hardening and network design.

-Josh More

On Tue, Apr 19, 2011 at 11:28 PM, prashant Kar <
kar.prashant@gmail.com> wrote:

Dear All,

Kindly guide me on how to do antivirus application security testing.

Any tools/methodology/approach/checklist that will help, please
suggest.

Best Regards,
Prashant

--
Technical Skill is the mastery of complexity,
while Creativity is the master of simplicity.....

The Future Belongs To Those Who Believe in The Beauty of Their Dreams.
Keep up the spirit!!!!

Prashant Kar


The Web Security Mailing List

WebSecurity RSS Feed
http://www.webappsec.org/rss/websecurity.rss

Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA

WASC on Twitter
http://twitter.com/wascupdates

websecurity@lists.webappsec.org

http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org

--
Wayne
Co-Founder, President & CTO
Armorize Technologies
http://www.armorize.com
+1-408-216-7893 ext 102


The Web Security Mailing List

WebSecurity RSS Feed
http://www.webappsec.org/rss/websecurity.rss

Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA

WASC on Twitter
http://twitter.com/wascupdates

websecurity@lists.webappsec.org
http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org

Hi Prashant Kar, If you like to bypass signature based Antivirus, you can easily do so using the "Splitting file method", it worked for me for +90% of the time. You can also try different and new packers. Have fun...! Kind Regards, Narkolayev Shlomi. Visit my blog: http://Narkolayev-Shlomi.blogspot.com On Thu, Apr 21, 2011 at 1:22 AM, Wayne Huang <wayne@armorize.com> wrote: > whitelisting++ > > secret weapon of some AV vendors these days > > > On Thu, Apr 21, 2011 at 5:35 AM, Josh More <guppie@starmind.org> wrote: > >> Wayne makes a very good point. I was thinking of the common desktop use >> case and completely ignoring API issues. >> >> One thing to add... if you are using this in a cloud environment and >> planning to tie into VShield, be aware that almost all of the vendors will >> be crippled. This technology allows you to schedule file-based scans and be >> extremely effective in your use of RAM. However, behavioural profiling, >> HIPS and stuff will not work. >> >> You can also shift the game entirely and look at application whitelisting. >> >> -Josh More >> >> >> On Wed, Apr 20, 2011 at 4:32 PM, Wayne Huang <wayne@armorize.com> wrote: >> >>> (We're not an antivirus vendor) >>> >>> We have had to select antivirus vendors to work with, and incorporate >>> their scanning engines. Because we leverage their engines to do large-scale >>> scanning, licensing fees are very expensive and therefore, we had to make >>> sure we make the right selection. >>> >>> (Of course we have our own technologies as well and don't just rely on AV >>> engines) >>> >>> So although the best over-all score may go to the bigger players that >>> Josh mentioned below, I was aware of some differences during my past tests: >>> >>> A. For most AV vendors, detection rates of their desktop versions differ >>> greatly with their API offerings (which is what we, as armorize, need). For >>> desktop versions some AV vendors hook into the browser, and this allows them >>> to see exactly what the browser is doing, what the javascript engine is >>> doing, and what the browser plugins (eg flash) are doing. So when they hit a >>> malware, even if it is heavily obfuscated and therefore their signatures >>> fail, they can still rely on behavior. >>> >>> However, very few have the same implementation for their API versions >>> because API versions run stand-alone without user environments, and often >>> under linux, and therefore, behavior capabilities are limited. Virus Total >>> results are based on API versions and not desktop versions. So if you are >>> looking at the API versions (like us) then Virus Total is a good reference; >>> if you're looking at the desktop versions then Virus Total current cannot >>> fully reflect capability differences. >>> >>> B. What are the objectives? If you can deal with false positives but >>> cannot accept false negatives, then another set of vendors, for example >>> Avira comes out top, especially when it comes to Web malware. If you're >>> doing mass scanning and cloud costs (servers) is a big issue then you'd have >>> to test out performance, and sometimes, vendors that excel at desktop-based >>> detection, have very slow and ineffective API implementations. Some vendors >>> don't have good signatures but have very good behavior and therefore for >>> desktop versions they actually do very very well, while their performance is >>> bad on Virus Total. At the same time, some vendors only focus on their API >>> versions and therefore do very well with it. >>> >>> This is our talk at blackhat / defcon focused on Web malware >>> (script-based) but not that much on PE malware: >>> http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detection >>> >>> <http://www.slideshare.net/wayne_armorize/drivesploit-circumventing-both-automated-and-manual-drivebydownload-detection>In >>> it you'll find a few tables comparing the AV vendors against drivesploit, an >>> open source drive-by download pack. >>> >>> -- >>> Wayne >>> Armorize Technologies >>> http://www.armorize.com >>> >>> >>> On Thu, Apr 21, 2011 at 1:17 AM, Josh More <guppie@starmind.org> wrote: >>> >>>> Don't bother. >>>> >>>> Seriously, the top players are: Symantec, McAfee, Trend Micro, >>>> Kaspersky and Sophos. Read the "independent" reviews and these five are >>>> always at the top. Look at the scores from places like >>>> http://www.virusbtn.com/ and these five are always there. Odds are >>>> that one of them will work for you just fine. (I usually pick Sophos for my >>>> clients.) >>>> >>>> Then look at the extra features. Learn why each one is necessary (note: >>>> they all exist to supplement flaws in the legacy signature-based system). >>>> Figure out which features you need and throw out the vendors that don't >>>> provide them. >>>> >>>> Then look at the UI's. If it will be difficult to use one of the >>>> systems in operations, throw it out. Find out if any of the admins are >>>> biased against a system (Symantec is a popular one for admins to hate.) You >>>> get more problems with malware from admins who resist caring for the system >>>> than you get from systems failing to catch stuff. >>>> >>>> Then look at the licensing. If you can't understand it or if they're >>>> nickel-and-diming you on price, throw them out. It's not worth the pain >>>> otherwise. >>>> >>>> If this process doesn't get you down to a single vendor, look at how >>>> they handle 24/7 support and make test support calls. If their support is >>>> poor, throw them out. If they don't offer 24/7, throw them out (malware >>>> doesn't wait for sun-up). If they force their people to work more than an >>>> eight hour shift, throw them out. >>>> >>>> This process will get you a solution that meets real world needs. If >>>> you try to test from a technical perspective, you're just going to be >>>> selecting the system that best protects against attackers that think just >>>> like you do... which you've already protected against through system >>>> hardening and network design. >>>> >>>> -Josh More >>>> >>>> On Tue, Apr 19, 2011 at 11:28 PM, prashant Kar < >>>> kar.prashant@gmail.com> wrote: >>>> >>>>> Dear All, >>>>> >>>>> Kindly guide me on how to do antivirus application security testing. >>>>> >>>>> Any tools/methodology/approach/checklist that will help, please >>>>> suggest. >>>>> >>>>> Best Regards, >>>>> Prashant >>>>> >>>>> -- >>>>> Technical Skill is the mastery of complexity, >>>>> while Creativity is the master of simplicity..... >>>>> >>>>> The Future Belongs To Those Who Believe in The Beauty of Their Dreams. >>>>> Keep up the spirit!!!! >>>>> >>>>> Prashant Kar >>>>> >>>>> _______________________________________________ >>>>> The Web Security Mailing List >>>>> >>>>> WebSecurity RSS Feed >>>>> http://www.webappsec.org/rss/websecurity.rss >>>>> >>>>> Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA >>>>> >>>>> WASC on Twitter >>>>> http://twitter.com/wascupdates >>>>> >>>>> websecurity@lists.webappsec.org >>>>> >>>>> http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> The Web Security Mailing List >>>> >>>> WebSecurity RSS Feed >>>> http://www.webappsec.org/rss/websecurity.rss >>>> >>>> Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA >>>> >>>> WASC on Twitter >>>> http://twitter.com/wascupdates >>>> >>>> websecurity@lists.webappsec.org >>>> >>>> http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org >>>> >>>> >>> >>> >> > > > -- > Wayne > Co-Founder, President & CTO > Armorize Technologies > http://www.armorize.com > +1-408-216-7893 ext 102 > > _______________________________________________ > The Web Security Mailing List > > WebSecurity RSS Feed > http://www.webappsec.org/rss/websecurity.rss > > Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA > > WASC on Twitter > http://twitter.com/wascupdates > > websecurity@lists.webappsec.org > http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org > >