More than FPs

Testing anti-malware software involves more than investigating how well the products handle threats. They need to handle legitimate software too.

Last week in Montreal, at the October AMTSO meeting, I presented Dennis Technology Labs' approach to running 'FP' tests (as they are generally known in the industry).

A false positive (FP) is when an anti-malware product incorrectly classifies a legitimate program as being malware. For example, if a product claims that a genuine notepad.exe file is a Trojan called W32.Netopad.AYX then that is an FP.

Our view is that FPs are only part of what needs to be tested. Any time that an anti-malware product fails to allow legitimate software to run without annoying or otherwise distracting the user needs to be taken into account.

Similarly, if the legitimate product is somehow limited in what it can do on the system then that is a factor for consideration.

When an anti-malware program does not simply allow legitimate software to run it is not performing optimally. Also, if it classifies legitimate software as anything other than 'safe' or similar then there is a problem.

In such cases we have a Non-Optimal Classification/Action (NOCA). FPs are a subset of NOCAs. FPs are, in fact, just about the worst type of NOCA. Other options are less serious and some are due to bugs in the anti-malware products rather than anything else.

The slides of the presentation are now available online in PDF format.