💾 Archived View for spam.works › mirrors › textfiles › virus › eval.vir captured on 2023-06-16 at 21:02:21.
-=-=-=-=-=-=-
Anti-Viral Product Evaluation
May 5, 1989
This evaluation paper has been written by Jim Goodwin, Lynn
Marsh and Tim Sankary. It is copyrighted, 1989, and is intended
for circulation among fellow members of the virus research
community who use IBM PCs or compatibles. We do not consider it
complete, since we did not evaluate every available product, and
it is not intended as a public guide to selecting antiviral
programs. We hope, however, that it will prove useful to other
members of the community who work with live viruses and need
ongoing protection for their systems. This document may be
freely copied and distributed providing the disclaimer and
copyright are kept intact, and no changes, additions or deletions
are made to the text.
We would like to acknowledge the ample research data
provided by Jim Bates and Rusty Davis in England, Ivan Grebert of
Acal Corporation in Paris, Colin Haynes of the International
Computer Virus Institute, and the many volunteer researchers from
the Silicon Valley area that contributed so much to our efforts.
We would also like to acknowledge the HomeBase users group for
providing their detailed log of infection occurrences and other
epidemiological data.
The Need for a Reasonable Evaluation:
In the April issue of PC Magazine you will find a review of
11 antiviral products. The review, while well intentioned,
tested products against only two viruses (plus one simulated
virus that was developed by the magazine). None of the viruses
were boot sector infectors (viruses which attach to the boot
sector) and none were among the most common viruses. Since the
vast majority of virus infections are boot sector infections, and
since most viruses are much more difficult to detect than the two
chosen, the results of the review were next to meaningless. The
PC Magazine review was similar to many others published in the
past year. It was performed without adequate access to the
viruses actually causing problems in the user community.
A second problem with these reviews, is that many of the
reviewers have had limited experience with the broad range of
infections that have occurred within the past 18 months. They
base evaluations on assumptions that do not hold for the real
world. This is not necessarily the fault of the reviewers.
Viruses are a new phenomenon and few people have dedicated their
time and resources to a long term study. A reviewer who has had
experience with only one or two viruses might naturally draw
incorrect conclusions about "generic" virus issues.
For example, a number of viruses infect programs using
common DOS calls (interrupt 21 or other interrupt call). This
type of infection can be easily detected and prevented. An
entire class of products, called Filters, has grown up around the
assumption that virus infections can be prevented by redirecting
certain interrupts and intercepting the infection replication
process. It works for a few viruses. The vast majority of
infections, though, are caused by viruses that use non-standard
I/O, and these infections cannot be prevented through interrupt
re-vectoring techniques. Thus, filter type products - included
among them are C-4 and Flu-Shot+ - are virtually useless against
most viruses. Yet many reviewers, and some product developers,
still believe that viruses can be stopped through re-directing
system interrupts.
The criteria:
A lot of time and effort has gone into the various checksum,
encryption, logging and chaining algorithms proposed as safe
techniques for detecting viruses. And much discussion and
argumentation has gone one regarding the various merits of high
security algorithms. Yet, every generic application infector
that we have seen to date could have been detected by merely
checking to see if the SIZE of the file had changed. Developing
such a virus detector requires less than an hour of programming
time and is as effective as available products costing hundreds
of dollars. We're not suggesting that size checking should be
the criteria for detecting viruses (we know better), we are
merely pointing out the vast gulf between theory and current
reality. We understand that viruses of today may not reflect the
situation two years from now, and we also understand that current
boot sector viruses and certain operating system viruses pose a
special case to our size example, but the first step in solving
any problem must be a solid understanding of the current state of
the problem. And the current problem is in a different world
from the theoretical solutions proposed for it.
An astute reader might ask at this point why we would be
concerned if the proposed solutions to viruses were overkill.
Isn't it better, you might think, to include as much protection
as is available, to get as close to 100% security as possible?
We think not. Beta testing of virus products in many
corporations and our own experience with these products over the
past year has shown that, beyond a certain point of
reasonableness, increased security functions begin to hinder the
computing process. Either increases in required run time, or
user constraints or annoying additions to the system make the
products so cumbersome to use that the user ultimately discards
them. Alternately, false alarms and questionable product
conditions desensitize the user, and thus real virus alarms, when
they occur, are disregarded.
Again, we are not saying that sound security principles
should not be included in a given product. We are only
suggesting that the search for the 100% solution must have its
limits. The theoretical discussions about batch file viruses,
viruses that can imbed themselves within a program without
changing initial branch addresses, and viruses that can infect
without making any modifications to a program are interesting and
entertaining. But if you are selecting a product based on the
ability to detect such viruses, then you will be disappointed.
In general then, our criteria for evaluating antiviral
programs are:
1. The program's effectiveness against existing viruses.
There are anywhere from two dozen to over 50 different
PC viruses (depending on how you classify them) that
can infect your system today. If the product cannot
detect these viruses, then it certainly cannot detect
tomorrow's viruses. We rated this criteria the
highest.
2. The techniques used by the program to anticipate new
viruses. We have to admit to some subjectivity here.
No-one really knows what virus may pop up tomorrow, but
reasonable people can make reasonable guesses (Tim
Sankary is the only member of this review team who
admits to being unreasonable). We do expect to see
viruses in the next few years that can imbed themselves
inside a generic COM or EXE program without changing
its size. We anticipate system infectors and other
program-specific viruses that can imbed themselves AND
not change initial branch instructions. (We feel these
viruses, however, will be limited to common programs
such as IBMBIO, IBMSYS, COMMAND.COM etc.). We
anticipate viruses that will encrypt themselves in such
a way that every infection will be different (1704
nearly achieves that now). We anticipate boot sector
viruses that will not need to save and execute the
original boot sector. We also expect viruses that will
entirely replace system modules, such as the command
interpreter.
3. The usability of the software. This is the most
subjective criteria and we accordingly weighted it the
least. We decided, however, that if we felt like
screaming, smashing the monitor or savagely beating the
family pets while trying to install or use the program,
then we would subtract points for lack of user
friendliness.
The Viruses:
Jim Goodwin insisted that there were 61 PC viruses and that
we should test them all. He includes in this list three versions
of the Pakistani Brain that differ only in the imbedded text and
volume label copyright display, and four identical versions of
the 1704 that differ only in their activation dates. Lynn
Marsh, who has a new beau, and, we suspected, would like to
spend time with him, suggested that there were only 14 base PC
viruses. Any modifications to these viruses, she insisted, were
inconsequential and should be ignored. A compromise was reached
along the following lines:
Any modification to a base virus that materially
altered its ability to be detected would be considered
a different virus for our testing purpose.
Frankly, the definition didn't help us much because we
continued to squabble, but it eventually worked itself out. It
became clear that certain modifications to base viruses did
indeed materially affect our test results. As an example, one
modification to the Israeli virus, called the New Jerusalem,
performs a format of the hard disk when it activates, and it
additionally does not have the EXE infector bug that the
original Israeli had. When this virus activated, one antiviral
products that was able to detect the original Israeli file-delete
activation and prevent it, was unable to detect the modified
virus's format attempt. There were numerous other such
examples. Even machine or configuration type changes (such as
the numerous 1704 modifications) had an effect on testing under
certain circumstances. We finally narrowed the field down to 27
distinct viruses, 11 of which were boot sector infectors.
We realize that our test base is skewed if you compare it
to infection reporting statistics (where over 80% of infections
are boot sector infections), but we feel the sampling will become
more valid over time, since the boot infector ratio appears to be
slowly declining.
The Testing:
All testing was performed on systems with fixed disks.
Where applicable, the infection was introduced onto the hard
disk. The only exceptions to this were five boot sector viruses
which would not replicate onto a fixed disk. When testing
against these floppy-only viruses, a 5 and 1/4 inch, 360KB
diskette was used. The test systems each contained over 300
executable programs, approximately 2/3 EXE programs and 1/3 COM
programs, arranged in multiple levels of directories. Programs
with overlay structures were also included. DOS 2.0 and 3.3 were
both used, and testing was performed with and without the memory
resident program and shell routine - Carousel and Norton
Commander. Monochrome and VGA graphics adaptors were also
included.
All product detection tests were made while boot sector
viruses were already in memory and in control. This was a
critical point for us. For example, the Pakistani Brain is a
trivial virus to detect if you insert an infected floppy into an
uninfected system and run a detection program against it. If you
boot from an infected diskette, however, the detection process
becomes much more difficult (since the virus traps all attempts
to read the boot sector). We found only one generic product that
was able to detect the Brain while it was active.
When testing against generic COM and EXE infectors, we used
two approaches. First, we loaded the protection software onto a
clean machine and then infected it. Second, we infected a
machine with the virus, then installed the protection software,
and then allowed the virus to continue the infection process.
Throughout the review process, we considered a product to be
ineffective against a given virus if any of the following
occurred:
- The program was unable to detect the presence of
infection activity during its normal check cycle.
- The system hung when the virus was introduced, or
during the check cycle, and no warning indication was
given by the program prior to the hang-up. (This
assumed, of course, that the virus ran normally without
the prevention product being present)
- A loss of data occurred during the checking process.
A product was considered to be effective against a given
virus if all of the following occurred:
- The product identified the presence of infection
activity.
- The product was able to identify each and every
infected component of the system, name each infected
program, and specify the program's directory path.
Usability ratings were loosely handled as follows:
1. Global detection products that required more than two
seconds per program for a system scan (ten minutes on
our test system) scored high on our aggravation scale.
2. Programs that required us to use new system command
structures or required us to modify the way in which we
normally interface with the operating system or our
application programs were placed in the questionable
category.
3. Programs that required constant attention to the user's
manual in order to be useful were frowned on.
(Allowances were made for Tim Sankary's slow thought
processes).
4. Programs that caused false alarms were given an
annoyance ratio proportional to the number of false
alarms.
5. Programs that installed in ten minutes and remained
invisible thereafter were well liked and much
appreciated.
Please don't mistake our lighthearted attitude to the user
friendly category. It's just that we could not come up with a
really objective measure here. No matter how hard we tried, it
usually ended up being a matter of personal opinion. Keep in
mind that we weighted the whole user interface area low in
importance.
The Products:
We were able to identify over twenty PC products being
distributed through vendor channels and through public
domain/shareware channels. We chose five to review that we felt
were the most commonly available and most widely used.
C-4
From McAfee Associates, 4423 Cheeney St, Santa Clara, CA 95054
408 988 3832