💾 Archived View for mirrors.apple2.org.za › archive › apple.cabi.net › Applications › HardPressedGS.… captured on 2024-02-05 at 11:17:59.
⬅️ Previous capture (2023-01-29)
-=-=-=-=-=-=-
Draft HardPressed Documentation
Andy McFadden
Revised: 18-Mar-93 (touched up a bit 1998/06/06)
I. Intro stuff
1. Congratulations on your purchase
- feature list
- minimum requirements (1MB RAM, 2MB+HD recommended - basically the
same stuff as for System 6.0)
2. How to turn the power on, put the disk in, and install it
II. About HardPressed
1. Basic operating principles
HardPressed works by compressing and uncompressing individual files.
Unlike products for other machines (e.g. The Stacker), you don't need
to create special partitions for the compressed data. All of your
files will continue to live in the same places.
Data compression is a CPU-intensive operation. By compressing files
you are trading off disk space for time. In most cases, access to
compressed files will be slower than it would be if the files weren't
compressed. Access may actually be faster in certain cases, such as
when you use a fast compression method on a slow disk device such as
an AppleDisk 3.5" or 5.25" drive.
You should not notice any reduction in speed when accessing
uncompressed files.
1a. compress me a file
With HardPressed, files can be compressed manually or automatically.
Manual compression is done by selecting the files from the Finder and
using the HardPressed Finder(tm) Extension to compress or uncompress
them.
Automatic compression is done by "marking" folders so that files which
are created or changed inside them are automatically compressed. The
marks can be added or removed from the CDev. You can also control
whether or not ANY file is automatically compressed by selecting one
of the following modes:
(1) Compress and Expand. Files will be expanded when accessed, and
files in folders which have been "marked" will be compressed
automatically when they are created or changed.
(2) Expand only. Files will be expanded when accessed, but will not
be re-compressed after they are created or changed.
(3) Inactive. Files will not be compressed or expanded. In addition,
they will appear as HardPressed files in the Finder instead of their
actual type.
The mode can be changed from the CDev or the CDA.
To specify how you want the files to be compressed, you may choose one
of several compression "profiles", which allow you to select greater
compression at a loss of speed, or greater speed with less
compression. These are chosen from the CDev.
If you want to be sure that the file has been compressed correctly,
you can choose a "verify" option which will automatically test the
compressed version before removing the original.
2. How to use HardPressed
HardPressed has four components: an INIT, a CDev (Control Panel), a
CDA (Classic Desk Accessory), and a Finder Extension.
The INIT is the workhorse of the system. It performs the actual
compression and uncompression of files. Because it works invisibly,
you don't need to interact with it; however, if it isn't around, none
of the other components will function.
HardPressed will not compress anything until you tell it to. Simply
installing HardPressed will not cause any compression to occur. There
are two ways to specify files, one manual and one automatic.
2a. The FExt
You can use the Finder(tm) extension to manually select files to
compress and expand. To use it, select the files you want to compress
from the Finder, and choose the "Compress" item from the "Extras"
menu. A dialog similar to the "copying files" dialog [picture?] will
inform you of your progress.
You can select files, folders, or entire disks. The disks can be
ProDOS, HFS, or any other format that GS/OS can write to.
The "Expand" option works similarly, but in reverse. Use this if you
decide that a file, folder, or disk should no longer be compressed.
It is NOT necessary to expand files to use them from GS/OS! (That
would defeat the purpose.) You may find it useful to compress and
expand ProDOS 8 files before using them; this can serve as a
convenient way to archive and extract them.
To see how much space you have saved, select some files and choose the
"Compression Info" item. The dialog will tell you the average
compression of the compressed files, and how much space was saved
overall. In addition, if you only select one file, the method of
compression used will be displayed.
Bear in mind that the savings here may not match the actual reduction
in disk usage. There are two big factors here:
(1) on most disks that GS/OS supports, space is allocated in 512 byte
chunks. This means that, if you only have 511 bytes of data in a
file, no amount of compression is going to reduce the amount of disk
space. On the other hand, if you have a 513 byte file, reducing the
size by one byte will cut the space needed in half.
(2) ProDOS 8 and the GS/OS ProDOS FST (File System Translator) support
"sparse" files. These are files which are often huge but almost
entirely empty. When GS/OS (more accurately, the ProDOS FST) goes to
write a block, it first checks to see if the block is full of zeros.
If so, it just makes a note that the block is empty, and doesn't
allocate one. Thus, a 100K file might only use 10K of space on the
disk, and while compression might reduce this to 5K, you will only
have a 50% reduction in disk usage even though the file itself was
compressed by 95%.
By and large, however, the results will be accurate to within a few
percent.
HardPressed will only compress and expand "locked" files if the
appropriate box in the CDev's "Preferences" screen is checked.
Sometimes HardPressed will think a file is locked even though the
Finder doesn't display it as such (I consider this a bug in the
Finder). If you seem to be banging your mouse against the wall and
getting nowhere, try locking and unlocking the file from the Finder's
"File Info" dialog (click on the file and hit OA-I).
2b. The CDev
The other way to tell HardPressed to compress your files is from the
CDev. The CDev has a number of other features, so it's probably best
to describe it frame-by-frame.
[ picture of initial screen ]
You may choose the mode (Compress and Expand, Expand only, and
Inactive) by clicking on the appropriate button.
IMPORTANT: Because of limitations in the Finder, it is not possible to
redraw the files after switching the system to or from "Inactive"
mode. For the icons to appear differently, the window must be closed
and then reopened. [ can we do this automatically? ] It is
recommended that you do not leave the system in Inactive mode to avoid
confusion.
The Profile pop-up menu displays a list of profiles found in the
"HP.Modules" folder (which in turn is in your */System/System.Setup
folder). Choosing one with the mouse will cause that profile to be
loaded. If you want to return to the Standard profile, you can just
hit Open-Apple-S.
The "Preferences..." button takes you to the preferences screen. You
can configure several different aspects of HardPressed from here.
By clicking on the "Verify" box, you are enabling verify mode. This
causes HardPressed to validate the compressed version of the file
before removing the original. This requires that the compressed
version be expanded, so selecting this option may cause operations to
take twice as long.
The "show progress with mouse cursor" [what'd I call it?] button lets
you decide whether or not the mouse cursor should change to a filling
circle while HP is compressing.
The Disk Caching pop-up menu lets you tell HardPressed how much memory
it should use to cache the files it is working on. This value is a
maximum limit; HardPressed does NOT permanently take away that much
memory from the system. If you set the cache size to "Off", no
caching will be done.
The lower half of the screen has the Finder Extension preferences.
You can choose whether or not locked files will be compressed and
expanded, and can choose which keys to use for the FExt commands.
The "Temp. Folder..." button brings up a small window that lets you
tell HardPressed how and where you want temporary files to be stored.
When compressing or expanding, HardPressed needs to have a copy of
both the expanded and compressed versions of the file. If it can't
fit in the disk cache, HardPressed will create a temporary file.
If you don't check the "Use a single folder" box, the temporary files
will be stored in the same folder as the file being compressed or
expanded.
If you do check the box, all of the temporary files will be stored in
a single folder. This is the recommended option, because it lets you
choose the fastest device you have (such as a RAM disk or fast hard
drive), and allows you to work with write-protected disks. The disk
used to hold the temporary files should have at least enough free
space to hold the largest file you plan to compress or expand.
You can select the folder in one of two ways. The first (and
recommended) method is to press the "Select..." button [key equiv
OA-S... actually, there are key equivs for just about everything].
This brings up a standard file dialog; move to the appropriate folder
and press "Accept".
The second way is to just type the name in. However, very little
error checking is performed, so if you make a typo HardPressed will be
unable to create temporary files. The lack of error checking allows
you to make mistakes, but it also allows you to do unusual things.
For example, the default temporary file folder is "*:", which is
shorthand for "use the volume directory of whatever disk the system
was booted from."
The "Folders..." button takes you to the disk selection screen. You
will be presented with a list of the disks that HardPressed was able
to find. Opening one of them (by double-clicking on the name or
selecting Open) takes you to the folder selection screen. You can
switch to another disk at any time by selecting the disk from the
pop-up menu at the top.
Here you will see a list of the folders present on the disk you chose.
By using Open and Close [key equivs OA-O, OA-down-arrow, Esc,
OA-up-arrow], you can open up or close the folders.
To mark a folder so that the files in it will automatically be
compressed after being created or changed, select the folder from the
list and hit the Mark button [M, OA-M, space toggles]. You can unmark
it with the UnMark button [U, OA-U, space].
An important feature of the folder selection screen is that what you
see is what you get. If a folder does not appear on the screen, then
its mark will be the same as the mark for its parent.
So if you have the following:
:MyDisk
Applications
X Data
Drawing
X WordProcessor
Games
Then only files that are in :MyDisk:Data and
:MyDisk:Data:WordProcessor will be compressed automatically. Files in
:MyDisk:Data:Drawing will NOT be, because the "Drawing" folder doesn't
have a mark next to it.
If on the other hand you had this (for the same disk, but the "Data"
folder is now closed):
:MyDisk
Applications
X Data
Games
Then everything under "Data" will be compressed, including those in
"Drawing" and "WordProcessor".
To reduce clutter, when you hit the "Done" button or switch to another
disk, the HardPressed CDev will automatically close any folder whose
children all have the same mark status. So if you had:
:MyDisk
Applications
X Data
X Drawing
X WordProcessor
Games
Then the "Data" folder would be closed, leaving:
:MyDisk
Applications
X Data
Games
Since the mark on "Data" affects all of the subdirectories, it isn't
necessary to explicitly mark "Drawing" and "WordProcessor".
After you mark some folders, the result will be saved in a file called
"HP.Vol.Prefs" on the disk in question. The file will be invisible
like the Finder's data files, so you will only see it if you set the
Finder preference which allows you to see the Finder's data files.
Pressing the "Revert" button will throw out any changes you have made,
and re-read the information from the disk. Your changes are saved
whenever you switch to a different disk with the pop-up menu or press
the "Done" button.
It is possible to select more than one folder at a time for any of
these operations [standard list selection stuff, OA-A for select all].
To expedite the process of opening all the folders inside a selected
folder, newly-opened folders will appear selected in the list. They
will also have the mark status of the folder they are in.
When you close the CDev, your preferences will be saved in a file
called "HP.Prefs" on the boot disk in the "System.Setup" folder. To
avoid the possibility of not being able to boot because some system
files were compressed, the CDev will NOT save "Inactive" status to the
preferences file. If you close the CDev while HardPressed is
Inactive, you will be in "Expand only" mode when you reboot.
If you want to boot the system without HardPressed, press 'h' right
after you see the GS/OS boot thermometer. The CDev's icon should
appear with a red 'X' over it.
2c. CDA
The CDA (Classic Desk Accessory) provides a quick interface to the
boxed items on the CDev screen. You can change the mode to Compress
and Expand, Expand only, or Inactive, and you can toggle Verify on and
off. Changes made here do NOT affect the HP preferences file, so this
is a good way to change settings temporarily.
The CDA controls will not be accessible from ProDOS 8.
2d. Notes & random stuff
It is important to remember that files will be recompressed ONLY if
the folder they are in is marked. This means that if you change a
compressed file in a folder that is marked as "Expand only", the file
will be stored without compression. This has some advantages, which
are explained later.
IMPORTANT: the Finder extension will ignore the on/off switch and will
override your folder settings. This is based on the assumption that
you have deliberately chosen to compress or expand the files you have
selected. This allows you to compress files without having to mark
the folder they're in, which is useful for files which don't change.
For example, you might have some sample source code or old database
files that you want to reference, but you don't want the stuff you're
working on to be compressed. You can leave the folder unmarked and
compress the sample files from the Finder; that way you get the
benefits of compression on the old files but nothing you edit or
create will be compressed. This is especially useful for compressing
stuff with schemes which compress slowly but extract very quickly,
such as LZSS.
Other examples include TrueType fonts, desk accessories, and
applications.
TIP: if you're working with some files in a marked folder, but you
decide at the last minute that don't want them to be compressed, you
don't need to go into the CDev to turn off the mark status. Instead,
use the CDA to change the mode to "Expand only" until you are done.
That way, none of the files will be compressed automatically,
regardless of the folder mark status.
HardPressed will always expand files that it has compressed, whether
or not the folder they are in is marked. In addition, the current
profile setting does not affect expansion. However, if you change the
mode to Inactive, it won't expand anything. This can be useful when
you want to copy a large collection of compressed files from one place
to another.
If you tried it the usual way, it would expand the files, and then
either store them without compression or re-compress them on the
destination disk. By first putting HardPressed into Inactive mode,
you can copy the files as compressed data, which is considerably
faster. (This is NOT recommended, of course, if you are copying the
files onto a disk to give to somebody who - for whatever reason -
doesn't own a copy of HardPressed.)
The HardPressed compression code is stored in individual "modules",
which are kept in the "HP.Modules" folder under "System.Setup". These
can be inactivated from the Finder's "File Info" menu item if you
don't want them to be loaded. This can be useful for infrequently
used modules. For example, the adaptive Huffman and differential
compression modules are inactive by default).
If you want to activate a module, it is not necessary to reboot. The
CDev will automatically check for new modules and newly activated
modules when it is opened. So, to activate a previously inactive
module, just remove the check from the "Info" box in the Finder and
open the CDev. This only applies when making a module active, not
when making it inactive... once a module has been loaded, it's there
to stay.
2e. incompatibilities
[ 1998/06/06: see the README file on the disk for a more accurate list ]
Because of the way HardPressed works, there are some programs with
which it will not get along. Here they are, from worst to least.
- ProSel-16. As of v8.81, ProSel did a lot of low-level disk I/O,
bypassing HardPressed. This is only a factor in some areas, notably
the utilities. It would be wise to change the mode to Inactive when
using ProSel's utilities, backup/restore, volume repair, and volume
optimization menus. The author has used ProSel-16 as a program
launcher throughout the development of HardPressed, and has had no
problems with that part. However, it is *VERY* important that you do
not try to copy compressed files with ProSel's utilities while
HardPressed is active.
- SoundStudio (freeware digitized sound utility). When loading
sounds, this program uses the file's block size to compute the sound's
length. Since HardPressed reduces the number of blocks that the file
requires on disk, SoundStudio thinks the files are smaller than they
really are, and only loads the first part of the file.
- GNOBug (debug INIT for the GNO environment). "gnobug" is used to
hook GNO in above GSBug, to make _osbrk work right. However, hooking
GNO above HardPressed can cause problems. The correct order is
HardPressed first, gnobug second, and GSBug third. [this may not be
the case anymore... Jawaid?]
- Old ORCA shells. Nothing serious here. The "CAT" command will show
the files as HardPressed files (LBR), but all commands and utilities
will function without any problems.
2f. Questions & Answers
Q. Sometimes when I get Icon Info from the Finder (OA-I), it expands
the file. Why?
A. Some files have version information in the resource fork. For
example, getting Icon Info on the HardPressed CDev will display the
full name of the program and a copyright notice. To get at this
information, the resource fork must be expanded first. The data fork
is not examined.
Q. I copied the "All: Adapative Huffman" profile out of the Goodies
folder, but I can't seem to compress anything with it.
A. The profiles tell HP which modules to use. If the needed modules
aren't loaded, HP can't use them. By default, the Adaptive Huffman
and Differential modules are marked as "inactive" in the Finder. You
can enable them with the Finder's Icon Info feature (click on the
module in the System/System.Setup/HP.Modules folder, hit OA-I, and
click on the "inactive" box so that the box is empty). Once you have
activated the module, it will be available every time you boot GS/OS.
HardPressed will load it when you reboot or reopen the CDev.
Q. When compressing a file with the Finder extension, why does the
little document icon empty and then fill up?
A. You have verify mode enabled. It empties during compression, and
fills during the verify. When expanding, it just fills.
III. How HardPressed Works
[ 1998/06/06: some is explained better in files in the "Goodies" folder ]
This chapter goes into greater technical detail than the previous.
You don't need to read this to use HardPressed, but you will pick up
some interesting tidbits even if you just skim it.
One item which should be stressed is that HardPressed is as safe as
GS/OS in theory, but may be less so in practice. This means that, if
GS/OS says "you are safe", you are still safe. If it says, "you risk
loss of data", then your risk is slightly higher.
For example, suppose you have a power failure. GS/OS caches parts of
files, so if you lose power while a file is open, you may lose data.
HardPressed has a larger file cache, so the chance that your data will
be in memory instead of on disk when the power goes down is higher.
Thus, HardPressed does not create a new opportunity for data loss, but
instead merely increases the odds of being caught in a bad situtation.
(Of course, you can avoid this particular problem by setting the cache
size to "Off".)
2a. file interface
HardPressed identifies its data files by file and auxtype. (Actually,
there are three auxtypes, indicating that the data fork, resource
fork, or both are compressed.) This is one of several possible
methods, and is the best for the //gs.
If you have already installed and run HardPressed, you will have
noticed that the files appeared to retain their original type. This
is because HardPressed intercepts the GS/OS calls which get the files'
types, and replaces the types with the original ones.
The original filetype, auxtype, and EOF (end of file - the length of
the file in bytes) are stored in the appropriate fork of the file
itself. This means that the file must be opened and read whenever a
program (such as the Finder) tries to access any of the above.
At this point you might be thinking, "yuck, there must be a better
way." Don't we wish. The main roadblock is the file EOF, which must
reflect the ORIGINAL size, NOT the compressed size. Suppose we have a
10K file, which compressed down to 5K. Your Favorite Word Processor
wants to edit the file, so it gets the file's length. If it sees
"5K", it will open the file, read 5K, and stop. However, HardPressed
will have uncompressed the file to its full 10K size, so you would
only get half of the file. Thus, we need to tell FWP that the file's
size is really 10K, or we may appear to lose data.
This can still be a problem with programs which read the disk directly
instead of making GS/OS file calls (ProSel-16 as of v8.81 did this).
Because they read it directly, HardPressed doesn't get a chance to
substitute the true file size, and the situation seen above happens.
It is important to remember where the filetype is being stored when
changing the access permissions on a file. If a file is
read-disabled, HardPressed will have great difficulty getting the
filetype. If it is write-disabled (i.e. "locked"), changing the type
will be difficult. I say "difficult" and not "impossible" because if
the file is locked, HardPressed will change the file's access bits,
read or write the necessary information, and then reset them
afterward. In general it is a very bad idea to disable read access,
because accessing the file will be impossible.
(An extreme example: if you open a locked file with read-only access,
change the filetype, and then close the file, HardPressed will close
the file, change the access bits, reopen the file with read/write
access, change the filetype, close the file, relock the file, and
reset the file's modification date to its previous value. How's THAT
for service?)
As I just hinted at, it IS possible to reliably change the filetype on
an open file. Normally you can't write anything to a file which is
already open. In this case, however, the file will have been opened
by HardPressed, which can just change its internal data structures and
save the information when it goes to close the file. This will not
work if you deliberately try to defeat it (by turning HardPressed off
from the CDev, opening the file, and then turning it back on again),
and it may not always work on AppleShare volumes, where a file could
have been opened by someone elsewhere on the network.
HardPressed does everything it can to avoid these problems, but there
are some situations which are simply impossible to resolve (for
example, the file/auxtype/EOF information could be corrupted). In
such cases, queries on the file information will return HardPressed's
filetype ($e0 - LBR) and the actual EOF, and attempts to access the
data inside file will fail with error $4A (invalid version).
2b. hard at work
Small bit of background for those unfamiliar with GS/OS... The "Open"
call prepares a file for reading or writing. You give it a pathname,
and it returns a "reference number" or "refNum". All further accesses
are done with the refNum instead of the pathname. The "Read" call
reads data from a file, and the "Write" call writes data. The "Close"
call tells the system that you are finished with the file.
NOTE: These are _not_ the same things as the Open and Close items
found in the "File" menu of desktop applications. Choosing "Open..."
from the menu bar may cause application programs to invoke several
GS/OS Open and Close calls. (Remember, you don't NEED to know all
this stuff; we just put it here in case you got curious.)
When an "Open" call is issued, HardPressed opens the file and checks
the filetype and auxtype to see if the data is compressed. If so, the
first few bytes of the file are read to find out how big it really is
and how it was compressed. If it doesn't know how to deal with the
file (perhaps a needed compression module was marked inactive), it
closes it and returns an "invalid version" error.
When the first Read or Write call is issued, the entire file is
expanded (more accurately, the entire FORK is expanded). Depending on
a number of factors, the uncompressed data will be stored either in
the HardPressed file cache or in a temporary file on disk. Future
Reads and Writes are made to the file in memory or the file on disk.
Note that, even if the temporary file was too big to store in the
cache, HardPressed will try to cache any Write calls which fit.
When the last Close call is issued (you can issue more than one Open
on a file, so you can have an equal number of Closes), HardPressed
checks to see if the file's contents or filetype have changed. If the
contents have changed, the file is recompressed from the data in the
cache or in the temp file. If only the type has changed, then just
the header will be rewritten.
If the file is smaller than 512 bytes uncompressed, it will always be
stored without compression. This is because the typical disk block
size is 512 bytes, and the file will occupy one full disk block
whether it's 5 bytes or 511. The Finder extension silently ignores
tiny files.
When an Open call is issued on an UNcompressed file, HardPressed
checks the mark status of the folder it's in and the file's access
bits. If it's writeable and in a "compress when changed" folder,
HardPressed will figure out how to compress it (explained next) and
add it to the list of files it's handling. From there it works like a
file which was already compressed: if it gets changed, it gets
compressed; if not, nothing is written.
2c. HowPressed [this might belong in the basic chapter, actually]
As mentioned earlier, HardPressed can compress your files in several
different ways. These are summarized in the table below.
[put the "cdref" file in a table, I guess]
A significant feature of all the compression modules is that they can
be chained together. For example, you could compute a CRC, pack the
file with RLE, and then pack it again with LZSS. Future compressors -
and other modules, including stuff like data encryption or error
correcting codes - can simply be chained on. The output is "piped"
directly between them, so you don't have to wait for disk access
between each stage.
Besides being able to add fancy stuff like CRCs or data encryption to
the compression, this enables you to compress files in ways which
would have been difficult otherwise. For example, digitized sound
files don't compress very well with standard algorithms. However, if
you perform a "differential" encoding pass followed by adaptive
Huffman, you can get fairly good results.
Another example is "sparse" files. Sparse files are large files
(perhaps several megabytes) which are almost entirely filled with
zeros. Typical compression algorithms will have spend a lot of time
compressing all the zeros before they can compress the data. However,
the RLE (Run Length Encoding) algorithm was designed to deal with
large stretches of zero bytes, so by chaining RLE and LZW together,
you can compress the zeros quickly but still get the benefits of LZW
compression.
The way that compression modules are chosen is through the compression
profiles you select from the CDev. Each profile contains a list of
"categories", which group files by filetype and auxtype. There's a
category for applications, one for graphics, one for word processing
files, and so on. Each category has up to eight modules associated
with it.
When a file is Opened, HardPressed determines which category the file
belongs in, and then looks up the list of modules for that category.
If there aren't any, the file will not be compressed. This applies to
files which are already compressed as well: if you recompress a file,
it will use whatever the system says to do now, NOT what was used when
the file was compressed.
There's a couple of good reasons for doing it that way. First, it
allows you to do some neat things, which will be explained next.
Second, it isn't always possible to determine what kind of compression
was used previously. For example, some applications will delete the
old data file before creating a new one. When the old file gets
deleted, all of the information about how it was compressed goes with
it. Other applications save the data to a temporary file, and then
when the save is completed they replace the original file with the
temporary one, so again there's no way to retrieve the previous
information.
There are several profiles provided for you. "Standard" gives you
good compression at moderate speed. "Faster" gives you less
compression, but uses faster modules and compresses fewer kinds of
files. "Smaller" gives you better compression at the cost of speed.
There are also some modules which compress absolutely everything
(including ProDOS 8 stuff!!) with one particular module. These are
useful if you want to compress a lot of data in one particular way.
If you examine the list of compression modules carefully, you'll
notice that LZSS is the best at packing files (marginally edging out
LZW), and unpacks quickly (blowing away LZW). However, it has truly
abysmal performance when compressing. This leads to an interesting
disk compression strategy.
Basically, you choose the "only LZSS" profile from the CDev, select
several folders worth of stuff from the Finder, select "Compress" from
the Extras menu, and go out to lunch (or to sleep if you have a large
hard drive). When you come back, all of your files will be tightly
compressed, and will extract quickly.
You should then switch back to one of the standard profiles.
Thereafter, when you change a file or create a new one, it will be
compressed with the method in the profile, NOT LZSS. This allows you
to get the benefits of tight compression with no reduction in speed in
daily operations.
2c.1. help, I've expanded and I CAN'T GET UP
If a file fails to compress (i.e. the file is larger after compression
than it was before), HardPressed will store the file without
compression. This can happen for certain kinds of files, such as
digitized sound or files which are already compressed.
One interesting feature is that, if you have included some modules
which are NOT compressors (such as data encryption or error
correction), they will not be included in the comparison, and they
will not be removed should the file expand.
Suppose you have some super duper encryption scheme which expands the
file by 50%. You compress it by 10% with RLE, and then encrypt it.
The resulting file is larger than the original, but because the
COMPRESSION modules made it smaller, it will be kept.
On the other hand, suppose RLE fails and increases the size by 10%.
In that case, HardPressed will re-encode it with ONLY the encryption
module.
At present, all of the modules are defined as compressors, including
the CRC module (this was done so that you wouldn't end up with a bunch
of stray files with just CRCs on them).
Assigning compression methods into categories is done by the
stand-alone application [which hasn't been written yet. Oh bother.]
2d. the big off
If you've been reading along, you now have enough background to
understand what the "off" switch in the CDev and CDA REALLY does.
There are two places in which the switch is examined. First, when
handling file information calls. When HardPressed is "off", all such
calls are passed directly to the system. This means you will get the
original filetype, auxtype, and EOF marker, and changes to the file's
type will affect its "true" type. If you accidentally mangle a file's
type, you can have it reset automatically from the stand-alone
application.
The second activity it affects is when deciding whether or not to
compress a file. If the switch is off, all opens are passed directly
to the system.
The switch does NOT affect any other calls, so Read, Write, and Close
will continue to behave as before. This means that files that were
open BEFORE you threw the switch will continue to be handled
correctly. However, if the switch is off when the Close call is
issued, they will be stored without compression.
The "Expand only" setting has a similar effect, except that it doesn't
affect the file information calls. It just changes the policy
decisions in Open and Close so that uncompressed files stay that way
and compressed files will not be recompressed.
2e. welcome to moduland
The compression modules are actually "filters"; any algorithm which
can operate on a continuous stream of data can be built into a module.
This means that data compression is only one possible use for them.
Data encryption, error detection (like the CRC module included), error
correction, and addition of extra file information (like a
modification time in seconds or a comment field) are all possible.
A "HardPressed module author's kit" will be made available on the
major online services. It will explain in detail how the modules
work, and will include some sample source code. In addition, lesson
12 of "Hacking Data Compression", an online course in the A2Pro
section of GEnie, will be all about writing your own modules.
One recurring daydream is to write a module which will encrypt a file
in a way that only works on your system. The modules could use the
message facilities in System 6 to communicate with a "key server"
running on your system as an invisible INIT. So long as the INIT is
there, your files are accessible. If not, they aren't. This would
allow you to have encrypted files accessible without having to type a
password all the time (useful in a networked environment where you can
guarantee the physical safety of your local machine), or at least have
the password program be indepdendent of the module itself. It could
also be used as a form of copy protection (boo, hiss!)
Well, back to Earth. The compression modules are stored along with
the profiles in a folder called "HP.Modules" under