Traveller
=========

The traveller programs help users find their way around gopher servers,
by enabling them to search for an title (much like veronica) and to
know what is new in a gopher graph (much like whatsnewd).  The information
is updated automatically, and that does pose some limitations to the tool:
if some part of the gopherspace is updated very often, it tends to shadow
the less frequent updates to other parts of the gopherspace.

The system can be explained better with the help of a metaphor.  Imagine
yourself in the old medieval days, when there were no newspapers, no
TV news, nor, in fact, any organized way to difuse information.  In those
times, the travellers carried the news.  When going from place to place,
they would hear and repeat stories at each stop.

That is very much what the traveller programs do.  A set of specialised
programs can travel through gopherspace, including ftp servers and local
filesystems, ommiting uninteresting items.  A different program
accumulates the results, and creates a memory of what items exist, and
when were they first seen.  Finally, a go4gw daemon answers questions.
The query language allows queries on the name, type, path, host, and
port, and also on depth and age of the item.

You can take a look at a running server in:

Type=1
Name=Traveller memories
Host=gopher.fct.unl.pt
Port=4320
Path=travel

DISTRIBUTION:

This is a first attempt to a documentation for the traveller programs.
Most files have vestigial manpages at their beginning. Refer to them
or, as a last resort, to the code, to solve the details.

The traveller programs are distributed in a tar file with the following
structure:

README     -- this file
LEGAL      -- just to lighten your heart
TODO       -- some indication of things that could be done
data       -- the data directory, plus miscelaneous scripts and kill files
daemon     -- the server programs
lib        -- libraries for the use of the server and the travellers
travellers -- the traveller programs

INSTALLATION:

Create a data directory somewhere, and give it the right permissions.
Copy the travellers to some bin directory;
Copy the g2travel daemon to where you have the other go4gw gateways,
and install it in go4gw.conf
Copy the libraries to some common place, accessible for both the daemon
and the travellers.

Edit g2travel: change $travel_dir to point to your travel data directory;
Edit the travel.conf file in the data directory.
... Now the server should be up and running, but it has no data to show!

ORGANIZATION OF THE DATA DIRECTORY:

travel.conf -- describes the existing travellers (as the user will see them).
               Assigns a file prefix to each one.

about -- global help file

For each "user" traveller, with prefix "pref", the daemon will use the
following files, if they exist:

pref.about  -- help file for the "pref" traveller (optional)
pref.gtimes -- the travel results, created by gtimes and the travellers
pref.pretty -- the road map created by gpretty (optional)

By convention, the following files may also exist, but are ignored by the
server:

pref.bak    -- backup of the last pref.gtimes, should something go wrong
pref.travel -- the last travel of some traveller
pref.kill   -- kill file for the "pref" traveller
pref.sh     -- shell script to run the "pref" traveller and update its
               corresponding pref.gtimes file

RUNNING THE TRAVELLERS:

Take a look at some ".sh" files to see how it runs.  The method depends
from how the data is to be collected.

The easiest way is to run gtravel.  It will walk through a gopher server,
and see exactly what the server sees.  If you don't use a good kill file,
however, you may try to explore the whole gopherspace, so please be
careful, and watch it run the first time.

ftravel is useful if you are serving a large file system; gtravel may
be too slow.

lstravel is useful if you connect to ftp servers; it will take an "ls -lR"
and turn it into a travel file.

All these programs create travel files.  After that, you run gtimes to
produce the final gtimes file.

THE KILL FILES:

These where somewhat inherited from whatsnewd.  Each line can start with
either "kill" or "keep" or "kill-subs-of" or "kill-subs-of". An item
is kept if it matches at least one "keep" line and does not match any
"kill" line; a menu link is explored (recursed into) if it matches at
least one "keep-subs-of" line and does not match any "kill-subs-of" line.

The expressions afterwords can make reference to the variables:
type, name, path, host, port, depth ... of the item
type0, name0, path0, host0, port0, depth0 .. of the item's father (but note
                                             that these ane not always
                                             available).
Operands include
= eq   -- equal
!= ne  -- not equal
~ =~   -- matches regular expression
!~     -- does not match regular expression

The values can be a single word (no white space), or something inside "".

Here are some useful examples:

keep name = my.host.address    --- only keep things at my host
keep name0 = my.host.address   --- my server plus things one hop away
kill path ~ ^./tmp/            --- kill everything under the top directory tmp

CHANGES:

v1.2 - 20-Aug-94:  Bug fixes, no new functionality.
v1.1 - 02-Dec-93:  Compressed *.gtimes files, added missing file.
v1.0 - 21-Sep-93:  First release.