💾 Archived View for gemini.spam.works › mirrors › textfiles › computers › DOCUMENTATION › jmodem.txt captured on 2022-06-12 at 06:34:42.
-=-=-=-=-=-=-
J M O D E M
File Transfer System
Compliments of
Richard B. Johnson
PROGRAM EXCHANGE
(303) 440-0786
December 31, 1988
General:
First Let me get this over with.
MS-DOS is a registered trademark of Microsoft Corporation
IBM, IBM-PC, IBM-XT, AT, are registered trademarks of
International Business Machines, Inc. WILDCAT! is a
trademark of Mustang Software. XMODEM is a public-domain
file-transfer protocol developed by Ward Christensen.
JMODEM is released into the public domain. As with most
public-domain protocols, you are advised that there is no
implied warranty of any kind. The source-code is provided so
that you may determine for yourself if this program may
serve a useful purpose. It is written in Microsoft MASM
Assembly-language using good standards of engineering
practice. It does not use any strange or "undocumented"
functions of the MS-DOS operating system.
PLEASE UPLOAD THIS FILE (The ARC file) to as many
BBS systems as you can so that it gets installed
all around the country.
Introduction:
JMODEM is a new file-transfer protocol developed to be used
as an "external protocol" on BBS systems and personal
computers using the IBM-PC/AT/XT structure and the MS-DOS
operating system.
This file-transfer system features:
o 16-bit CRC for verification
o File size is exactly maintained
o Data compression.
o Rapid host/remote synchronization.
o Variable-length transmission blocks which,
if there are few errors, increase to 8192
data-bytes in length.
o Flow control (automatic)
o Hangup protection
o Aborted files are saved
o Files being overwritten are renamed to
".OLD", rather than deleted.
o COM1 - COM4 support.
o Interrupt on received characters allows data
to be received while the previous block is
being written to disk. This provides almost
continuous data transmission without long
waits between blocks.
- 1 -
JMODEM Protocol Page 2
JMODEM is not for everyone! It does not have any pretty
screens to dazzle the user. It is designed to maximize the
amount of data that can be transferred in a given time (and
reduce telephone cost). It does this by sending very long
blocks of data and encoding (compressing) the data wherever
possible. Since long blocks of data are subject to many
transmission errors, a 16 bit CRC is used to determine the
data integrity. As many as ten retries are made if the data
is not correctly received. If you have a noisy telephone
circuit, you will find that JMODEM will abort more often
than the XMODEM protocol which sends very short blocks. A
future version that will be downward-compatible with the
existing version is being developed that will do "heroic"
retries and will even go down to a 16-byte block-lengths if
that's what is necessary to get the data transferred.
Once synchronization between the remote computer and the
host are established, JMODEM paints a status block on the
screen that shows how the file transfer is going. The status
screen shows the block being transmitted (or the last re-
ceived), the length of the block, and the total bytes having
been transmitted (or received). A special synchronization
routine is used so that the first block is not thrown away
as happens so often in XMODEM type routines. During the
synchronization routine, where the host is waiting for the
user to enter the proper file parameters (a 30 second wait).
You can abort immediately by sending a control X (^X). After
actual file transfer begins, ie. when you see the status
window on the screen, no input from the keyboard is pos-
sible. You send a control BREAK to abort the transmission
(or ^C). In this case, the program will abort after the
block being sent/received is complete. This could take 15 or
more seconds with long blocks so be patient.
In the event that carrier is lost (the user disconnected),
the file-transfer program will also abort. This, too, could
take as long as 15 seconds. JMODEM has logic to determine if
the modem carrier was present when it started. This will
allow you to transfer files between computers with a wire
without having to tie the RLSD lead high.
How it works:
The block size starts out at 512 bytes (or the actual bytes
in the file -- whichever is less). To this is added a 6-byte
overhead. If the block transfer occurred without any
retries, the block length is increased by 512 bytes to 1024
bytes. As long as the transmission was successful without
incurring any re-tries, the block-length increases to a
maximum of 8192 bytes. There is still the same 6-byte
overhead so the maximum block length will actually be 8198
bytes. Any time there are retries, the block length (on the
next new string) is decreased by 512 bytes. The string-
length is never reduced to less than 512 bytes due to
- 2 -
JMODEM Protocol Page 3
errors. When the last bytes are read from the file, the
block length may be as little as 7 bytes (one data byte,
plus the 6 byte overhead). The file size as received will be
exactly the file size as transmitted. XMODEM will "round-up"
the file size to the next higher block. This means that MS-
DOS's COMP (compare) will always fail when you attempt to
check files that have been sent by XMODEM and many other
protocols.
When the file is read, an attempt is made to compress the
data using the well-known RLL process where multiple bytes
are compressed into a 4-byte statement.
For instance a string that looks like this (hex):
A0 A0 A0 A0 A0 A0 A0 A0 A0 A0 A0 A0 A0 A0 A0 A0
Gets compressed into this:
BB 0F 00 A0
| | | |__________ Byte to repeat
| | |_____________ High byte of repeat length
| |________________ Low byte of repeat length
|___________________ Sentinel Byte
The sentinel-byte (BBH), when encountered in a data stream
will get expanded to four bytes. Therefore, it is possible
for the "compressed" data string to actually be longer than
the original. If this occurs, the original string is sent
rather than the longer encoded one. Since the kind of data
sent can be different from block-to-block, it is necessary
to send a control-byte along with the data so the receiver
had determine how to operate on the data.
This is the control structure:
00 02 00 0B 01 02 03 04 05 06 07 08 09 0A .... 0A EA
| | | | | | |_ CRC
| | | | | |____ CRC
| | | | |______________________________________ data
| | | |_________________________________________ rec.
| | |____________________________________________ control
| |_______________________________________________ length
|__________________________________________________ Length
Two bytes are used for the string length and two bytes are
used for the CRC. As is standard in memory, the high-byte
looks "to people using DEBUG" swapped with the low byte. The
data is transmitted exactly as the memory image.
The length (a word) begins the string so the receiver may
know exactly how may bytes to receive.
The control byte is bit-mapped to 8 possibilities. The ones
most important are:
NORMAL DATA
- 3 -
JMODEM Protocol Page 4
COMPRESSED DATA
END OF FILE
ABORT
This is now the receiver "knows" what to do with the data.
The CRC is generated using this polynomial:
X = X + X^(2(n-mod 7))....... Where n = t(n-1)
And t = string length
It has the advantage of simplicity in assembly-language
programming and will detect errors with a probability of
about one undetected error in 2^132 (which is a very large
number). It does not correct errors so its not important to
use some "standard" function to generate the CRC.
Support for COM3 and COM4 have been added on revision level
V1.05 . Note that the standards for port locations are de-
facto standards only and may not be the ports actually used
in your computer. Please modify the communications port
structures at the beginning of the assembly-language program
to match your system parameters if they are different. The
modifications should be done to the STRUCTURES, not to the
EQUATES!
Revision level V1.09 brings musical exit status. If the file
transfer was successful, the computer plays "victory". If
the file transfer was aborted, the computer plays "retreat".
Since the BBS SysOp is unlikely to want his computer to play
bugle-calls well into the evening, the user has the option
of turning the music off. This is done by setting an
environment variable string:
SET JMODEM=SHUTUP
... Will accomplish the desired result.
Usage:
This program uses parameters on the command line.
JMODEM S <filename.typ> ( Sends a file to COM1:)
JMODEM R <filename.typ> ( Receives a file from COM1:)
JMODEM S1 <filename.typ> ( Sends a file to COM1:)
JMODEM R1 <filename.typ> ( Receives a file from COM1:)
JMODEM S2 <filename.typ> ( Sends a file to COM2:)
JMODEM R2 <filename.typ> ( Receives a file from COM2:)
After version 1.05, JMODEM supported communications adapter
ports 1 through 4.
In a batch file, <filename.typ> may be a substitution
character.
JMODEM S2 %1 ( Sends a file to COM2:)
JMODEM R2 %1 ( Receives a file from COM2:)
- 4 -
JMODEM Protocol Page 5
- 5 -
JMODEM Protocol Page 6
Setting up a Communications Program
External File-Transfer Protocol.
On my system, TELIX resides in the C:\TELIX directory. A
copy of JMODEM.COM is also in that directory. TELIX passes
the filename as the %3 parameter. Therefore the contents of
JUP.BAT is:
C:\TELIX\JMODEM S1 %3
The contents of JDOWN.BAT are:
C:\TELIX\JMODEM R1 %3
If I wished to receive in the "batch" mode, I could make a
file like this. Notice that some communications programs do
not allow multiple file names. Note that the comments "!"
are NOT ALLOWED in a DOS batch file.
:DO_LOOP ! Return here
IF "%3" == "" GOTO DONE ! More parameters?
C:\TELIX\JMODEM R1 %3 ! Execute JMODEM
IF ERRORLEVEL 1 GOTO DONE ! Abort on error
SHIFT ! %4 becomes %3
GOTO DO_LOOP ! Continue
:DONE ! Exit batch file
If you do not know what "%" parameters are used to pass the
file name, all you have to do is make a "dummy" batch file
that contains the following:
@ECHO OFF
ECHO %1
ECHO %2
ECHO %3
ECHO %4
ECHO %5
PAUSE
When this is executed, you will see something like this:
1200
1
FILENAME.TYP
ECHO is off
ECHO is off
Strike a key when ready . . .
The first line contains "1200" which is the baud rate. This
means that the %1 parameter contains the baud rate.
The second line contains "1" which is the communications
adapter port being used. This means that the port is being
passed as the %2 parameter.
- 6 -
JMODEM Protocol Page 7
The third line contains "FILENAME.TYP" which is the file
name. This means that the file name is being passed as the
%3 parameter.
The fourth and fifth lines contain nothing to echo so DOS
replies the current state of the echo function which is
"off".
PCPLUS handles the file name passing a little bit different.
If I execute the same "dummy" batch file from the PCPLUS
directory, the response is:
FILENAME.TYP
ECHO is off
ECHO is off
ECHO is off
ECHO is off
Strike a key when ready . . .
This shows us that PCPLUS passes the file name as the first
parameter and there are no other parameters. However, If I
put more parameters on the command line within PCPLUS, they
will get sent to the batch file. The response is:
FILENAME.001
FILENAME.002
FILENAME.003
FILENAME.
ECHO is off
Strike a key when ready . . .
Therefore PCPLUS allows up to four file names to be passed
providing there's room on the command line.
Notice that these two communications programs check the
default directory for the external protocol batch file
FIRST! Therefore you must make certain that there are no
other similarly-named batch files in the current directory
or within the current path. Failure to do so will cause the
improper execution of the wrong batch file. Lets say that
the path was "C:\DOS;C:\TOOLS;C:\PCPLUS;C:\TELIX;C:\QMODEM".
If you named all your JMODEM external protocol batch files
with the same name, and you were attempting to use an
external file transfer protocol from QMODEM, the batch file
designed to operate with PCPLUS would be the first one
"found" and executed since the search-path will search the
\PCPLUS directory before the \QMODEM directory. You prevent
the execution of the incorrect batch file by calling them
slightly different names.
When setting up external protocols, remember to configure
the communications program so that it prompts you for the
file names. Unlike some protocols, JMODEM does not transfer
- 7 -
JMODEM Protocol Page 8
the file name. You can use any file name that you wish. You
must pass the file name to JMODEM since it must know the
name of the file being transmitted or received. There are no
defaults.
- 8 -
JMODEM Protocol Page 9
Setting up a BBS System
External File-Transfer Protocol.
If you are running a WILDCAT! bulletin board, the external
protocol files can be set up like this:
(JUP.BAT)
CD D:\WILDCAT\PROTOCOL
IF EXIST TRANSFER.BAD DEL TRANSFER.BAD
JMODEM R1 %3
IF ERRORLEVEL 1 GOTO END
COPY %3 %4
:END
DEL %3
(JDOWN.BAT)
CD D:\WILDCAT\PROTOCOL
IF EXIST TRANSFER.BAD DEL TRANSFER.BAD
JMODEM S1 %3
IF ERRORLEVEL 1 GOTO BAD
GOTO END
:BAD
COPY ALL.OK TRANSFER.BAD
:END
There are many variations available. Since WILDCAT! supports
batch-mode downloading, you could set up the batch file like
this:
(JDOWN.BAT)
CD D:\WILDCAT\PROTOCOL
IF EXIST TRANSFER.BAD DEL TRANSFER.BAD
:DO_LOOP
IF "%3" == "" GOTO END
JMODEM S1 %3
IF ERRORLEVEL 1 GOTO BAD
SHIFT
GOTO DO_LOOP
:BAD
COPY ALL.OK TRANSFER.BAD
:END
WILDCAT! checks the \PROTOCOL directory to see if the file
TRANSFER.BAD has been created. If it exists, it announces
that the file transfer has failed. It also announces "Error
with external protocol .. ". It does this when, in fact,
WILDCAT! has made an error itself. In many cases WILDCAT!
will "find" the file TRANSFER.BAD when it DOES NOT EXIST! In
spite of this bug, WILDCAT! is one of the most reliable BBS
systems supporting external protocols.
When setting up batch files remember that there is also a
bug in all DOS versions. The "IF ERRORLEVEL " statement does
- 9 -
JMODEM Protocol Page 10
NOT test the actual value of ERRORLEVEL! Instead it checks
to see if the returned value is EQUAL or GREATER than the
tested value. If you were to put the statement:
IF ERRORLEVEL 0 GOTO GOOD
.... in a batch file, the execution would ALWAYS branch to
label "GOOD" regardless of the actual ERRORLEVEL returned!
More about bugs when we get to the "BAD BBS" section towards
the end.
JMODEM does not require any information about handshaking.
It will look at the modem port and figure out for itself
what to do.
In the event that the modem carrier is lost, JMODEM will
abort. Since JMODEM only checks the modem port occasionally,
it may take several seconds to abort when the carrier is
lost. It is impossible for a user to get at the DOS level
through JMODEM. Do NOT use the CTTY command in the external
protocol batch files. JMODEM returns ERRORLEVEL 1 if there
was any error in transmission or reception. It returns
ERRORLEVEL 0 (no error) otherwise. It does not delete files
that have been partially received although it properly
closes them. The system operator can arrange the batch files
to delete them if required.
When JMODEM attempts to create a file that already exists it
can't ask the user if the old one should be deleted since
the user is probably not in a terminal program at the time.
Therefore, JMODEM renames the other file to <filename.OLD>
and creates the new file. In the event that <filaname.OLD>
exists, it is deleted before the rename operation occurs.
If you don't know what parameters are being sent to external
protocols, you can make a dummy batch file to check them
using the DOS's echo command just as explained in the user
interface previous to this "BBS" section. You can't see the
parameters being echoed from a remote terminal. You must be
present at the BBS board terminal to test those parameters
unless you modify the dummy command file like this:
@ECHO OFF
ECHO %1>COM1
ECHO %2>COM1
ECHO %3>COM1
ECHO %4>COM1
ECHO %5>COM1
If you find that your system passes the file name as %3,
your "upload" (receive) batch file would contain this:
JMODEM R1 %3
- 10 -
JMODEM Protocol Page 11
Your "download" (send) batch file would contain this:
JMODEM S1 %3
In these examples, it is assumed that you are using
communications adapter port "1".
- 11 -
JMODEM Protocol Page 12
The BAD BBS
Problem.
Believe it or not, there are several very fine running BBS
systems in use that do not properly handle external proto-
cols. MS-DOS provides the proper mechanism for loading and
executing "child" programs from within the "parent"
programs. This is function 4BH of the "DOS" INT 21H DOS
interface. Instead of using this function, these poorly-
behaved programs perform the external file-transfer protocol
in the following (or similar) manner:
(1) Make a DOS call to find the file-size of the
external file-transfer protocol.
(2) Free up an array of memory from "string-space"
within the program that is large enough to copy the external
file transfer contents into it. As assumption about the
data-space required by the external file transfer program is
made based upon the "block-size" information that has been
entered during configuration.
(3) Loads the file into string-space memory.
(4) If its an ".EXE" file, ignore the header.
(5) CALL the first byte of code!
If JMODEM is run in this environment, it WILL crash
the system. JMODEM assumes that it has been placed in memory
by MS-DOS and that an entire segment (64k) is available to
run. JMODEM uses two buffers that are almost 32k in length!
One of the buffers is used to support data compression and
expansion. The other is used for the interrupt buffer.
If you have such a BBS system and you wish to run JMODEM,
you can make a simple modification to the source-code, and
re-compile to produce a version which is a bit slower to
initialize and exit, but is guaranteed to leave all memory
and registers EXACTLY as they were when JMODEM got control.
This is done by saving and restoring all registers.
Additionally, any data space that will be modified is copied
to a file called VIRTUAL.MEM, then restored from that file
just before JMODEM exits.
You modify the source-code by finding the "BAD_BBS" equate
near the beginning of the file. This is normally set to
"FALSE". You set this to "TRUE". Then you recompile in the
following manner:
MASM JMODEM;
LINK JMODEM;
EXE2BIN JMODEM.EXE JMODEM.COM
DEL JMODEM.EXE
- 12 -
JMODEM Protocol Page 13
Do NOT attempt to execute the ".EXE" version. You must
change JMODEM to a ".COM" file. If you do not have
EXE2BIN.COM to make the change, you can use DOS's DEBUG to
do the same thing. You do it this way:
F:\DEV> debug jmodem.exe ; DOS command line
-rcx ; Examine CX register
CX 1239 ; Debug says the size was 1239
:
-h 1239,100 ; Calculate 1239H - 100H
1339 1139 ; Sum = 1339H, dif = 1139H
-rcx ; Examine register again
CX 1239 ; Is 1239H
:1139 ; Change to 1139H
; .. subtract 100H
-njmodem.com ; Name new SAVE file name
-w ; Write to file
Writing 1139 bytes ; Debug prompts
-q ; Exit
F:\DEV> ; Back to DOS
When you use this version of JMODEM, it will take a little
while longer to load and exit because it must write a 64k
block of memory to a file and read / delete in upon exit.
- 13 -
JMODEM Protocol Page 14
When things don't work!
The early versions of JMODEM assumed that it was being
properly loaded either by COMMAND.COM, the resident command
processor, or by a "parent" process that properly executes
the DOS function 1BH (the EXEC function). Much to my
surprise, I found that much BBS system software is not
written properly. Starting at version V1.10, I included the
"BAD_BBS" routines to help overcome some of the loading
problems that these BBS systems have. In some cases this
work-around was successful and in others it was not. In
version V1.13, I included some tests that verify whether or
not JMODEM has been loaded properly. Instead of crashing the
system, JMODEM now prints an error message and exits. These
error messages can help you find out what the problem is
and, hopefully, obtain a solution.
Presently, when JMODEM is loaded one of the first things it
does is make a DOS call to give up all memory except that
which contains the code. If DOS returns an error, it means
that JMODEM was not loaded properly and therefore doesn't
own any memory to give up! If an error message appears
telling you that JMODEM doesn't own any memory, you must
contact the writer of your BBS system software to obtain a
version that has this bug fixed. There will be no way to
execute JMODEM in this kind of environment without risking
serious system crashes.
The next thing JMODEM does is make a DOS call, requesting
64k (-1) bytes of memory for its necessary buffer space. If
this DOS call fails, a message is printed stating that there
is not enough memory available for JMODEM. In this case, you
simply reconfigure your BBS software to provide enough
memory for the external protocol. On some BBS systems, this
is done by defining "block-size". Generally, you just
specify the largest number that your BBS software will
accept and therefore force the BBS system software to
allocate a large block of memory for the external protocol.
Although the code size of JMODEM is small (around 4k), it
requires just as much memory as other external protocols.
The memory is used for three major buffers:
(1) Data buffer 8198 bytes
(2) Encode/decode buffer 32,767 bytes
(3) Interrupt buffer 20,000 bytes
These buffers are put in a separate segment so a wild bit-
stream input cannot cause a system crash. The index used for
addressing memory in the interrupt buffer simply wraps
around past 64k, back to zero. Of course you get data errors
when this happens, but no crash, and if the incoming bit-
stream ends before a time-out, the error is recoverable.
When satellite links lose lock, the result is usually a 20
to 30 second burst of noise (random bit-stream). At 9600
- 14 -
JMODEM Protocol Page 15
baud, you can easily overflow a buffer if it is not large
enough or is not allowed to wrap. Alternative methods are to
limit the size of the buffer and check the limit every time
a byte is put in the buffer. JMODEM doesn't do this for two
reasons. The first is the increased software overhead (the
check must be made for every byte in the interrupt service
routine) and the second is that you may be waiting for an
ACK. JMODEM always preserves the last byte in the buffer so
that, even with a noisy reverse-channel, the ACK/NAK can be
extracted.
These are the error messages and what they mean.
JMODEM error message:
Specified file "" not found
This means that no file name was passed to JMODEM on the
command line during a download.
JMODEM error message:
Specified file "\D:PATH\FILENAME.TYP" not found
This means that an incorrect file name was passed to JMODEM
on the command line during a download.
JMODEM error message:
Specified file "" Can't be created.
This means that no file name was passed to JMODEM on the
command line during an upload.
JMODEM error message:
Specified file "\D:PATH\FILENAME.TYP" Can't be created.
This means that an incorrect file name was passed to JMODEM
on the command line during an upload or:
o The path doesn't exist
o The drive doesn't exist
o Not enough file handles
Put FILES=40 in CONFIG.SYS and reboot
o Attempt to write to a network drive
If you are not networked, do NOT install SHARE
JMODEM error message:
File transfer aborted!
This is a normal abort. If the user didn't abort then the
active communications adapter port is not being passed to
JMODEM correctly.
- 15 -
JMODEM Protocol Page 16
JMODEM error message:
Modem carrier failed.
Modem either was not online or the user hung up during
protocol execution.
JMODEM error message:
Can't execute, no free RAM!
A poorly-written BBS system attempted to load the program as
a subroutine and execute it. Contact the BBS software
writer.
JMODEM error message:
This program was already loaded over resident code.
The system will probably crash!
A poorly-written BBS system just loaded JMODEM over its own
code!
JMODEM error message: Can't create file VIRTUAL.MEM
JMODEM error message: Can't write file VIRTUAL.MEM
JMODEM error message: Can't close file VIRTUAL.MEM
JMODEM error message: Can't open file VIRTUAL.MEM
JMODEM error message: Can't delete file VIRTUAL.MEM
When compiled with the BAD_BBS system conditional set to
"TRUE", JMODEM was unable to do the required file I/O
because something was screwed up by the BBS software or
there were not enough file handles available. Enter FILES=40
in CONFIG.SYS and reboot. This can also happen if SHARE is
installed and no network exists. Do NOT use SHARE if you are
not networked!
JMODEM error message:
DOS reports that JMODEM was improperly loaded and does not
own the memory it is using!
The BBS system software is either improperly configured or
improperly written for external protocols. Contact the
writer of the BBS system software.
JMODEM error message:
Not enough free memory for JMODEM to use!
The BBS system software is not properly configured to give
JMODEM the memory it requires to execute. Check the BBS
system software documentation and reconfigure.
If you have problems using JMODEM with your system, you can
call the PROGRAM EXCHANGE and leave a message You can also
test your communications program's external file-transfer
protocols by transferring files (hopefully uploading) to the
PROGRAM EXCHANGE. Currently there are hundreds of boards
- 16 -
JMODEM Protocol Page 17
that are using this protocol and the number is growing every
day. Most problems encountered are found to be caused by
incorrect file names being sent to JMODEM (the wrong "%"
parameters). A simple batch file to test these parameters
will go a long way towards solving the problems.
Richard B. Johnson
PROGRAM EXCHANGE
(303) 440-0786
Boulder Colorado
80303
- Finis -
- 17 -