💾 Archived View for retroforth.org › nga › docs › RETRO-Book.md captured on 2023-04-26 at 13:06:43.
-=-=-=-=-=-=-
# Retro: a Modern, Pragmatic Forth Welcome to Retro, my personal take on the Forth language. This is a modern system primarily targeting desktop, mobile, and servers, though it can also be used on some larger (ARM, MIPS32) embedded systems. The language is Forth. It is untyped, uses a stack to pass data between functions called words, and a dictionary which tracks the word names and data structures. But it's not a traditional Forth. Retro draws influences from many sources and takes a unique approach to the language. Retro has a large vocabulary of words. Keeping a copy of the Glossary on hand is highly recommended as you learn to use Retro. This book will hopefully help you develop a better understanding of Retro and how it works. # Obtaining Retro ## Stable Releases I periodically make stable releases. This will typically happen quarterly. - http://forthworks.com/retro - http://forth.works ## Snapshots A lot of development happens between releases. I make snapshots of my working source tree nightly (and often more often). The latest snapshot can be downloaded from the following stable URLs:
retro
Or:
retro -i
This should be sufficient for most uses. ## Using In a Pipe Retro will work with piped input. E.g.,
echo "'lol s:put nl" | retro
## Running A Program In A File You can run code in a file very easily. This is simply:
retro filename
You can follow the filename with any arguments that it may need. These will be accessible to the program via the `script:arguments` and `script:get-argument` words. Source files must be written in Unu format. ## Scripting You can use Retro to write scripts. Add a shebang:
And make the file executable.
Source files must be written in Unu format.
## Command Line Arguments
For a summary of the full command line arguments available:
Scripting Usage:
retro filename [script arguments...]
Interactive Usage:
retro [-h] [-i] [-f filename] [-t]
-h Display this help text
-i Interactive mode (line buffered)
-f filename Run the contents of the specified file
-t Run tests (in ``` blocks) in any loaded files
# Basic Interactions
Start Retro in interactive mode:
retro -i
You should see something similar to this:
Retro 12 (2021.7)
8388608 MAX, TIB @ 1025, Heap @ 9374
At this point you are at the *listener*, which reads and
processes your input. You are now set to begin exploring
Retro.
Retro is normally silent; unlike other Forth systems, it
does not display an "ok" prompt.
To exit, run `bye`:
bye
# Unu: Simple, Literate Source Files
Retro is written in a literate style. Most of the sources
are in a format called Unu. This allows easy mixing of
commentary and code blocks, making it simple to document
the code.
As an example,
# Determine The Average Word Name Length
To determine the average length of a word name two values
are needed. First, the total length of all names in the
Dictionary:
~~~
#0 [ d:name s:length + ] d:for-each
~~~
And then the number of words in the Dictionary:
~~~
#0 [ drop n:inc ] d:for-each
~~~
With these, a simple division is all that's left.
~~~
/
~~~
Finally, display the results:
~~~
'Average_name_length:_%n\n s:format s:put
~~~
This illustrates the format. Only code in the fenced blocks
(between \~~~ pairs) get extracted and run.
(Note: this only applies to source files; fences are not used
when entering code interactively).
## On The Name
The name Unu comes from the Maori language, where it means:
(verb) (-hia) pull out, withdraw, draw out, extract.
Taken from https://maoridictionary.co.nz/
# Retro's Markdown Syntax
I use a variation of Markdown for writing documentation and
when commenting code written in Retro. The syntax is
described below.
## Basic Syntax
### Headings
Headings start with one or more number (`#`) signs. The
number of number signs should correspond to the heading
level.
# Heading 1
## Heading 2
### Heading 3
#### Heading 4
My Markdown does not support the alternate underline
format for headings.
### Paragraphs & Line Breaks
To create paragraphs, use a blank line to separate one or
more lines of text.
Do not add spaces or tabs at the start of a paragraph as
this may cause the Markdown tools to interpret the line
improperly.
Line breaks are entered at the end of each line.
### Emphasis
#### Bold
To make text bold, surround it with asterisks.
The *bold* word.
#### Italic
To make text italic, surround it with front slashes.
The /italic words/.
#### Underline
To underline text, surround it with underscores.
Underline _some text_.
### Horizontal Rules
Horizontal rules can be inserted by starting a line with a
sequence of four or more dashes (`-`) or four or more alternating
dash and plus (`-+-+`) characters.
----
## Lists
Lists start with a `-` or `*`, followed by a space, then the item
text. Additionally, nested lists starting with two spaces before
the list marker can be used.
- this is a list item
- so is this
- this will be indented
- likewise
- back to the standard level
## Code
### Code Blocks
Code blocks start and end with ~~~ on a line by themselves.
Sum the values.
~~~
{ #10 #20 #13 #4 #22 } #0 [ + ] a:reduce
~~~
You can also denote code by starting the line with four spaces.
This line will be treated as code.
### Test Blocks
Unit testing blocks start and end with ``` on a line by
themselves.
```
{ #10 #20 #13 #4 #22 } #0 [ + ] a:reduce
```
### Inline Code
To mark a sequence as inline code, surround it with backticks.
For instance, look at the value in `Compiler` to see if
the colon compiler is active.
## Escaping
You can preceed a character with a backslash (\\) to have it
not be processed as a Markdown element.
# A Quick Tutorial
Programming in Retro is all about creating words to solve
the problem at hand. Words operate on data, which can be
kept in memory or on the stack.
Let's look at this by solving a small problem: writing a
word to determine if a string is a palindrome.
A palindrome is a phrase which reads the same backward
and forward.
We first need a string to look at. Starting with something
easy:
'anna
Looking in the Glossary, there is a `s:reverse` word for reversing a string. We can find `dup` to copy a value, and `s:eq?` to compare two strings. So testing:
'anna dup s:reverse s:eq?
This yields -1 (`TRUE`) as expected. So we can easily name it:
:palindrome? dup s:reverse s:eq? ;
Naming uses the `:` sigil to add a new word to the dictionary. The words that make up the definition are then placed, with a final word (`;`) ending the definition. We can then use this:
'anna palindrome?
Once defined there is no difference between our new word and
any of the words already provided by the Retro system.
# Syntax
Retro has more syntax than a traditional Forth due to ideas
borrowed from ColorForth and some design decisions. This has
some useful traits, and helps to make the language more
consistent.
## Tokens
Input is divided into a series of whitespace delimited tokens.
Each of these is then processed individually. There are no
parsing words in Retro.
Tokens may have a single character *sigil*, which Retro will
use to decide how to process the token.
## Sigils
Sigils are single characters added to the start of a token
to guide the compiler. The use of these is a major way in
which Retro differs from traditional Forth.
When a token is passed to `interpret`, Retro first takes the
initial character and looks to see if there is a word that
matches this. If so, it will pass the rest of the token to
that word to handle.
In a traditional Forth, the interpret process is something
like:
get token
is token in the dictionary?
yes:
is it immediate?
yes: call the word.
no: are we interpreting?
yes: call the word
no: compile a call to the word
no:
is it a number?
yes: are we interpreting?
yes: push the number to the stack
no: compile the number as a literal
no: report an error ("not found")
In Retro, the interpret process is basically:
get token
does the first character match a `sigil:` word?
yes: pass the token to the sigil handler
no: is token a word in the dictionary?
yes: push the XT to the stack and call the
class handler
no: report an error ("not found")
All of the actual logic for how to deal with tokens is moved
to the individual sigil handlers, and the logic for handling
words is moved to word class handlers.
This means that sigils are used for a lot of things. Numbers?
Handled by a `#` sigil. Strings? Use the `'` sigil. Comments?
Use `(`. Making a new word? Use the `:` sigil.
The major sigils are:
| Sigil | Used For |
| ------ | ----------------------------- |
| @ | Fetch from variable |
| ! | Store into variable |
| & | Pointer to named item |
| # | Numbers |
| $ | ASCII characters |
| ' | Strings |
| ( | Comments |
| : | Define a word |
The individual sigils will be covered in more detail in the
later chapters on working with different data types.
## Word Classes
Word classes are words which take a pointer and do something
with it. These are covered in detail in their own chapter,
but essentially they decide *how* to execute or compile specific
types of words.
# Additional Tools
In addition to the core `retro` binary, the `bin` directory
will contain a few other tools.
## retro
This is the main RETRO binary.
## retro-describe
This is a program that looks up entries in the Glossary.
At the command line, you can use it like:
retro-describe s:for-each
You can pass multiple word names to it:
retro-describe s:for-each nl d:words
## retro-embedimage
This is a program which generates a C file with the ngaImage
contents. It's used when building `retro`.
retro-embedimage ngaImage
The output is written to stdout; redirect it as needed.
## retro-extend
This is a program which compiles code into the ngaImage.
It's used when building `retro` and when you want to make a
standalone image with custom additions.
Example command line:
retro-extend ngaImage example/rot13.forth
Pass the image name as the first argument, and then file names
as subsequent ones. Do *not* use this for things relying on I/O
apart from the basic console output as it doesn't emulate other
devices. If you need to load in things that rely on using the
optional I/O devices, see the **Advanced Builds** chapter.
## retro-muri
This is the assembler for Nga. It's used to build the initial
RETRO kernel and can be used by other tools as well.
retro-muri retro.muri
## retro-tags and retro-locate
These tools are intended to be used together. The first tool,
`retro-tags`, will recursively scan the current directory for
RETRO source files and extract the locations of words defined
in them. These will be written to disk in a `tags` file, using
the standard ctags format.
`retro-locate` takes a word name, and returns the location(s)
where it is defined. This requires a `tags` file to be present.
Create the `tags` file:
retro-tags
Locate a word:
retro-locate n:square
## retro-unu
This is the literate source extraction tool for RETRO. It
is used in building `retro`.
Example usage:
retro-unu literate/RetroForth.md
Output is written to stdout; redirect as neeeded.
# The Optional Retro Compiler
In addition to the base system, users of RETRO on Unix hosts
with ELF executables can build and use the `retro-compiler`
to generate turnkey executables.
## Requirements
- Unix host
- ELF executable support
- `objcopy` in the $PATH
## Building
make bin/retro-compiler
## Installing
Copy `bin/retro-compiler` to somewhere in your $PATH.
## Using
`retro-compiler` takes two arguments: the source file to
compile and the name of the word to use as the main entry
point.
Example:
Given a `hello.forth`:
~~~
:hello 'Hello_World! s:put nl ;
~~~
Use:
retro-compiler hello.forth hello
The compiler will generate an `a.out` file which you can
then rename.
## Known Limitations
This does not provide the scripting support for command line
arguments that the standard `retro` interface offers.
A copy of `objcopy` needs to be in the path for compilation
to work.
The current working directory must be writable.
This only supports hosts using ELF executables.
The output file name is fixed to `a.out`.
RETRO(1) General Commands Manual RETRO(1)
RETRO
retro - a modern, pragmatic forth development system
SYNOPSIS
retro [-h] [-i] [-t] [-f filename] [-u filename] [-r filename]
[filename script-args]
DESCRIPTION
RETRO is a modern, pragmatic Forth drawing influences from many sources.
It's clean, elegant, tiny, and easy to grasp and adapt to various uses.
retro is the main interface for interacting with Retro. It provides both
an interactive and a scripting model.
OPTIONS
-h Display a help screen.
-i Start Retro in interactive mode.
-s Start Retro in interactive mode and supress the startup message.
-t Run any test blocks in the loaded files.
-f filename
Run any code blocks in the specified file.
-u filename
Load and use the specified image file rather than the integral
one.
-r filename
Load and run the code in the specified image file rather than
the integral one.
filename script-args
Run code blocks in a single file. Pass script-args to the code
being run.
AUTHORS
Charles Childers <crc@forthworks.com>
OpenBSD 6.4 September 2019 OpenBSD 6.4
RETRO-DESCRIBE(1) General Commands Manual RETRO-DESCRIBE(1)
RETRO-DESCRIBE
retro-describe - a modern, pragmatic forth development system
SYNOPSIS
retro-describe wordname [additional wordnames]
DESCRIPTION
RETRO is a modern, pragmatic Forth drawing influences from many sources.
It's clean, elegant, tiny, and easy to grasp and adapt to various uses.
retro-describe is a tool for looking up the description and stack
comments for words in the core language and extensions. It will write
output to stdout.
AUTHORS
Charles Childers <crc@forthworks.com>
OpenBSD 6.4 May 2019 OpenBSD 6.4
RETRO-DOCUMENT(1) General Commands Manual RETRO-DOCUMENT(1)
RETRO-DOCUMENT
retro-document - a modern, pragmatic forth development system
SYNOPSIS
retro-document filename
DESCRIPTION
RETRO is a modern, pragmatic Forth drawing influences from many sources.
It's clean, elegant, tiny, and easy to grasp and adapt to various uses.
retro-document is a tool for generating a listing of the descriptions and
stack comments for all standard word used in a source file. It will write
output to stdout.
AUTHORS
Charles Childers <crc@forthworks.com>
OpenBSD 6.4 May 2019 OpenBSD 6.4
RETRO-EMBEDIMAGE(1) General Commands Manual RETRO-EMBEDIMAGE(1)
RETRO-EMBEDIMAGE
retro-embedimage - a modern, pragmatic forth development system
SYNOPSIS
retro-embedimage [filename]
DESCRIPTION
RETRO is a modern, pragmatic Forth drawing influences from many sources.
It's clean, elegant, tiny, and easy to grasp and adapt to various uses.
retro-embedimage loads the specified image (or `ngaImage` from the
current directory if none is specified). It converts this into C code
that can be compiled for inclusion in a RETRO executable. It will write
the output to stdout.
AUTHORS
Charles Childers <crc@forthworks.com>
OpenBSD 6.4 February 2019 OpenBSD 6.4
RETRO-EXTEND(1) General Commands Manual RETRO-EXTEND(1)
RETRO-EXTEND
retro-extend - a modern, pragmatic forth development system
SYNOPSIS
retro-extend image filename [filenames]
DESCRIPTION
RETRO is a modern, pragmatic Forth drawing influences from many sources.
It's clean, elegant, tiny, and easy to grasp and adapt to various uses.
retro-extend is a tool to load additional code into an image file. It
takes the name of an image file and one or more source files to load into
the image. After completion the image file will be updated with the
changes.
CAVEATS
retro-extend only emulates the minimal console output device. If the
source files require additional I/O to be present, the extend process
will likely fail to work correctly.
AUTHORS
Charles Childers <crc@forthworks.com>
OpenBSD 6.4 January 2021 OpenBSD 6.4
RETRO-LOCATE(1) General Commands Manual RETRO-LOCATE(1)
RETRO-LOCATE
retro-locate - a modern, pragmatic forth development system
SYNOPSIS
retro-locate wordname
DESCRIPTION
RETRO is a modern, pragmatic Forth drawing influences from many sources.
It's clean, elegant, tiny, and easy to grasp and adapt to various uses.
retro-locate searches the tags file generated by retro-tags for the
desired word name. Any matches are displayed, along with the line number.
AUTHORS
Charles Childers <crc@forthworks.com>
OpenBSD 6.6 January 2020 OpenBSD 6.6
RETRO-MURI(1) General Commands Manual RETRO-MURI(1)
RETRO-MURI
retro-muri - a modern, pragmatic forth development system
SYNOPSIS
retro-muri filename
DESCRIPTION
RETRO is a modern, pragmatic Forth drawing influences from many sources.
It's clean, elegant, tiny, and easy to grasp and adapt to various uses.
retro-muri is an assembler for Nga, the virtual machine at the heart of
Retro. It is used to build the image file containing the actual Retro
language.
This will extract the code blocks in the specified file and generate an
image file named `ngaImage`.
AUTHORS
Charles Childers <crc@forthworks.com>
OpenBSD 6.4 February 2019 OpenBSD 6.4
RETRO-TAGS(1) General Commands Manual RETRO-TAGS(1)
RETRO-TAGS
retro-tags - a modern, pragmatic forth development system
SYNOPSIS
retro-tags
DESCRIPTION
RETRO is a modern, pragmatic Forth drawing influences from many sources.
It's clean, elegant, tiny, and easy to grasp and adapt to various uses.
retro-tags is a tool for extracting code from fenced blocks in literate
sources and generating a tags file compatible with ctags.
AUTHORS
Charles Childers <crc@forthworks.com>
OpenBSD 6.4 August 2019 OpenBSD 6.4
# Naming Conventions
Word names in RETRO generally follow the following conventions.
## General Guidelines
{ #1 #2 #3 #4 }
{ 'this 'is 'an 'array 'of 'strings }
{ 'this 'is 'a 'mixed 'array #1 #2 #3 }
You can also make an array from a quotation which returns values and the number of values to store in the a:
[ #1 #2 #3 #3 ] a:counted-results
[ #1 #2 #3 #3 ] a:make
## Accessing Elements You can access a specific value with `a:th` and `fetch` or `store`:
{ #1 #2 #3 #4 } #3 a:th fetch
## Find The Length Use `a:length` to find the size of the array.
{ #1 #2 #3 #4 } a:length
## Duplicate Use `a:dup` to make a copy of an a:
{ #1 #2 #3 #4 } a:dup
## Filtering RETRO provides `a:filter` which extracts matching values from an array. This is used like:
{ #1 #2 #3 #4 #5 #6 #7 #8 } [ n:even? ] a:filter
The quote will be passed each value in the array and should return TRUE or FALSE. Values that lead to TRUE will be collected into a new array. ## Mapping `a:map` applies a quotation to each item in an array and constructs a new array from the returned values. Example:
{ #1 #2 #3 } [ #10 * ] a:map
## Reduce `a:reduce` takes an array, a starting value, and a quote. It executes the quote once for each item in the array, passing the item and the value to the quote. The quote should consume both and return a new value.
{ #1 #2 #3 } #0 [ + ] a:reduce
## Search RETRO provides `a:contains?` and `a:contains/string?` to search an array for a value (either a number or string) and return either TRUE or FALSE.
'test { 'abc 'def 'test 'ghi } a:contains/string?
## Implementation
In memory, an array is a count followed by the values. As an
example, if you have an array:
{ #10 #20 #30 }
In memory this would be setup as:
| Offset | Value |
| ------ | ----- |
| 000 | 3 |
| 001 | 10 |
| 002 | 20 |
| 003 | 30 |
You can construct one on the fly by keeping a pointer to
`here` and using `,` to place the values. E.g.,
here [ #3 , #10 , #20 , #30 , ] dip
An example of this can be seen in this excerpt from an example
(*example/Primes.forth*):
:create-set (-a)
here #3000 , #2 #3002 [ dup , n:inc ] times drop ;
# Working With Assembly Language
RETRO runs on a virtual machine called Nga. It provides a
standard assembler for this called *Muri*.
Muri is a simple, multipass model that's not fancy, but
suffices for RETRO's needs.
## Assembling A Standalone File
A small example (*test.muri*)
~~~
i liju....
r main
: c:put
i liiire..
i 0
: main
i lilica..
d 97
i liju....
r main
~~~
Assembling it:
retro-muri test.muri
So breaking down: Muri extracts the assembly code blocks to
assemble, then proceeds to do the assembly. Each source line
starts with a directive, followed by a space, and then ending
with a value.
The directives are:
: value is a label
i value is an instruction bundle
d value is a numeric value
r value is a reference
s value is a string to inline
Instructions for Nga are provided as bundles. Each memory
location can store up to four instructions. And each instruction
gets a two character identifier.
From the list of instructions:
0 nop 5 push 10 ret 15 fetch 20 div 25 zret
1 lit 6 pop 11 eq 16 store 21 and 26 halt
2 dup 7 jump 12 neq 17 add 22 or 27 ienum
3 drop 8 call 13 lt 18 sub 23 xor 28 iquery
4 swap 9 ccall 14 gt 19 mul 24 shift 29 iinvoke
This reduces to:
0 .. 5 pu 10 re 15 fe 20 di 25 zr
1 li 6 po 11 eq 16 st 21 an 26 ha
2 du 7 ju 12 ne 17 ad 22 or 27 ie
3 dr 8 ca 13 lt 18 su 23 xo 28 iq
4 sw 9 cc 14 gt 19 mu 24 sh 29 ii
Most are just the first two letters of the instruction name. I
use `..` instead of `no` for `NOP`, and the first letter of
each I/O instruction name. So a bundle may look like:
dumure..
(This would correspond to `dup multiply return nop`).
## Runtime Assembler
RETRO also has a runtime variation of Muri that can be used
when you need to generate more optimal code. So one can write:
:n:square dup * ;
Or:
:n:square \dumure.. ;
The second one will be faster, as the entire definition is one
bundle, which reduces memory reads and decoding by 2/3.
Doing this is less readable, so I only recommend doing so after
you have finalized working RETRO level code and determined the
best places to optimize.
The runtime assembler has the following directives:
i value is an instruction bundle
d value is a numeric value
r value is a reference
Additionally, in the runtime assembler, these are reversed:
'dudumu.. i
Instead of:
i dudumu..
The runtime assembler also provides three sigils for use in
inlining machine code into a definition. These are:
\ Treat token as an assembly sequence
` Treat token as a numeric value
^ Treat token as a reference
E.g., instead of doing something like:
:n:square as{ 'dumu.... i }as ;
:test as{ 'lilica.... i #22 d 'n:square r }as ;
Just write:
:n:square \dumu.... ;
:test \lilica.. `22 ^n:square ;
# Working With a Buffer
RETRO provides words for operating on a linear memory area.
This can be useful in building strings or custom data
structures.
## Namespace
Words operating on the buffer are kept in the `buffer:`
namespace.
## Implementation
A buffer is a linear sequence of memory. The buffer words
provide a means of incrementally storing and retrieving
values from it.
The buffer words keep track of the start and end of the
buffer. They also ensure that an `ASCII:NULL` is written
after the last value, which make using them for string
data easy.
## Limitations
Only one buffer can be active at a time. RETRO provides a
`buffer:preserve` combinator to allow using a second one
before returning to the prior one.
## Set The Active Buffer
To set a buffer as the active one use `buffer:set`. This takes
an address.
The buffer will be assumed to be empty. The initial value will
be set to ASCII:NULL.
## Add Value
Use `buffer:add` to append a value to the buffer. This takes
a single value and will also add an ASCII:NULL after the end
of the buffer.
## Fetch Last Value
To return the last value in the buffer you can use `buffer:get`.
This removes the value and sets an ASCII:NULL in the memory
location the returned value occupied.
## Get Data About The Buffer
RETRO provides `buffer:start` to get the initial address in
the buffer, `buffer:end` to get the last address (ignoring the
ASCII:NULL), and `buffer:size` to return the number of values
in the buffer.
## Reset
You can reset a buffer to the empty state using `buffer:empty`.
## Example
To begin, create a memory region to use as a buffer.
'Test d:create #1025 allot
Then you can set this as the current buffer:
&Test buffer:set
When a buffer is set, the vocabulary sets an internal index to the first address in it. This will be incremented when you add data and decremented when you remove data. Let's add some stuff using `buffer:add`:
And then retrieve the values:
buffer:get n:put nl
buffer:get n:put nl
buffer:get n:put nl
You can remove all values using `buffer:empty`:
buffer:empty
And ask the buffer how many items it contains:
buffer:size n:put nl
buffer:size n:put nl
buffer:empty
The other functions are `buffer:start`, which returns the address of the buffer, `buffer:end`, which returns the address of the last value, and `buffer:preserve`. The first is easy to demo:
buffer:start Test eq? n:put nl
The last one is useful. Only one buffer is ever active at a given time. The `buffer:preserve` combinator lets you execute a word, saving and restoring the current buffer indexes. So the word could assign and use a new buffer and this will reset the previous one after control returns. There are a few notes that need to be considered. The preserve combinator saves the start and current index but *not* the contents. If the word you call uses the same buffer, the contents will remain altered. Finally, the buffer words have one interesting trait: they store an ASCII NULL after adding each item to the buffer. This lets one use them to build strings easily.
Test buffer:set
$h buffer:add
$e buffer:add
$l buffer:add
$l buffer:add
$o buffer:add
$, buffer:add
$w buffer:add
$o buffer:add
$r buffer:add
$l buffer:add
$d buffer:add
buffer:start s:put nl
# Working With Characters RETRO provides words for working with ASCII characters. ## Sigil Character constants are returned using the `gemini - kennedy.gemi.dev sigil. ## Namespace Words operating on characters are in the `c:` namespace. ## Classification RETRO provides a number of words to determine if a character fits into predefined groups. The primary words for this are:
$a c:put
With the default system on BSD, Linux, and macOS (and other
Unix style hosts), `c:get` is provided to read input. This
may be buffered, depending on the host.
# Defining Words
Words are named functions. To start a word, preceed it's name
with a colon. Follow this by the definition, and end with a
semicolon.
E.g.,
:do-nothing ;
:square dup * ;
# Working With The Dictionary
The Dictionary is a linked list containing the dictionary
headers.
## Namespace
Words operating on the dictionary are in the `d:` namespace.
## Variables
`Dictionary` is a variable holding a pointer to the most recent
header.
## Header Structure
Each entry follows the following structure:
Offset Contains
------ ---------------------------
0000 Link to Prior Header
0001 Link to XT
0002 Link to Class Handler
0003 Source Identifier
0004+ Word name (null terminated)
RETRO provides words for accessing the fields in a portable
manner. It's recommended to use these to allow for future
revision of the header structure.
## Accessing Fields
Given a pointer to a header, you can use `d:xt`, `d:class`,
and `d:name` to access the address of each specific field.
There is no `d:link`, as the link will always be the first
field.
## Shortcuts For The Latest Header
RETRO provides several words for operating on the most recent
header.
`d:last` returns a pointer to the latest header. `d:last.xt`
will give the contents of the `d:xt` field for the latest
header. There are also `d:last.class` and `d:last.name`.
## Adding Headers
Two words exist for making new headers. The easy one is
`d:create`. This takes a string for the name and makes a
new header with the class set to `class:data` and the XT
field pointing to `here`.
Example:
'Base d:create
The other is `d:add-header`. This takes a string, a pointer to the class handler, and a pointer for the XT field and builds a new header using these. Example:
'Base &class:data #10000 d:add-header
## Searching RETRO provides two words for searching the dictionary. `d:lookup` takes a string and tries to find it in the dictionary. It will return a pointer to the dictionary header or a value of zero if the word was not found. `d:lookup-xt` takes a pointer and will return the dictionary header that has this as the `d:xt` field, or zero if no match is found. ## Iteration You can use the `d:for-each` combinator to iterate over all entries in the dictionary. For instance, to display the names of all words:
[ d:name s:put sp ] d:for-each
For each entry, this combinator will push a pointer to the entry to the stack and call the quotation. ## Listing Words Most Forth systems provide WORDS for listing the names of all words in the dictionary. RETRO does as well, but this is named `d:words`. This isn't super useful as looking through several hundred names is annoying. RETRO also provides `d:words-with` to help in filtering the results. Example:
'class: d:words-with
# Working With Floating Point
Some RETRO systems include support for floating point numbers.
When present, this is built over the system `libm` using the
C `double` type.
Floating point values are typically 64 bit IEEE 754 double
precision (1 bit for the sign, 11 bits for the exponent, and
the remaining 52 bits for the value), i.e. 15 decimal digits
of precision.
## Sigil
Floating point numbers start with a `.`
Examples:
Token Value
.1 1.0
.0.5 0.5
.-.4 -0.4
.1.3 1.3
## Namespace
Floating point words are in the `f:` namespace. There is also
a related `e:` namespace for *encoded values*, which allows
storing of floats in standard memory.
## Operation
Floating point values exist on a separate stack, and are bigger
than the standard memory cells, so can not be directly stored
and fetched from memory.
The floating point system also provides an alternate stack that
can be used to temporarily store values.
The following words exist for arranging values on the floating
point stack. These are direct analogs to the non-prefiexd words
for dealing with the data stack.
- `f:nip`
- `f:over`
- `f:depth`
- `f:drop`
- `f:drop-pair`
- `f:dup`
- `f:dup-pair`
- `f:dump-stack`
- `f:tuck`
- `f:swap`
- `f:rot`
For the secondary floating point stack, the following words are
provided:
- `f:push`
- `f:pop`
- `f:adepth`
- `f:dump-astack`
## Constants
| Name | Returns |
| -------- | ----------------- |
| `f:E` | Euler's number |
| `f:-INF` | Negative infinity |
| `f:INF` | Positive infinity |
| `f:NAN` | Not a Number |
| `f:PI` | PI |
## Comparisons
The basic set of comparators are the same as those for
operating on integers. These are:
- `f:-eq?`
- `f:between?`
- `f:eq?`
- `f:gt?`
- `f:lt?`
- `f:negative?`
- `f:positive?`
- `f:case`
There are also a few additions for comparing to special values
like infinity and NaN.
- `f:-inf?`
- `f:inf?`
- `f:nan?`
## Basic Math
- `f:*`
- `f:+`
- `f:-`
- `f:/`
- `f:abs`
- `f:floor`
- `f:inc`
- `f:limit`
- `f:max`
- `f:min`
- `f:negate`
- `f:power`
- `f:ceiling`
- `f:dec`
- `f:log`
- `f:sqrt`
- `f:square`
- `f:round`
- `f:sign`
- `f:signed-sqrt`
- `f:signed-square`
## Geometry
RETRO provides a small number of words for doing geometric
related calculations.
| Word | Returns |
| -------- | ------------ |
| `f:acos` | arc cosine |
| `f:asin` | arc sine |
| `f:atan` | arc tangent |
| `f:cos` | cosine |
| `f:sin` | sine |
| `f:tan` | tangent |
## Storage and Retrieval
By leveraging the encoded value functions, RETRO is able to
allow storage of floating point values in memory. This does
have a tradeoff in accuracy as the memory cells are considerably
smaller than a full floating point size.
You can use `f:fetch` to fetch a floating point value and
`f:store` to store one.
If you need more precision, try Kiyoshi Yoneda's FloatVar
example (`example/FloatVar.forth`), which includes words to
store and retrieve values using multiple cells.
- `f:to-number`
- `f:to-string`
## I/O
The floating point vocabulary has a single I/O word, `f:put`,
for the display of floating point numbers.
## Encoded Values
RETRO provides a means of encoding and decoding floating point
values into standard integer cells. This is based on the paper
"Encoding floating point values to shorter integers" by Kiyoshi
Yoneda and Charles Childers.
- `f:E1`
- `f:to-e`
- `e:-INF`
- `e:-inf?`
- `e:INF`
- `e:MAX`
- `e:MIN`
- `e:NAN`
- `e:clip`
- `e:inf?`
- `e:max?`
- `e:min?`
- `e:n?`
- `e:nan?`
- `e:put`
- `e:to-f`
- `e:zero?`
# Working With Files
On Unix and Windows systems RETRO provides a set of words for
working with files. As a pragmatic choice these are mostly
modeled after the file functions in libc.
The file words are in the `file:` namespace.
## File Access Modes
You can open a file for various operations. The functionality
allowed depends on the file access mode. Valid modes in RETRO
are:
file:A Open for appending; file pointer set to end of file
file:R Open for reading; file pointer set to start of file
file:R+ Open for reading and writing
file:W Open for writing
## Opening A File
To open a file, pass the file name and a file mode to `file:open`.
'/etc/motd file:R file:open
On a successful open this will return a file handle greater than
zero.
Additionally, RETRO provides a few other forms for opening files.
To open a file for reading:
'/etc/motd file:open-for-reading
This will return the size of the file (as NOS) and the file handle
(as TOS).
To open a file for writing:
'/tmp/test file:open-for-writing
This returns the file handle.
To open a file for append operations:
'/tmp/test file:open-for-append
As with `file:open-for-reading`, this returns both the size of
the file and the file handle.
## Closing A File
To close a file, pass the file handle to `file:close`.
'/etc/motd file:A file:open file:close
## Reading From A File
To read a byte from an open file, pass the file handle to the
`file:read` word.
@FID file:read n:put
To read a line from a file, pass the file handle to the word
`file:read-line`.
@FID file:read-line s:put
The line is read into a temporary string buffer. Move the
text to a safe place if you aren't using it quickly or if
the length of the line is bigger than the size of a temporary
string.
## Writing To A File
To write a byte to a file, pass it and the file handle to
`file:write`.
$h @FID file:write
$e @FID file:write
$l @FID file:write
$l @FID file:write
$o @FID file:write
Though cells are 32 or 64 bits in size, only the byte value will
be written to the file.
## Deleting Files
You can delete a file by passing the file name to `file:delete`.
/tmp/test file:delete
## Check For File Existance
Use `file:exists?` to detect the existance of a file. Pass it a
file name and it will return `TRUE` if existing or `FALSE` if
it does not.
'/etc/motd file:exists?
This will also return `TRUE` if the filename is a directory.
## Flush Caches
Use `file:flush` to flush the system caches for a file. Pass a
file handle to this.
@FID file:flush
## Seek A Position Within A File
You can use `file:seek` to move the internal file pointer
for a given file. Pass this the new location and a file.
#100 @FID file:seek
The location for the file pointer is a fixed offset from the
start of the file, not a relative offset.
## Get The Current Position Within A File
To find the current value of the file pointer within a file
just pass the file handle to `file:tell`.
@FID file:tell
This returns a number that is the number of bytes into the file
that the file pointer is currently at.
## Determine The Size Of A File
Use `file:size` to return the size of a file. Pass this a file
handle and it will return the size of a file, or 0 if empty. If
the file is a directory, it returns -1.
@FID file:size
## Reading An Entire File
If you want to read an entire file into memory you can use
`file:slurp`. This takes the starting address of a memory
region and the name of the file.
here '/etc/motd file:slurp
Take care that the memory buffer is large enough for the file
being read or you will run into problems.
## Writing A String To A File
If you have a string that you want to write to a file, replacing
any existing contents, you can use `file:spew`. This takes the
string to write and a file name.
'hello_world '/tmp/test.txt file:spew
## Iterating Over A File, Line By Line
You can easily iterate over each line in a file using the word
`file:for-each-line`. This will take a file name and a quote,
read each line into a temporary string, then pass this string to
the quote.
'/etc/motd [ s:put nl ] file:for-each-line
# Loops
RETRO provides several words for creating loops.
## Unconditional Loops
An unconditional loop begins with `repeat` and ends with `again`.
:test repeat #1 n:put sp again ;
test
Unconditional loops must be inside a definition or quote. To exit
one of these, use `0;`, `-if;` or `if;`.
:test #100 repeat 0; dup n:put sp n:dec again ;
test
:test #100 repeat dup #50 eq? [ 'done! s:put nl ] if; n:dec again ;
test
You can also achieve this via recursion:
:test 0; dup n:put sp n:dec test ;
#100 test
Be careful with recursion as the virtual machine will have a limited
amount of space for the address stack and recursing too many times
can cause a stack overflow.
## Conditional Loops
There are two conditional looping combinators: `while` and `until`.
Both take a quote and execute it, checking a returned flag to decide
when to stop running.
#0 [ dup n:put sp n:inc dup #10 eq? ] until
#10 [ dup n:put sp n:dec dup n:-zero? ] while
## Counted Loops
There are two combinators for counted loops. These are `times` and
`indexed-times`.
#0 #10 [ dup n:put sp n:inc ] times nl
#10 [ I n:put sp ] indexed-times
The `indexed-times` provides an index via the `I`, `J`, and
`K` words. `I` will be the index of the current loop, with `J` and
`K` being the indexes of the next two older loops.
The loop indexes can be accessed outside the loop body:
:display I n:square n:put sp ;
:squares [ display ] indexed-times nl ;
#100 squares
## Tradeoffs
The unconditional loop form is more efficient as it's just a
simple jump operation. The `times` counted loops are a little
slower, but can be cleaner and more readable in many cases. The
`indexed-times` form is significantly slower than the other
two forms.
# Working With Numbers
Numbers in RETRO are signed integers.
## Sigil
All numbers start with a `#` sigil.
## Namespace
Most words operating on numbers are in the `n:` namespace.
## Range of Values
A default RETRO system with 32 bit cells provides a range of
-2,147,483,648 to 2,147,483,647. For 64 bit systems, the range
will be -9,223,372,036,854,775,807 to 9,223,372,036,854,775,806.
You can check the range your VM and image support using:
n:MIN
n:MAX
These will return the limits for your system.
## Comparisons
RETRO provides a number of comparison words for numeric values.
The basic comparators are:
-eq?
eq?
lt?
lteq?
gt?
gteq?
Additionally RETRO also provides:
n:-zero?
n:between?
n:even?
n:negative?
n:odd?
n:positive?
n:strictly-positive?
n:zero?
## Basic Operations
+
-
*
/
mod
/mod
n:abs
n:dec
n:inc
n:limit
n:max
n:min
n:negate
n:pow
n:sqrt
n:square
## Conversions
You can convert a number to a string with `n:to-string` or
to a floating point value with `n:to-float`.
#123 n:to-float f:put
#123 n:to-string s:put
## Display
To display a number, use `n:put`.
#123 n:put
# Working With Pointers
## Sigil
Pointers are returned by the `&` sigil.
## Examples
'Base var
&Base fetch
## Notes The use of `&` to get a pointer to a data structure (with a word class of `class:data`) is not required. I like to use it anyway as it makes my intent a little clearer. Pointers are useful with combinators. Consider:
:abs dup n:negative? [ n:negate ] if ;
Since the target quote body is a single word, it is more efficient to use a pointer instead:
:abs dup n:negative? &n:negate if ;
The advantages are speed (saves a level of call/return by
avoiding the quotation) and size (for the same reason).
This may be less readable though, so consider the balance
of performance to readability when using this approach.
# Quotations
Quotes are anonymous functions. RETRO uses these as the basis for
executable flow control and combinatorial logic.
## Using Quotations
To make a quotation, surround the code with square brackets. E.g.,
#1 #2 eq? [ 'No_match s:put nl ] -if
Quotes can be nested:
[ #3 [ #4 ] dip ] call
After creation, a pointer to the quotation is left on the stack
(or is compiled into the current definition).
## Combinators
Words operating on quotations are called combinators; these are
discussed in *Using Combinators*.
## Implementation
A quotation is compiled as:
... code before quotation ...
i liju.... (if_compiling_only)
d address after quotation (if_compiling_only)
... code for quotation
i re...... (this_is_where_the_quote_ends)
i li......
d address of code for quotation
... code after quotation ....
## Other Notes
Quotations are used heavily in RETRO. They give the source a
feel that's different from traditional Forth, and allow for
a more consistent syntax.
For instance, in a traditional Forth, you might have some
conditionals:
IF ... THEN
IF ... ELSE ... THEN
IF ... EXIT THEN
RETRO uses conditional combinators for these:
[ ... ] if
[ ... ] [ ... ] choose
[ ... ] if;
Or loops:
FOR ... NEXT
Is replaced by:
[ ... ] times
This can also extend to stack flow. Sequences like:
>R ... >R
DUP >R ... >R
Become:
[ ... ] dip
[ ... ] sip
And forms like:
1 2 3 * swap 3 * swap
Can be replaced with a combinator like:
#1 #2 [ #3 * ] bi@
While there is a different set of words to learn, I find that
overall there's less noise from low level stack shuffling words
and the added consistency with regards to overall syntax has
been nice as I was never fond of the multiple forms that existed
in traditional Forth.
# Sockets
On Unix hosts, RETRO provides an optional set of words for using
network sockets.
## Create a Socket
To create a new socket, just run:
socket:create
This will return a socket handle.
## Bind To A Port
To bind to a port, pass the port number and socket handle
to `socket:bind`. The port should be a string. This will return
0 if successful, -1 if not successful, and an error code.
'9998 @Sock socket:bind
## Configure To Allow Incoming Connections
To prepare a socket for incoming connections use socket:listen. This
will take a backlog count and a socket handle. It returns a flag
(0 success, -1 failed) and an error code.
#3 @Sock socket:listen
## Accept Connections
To accept connections pass the socket handle to `socket:accept`.
This returns a new socket for the connection and an error code.
@Sock socket:accept
## Make A Connection
To connect to a server using the socket:
'forth.works '70 socket:configure
@Sock socket:connect
`socket:connect` will return a status code and an error code.
## Writing To A Socket
To write a string to a socket, use `socket:send`. This will
take a string and a socket handle and will return the number
of bytes sent and an error code.
'test @Sock socket:send
## Reading From A Socket
To read data from a socket pass an address, a maximum number of
bytes, and the socket handle to `socket:recv`. This will return
the number of bytes received and an error code. The bytes will
be stored in memory starting at the specified address.
here #1024 @Sock socket:recv
## Close a Socket
To close a socket, pass the socket handle to `socket:close`.
@Socket socket:close
# Unix Scripting
RETRO on Unix hosts is designed to play well with scripting.
Shebang
To run an entire program directly, start the file with the standard
shebang and make the file executable:
#!/usr/bin/env retro
This requires the retro binary to be in your path.
## Arguments
RETRO provides several words in the `script:` namespace for accessing
command line arguments.
The number of arguments can be accessed via `script:arguments`. This
will return a number with the arguments, other than the script name.
script:arguments '%n_arguments_passed\n s:format s:put
To retreive an argument, pass the argument number to `script:get-argument`:
script:arguments [ I script:get-argument s:put nl ] indexed-times
And to get the name of the script, use `script:name`.
script:name s:put
## Mixing
With use of the Unu literate format, it's possible to mix both shell
and RETRO code into a single script. As an example, this is a bit of
shell that runs itself via retro for each .retro file in the current
directory tree:
#!/bin/sh
# shell part
find . -name '*.retro' -print0 | xargs -0 -n 1 retro $0
exit
# retro part
This will scan a source file and do something with it:
~~~
... do stuff ...
~~~
# Working With Strings
Strings in RETRO are NULL terminated sequences of values
representing characters. Being NULL terminated, they can't
contain a NULL (ASCII 0).
The character words in RETRO are built around ASCII, but
strings can contain UTF8 encoded data if the host platform
allows. Words like `s:length` will return the number of bytes,
not the number of logical characters in this case.
## Sigil
Strings begin with a single `'`.
'Hello
'This_is_a_string
'This_is_a_much_longer_string_12345_67890_!!!
RETRO will replace spaces with underscores. If you need both
spaces and underscores in a string, escape the underscores and
use `s:format`:
'This_has_spaces_and_under\_scored_words. s:format
## Namespace
Words operating on strings are in the `s:` namespace.
## Lifetime
At the interpreter, strings get allocated in a rotating buffer.
This is used by the words operating on strings, so if you need
to keep them around, use `s:keep` or `s:copy` to move them to
more permanent storage.
In a definition, the string is compiled inline and so is in
permanent memory.
You can manually manage the string lifetime by using `s:keep`
to place it into permanent memory or `s:temp` to copy it to
the rotating buffer.
## Mutability
Strings are mutable. If you need to ensure that a string is
not altered, make a copy before operating on it or see the
individual glossary entries for notes on words that may do
this automatically.
## Searching
RETRO provides four words for searching within a string.
:acc (n-) here swap , [ dup v:inc fetch ] curry ;
This would create an accumulator function, which takes an initial value and returns a quote that will increase the accumulator by 1 each time it is invoked. It will also return the latest value. So:
dup call n:put
dup call n:put
dup call n:put
## Execution Flow Combinators of this type execute other functions. ### Fundamental `call` takes a quote and executes it immediately.
[ #1 n:put ] call
&words call
### Conditionals RETRO provides three primary combinators for use with conditional execution of quotes. These are `choose`, `if`, and `-if`. `choose` takes a flag and two quotes from the stack. If the flag is true, the first quote is executed. If false, the second quote is executed.
#0 [ 'true s:put ] [ 'false s:put ] choose
`if` takes a flag and one quote from the stack. If the flag is true, the quote is executed. If false, the quote is discarded.
#0 [ 'true s:put ] if
`-if` takes a flag and one quote from the stack. If the flag is false, the quote is executed. If true, the quote is discarded.
#0 [ 'false s:put ] -if
RETRO also provides `case` and `s:case` for use when you have multiple values to check against. This is similar to a `switch` in C. `case` takes two numbers and a quote. The initial value is compared to the second one. If they match, the quote is executed. If false, the quote is discarded and the initial value is left on the stack. Additionally, if the first value was matched, `case` will exit the calling function, but if false, it returns to the calling function. `s:case` works the same way, but for strings instead of simple values.
:test (n-)
#1 [ 'Yes s:put ] case
#2 [ 'No s:put ] case
drop 'No idea s:put ;
### Looping Several combinators are available for handling various looping constructs. `while` takes a quote from the stack and executes it repeatedly as long as the quote returns a true flag on the stack. This flag must be well formed and equal -1 or 0.
`times` takes a count and quote from the stack. The quote will be executed the number of times specified. No indexes are pushed to the stack.
There is also a `indexed-times` variation that provides access to the loop index (via `I`) and parent loop indexes (via `J` and `K`).
## Data Flow These combinators exist to simplify stack usage in various circumstances. ### Preserving Preserving combinators execute code while preserving portions of the data stack. `dip` takes a value and a quote, moves the value off the main stack temporarily, executes the quote, and then restores the value.
Would yield the following on the stack:
11 20
`sip` is similar to `dip`, but leaves a copy of the original value on the stack during execution of the quote. So:
Leaves us with:
11 10
### Cleave Cleave combinators apply multiple quotations to a single value or set of values. `bi` takes a value and two quotes, it then applies each quote to a copy of the value.
`tri` takes a value and three quotes. It then applies each quote to a copy of the value.
### Spread Spread combinators apply multiple quotations to multiple values. The asterisk suffixed to these function names signifies that they are spread combinators. `bi*` takes two values and two quotes. It applies the first quote to the first value and the second quote to the second value.
`tri*` takes three values and three quotes, applying the first quote to the first value, the second quote to the second value, and the third quote to the third value.
### Apply Apply combinators apply a single quotation to multiple values. The @ sign suffixed to these function names signifies that they are apply combinators. `bi@` takes two values and a quote. It then applies the quote to each value.
`tri@` takes three values and a quote. It then applies the quote to each value.
RETRO also provides `for-each` combinators for various data structures. The exact usage of these varies; consult the Glossary and relevant chapters for more details on these. # Word Classes Word classes are one of the two elements at the heart of RETRO's interpreter. There are different types of words in a Forth system. At a minimum there are data words, regular words, and immediate words. There are numerous approaches to dealing with this. In RETRO I define special words which receive a pointer and decide how to deal with it. These are grouped into a `class:` namespace. ## How It Works When a word is found in the dictionary, RETRO will push a pointer to the definition (the `d:xt` field) to the stack and then call the word specified by the `d:class` field. The word called is responsible for processing the pointer passed to it. As a simple case, let's look at `immediate` words. These are words which will always be called when encountered. A common strategy is to have an immediacy bit which the interpreter will look at, but RETRO uses a class for this. The class is defined:
:class:immediate (a-) call ;
Or a normal word. These should be called at interpret time or compiled into definitions. The handler for this can look like:
:class:word (a-) compiling? [ compile:call ] [ call ] choose ;
## Using Classes The ability to add new classes is useful. If I wanted to add a category of word that preserves an input value, I could do it with a class:
:class:duplicating (a-)
compiling? [ &dup compile:call ] [ &dup dip ] choose
class:word ;
:duplicating &class:duplicating reclass ;
:. n:put nl ; duplicating
# Checking The Version RETRO releases add and change things. You can use the `Version` variable to determine the version in use and react accordingly.
@Version #201906 eq? [ 'Needs_2019.6! s:put nl bye ] if
This can be also be used to conditionally load compatibility files:
(If_newer_than_2016.6,_load_aliases_for_renamed_words)
@Version #201906 gt? [ 'Renamed_2019.6.forth include ] if
## Version Number Format
The version is a six digit number encoding the year and month of
the release. So:
201901 is 2019.1
201906 is 2019.6
201911 is 2019.11
A `#100 /mod` will suffice to split these if needed.
# Errors
RETRO does only minimal error checking.
## Non-Fatal
A non-fatal error will be reported on *word not found* during
interactive or compile time. Note that this only applies to
calls: if you try to get a pointer to an undefined word, the
returned pointer will be zero.
## Fatal
A number of conditions are known to cause fatal errors. The
main ones are stack overflow, stack underflow, and division
by zero.
On these, RETRO will generally exit. For stack depth issues,
the VM will attempt to display an error prior to exiting.
In some cases, the VM may get stuck in an endless loop. If this
occurs, try using CTRL+C to kill the process, or kill it using
whatever means your host system provides.
## Rationale
Error checks are useful, but slow - especially on a minimal
system like RETRO. The overhead of doing depth or other checks
adds up quickly.
As an example, adding a depth check to `drop` increases the
time to use it 250,000 times in a loop from 0.16 seconds to
1.69 seconds.
# Lexical Scope
RETRO has a single dictionary, but does provide a means of using
lexical scope to keep this dictionary clean.
## Example
{{
'A var
:++A &A v:inc ;
---reveal---
:B ++A ++A @A n:put nl ;
}}
In this example, the lexical namespace is created with `{{`. A
variable (`A`) and word (`++A`) are defined. Then a marker is
set with `---reveal---`. Another word (`B`) is defined, and the
lexical area is closed with `}}`.
The headers between `{{` and `---reveal---` are then hidden from
the dictionary, leaving only the headers between `---reveal---`
and `}}` exposed.
If you wish to hide all headers in a `{{` ... `}}` block, leave
out the `---reveal---`.
{{
:a #3 ;
:b a dup * ;
}}
## Notes This only affects word visibility within the scoped area. As an example:
:a #1 ;
{{
:a #2 ;
---reveal---
:b 'a s:evaluate n:put ;
}}
In this, after `}}` closes the area, the `:a #2 ;` is hidden and
the `s:evaluate` will find the `:a #1 ;` when `b` is run.
## A Word of Warning
Use of these words can result in a corrupt dictionary and system
crashes. Specifically, use of `---reveal---` with an empty private
or public section will result in dictionary corruption.
If you don't need private words, don't put them in a scope. And if
you don't need public words, don't include the `---reveal---`.
# The Stacks
The stacks are a defining feature of Forth. They are are used
to pass data between words and to track return addresses for
function calls.
RETRO always has two stacks, and optionally (if built with
floating point support) a third.
## Data Stack
This is the primary stack. Values are placed here, passed to
words which consume them and then return results. When I
refer to "the stack", this is the one I mean. Learning to use
the stack is a crucial part to making effective use of RETRO.
### Placing Values On The Stack
Values can be placed on the stack directly.
| Example | Action |
| -------------- | ---------------------------------------- |
| `#300123` | Push the number `300123` to the stack |
| `$h` | Push the ASCII code for `h` to the stack |
| `'hello_world` | Push a pointer to a string to the stack |
| `&fetch` | Push the address of `fetch` to the stack |
### Reordering The Stack
RETRO provides a number of *shufflers* for reordering items
on the stack.
Some of the most common ones are:
| Word | Before | After |
| ------- |--------- | -------- |
| dup | #1 | #1 #1 |
| drop | #1 #2 | #1 |
| swap | #1 #2 | #2 #1 |
| over | #1 #2 | #1 #2 #1 |
| tuck | #1 #2 | #2 #1 #2 |
| nip | #1 #2 | #2 |
| rot | #1 #2 #3 | #3 #1 #2 |
You can use `push` and `pop` to move values to and from the
address stack. Make sure you `pop` them back before the word
ends or RETRO will crash. These two words can not be used
at the interpreter.
There is also a special one, `reorder`, which allows for big
stack restructuring. This is slow but can be very useful.
As an example, let's say we have four values:
#1 #2 #3 #4
And we want them to become:
#4 #3 #2 #1
Doing this with the basic shufflers is difficult. You could end
up with something similar to:
swap rot push rot pop swap
But with `reorder`, you can just express the before and after
states:
'abcd 'dcba reorder
### Resetting The Stack
If you need to quickly empty the stack, use `reset`.
### Get The Stack Depth
To find out how many items are on the stack, use `depth`.
### Displaying The Stack
You can display the stack by running `dump-stack`.
### Data Flow Combinators
RETRO provides *combinators* for working with data order on
the stack. These are covered in a later chapter and are worth
learning to use as they can help provide a cleaner, more
structured means of working.
### Tips
The stack is *not* an array in addressable memory. Don't try
to treat it like one.
## Address Stack
This stack primarily holds return addresses for function calls.
You normally won't need to directly interact with this stack,
but you can use `push` and `pop` to move values between the
data stack and this.
## Floating Point Stack
If you are using a build with floating point support a third
stack will be present. Floating point values are kept and
passed between words using this.
See the Floating Point chapter for more details on this.
## Tips
I recommend keeping the data stack shallow. Don't try to juggle
too much; it's better to factor definitions into shorter ones
that deal with simpler parts of the stack values than to have
a big definition with a lot of complex shuffling.
## Notes
The standard system is configured with a very deep data stack
(around 2,000 items) and an address stack that is 3x deeper.
In actual use, your programs are unlikely to ever need this,
but if you do, keep the limits in mind.
# Multiple Cores
Nga has optional support for multiple virtual cores. These
are not directly comparable to actual CPU cores, but do
allow for a degree of multitasking.
Cores share a single address space for RAM, but each gets
its own data and address stacks, as well as 24 internal
registers for private data storage. The floating point stack
is shared across all cores.
Execution is handled in a round-robin fashion. After an
instruction bundle is processed, Nga will pass control to the
next active core. Note that this means that execution will
stall when an I/O operation is running.
## Enabling
To make use of the multicore support, edit the `Makefile` and
uncomment the lines reading:
# ENABLED += -DENABLE_MULTICORE
# DEVICES += interface/multicore.retro
And rebuild.
## Scheduling
Nga switches to the next active core after execution of an
instruction bundle completes on the current core.
## Initialization
To initialize a core, pass the core identifier (0 through 7)
to `core:init`. This will zero out its internal registers and
set it to.
Example:
#4 core:init
## Starting a Core
Initialization does not activate a core. To do this, you need
to use `core:start`. Pass this the address of the word to
run and the core number.
Example:
:a [ $a c:put ] forever ;
&a #4 core:start
## Pausing a Core
Two words are provided for suspending a core. The first is
`core:pause`. Pass this the core number to pause.
Example:
#4 core:pause
The second option is intended if you need the currently
active core to pause. This word is `core:pause-current`.
Example:
core:pause-current
## Resuming a Core
To reactive a core, use `core:resume`. This takes the core
number to activate.
Example:
#4 core:resume
## Registers
Each core has 24 internal memory spaces. You can read these
with `core:rr` and modify them with `core:wr`. (These are
used identically to `fetch` and `store`). Registers are
numbered as 0 through 23.
A core can not access the registers in another core.
## Napia
The implementation here is based on the multicore model used
in Napia, the virtual machine being written for the Retro
on smaller targets. Code written to work with this will be
usable on Retro/Napia with minimal changes.
The main differences are that under Nga, this is an optional
extension, but in Napia, it is part of the standard system.
## Other Notes
On startup, execution occurs on core 0, with only core 0 being
initialized.
I/O is run on the currently active core. Since I/O is blocking,
waiting for an interaction to occur will prevent other cores
from running until the operation is complete.
# Internals: Nga Virtual Machine
## Overview
At the heart of Retro is a simple MISC (minimal instruction
set computer) processor for a dual stack architecture.
This is a very simple and straightforward system. There are
30 instructions. The memory is a linear array of signed 32
bit values. And there are two stacks: one for data and one
for return addresses.
## Instruction Table
| Stacks |
| Opcode | Muri | Full Name | Data | Address |
| ------ | ---- | ------------------ | ----- | ------- |
| 0 | .. | nop | - | - |
| 1 | li | lit | -n | - |
| 2 | du | dup | n-nn | - |
| 3 | dr | drop | n- | - |
| 4 | sw | swap | xy-yx | - |
| 5 | pu | push | n- | -n |
| 6 | po | pop | -n | n- |
| 7 | ju | jump | a- | - |
| 8 | ca | call | a- | -A |
| 9 | cc | conditional call | af- | -A |
| 10 | re | return | - | A- |
| 11 | eq | equality | xy-f | - |
| 12 | ne | inequality | xy-f | - |
| 13 | lt | less than | xy-f | - |
| 14 | gt | greater than | xy-f | - |
| 15 | fe | fetch | a-n | - |
| 16 | st | store | na- | - |
| 17 | ad | addition | xy-n | - |
| 18 | su | subtraction | xy-n | - |
| 19 | mu | multiplication | xy-n | - |
| 20 | di | divide & remainder | xy-rq | - |
| 21 | an | bitwise and | xy-n | - |
| 22 | or | bitwise or | xy-n | - |
| 23 | xo | bitwise xor | xy-n | - |
| 24 | sh | shift | xy-n | - |
| 25 | zr | zero return | n-? | - |
| 26 | ha | halt | - | - |
| 27 | ie | i/o enumerate | -n | - |
| 28 | iq | i/o query | n-xy | - |
| 29 | ii | i/o invoke | ...n- | - |
## Encoding
Up to four instructions can be packed into each memory cell.
As an example,
Opcode 4 Opcode 3 Opcode 2 Opcode 1
00000000:00000000:00000000:00000000
If we have a bundle of `duliswst`, it would look like:
st sw li du
00010000:00000100:00000001:00000010
Each `li` should have a value in the following cell(s). These
values will be pushed to the stack. E.g., `lili....` and
1, 2:
00000000:00000000:00000001:00000001
00000000 00000000 00000000 00000001 (1)
00000000 00000000 00000000 00000010 (2)
## Shifts
`sh` performs a bitwise arithmetic shift operation.
This takes two values:
xy
And returns a single one:
z
If y is positive, this shifts `x` right by `y` bits. If negative,
it shifts left.
## Queries: Memory, Stacks
The `fe` instruction allows queries of some data related to
the Nga VM state. These are returned by reading from negative
addresses:
| Address | Returns |
| ------- | ---------------------- |
| -1 | Data stack depth |
| -2 | Address stack depth |
| -3 | Maximum Image Size |
| -4 | Minimum Integer Value |
| -5 | Maximum Integer Value |
## I/O Devices
Nga provides three instructions for interacting with I/O devices.
These are:
ie i/o enumerate returns the number of attached devices
iq i/o query returns information about a device
ii i/o interact invokes an interaction with a device
As an example, with an implementation providing an output source,
a block storage system, and keyboard:
ie will return `3` since there are three i/o devices
0 iq will return 0 0, since the first device is a screen
(device class 0, version of 0)
1 iq will return 1 3, since the second device is a block
storage (class 3), with a version of 1
2 iq will return 0 1, since the third device is a keyboard
(class 1), with a version of 0
In this case, some interactions can be defined:
: c:put
i liiire..
d 0
: c:get
i liiire..
d 2
Setup the stack, push the device handle to the stack, and then use
`ii` to invoke the interaction.
A Retro system requires one I/O device (a generic output for a
single character). This must be the first device, and must have
a device class of 0.
All other devices are optional and can be specified in any order.
The currently supported and reserved device identifiers are:
| ID | Device Type | Notes |
| ---- | ---------------- | -------------------------- |
| 0000 | Generic Output | Always present as device 0 |
| 0001 | Keyboard | |
| 0002 | Floating Point | |
| 0003 | Block Storage | Raw, 1024 cell blocks |
| 0004 | Filesystem | Unix-style Files |
| 0005 | Network: Gopher | Make gopher requests |
| 0006 | Network: HTTP | Make HTTP requests |
| 0007 | Network: Sockets | |
| 0008 | Syscalls: Unix | |
| 0009 | Scripting Hooks | |
| 0010 | Random Number | |
This list may be revised in the future. The only guaranteed
stable indentifier is 0000 for generic output.
# Internals: Interface Layers
Nga provides a virtual processor and an extensible way of adding
I/O devices, but does not provide any I/O itself. Adding I/O is
the responsability of the *interface layer*.
An interface layer will wrap Nga, providing at least one I/O
device (a generic output target), and a means of interacting
with the *retro image*.
It's expected that this layer will be host specific, adding any
system interactions that are needed via the I/O instructions.
The image will typically be extended with words to use these.
# Internals: I/O
Retro provides three words for interacting with I/O. These are:
io:enumerate returns the number of attached devices
io:query returns information about a device
io:invoke invokes an interaction with a device
As an example, with an implementation providing an output source,
a block storage system, and keyboard:
io:enumerate will return `3` since there are three
i/o devices
#0 io:query will return 0 0, since the first device
is a screen (device class 0) with a version
of 0
#1 io:query will return 1 3, since the second device is
block storage (device class 3), with a version
of 1
#2 io:query will return 0 1, since the last device is a
keyboard (device class 1), with a version
of 0
In this case, some interactions can be defined:
:c:put #0 io:invoke ;
:c:get #2 io:invoke ;
Setup the stack, push the device handle, and then use `io:invoke`
to invoke the interaction.
A Retro system requires one I/O device (a generic output for a
single character). This must be the first device, and must have
a device class and handle of 0.
All other devices are optional and can be specified in any
order.
# I/O Devices
I/O devices on Nga are exposed via three instructions:
ie enumerate i/o devices
iq query i/o device for class and version
ii invoke i/o interaction
All devices are registered with the VM. How this occurs is
implementation dependent.
## Counting Devices
Use the `ie` instruction to return the number of attached devices.
i ie......
Upon running, the stack will contain the number of devices. You
can then query these by passing the device number to `iq`.
## Query Devices
Use `iq` to query an attached device. This will return two values,
a device class and a revision number.
The device class will be the top value on the stack.
## Invoking a Device
You can trigger an I/O operation by passing the device handle to
the `ii` instruction.
E.g., to display a character (ASCII code 98 in this case):
i liliii..
d 98
d 0
Be sure to pass the device handle, not the device class.
## Device Class
Ultimately devices are implementation-specific, but the
standard system provides or reserves the following:
ID | Device Class | Notes |
-----+------------------+----------------------------+
0000 | Generic Output | Always present as handle 0 |
0001 | Keyboard | |
0002 | Floating Point | |
0003 | Block Storage | Raw, 1024 cell blocks |
0004 | Filesystem | Unix-style Files |
0005 | Clock | |
0006 | | |
0007 | Network: Sockets | |
0008 | Syscalls: Unix | |
0009 | Scripting Hooks | |
0010 | Random Number | |
1000 | Image Saving | |
It must be noted here that nothing forces devices to use these
class identifiers, and one must take care to use an Nga
implementation that provides the devices they need.
## Device Revisions
Over time, the functionality a device provides may change. To allow
detection of this, the query functionality provides a revision number.
Your code can use this to ensure that the device provided supports
the level of functionality you need.
## Device Class Details
### 0000: Generic Output
Supported by all Nga implementations. This is required to be the
first device, and is the only one guaranteed to be provided. It
consumes a value from the stack, writing to to the host-specific
output. (This does not need to be a screen).
### 0001: Keyboard
Read and return a keypress.
Consumes no data, returns a single value representing the
character that was read.
No subcommands are defined.
### 0002: Floating Point
The current revision is 1.
It currently provides:
n:to-float (n-_f:-n)
s:to-float (s-_f:-n)
f:to-number (f:a-__-n)
f:to-string (f:n-__-s)
f:+ (f:ab-c)
f:- (f:ab-c)
f:* (f:ab-c)
f:/ (f:ab-c)
f:floor (f:ab-c)
f:ceiling (f:f-f)
f:sqrt (f:f-f)
f:eq? (f:ab-c)
f:-eq? (f:ab-c)
f:lt? (f:ab-c)
f:gt? (f:ab-c)
f:depth (-n)
f:dup (f:a-aa)
f:drop (f:a-)
f:swap (f:ab-ba)
f:log (f:ab-c)
f:power (f:ab-c)
f:sin (f:f-f)
f:cos (f:f-f)
f:tan (f:f-f)
f:asin (f:f-f)
f:acos (f:f-f)
f:atan (f:f-f)
f:push (f:f-)
f:pop (f:-f)
f:adepth (-n)
### 0003: Block Storage
Reserved for future use.
### 0004: Filesystem
Currently at revision 0.
This implements a device providing traditional Unix-like files.
Takes a value indicating an operation, and each operation takes
additional values.
| Operation | Stack | Action |
| --------- | ----- | -------------------------------- |
| 0 | sm-h | Open a file |
| 1 | h- | Close a file |
| 2 | h-c | Read a byte from a file |
| 3 | ch- | Write a byte to a file |
| 4 | h-n | Return current pointer into file |
| 5 | nh- | Move pointer in a file |
| 6 | h-n | Return the size of a file |
| 7 | s- | Delete a file |
| 8 | h- | Flush pending writes |
### 0010: Random Number Generator
This is currently at revision 0.
On invocation, this returns a random number.
## Implementation Details (C)
On the C implementation, each I/O device has the needed support
functions defined, then a query function and invocation function
defined.
As an example, to add a device that has two functions, I might do:
void one() {
stack_push(100);
}
void two() {
stack_push(200);
}
Handler device_actions[] = {
one, two
}
void io_device() {
device_actions[stack_pop()]();
}
void query_device() {
stack_push(0); /* Revision */
stack_push(1234); /* Device Class */
}
Then add pointers to `io_device` to `IO_deviceHandlers` and
`query_device` to `IO_queryHandlers` and increase the `NUM_DEVICES`
by one.
You will then need to write a set of Retro words to use the new
device.
:device:one #1 #1234 io:scan-for io:invoke ;
:device:two #2 #1234 io:scan-for io:invoke ;
Rebuild the VM, adding these to image.
# Internals: The Retro Image
The actual Retro language is stored as a memory image for Nga.
## Format
The image file is a flat, linear sequence of signed 32-bit
values. Each value is stored in little endian format. The
size is not fixed. An interface should check when loading to
ensure that the physical image is not larger than the emulated
memory.
## Header
The image will start with two cells. The first is a liju....
instruction, the second is the target address for the jump.
This serves to skip over the rest of the data and reach the
actual entry point.
This is followed by a pointer to the most recent dictionary
header, a pointer to the next free address in memory, and
then the Retro version number.
| Offset | Contains |
| ------ | --------------------------- |
| 0 | lit call nop nop |
| 1 | Pointer to main entry point |
| 2 | Dictionary |
| 3 | Heap |
| 4 | Retro version |
The actual code starts after this header.
The version number is the year and month. As an example,
the 12.2019.6 release will have a version number of
`201906`.
## Layout
Assuming an Nga built with 524287 cells of memory:
| RANGE | CONTAINS |
| --------------- | ---------------------------- |
| 0 - 1024 | Retro Core kernel |
| 1025 - 1535 | token input buffer |
| 1536 + | start of heap space |
| ............... | free memory for your use |
| 506879 | buffer for string evaluate |
| 507904 | temporary strings (32 * 512) |
| 524287 | end of memory |
The buffers at the end of memory will resize when specific
variables related to them are altered.
# Calling Retro from C
The C implementation of Retro provides several functions for
interacting with code written in Retro.
## Dictionary
The dictionary is a linked list, with a pointer to the most
recent entry stored in address 2.
You can access the fields for each entry using:
d_link
d_xt
d_class
d_name
Each takes a dictionary header address (the "dictionary token")
and returns a pointer to the Retro address for the desired data.
To find a dictionary token, use `d_lookup`. This takes the address
of the dictionary to search (`memory[2]` in most cases) and the
name of the word to find.
There is also `d_xt_for()` which takes a name and a dictionary
pointer and returns the execution token for the specified word.
## Strings
Like C, Retro uses NUL terminated strings. But, since all
addressing is 32-bit (or 64-bit, depending on your configuration),
some conversion is needed.
To get a C version of a string, use `string_extract()`. This takes
a Retro address and returns a pointer to a C string.
Example:
// Get the name of the most recently defined word
string_extract(d_name(memory[2]));
To push a C string into Retro memory, use `string_inject()`. This
takes a C string and a Retro address.
// Copy a string to the TIB buffer
string_inject("hello", 1024);
## Stack
You can push values to the stack with `stack_push()` and pop them
off with `stack_pop()`.
## Interacting
If you have a word named `hello` that you wish to run:
execute(d_xt_for("hello", memory[2]));
If you want to evaluate a token as if it was typed into a Retro
listener:
string_inject("token", 1024);
stack_push(1024);
execute("interpret", memory[2]);
The `interpret` word handles things like sigils, so this is
needed if you want to run something that needs those.
# Historical Papers and Notes
## On the Naming of Retro
Taken from http://lists.tunes.org/archives/tunes-lll/1999-July/000121.html
On Fri, Jul 30, 1999 at 07:43:54PM -0400, Paul Dufresne wrote:
> My brother did found it funny that Retro is called like that.
> For him retro means going back (generally in time) so this
> does not looks like a name of a OS to come. So he'd like to
> know from where the name came.
Heheh, here's the story: When I started playing with OS stuff
last year (not seriously), I was reading about some old things
like FORTH and ITS, dating back to the 1960's and 70's. The
past few years in America, there's been a revival of disco
music (along with bell bottoms, platform shoes, and all that
crap) and they call it "retro". Now, my OS was named by
musicians.. I was telling a fellow musician about my ideas,
how it would be cool to have a small OS that isn't bloated and
unmanageable like Windows... go back to the 70's and resurrect
a line of software that died out. He goes "hmm.. sounds kinda
retro.."
I think it sounds kinda rebellious, which is a Good Thing now
that everybody hates the M$ empire. :) It seems like other
people are as sick of the future as I am. Look at TUNES, the
idea there isn't to make some great new invention, just take
some decades-old ideas and combine them in one OS. The first
time I saw Knuth's "Art of Computer Programming" in the library
I thought "god that looks old.. 1973!!! nevermind.." Now it's
my programming bible. Find me something better published in
the 90's.. if such a thing exists, it'll be like a needle in a
haystack. "Newer" doesn't necessarily mean "better".
New cars = flimsier
New farming methods = more devastating
New version of Netscape = more bloat, more bullshit
One thing is better now: computer hardware. Give me 70's
software on 90's and 00's hardware :)
- Tom Novelli <tcn@tunes.org>
## The Design Philosophy of Retro Native Forth
Computer software is a technology in its infancy, a mere fifty years
old. The last 25 years in particular have seen an explosion in the
software business. However, software has seen little innovation while
hardware technology has improved phenomenally (notwithstanding the advent
of lousy slave-made parts). Proven software techniques of forty years ago
have yet to reach widespread use, in deference to the "latest and
greatest" proprietary solutions of dubious value. Thanks to agressive
marketing, we make huge investments in these dead-end technologies
(through our businesses and governments, if not personally) and we end up
with a reliance on a heap of complicated, error-prone, poorly understood
junk software.
Complexity will dominate the software industry for the foreseeable
future. The Retro philosophy is a simple alternative for those willing to
make a clean break with legacy software. A Retro system can communicate
with other systems, but it won't run much legacy software, especially
proprietary software without source code. An emulation layer could be
added, but doing so would defeat the purpose of a simple operating system.
I think TCP/IP support is all the compatibility that's needed.
At first Retro will appeal to computer hobbyists and electronic
engineers. Once the rough edges are smoothed out, it could catch on with
ordinary folks who don't like waiting five minutes just to check their
email (not to mention the long hours of setup and maintenance). Game
programmers who take their craft seriously may also be interested.
Businesses might even see a use for it, if the managers decide it's more
cost-effective to carefully design software for specific needs, rather
than buying off-the-shelf crap and spending countless manhours working
around the bugs. Since it's not practical for businesses to make a clean
break, my advice is to run Retro (and its ilk) on separate machines
connected by a network. Retro is efficient enough to run on older
machines that would otherwise sit idle, being too slow for the latest
Microsoft bloatware (or Linux, for that matter).
I strive to avoid the extraneous. That applies even to proven
technologies, if I don't need them. If my computer isn't set up for
people to log in over the network, I don't want security features; they
just get in the way. If I'm only running programs I wrote, I should be
able to run them with full access to the hardware; I don't need protection
from viruses. If I download something I don't trust, then I can run it in
an isolated process, which is customary with Unix and kin. But that's not
core functionality. All that's needed is the flexibility to add things
like security, graphical interfaces, and distributed processing - if the
need ever arises.
In programming languagues, I was misled. It's the Tower of Babel all
over again. The thousands of languages in existence all fall into a
handful of archetypes: Assembler, LISP, FORTRAN and FORTH represent the
earliest descendants of nearly all languages. I hesitate to name a
definitive "object-oriented" language, and here's why: Object-Oriented
programming is just a technique, and any language will suffice, even
Assembler. The complexites of fancy languages like Ada and C++ are a
departure from reality -- the reality of the actual physical machine.
When it all boils down, even LISP, FORTRAN and FORTH are only extensions
of the machine.
I chose FORTH as the "native tongue" of Retro. LISP, FORTRAN, and
other languages can be efficiently implemented as extensions of FORTH, but
the reverse isn't so efficient. Theoretically all languages are
equivalent, but when design time, compilation time, and complexity are
accounted for, FORTH is most efficient. FORTH also translates most
directly to the hardware. (In fact, FORTH has been implemented in
hardware; these "stack machines" are extremely efficient.) FORTH is also
the easiest language to implement from scratch - a major concern when
you're trying to make a clean break. So with simplicity in mind, FORTH
was the obvious choice.
I'm perfectly happy working with text only, and I go to great lengths
to avoid using the standard graphical environments, which have major
problems: windows, pulldown menus, and mice. Windows can't share the
screen nicely; that idea is hopeless. Pulldowns are tedious. Mice get in
the way of typing without reducing the need for it; all they give me is
tendonitis. Their main use is for drawing.
Some of my favorite interfaces: Telix, Telegard BBS, Pine, Pico, Lynx,
and ScreamTracker. All "hotkey" interfaces where you press a key or two
to perform an action. Usually the important commands are listed at the
bottom of the screen, or at least on a help screen. The same principles
apply to graphical interfaces: use the full screen, except for a status
and menu area on one edge. Resist the temptation to clutter up the
screen.
As for switching between programs, the Windows methods suck; the only
thing worse is Unix job control (jobs, fg, and such). The Linux method is
tolerable: Alt-Arrows, Alt-F1, Alt-F2, etc. Still, things could be
better: F11 and F12 cycle back and forth through all open programs; Alt-F1
assigns the currently selected program to F1, and likewise for the other
function keys. Programs just won't use function keys - Control and Alt
combinations are less awkward and easier to remember, besides. I'll also
want a "last channel" key and a "task list" key; maybe I'll borrow those
stupid Win95 keys. The Pause key will do like it says - pause the current
program - and Ctrl-Pause (Break) will kill it.
One more thing: consistency. I like programs to look different so I
can tell them apart, but the keys should be the same as much as possible.
Keys should be configured in one place, for all programs. Finally,
remember the most consistent interface, one of the few constants
throughout the history of computing - the text screen and keyboard, and
the teletypewriter before that. Don't overlook it.
More to come, maybe... :)
"If it's on line, it's a work in progress."
Tom Novelli, 3/4/2000
## Metacompilation and Assembly
Retro 10 and 11 were written in themselves using a metacompiler.
I had been fascinated by this idea for a long time and was able
to explore it heavily. While I still find it to be a good idea,
the way I ended up doing it was problematic.
The biggest issue I faced was that I wanted to do this in one
step, where loading the Retro source would create a new image
in place of the old one, switch to the new one, and then load
the higher level parts of the language over this. In retrospect,
this was a really bad idea.
My earlier design for Retro was very flexible. I allowed almost
everything to be swapped out or extended at any time. This made
it extremely easy to customize the language and environment, but
made it crucial to keep track of what was in memory and what had
been patched so that the metacompiler wouldn't refer to anything
in the old image during the relocation and control change. It
was far too easy to make a mistake, discover that elements of
the new image were broken, and then have to go and revert many
changes to try to figure out what went wrong.
This was also complicated by the fact that I built new images
as I worked, and, while a new image could be built from the last
built one, it wasn't always possible to build a new image from
the prior release version. (Actually, it was often worse - I
failed to check in every change as I went, so often even the
prior commits couldn't rebuild the latest images).
For Retro 12 I wanted to avoid this problem, so I decided to go
back to writing the kernel ("Rx") in assembly. I actually wrote
a Machine Forth dialect to generate the initial assembly, before
eventually hand tuning the final results to its current state.
I could (and likely will eventually) write the assembler in
Retro, but the current one is in C, and is built as part of the
standard toolchain.
My VM actually has two assemblers. The older one is Naje. This
was intended to be fairly friendly to work with, and handles
many of the details of packing instructions for the user. Here
is an example of a small program in it:
:square
dup
mul
ret
:main
lit 35
lit &square
call
end
The other assembler is Muri. This is a far more minimalistic
assembler, but I've actually grown to prefer it. The above
example in Muri would become:
i liju....
r main
: square
i dumure..
: main
i lilica..
d 35
r square
i en......
In Muri, each instruction is reduced to two characters, and the
bundlings are listed as part of an instruction bundle (lines
starting with `i`). This is less readable if you aren't very
familiar with Nga's assembly and packing rules, but allows a
very quick, efficient way of writing assembly for those who are.
I eventually rewrote the kernel in the Muri style as it's what
I prefer, and since there's not much need to make changes in it.
## The Path to Self Hosting
Retro is an image based Forth system running on a lightweight
virtual machine. This is the story of how that image is made.
The first Retro to use an image based approach was Retro 10.
The earliest images were built using a compiler written in
Toka, an earlier experimental stack language I had written.
It didn't take long to want to drop the dependency on Toka,
so I rewrote the image compiler in Retro and then began
development at a faster pace.
Retro 11 was built using the last Retro 10 image and an
evolved version of the metacompiler. This worked well, but
I eventually found it to be problematic.
One of the issues I faced was the inability to make a new
image from the prior stable release. Since I develop and
test changes incrementally, I reached a point where the
current metacompiler and image required each other. This
wasn't a fatal flaw, but it was annoying.
Perhaps more critical was the fragility of the system. In
R11 small mistakes could result in a corrupt image. The test
suite helped identify some of these, but there were a few
times I was forced to dig back through the version control
history to recover a working image.
The fragile nature was amplified by some design decisions.
In R11, after the initial kernel was built, it would be
moved to memory address 0, then control would jump into the
new kernel to finish building the higher level parts.
Handling this was a tricky task. In R11 almost everything
could be revectored, so the metacompiler had to ensure that
it didn't rely on anything in the old image during the move.
This caused a large number of issues over R11's life.
So on to Retro 12. I decided that this would be different.
First, the kernel would be assembly, with an external tool
to generate the core image. The kernel is in `image/retro.muri`
and the assembler is `Muri`. To load the standard library, I
wrote a second tool, `Retro-extend`. This separation has allowed
me many fewer headaches as I can make changes more easily and
rebuild from scratch when necessary.
But I miss self-hosting. So last fall I decided to resolve
this. And today I'm pleased to say that it is now done.
There are a few parts to this.