Communication between processes happens through message passing. If you know the address of a process, there is a system call for sending it messages, which can be any Swail object; conversely you can perform a system call to ask the kernel whether the current process has received any messages and/or block until you have received one. There is a standard protocol used by built-in processes, which you need to follow if you want to communicate with built-in processes. For your own processes, you are completely free to design your own message protocol.
XukutOS manual → Processes: more information about processes and process addresses
XukutOS manual → System calls: an overview of all system calls
When a process sends an object as a message, and another receives it, both processes will have access to the sent object. Thus, either process could potentially perform modifications, so that the other process sees the object changing from underneath it. Code needs to be very carefully designed to handled such modifications, and indeed much of XukutOS has not been designed to handle this. Moreover, memory operation reordering can mean that changes appear out of serialized order, making everything that much more difficult to handle. Therefore, unless you really know what you are doing, you should not modify the sent object from the sending process, and if the receiving process modifies the object, the sending process should not read the object after sending. Unless explicitly noted, built-in processes will only read received messages and not read or modify sent messages.
Any receive operation done by a process will receive all the messages sent; the kernel does not support creation of multiple mailboxes for one process or filtering of messages. This is by design: there are many ways to specify a `select'-like operation, which you should be able to do per-process. If you want multiple mailboxes, create multiple processes that forward their incoming messages. If you want to filter messages, you can store the rejected messages in a global `mailbox' variable. Indeed, those two abilities are (will be) built-in functionality available in Swail.
The kernel does not notify after sending when a message could not be delivered, or indicate when receiving which process sent the message. The lack of these features makes transparent forwarding much simpler: when sending a message to a forwarder is successful, but forwarding fails, the behaviour should be (mostly) indistinguishable from failing to send a message to a forwarder; similarly the sender of a forwarded message should be distinguished from the forwarder of that message. Of course, it is possible to build read receipts, time-outs when waiting for replies and to send your process address along with your message. (And if so, make sure to share them with the rest of us!)
An important process-side abstraction on messages provided by Swail is the mailbox. Instead of working with "the list of messages since the last systemcall to `receive'", each process has a mailbox of unopened messages. Sending messages adds them to the receiver's mailbox, and the receiver can open a subset of messages from the mailbox, leaving the rest to be opened at a later time.
All of these functions use the dynamically-bound syscall wrappers, in case you need to intercept syscalls.
XukutOS manual → System calls: an overview of all system calls
Message passing in XukutOS does not have many guarantees on the order in which messages will arrive. For example, if process A sends two messages to process B, it is possible process B receives the messages in opposite order. For example, the messages may be sent over an unreliable link that went down during the sending of the first message, which was therefore resent after the second message was already received. For communication between processes on the same computer, the kernel guarantees that sending and receiving act as if the computer executes one process at a time, switching between processes randomly.
However, I am not an expert on concurrency, so I might have made a mistake in these descriptions. In any case, the intention is approximately the level of consistency Erlang lets you get away with in an Erlang cluster.
Share the object `message' with the given process, putting it in the process's mailbox.
If the process address does not refer to an existing process, nothing happens.
If the process address refers to a process that is running on this computer, the message will be added to the mailbox of that process: the next receive operation of that process will have access to that message.
Planned: If the process address refers to a process that is available through the network on another computer, the system will do its best to send the message to that process. Depending on the connection information in the process address, "doing its best" can vary from doing one attempt at delivery and giving up at the first sign of trouble, to guaranteed delivery as long as the network connection will eventually be up for an arbitrary amount of time. (Of course, if the computer explodes between sending a message from one process on this computer and that process receiving it, that process will not be able to receive it either: this limitation is not really unique for sending messages over the network.)
This is the function called by the library when sending a message to a process on this computer, and performs the systemcall to do so.
XukutOS manual → System calls: an overview of all system calls
You can rebind this definition locally, for example if you want to test that your code will indeed send a message.
Remove all messages from the mailbox, giving them as a list.
If the mailbox is empty at the moment of calling, the process waits for up to the given time (TODO: specify which units we use for this time). When messages have arrived in the mailbox, these are returned and the waiting stops. Otherwise, when the waiting time is up, the result is an empty list.
The order of messages in the list is oldest last.
After performing this operation, the mailbox will be empty.
Add the given message to this process's mailbox.
One possible implementation is `(xukut:message:send! (xukut:sys:self) msg)', although more efficient implementations are definitively possible.
Remove all messages satisfying the predicate `filter' from the mailbox, giving them as a list. The filter is a function taking one argument, a message from the mailbox, and if its result converts to #tt, the message is included in the list of returned messages.
For example, this will allow you to only get messages of a specific format, or only those sent by a given process.
If no message in the mailbox satisfies the filter, the process waits for up to the given time (TODO: specify which units we use for this time). When messages have arrived in the mailbox that satisfy the filter, these are returned and the waiting stops. Otherwise, when the waiting time is up, the result is an empty list.
The order of messages in the list is oldest last.
It is unknown what happens if you try to receive messages in `filter'.
Remove the oldest message from the mailbox and return it.
If the mailbox is empty, the process waits for up to the given time (TODO: specify which units we use for this time). When messages arrive during waiting, the oldest is returned and the waiting stops. Otherwise, when the waiting time is up, the result is `else'.
Pattern-match messages in the mailbox according to `match-args'. The result is the result of pattern-matching for the first message that matches one of the cases.
XukutOS manual → Swail → Quote: more information about quoting and matching
If none of the messages in the mailbox match, the process waits for up to the given time (TODO: specify which units we use for this time). When messages arrive during waiting, each is pattern-matched in turn and, if successful, the result is returned. Otherwise, when the waiting time is up, the result is the default case for the pattern-match.
When using this macro as the only way to receive messages, you should have a fallback case discarding any message that does not match any "correct" pattern (perhaps replying with a complaint to the offending sender). Otherwise, all the non-matching messages will stick around and waste memory.
Interactively receive messages. `fold' should have type `(fold curr-result curr-msg stop go-next wait-next) -> irr.'. It will be repeatedly applied to the result up to now and the oldest unapplied message in the mailbox. `stop', `go-next' and `wait-next' all have type `(fn result (keep? 'default #ff)) -> irr.', and `fold' should call one of these to indicate how the receive operation should continue.
For each message, `fold' is called in order to determine what to do next: by calling `(stop result)', we return `result' from `fold-receive!'; by calling `(go-next result)' or `(wait-next result)', we continue by processing the next message. The difference is in what to do when all messages have been processed: `go-next' will return from `fold-receive!' while `wait-next' will wait for more messages to arrive.
For example, you can implement `receive-oldest!' as:
(def receive-oldest! (else (waiting-time 'default 0)) (fold-receive! (fn (curr-result curr-msg stop go-next wait-next) (stop curr-msg)) else waiting-time))
and `receive-filtered!' as:
(def receive-filtered! (filter (waiting-time 'default 0)) (fold-receive! (fn (curr-result curr-msg stop go-next wait-next) (if (filter curr-msg) (go-next (cons curr-msg curr-result)) (go-next curr-result #tt))) () waiting-time))
The following imperative algorithm is a precise description of what goes on.
- let the message list be the current contents of the mailbox with the oldest message first, the new mailbox be an empty list and the current result be `init'.
- when we are done, set the process mailbox to the message list reversed (so the oldest message is last) concatenated to the mailbox; return from `fold-receive!' the current result.
- if the message list is empty and `init-wait' is false, we are done.
- if the message list is empty and `init-wait' is true, wait up to the waiting time until messages are added to the process mailbox, and set the message list to the added messages.
- repeat the following steps:
- if the message list is empty, we are done.
- let the current message be the head of the message list (which is not empty) and the message list be the tail of the message list.
- call `fold' with the current result, the current message, `stop', `go-next' and `wait-next'.
- if `fold' returns a result without calling `stop', `go-next' and `wait-next', call `stop' on that result (and the default value of `keep?').
- continue until one of the three functions `stop', `go-next' or `wait-next' was called, with an argument for `result' and `keep?'.
- the current result is `result'.
- if `keep?' is true, add the current message to the head of the new mailbox (making it the newest message in the mailbox).
- if `stop' was called, we are done.
- if `wait-next' was called and the message list is empty, wait up to the (remaining) waiting time until messages are added to the process mailbox, and set the message list to the added messages.
- otherwise, go back to the start of the loop
It is unknown what happens if you try to receive messages in `fold'.
This is the function called by the library when receiving messages, and performs the systemcall to do so.
XukutOS manual → System calls: an overview of all system calls
You can rebind this definition locally, for example if you want to enter predetermined data when testing some code. You should probably also re-bind `mailbox' at the same time.
This variable contains the mailbox at the moment the previous receive operations finished. The "real" contents of the mailbox at any point in time are given by the concatenation of `mailbox' with the result of `((dyn receive!) 0)'.
You can rebind this definition locally, for example if you want to enter predetermined data when testing some code. You should probably also re-bind `receive!' at the same time.
This section describes the standard protocol used by built-in processes, which you need to follow if you want to communicate with built-in processes. For your own processes, you are completely free to design your own message protocol. The protocol distinguishes the start of a conversation and replies: the start of a conversation needs to fit some extra criteria to ensure the receiver of that message knows what is going on. The sender of the start of the conversation should know which protocols the receiver supports; if not then what can you possibly hope to accomplish by sending that message?
In general, a message will match the following structure:
(match message (error) (,msg-id ,version ,nonce ,sender ,@rest) _)
The list structure, and (if not mentioned otherwise) the contents of each message, are newly created at the moment of sending, excepting the immutable objects such as numbers (it is not specified whether that kind of objects are newly created in general).
The `msg-id', `version' and `nonce' put together inform the receiver what the rest of the message will look like. In general, `msg-id' and `nonce' will be an easily comparable object like numbers or symbols; `version' will be an integer; `sender' will be a process address.
`msg-id' is used by the sender to identify which kind of message this is. XukutOS tries to avoid re-using `msg-id' for different purposes, but independently written code will not be able to avoid this completely. In co-operation with the `nonce', it should be possible to avoid ambiguity.
The `version' is used per value of `msg-id' to indicate which extra data is carried by the message. Each message with equal `msg-id' but higher `version' will carry something extra compared to messages with a lower `version'. That is: every call to `(index message n)' that gave sensible information for a `message' with version `v', will give the same information for versions `w > v'. The length of the message may vary with different versions, ensure your pattern-matches discard the tail of the message in order to be forwards-compatible. It is also possible that version numbers increment without changing the representation of data carried in the message: this should allow some level of protocol negotiation.
The `nonce' provides the other half of the ability to distinguish messages: where the `msg-id' is more or less chosen by the sender, the `nonce' is more or less chosen by the receiver. In replies, the replying party will contain the nonce provided (in a protocol-dependent way) by the previous sender. Thus, the receiver of the reply is able to determine which message the reply is in reply to. At the start of a conversation, the choice of `nonce' is made in the protocol itself.
The `sender' is a process address object indicating which process should receive the reply to this message. If the address is `nil', no reply is expected.
XukutOS manual → Processes: more information about processes and process addresses
In summary, when starting a conversation, the fields are chosen as follows:
- msg-id: by the protocol that the sender knows the receiver supports
- version: by the protocol that the sender supports
- nonce: by the protocol that the sender knows the receiver supports
- sender: either the result of the `xukut:self-id' call, or `nil'
In a reply, the fields are determined as follows:
- msg-id: by the sender (as determined in the protocol chosen by the start of the conversation)
- version: by the protocol that the sender supports
- nonce: by the receiver (as determined in the protocol chosen by the start of the conversation)
- sender: either the result of the `xukut:self-id' call, or `nil'
Finally, the `rest' is a list with in principle arbitary data whose contents depend on the kind of message (determined above). Programs should be prepared for the length of this list to be more than their expected amount of data, discarding anything that they are not ready to handle.
This is an overview of the messages sent and received by the built-in processes. They are categorized by process, `msg-id', direction and version. For direction, `i' (in) means sent *to* that process, `o' (out) means sent *by* that process.
The kernel is not a real process and can therefore never receive a message. Use a system call to communicate information with the kernel.
(1 ,version ,nonce 0 ,interrupt-code ,interrupt-data ,@rest)
Sent to a process after it performs a `subscribe' system call, whenever the specified interrupt has occurred.
* `nonce' is the nonce passed as an argument to the `subscribe' system call
* `interrupt-code' is a number containing a machine-specific interrupt code
* `interrupt-data' is a span of machine-specific interrupt data
(18 ,version ,nonce 0 ,pid ,@rest)
Sent to a process after it performs a `monitor' system call, when the specified process stops.
* `nonce' is the nonce passed as an argument to the `monitor' system call
* `pid' is the process id of the process that stopped
(3 ,version 0 ,sender ,drawer-nonce (,x ,y ,w ,h) ,@rest)
A process wants to reserve a new area on the screen for drawing to it, typically a window. The area is rectangular, top left corner is `(,x . ,y)', width is `w', height is `h' (all non-negative). The compositor will send responses to this message with message `4o' to `sender', using the specified `drawer-nonce'.
If the `sender' + `drawer-nonce' combination has been sent before, the area will be updated rather than created, or equivalently, any previous areas with the same `sender' + `drawer-nonce' are deleted.
(4 ,version ,nonce ,sender ,list-drawings ,@rest)
A process wants to fill (some of) its reserved area (see message `3i') with some pixels. It is not necessarily the case that all these pixels will appear on the screen: if other areas are higher priority, they will be drawn "on top". The nonce used in this message is the nonce associated to the area, as received in a previous `4o' message. (You can send as many messages `4i' as you want, after you have received the initial `4o'.)
The list of drawings is a list whose elements have the form `((,x ,y ,w ,h) . ,span)', where the four coordinates indicate the location relative to the area indicated by the `nonce', and `span' contains `w' x `h' x 4 bytes of pixels.
(4 ,version ,nonce ,compositor-id ,area-nonce ,list-rects ,@rest)
The compositor asks the process to answer with message `4i', for the indicated rectangles. The nonce is the `drawer-nonce' of message `3i' and the answer, message `4i', should use `area-nonce'. The `area-nonce' and `drawer-nonce' will not change and stay in 1:1 correspondence.
The list of drawings is a list whose elements have the form `(,x ,y ,w ,h)', where the four coordinates indicate the location relative to the area indicated by the `nonce'.
(9 ,version ,nonce ,sender ,@rest)
The area with given nonce will not be drawn to anymore and should be removed from the screen.
(10 ,version 0 ,sender ,nonce ,condition-context ,@rest)
The sending process has encountered a condition that it cannot handle. It asks the debugger to respond with message 11o, which contains code that will be used to resolve the condition.
(11 ,version ,nonce ,sender ,handler ,@rest)
Response to message 10i using the same `nonce'. The handler is a function to be called as `(handler condition-context)'.
(2 ,version 0 ,subscriber ,subscriber-nonce ,modifiers ,@rest)
A process wants to receive key updates. After receiving this message, the keyboard driver will start sending message 6o to the subscriber. (Planned: Maybe you can request a message 6o for all keys currently pressed?)
The list element for modifiers was added in version 1. On version 0, this will be interpreted as `nil'.
(6 ,version ,nonce ,keyboard-id ,scancode ,down? ,modifiers ,@rest)
The keyboard driver sends this message to the subscribed processes whenever a key on the keyboard changes state.
The scan code is a number with a device-specific key ID. `down?' is a boolean, `tt' if the key became pressed and `ff' if the key became released.
The modifiers are a list of scan codes that are being pressed at the moment of press/release, out of the list of modifiers requested in message 2i. You can use this to implement modifier keys.
In principle, two subsequent messages of type 6o will not have the same value for scancode and down?.
The list element for modifiers was added in version 1.
Some more details on working with the keyboard in XukutOS.
(12 ,version 0 ,sender ,event ,text ,@rest)
Record the fact that `sender' experienced a certain event. The `event' is an arbitrary object. The `text' is a human-readable representation of the event. The specific meaning of either is left very vague and up to the sender, in the expectation that the system develops some good conventions.
(2 ,version 0 ,subscriber ,subscriber-nonce ,@rest)
A process wants to receive mouse clicks. After receiving this message, the mouse driver will start sending message 7o to the subscriber. (Planned: Maybe you can request a message 7o for the current mouse position? Or to get an update of the mouse position?)
(7 ,version ,nonce ,mouse-id ,x ,y ,buttons ,@rest)
The mouse driver sends this message to the subscribed processes whenever a mouse button is pressed.
The buttons are sent as a list of numbers, each number being associated to a specific mouse button (in a device-dependent way).
(13 ,version ,nonce ,sender ,span ,@rest)
A UART received some data, as contained in the `span'.
(5 ,version ,nonce ,usb-id ,span ,@rest)
The USB driver sends various other drivers a notification when "interesting" events happen. Which drivers and events this means, is currently hard-coded, mostly because I can't be bothered to implement something smarter.
The span is specific to this event, generally it will be along the lines of "the data recently sent by the device".
(3 ,version 0 ,sender ,drawer-nonce ,@rest)
A process wants to create a new window. The window manager allocates an area of the screen to put the window on, and sends keyboard/mouse events to the window if it is in focus.
The dimensions of the window and the compositor's nonce are indicated through a subsequent message compositor `4o' (not necessarily sent by the window manager process, but with the correct nonce).
The process sending this message starts receiving the following messages (not necessarily from the window manager process, but with the correct nonce): compositor `4o', keyboard `6o', mouse `7o', window `8o'. You can reply to the sender of these messages in the same way you would to e.g. a compositor's `4o' that is actually sent by the compositor.
(8 ,version ,nonce ,sender (,x ,y ,w ,h) ,@rest)
The window manager has resized (and/or moved) the window. The new area is rectangular, top left corner is `(,x . ,y)', width is `w', height is `h' (all non-negative). Expect a message `4o' using the new dimensions.
(9 ,version ,nonce ,sender ,@rest)
The user wants the window to close. The receiver should clean up its activities relating to this window (e.g. the editor will quit) and send message 9i to the compositor.
TODO: consider the semantics of this message a bit better!
Any questions? Contact me:
By email at vierkantor@vierkantor.com