💾 Archived View for gemini.complete.org › using-filespooler-over-nncp captured on 2024-08-31 at 12:08:33. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2024-07-09)
-=-=-=-=-=-=-
NNCP[1] is a powerful tool for building Asynchronous Communication[2] networks. It features end-to-end Encryption[3] as well as all sorts of other features; see my NNCP Concepts[4] page for some more ideas.
2: /asynchronous-communication/
NNCP has two fundamental communcation types:
You might say, "this sounds a lot like Filespooler", and you're right. I designed Filespooler to be a generic tool, and it was specifically designed because of some pain points I had with NNCP. Specifically:
Filespooler[5] has neither of those limitations. It makes a perfect companion to NNCP, in fact. Let's take a look at how.
There are two easy enough ways to get data to a Filespooler queue using NNCP:
1. `nncp-file` can directly write it into the queue's jobs directory, as described at Guidelines for Writing To Filespooler Queues Without Using Filespooler[6].
2. Or, you can pipe the prepared job to `nncp-exec` that will call `fspl queue-write` on the remote.
6: /guidelines-for-writing-to-filespooler-queues-without-using-filespooler/
Since nncp-file's writing algorithm is safe and compatible with Filespooler, you can really choose either option. I tend to prefer the second just because it frees me from ever having to generate filenames and such.
First, we need to grant permission to run `fspl queue-write`. On the receiver, we'll edit `nncp.hjson` to allow a specific sender to run this command. In the `neigh` block for that sender, add something like this:
exec: {
enqueue: ["fspl", "queue-write", "/srv/nncp/queue"]
}
Of course, use whatever path is appropriate there, taking care to have correct permissions (or use sudo). You can easily support multiple queues this way by giving them each a unique NNCP command name and path.
Now, on the sender, you would do something like this:
echo hi | fspl prepare -s ~/seqfile -i - | \ nncp-exec othernode enqueue
And that's it! Simple!
This is also pretty easy. To make this work, on the receiver, the incoming path has to include the queue (or, at the very least, the jobs subdirectory within it). You get a little less control over things this way.
On the receiver side, set `incoming` within the `neigh` section to an appropriate path.
On the sending side, you can use something like this:
echo hi | fspl prepare -s ~/seqfile -i | \ nncp-file - "othernode:queuepath/jobs/$(fspl gen-filename)"
The `fspl gen-filename` command will print a random filename that mathes the Filespooler pattern to stdout. This allows us to use it as a target filename. Since `nncp-toss` will write the file in a safe manner, this is all it takes to deliver a queue file to NNCP with Filespooler!
Another way to use Filespooler is to allow you to pass environment variables along with nncp-exec requests, even if you never use a queue. See Using Filespooler Without Queues to Pass More Metadata[7] for more info on this technique.
7: /using-filespooler-without-queues-to-pass-more-metadata/
--------------------------------------------------------------------------------
8: /building-an-asynchronous-internet-optional-instant-messaging-system/
I loaded up this title with buzzwords. The basic idea is that IM systems shouldn't have to only use the Internet. Why not let them be carried across LoRa radios, USB sticks, local Wifi networks, and yes, the Internet? I'll first discuss how, and then why.
9: /using-filespooler-without-queues-to-pass-more-metadata/
One frustration people sometimes have with ssh or NNCP[10] is that they'd like to pass along a lot of metadata to the receiving end. Both ssh and nncp-exec allow you to pass along command-line parameters, but neither of them permit passing along more than that. What if you have a whole host of data to pass? Maybe a dozen things, some of them optional? It would be very nice if you could pass along the environment.
11: /guidelines-for-writing-to-filespooler-queues-without-using-filespooler/
Filespooler[12] provides the `fspl queue-write` command to easily add files to a queue. However, the design of Filespooler intentionally makes it easy to add files to the queue by some other command. For instance, Using Filespooler over Syncthing[13] has Syncthing do the final write, the nncp-file (but not the nncp-exec) method in Using Filespooler over NNCP[14] had NNCP do it, and so forth.
13: /using-filespooler-over-syncthing/
14: /using-filespooler-over-nncp/
15: /introduction-to-filespooler/
It seems that lately I've written several shell implementations of a simple queue that enforces ordered execution of jobs that may arrive out of order. After writing this for the nth time in bash, I decided it was time to do it properly. But first, a word on the *why* of it all.
16: /one-to-many-with-filespooler/
In some cases, you may want to use Filespooler[17] to send the data from one machine to many others. An example of this could be using gitsync-nncp over Filespooler[18] where you would like to propagate the changes to many computers.
18: /gitsync-nncp-over-filespooler/
Filespooler lets you request the remote execution of programs, including stdin and environment. It can use tools such as S3, Dropbox, Syncthing[20], NNCP[21], ssh, UUCP[22], USB drives, CDs, etc. as transport; basically, a filesystem is the network for Filespooler.
Filespooler is particularly suited to distributed and Asynchronous Communication[23].
23: /asynchronous-communication/
NNCP lets you securely send files, or request remote execution, between systems. It uses asynchronous communication[25], so the source and destination need never be online simultaneously. NNCP can route requests via intermediate devices -- other NNCP nodes, USB sticks, tapes, radios, phones, cloud services, whatever -- leading to a network that is highly resilient and flexible. NNCP makes it much easier to communicate with devices that lack Internet connectivity, or have poor Internet.
25: /asynchronous-communication/
(c) 2022-2024 John Goerzen