💾 Archived View for complete.org › asynchronous-communication captured on 2024-08-24 at 23:36:32. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2024-08-18)
-=-=-=-=-=-=-
Asynchronous communication is communication between two endpoints that doesn't have to happen in real time or near-real-time.
Compared to synchronous communication, asynchronous communication lets you trade latency for reliability.
Asynchronous communication is closely related to the concept of store-and-forward networking and delay/disruption-tolerant networking. In fact, most asynchronous implementations are also store-and-forward and delay-tolerant, so these terms are often used interchangably.
Let's examine the difference with synchronous communication, which we are probably more familiar with:
With synchronous communication -- such as web sites -- the web browser and the web server must be in live communication. It make take 20 hops across Internet routers to reach the other end, but those routers don't take responsibility for the onward delivery of packets; it's up to the two endpoints to detect if packets were dropped. If the delay from one end to the other exceeds even one second, things can start to get difficult. An interruption at any point along the path from source to destination will cause the communication to fail entirely.
Asynchronous communication, by contrast, only insists that a packet be able to advance to the next hop in real time. The next hop could be anything from another node to a USB drive. Once the packet is delivered to the next hop, the next hop takes responsibility to advance it onward to its destination. This lets the overall communication path be more reliable, because an interruption at any point simply causes traffic to be delayed, not lost. The tradeoff is that it has higher latency than synchronous communication.
Email[1] -- or more particularly, the way SMTP is used -- is a widely-known system. With email, SMTP is the hop-to-hop protocol. Each mail server is only responsible for delivering the message to the next hop. Once delivered to the next hop, that server takes responsibility to advance it to its destination.
NNCP[2] and the older UUCP[3] are ground-up asynchronous protocols. NNCP in particular is a modern asynchronous system, supporting a variety of transports from the Internet to USB drives, and using encryption and onion routing.
Syncthing[4] isn't usually thought of as asynchronous, but its syncing protocol certainly can and does operate in that way.
--------------------------------------------------------------------------------
Sometimes we want better-than-firewall security for things. For instance:
6: /an-asynchronous-rsync-with-dar/
In my writing about dar[7], I recently made that point that dar is a filesystem differ and patcher.
8: /roundup-of-data-backup-and-archiving-tools/
Here is a comparison of various data backup and archiving tools. For background, see my blog post[9] in which I discuss the difference between backup and archiving. In a nutshell, backups are designed to recover from a disaster that you can fairly rapidly detect. Archives are designed to survive for many years, protecting against disaster not only impacting the original equipment but also the original person that created them. That blog post goes into a lot of detail on what makes a good backup or archiving tool.
dar is a Backup[11] and archiving tool. You can think of it as as more modern tar. It supports both streaming and random-access modes, supports correct incrementals (unlike GNU tar's incremental mode), Encryption[12], various forms of compression, even integrated rdiff deltas.
gitsync-nncp is a tool for using Asynchronous Communication[14] tools such as NNCP[15] or Filespooler[16], or even (with some more work) Syncthing[17] to synchronize git[18] repositories.
14: /asynchronous-communication/
A distributed version control system. With gitsync-nncp[20], can be used over Asynchronous Communication[21].
21: /asynchronous-communication/
22: /gitsync-nncp-over-filespooler/
You can use gitsync-nncp[23] (a tool for Asynchronous[24] syncing of git[25] repositories) atop Filespooler[26]. This page shows how. Please consult the links in this paragraph for background on gitsync-nncp and Filespooler.
24: /asynchronous-communication/
27: /using-filespooler-for-backups/
Filespooler[28] makes an *excellent* tool for handling Backups[29]. In fact, this was the case the prompted me to write it in the first place.
30: /introduction-to-filespooler/
It seems that lately I've written several shell implementations of a simple queue that enforces ordered execution of jobs that may arrive out of order. After writing this for the nth time in bash, I decided it was time to do it properly. But first, a word on the *why* of it all.
31: /using-filespooler-over-nncp/
NNCP[32] is a powerful tool for building Asynchronous Communication[33] networks. It features end-to-end Encryption[34] as well as all sorts of other features; see my NNCP Concepts[35] page for some more ideas.
33: /asynchronous-communication/
Filespooler lets you request the remote execution of programs, including stdin and environment. It can use tools such as S3, Dropbox, Syncthing[37], NNCP[38], ssh, UUCP[39], USB drives, CDs, etc. as transport; basically, a filesystem is the network for Filespooler.
Filespooler is particularly suited to distributed and Asynchronous Communication[40].
40: /asynchronous-communication/
I sometimes see people read about NNCP[42] and wonder "This sounds great! But... what can I do with it?" This page aims to answer those questions.
This page describes some basic concepts of NNCP[44].
45: /tools-for-communicating-offline-and-in-difficult-circumstances/
When things are difficult -- maybe there's been a disaster, or an invasion (this page is being written in 2022 just after Russia invaded Ukraine), or maybe you're just backpacking off the grid -- there are tools that can help you keep in touch, or move your data around. This page aims to survey some of them, roughly in order from easiest to more complex.
Here are some (potentially) interesting topics you can find here:
Syncthing is a serverless, peer-to-peer file synchronization tool. It is often compared to Dropbox. However, unlike Dropbox, there is no central server with Syncthing; your devices talk directly to each other to sync data. Syncthing has various effective methods for firewall traversal, including public relays for the worst case. All Syncthing traffic is fully encrypted and authenticated.
Email is one of the most common examples of Asynchronous Communication[49] people are familiar with today. It is a store-and-forward approach which is tolerant of temporary disruptions to services and supports multiple hops.
49: /asynchronous-communication/
50: /quux-org-nncp-public-relay/
According to the NNCP documentation[51], NNCP[52] is intended to help build up small size ad-hoc friend-to-friend (F2F) statically routed darknet delay-tolerant[53] networks for fire-and-forget secure reliable files, file requests, Internet Email[54] and commands transmission. All packets are integrity checked, end-to-end Encrypted[55], explicitly authenticated by known participants public keys. Onion encryption is applied to relayed packets. Each node acts both as a client and server, can use push and poll behaviour model. Also there is multicasting area support.
51: https://nncp.mirrors.quux.org/
53: /asynchronous-communication/
This page gives you references to software by John Goerzen[57].
58: /the-pc-internet-revolution-in-rural-america/
Inspired by several others (such as Alex Schroeder's post[59] and Szczeżuja's prompt[60]), as well as a desire to get this down for my kids, I figure it's time to write a bit about living through the PC and Internet revolution where I did: outside a tiny town in rural Kansas. And, as I've been back in that same area for the past 15 years, I reflect some on the challenges that continue to play out.
59: https://alexschroeder.ch/wiki/2021-11-14_The_early_years_on_the_net
60: https://mastodon.online/@szczezuja/108902027541781265
UUCP is a system for exchanging data and requesting remote execution. It dates back to 1979, and was primarily used over Modem[62]s using telephone landlines for most of its days of popularity. It is an Asynchronous Communication[63] system, which transmits data from one machine to the next on the way to its destination. Each intermediate node may store the data before passing it on to the next.
63: /asynchronous-communication/
NNCP lets you securely send files, or request remote execution, between systems. It uses asynchronous communication[65], so the source and destination need never be online simultaneously. NNCP can route requests via intermediate devices -- other NNCP nodes, USB sticks, tapes, radios, phones, cloud services, whatever -- leading to a network that is highly resilient and flexible. NNCP makes it much easier to communicate with devices that lack Internet connectivity, or have poor Internet.
65: /asynchronous-communication/
(c) 2022-2024 John Goerzen