💾 Archived View for gemini.hitchhiker-linux.org › gemlog › agis_updates.gmi captured on 2024-06-20 at 11:50:35. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2024-06-16)
-=-=-=-=-=-=-
I've had very little time recently to devote to programming, but there have been a few things that have received some attention in the past few months which are beginning to bear some fruit in more tangible ways. I've been focusing more on library code recently than applications. A side effect is that I'm finding that often an application that I wrote in the past might benefit from applying that library code.
Case in point is my Spartan server, Agis. What is Agis? It's a multithreaded Spartan protocol server written in Rust with some nice features. It serves this capsule over Spartan, actually. Being written in Rust has benefits and drawbacks. One of the benefits of writing something in Rust is having access to the entire crates ecosystem. One of the serious drawback of writing an application in Rust is having access to the entire crates ecosystem, because adding one dependency is generally going to pull in 3, 4, or 50 transient dependencies. This morning I finished swapping out two of the original dependencies in Agis with my own code as an experiment to see how much the dependency graph is brought down. The results are promising.
OsRand is a tiny crate I wrote a while back for random number generation. Osrand had the explicit goals of simplicity and no dependencies beyond `std`. Pretty much every RNG on crates.io generates randomness via cryptographic code. That's fine, and it's even good if you want to be able to use it in embedded applications without an OS, but what about when your application is running on some form of Unix? In Unix we already have an RNG at /dev/random and a psuedo-RNG at /dev/urandom. In OsRand I just use the rust std::io::Read interface to read random bytes from one of those device files, configurable at compile time through a feature flag. So yeah, it's tiny and simple.
But one of the most common use cases for an RNG isn't numerical at all actually. No, a lot of the time we want a random string rather than a random number. So I included a random string generator in OsRand, and to the best of my knowledge it's the only RNG on crates.io which has this capability. Seems like a huge oversight to me, but it's the way that the ecosystem developed I guess. Anyway, one of the original deps for Agis was the `tempfile` crate, which was used to create temporary files with a bit of randomness in their pathname. But tempfile, as I mentioned before, commits the sin of pulling in five other crates when you add it to your Cargo.toml. By replacing tempfile with OsRand I was able to remove five transient dependencies and 8k of binary size.
Agis has built in logging which is pretty verbose. Every log entry gets the time prepended. As I originally built the program I was using the chrono crate to get the time and make it human readable. Now chrono is a great library that covers 99.99% of anything related to timekeeping. It's a great library. But pulling in all of chrono, plus it's dependencies, was overkill for Agis. My own crate `epoch` does about 70% of what chrono does in a fraction of the code size and with no dependencies beyond rust std. The only drawbacks right now are that I haven't published it to crates.io so you have to get it from git, and I haven't resolved completely getting the current time for a completely stupid reason. Allow me to explain..
Unix records time using timestamps which are counted as the number of seconds since the Unix epoch, which occurred on Jan 1, 1970. Incidentally that's where the name of the crate came from. Anyway, those seconds were originally recorded as a signed 32-bit integer, or i32, which was way too small and was definitely going to overflow if not made larger. That's a solved problem now, because we all settled on using an i64 instead. By 'all' I mean every extant Unix derivative or clone. But here comes Rust and they decided in their infinite wisdom that we aren't going to care about dates previous to the Unix epoch and we want to go even further into the future, so we're going to have the standard library timekeeping functions use an unsigned 64 bit integer, but we're also going to hide that in an opaque struct and make you call a member function to compare two of those structs and return the difference, the second version being derived from the Unix Epoch, just to get at that internal u64. Fucking. Dumb. But now we're stuck with this shitty interface because stability.
Anyway, right now the `DateTime::now()` constructor relies on this shitty interface, which leaves a messy error situation where we now have a potentially fallible conversion from u64 back to i64, which won't ever actually occur anyway because on Unix the rust std interface uses gettimeofday from the C library anyway which returns an i64 (as I've already explained). This is shit code. I'm going to bypass this trainwreck and just get the timestamp from the OS directly. I just haven't done it yet, which is why the crate remains unpublished right now. I'm going to give three options for the `now` constructor, behind a feature flag. Option one is going to be using the libc crate. Option two will be to use a system call via my fork of the `sc` crate, and it's going to specifically be my fork because I've been adding support for other OS and architecture combinations since the only non-linux Unix that upstream seems to care about is x86_64 on FreeBSD and they haven't responded to pull requests in any way. But option two is nice because it's not pulling in the rather large `libc` crate for ONE function. Option three will be to exclude the `now` constructor altogether, because I don't like external dependencies.
Now that I've written that, I suppose there is a simpler way. I could create a binding that only includes the gettimeofday function, bypassing the libc crate. Something to ponder. Why include bindings to every part of libc when I just need that function? Well because libc is pulled in by other deps that I haven't gotten around to removing. Shit.
Anyway, just those two crate replacements reduce the dependency graph from 51 crates in a release build down to 41, and shave 12k (12,264 bytes) from the binary size. Now, 12k is a LOT of dead code. This is a lot of the reason I've been souring on Rust in the first place. The Zig compiler would happily just optimize all that dead code away, for instance, and the binary sizes after this sort of twiddling wouldn't be much different. But even ignoring the dead code, which is pretty much unreachable from the binary so not really a security issue, those extra ten dependencies I consider a huge win because the attack surface has shrunk by a fifth in one measure. That is to say, the number of external maintainers being trusted not to have screwed something up which might cause a security issue has been reduced by a fifth. It's still shit in that by including 11 total dependency entries in Cargo.toml I wind up with 41 dependencies due to transients, but it's better.
All content for this site is licensed as CC BY-SA.
© 2024 by JeanG3nie