💾 Archived View for chirale.org › 2019-05-15_5570.gmi captured on 2024-07-08 at 23:37:32. Gemini links have been rewritten to link to archived content

-=-=-=-=-=-=-

Sort of CDN to serve client-side libraries via an auto-pull git repo on tmpfs

This configuration will allow to install on a Debian-based system a fast server for client libraries. Key technologies used are:

tmpfs to serve files from volatile memory
git / mercurial from github / bitbucket to get files from a public or private repository
systemd units to mount tmpfs and sync
nginx to serve files to user

On this first step you’ll create a service to reserve some RAM for static files, pulling them from a private or public repo.

Mount tmpfs with systemd

To serve files directly from RAM, you have to mount a tmpfs directory. You can do it on fstab:

/etc/fstab

 tmpfs /mnt/cdn tmpfs rw,nodev,nosuid,size=300M 0 0

Or with a systemd unit:

/etc/systemd/system/mnt-cdn.mount

 [Unit] Description=Mount empty CDN directory on volatile memory [Mount] What=tmpfs Where=/mnt/cdn Type=tmpfs Options=defaults,noatime,size=300M [Install] WantedBy=multi-user.target

noatime will disable last access on contained files, reducing write on disk
size will reserve 300MB for /mnt/cdn partition on RAM (increase as needed)
WantedBy=multi-user.target mount the partition on runlevel 3 (multi-user mode with networking)

Create two units on a local path like /usr/local/share/systemd then create a symlinks on /etc/systemd/system or create directly them on /etc/systemd/system. You can also directly create them on /usr/local/share/systemd.

Create the pull service

When the /mnt/cdn is successfully loaded, pull static files from your repository.

/etc/systemd/system/cdn-pull.service

 [Unit] Description=Pull on CDN directory. After=network-online.target [Service] User=youruserhere Group=youruserhere ExecStart=/usr/local/share/systemd/cdn-pull.sh [Install] WantedBy=mnt-cdn.mount

Clone the git repository with a user on system using a key with an alias
Change youruserhere to the user who cloned the repository
Add to /root/.ssh/config and to /root/.ssh/my_private_key the private key to do the pull

Meaning:

WantedBy=mnt-cdn.mount copy the files to RAM only after the /mnt/cdn is created
After=network-online.target pull the repository only when the network is ready

On pull, all files will be written by root as youruserhere:youruserhere.

After the pull, to reduce RAM occupation, this script doesn’t download directly to RAM .git directory but copy them with rsync excluding them:

/usr/local/share/systemd/cdn-pull.sh

 #!/bin/bash # stop on first error set -e cd /srv/cdn-all git pull exec rsync -a --exclude=.git --exclude=.gitignore /srv/cdn-all/* /mnt/cdn/

Get systemd to know about the mount and service

To reload systemd units, you have to

 systemctl daemon-reload

Then do the mount via the systemd unit:

 systemctl start mnt-cdn.mount

Enable on boot

Since the cdn-pull.service is tied to mnt-cdn.mount, both have to be enabled to run:

 systemctl enable mnt-cdn.mount systemctl enable cdn-pull.service

When the system is ready create the tmpfs on /mnt/cdn/ After tmpfs is successfully created by the unit, the file will be automatically synced through cdn-pull.service.

Mount will auto-start sync

Start only the mnt-cdn.mount:

 systemctl start mnt-cdn.mount

And then ask for info about both services:

 systemctl status mnt-cdn.mount systemctl status cdn-pull.service

mnt-cdn.mount have to be active (mounted)
cdn-pull.service should be active (script is running) or inactive (sync is completed). In both cases, it’s ok.

With this set-up, when you restart the mnt-cdn.mount files will be automatically pulled and synced to RAM when system starts and when you start or restart mnt-cdn.mount service.