š¾ Archived View for nytpu.com āŗ gemlog āŗ 2021-03-07.gmi captured on 2022-01-08 at 13:43:06. Gemini links have been rewritten to link to archived content
ā¬ ļø Previous capture (2021-11-30)
ā”ļø Next capture (2022-03-01)
-=-=-=-=-=-=-
Disclaimer: Iām no sysadmin extraordinaire, please please please correct me if thereās something blatantly wrong here:
Also, I have to thank a whole bunch of resources that helped me put this together:
juneās post on their cgit setup
the git book - chapter 4 and 10 for this in particular, but the whole thing is great
zx2c4 for writing cgit, but also for writing actually good documentation
Finally, to see my setup in action before we get started:
Well, Iām sure most people here on Gemini recognize the value of having control over your own software and data, but for git specifically thereās enough possible objections to where I feel I should address them.
The first major objection Iāve heard (and had myself in the distant past) is āwell how do people contribute to your code?ā The answer is actually how git was originally intended to be used prior to githubās idea of locking people in and forcing them to make an account. Just use email! On your READMEs just make a note of what email you want people to send patches to, customarily a mailing list, but a private email works just fine.
git-send-email.io is a great tutorial!
Submitting an email is really scary, I know I was really worried I messed something up the first time I sent a patch. I actually didnāt that time, but in later times I did. But, maintainers are always nice and donāt mind if you mess something up, and itās trivial to resend a fixed patch again, no more stressful than editing a pull request.
Before we can get to setting stuff up, we need to go over some prerequisite information about how cloning/pulling and pushing works in git.
The local protocol is what is used when you do something like `git clone ../repo`, all it does is /essentially/ copy the repo from one place to another. Thereās a bit of other stuff that goes on, but in principle it really is just copying the folder. The interesting thing is that you can actually push and pull from that the same you could as any other protocol. Not /particularly/ useful, but you could have aĀ full āstableā repo instead of just a branch if you wanted to, or you could have the āremoteā on a network disk.
Now, the ssh protocol works exactly the same as the local protocol, just over an ssh connection. The same way you can `rsync local1 local2` or `rsync local1 user@remote:remote1`, or `borg create localrepo::name ~` and `borg create user@remote:remoterepo::name ~`, you can do with git. Just use a ssh name and path instead of just a path, and youāve got it working.
For something like a code forge, they use a special shell that comes with git called the `git-shell`, and all it does is allow you to do is `git receive-pack`, `git upload-pack`, and `git upload-archive` (these are what is used behind the scenes for pushing and pulling), and it doesnāt allow interactive use which is why you canāt `ssh git@git.sr.ht` and start issuing commands. However, in principle itās the exact same. And, on your self-hosted git, you actually can just use your regular ssh user the same way. For instance, my remotes are alex@nytpu.com:pub/<repo_name>.git, instead of a special user like `git@nytpu.com`.
Edit (2021-03-08, suggested by Drew DeVault): If you want to make it read-only, you have to give ssh access to the machine still, but since it respects unix file permissions you can simply make the repos read-only and that'll stop pushing.
The git protocol works the same way as the ssh protocol, but doesnāt require authentication or encryption, and is restricted to only what actions a git-shell can do. And yes, it supports all the actions, you can technically make a push-able git: repo, but you just never would because itās publicly modifiable. Itās generally just used as an alternative to the http protocols when cloning.
The ādumbā http protocol literally just expects a bare git repo to be served as raw files by a web server. The client just reads the files in the bare repo and constructs diffs, figures out what content to pull, etc. This is the only method cgit supports on itās own without anything else. Itās also /really/ slow, because the client has to go and parse and create the logs, graph, history, the whole thing every time so it can figure out what to get.
āSmartā http is what you see used on something like github for pulling and pushing, but even if itās a read-only http link you usually still use the smart http because itās so much faster than the ādumbā is. Itās essentially like the ssh protocol, but instead of shoving {recieve,upload}-pack and upload-archive into an ssh connection, theyāre instead shoved into an http connection. Otherwise itās the same principle, authenticate in a protocol-specific way then shove the data from these git commands into the stream. The main difference is that smart http lets you have a mix of authenticated and unauthenticated users, unlike the completely unauthenticated git and dumb http protocols. This mix of authenticated and unauthenticated is usually used to make it read-only for everyone, and read-write for authorized users, but you could technically make it work like the git protocol and let everyone write, or make it like dumb http and make it completely read-only.
So thatās all the background, basically all you need to know is that thereās four major protocols for accessing remote git repositories:
Weāre going to get set up with smart http (but still read-only), and ssh for our read-write access. We can also optionally make them read-only accessible over the git protocol.
Just install it. Iām going to be using arch package names for this whole thing, but it should be trivial to find what you need for your distro. `sudo pacman -S cgit` and youāre ready. We need to configure it, but thereās no default or example configuration file! Luckily, Iām providing you all with my cgitrc, and if you need to make any changes the cgitrc(5) man page is very good:
https://git.zx2c4.com/cgit/tree/cgitrc.5.txt
Also, the cgitrc references a custom.css, given here:
Finally, cgit has these thingās called āfilters,ā that are used as an html preprocessor before displaying the file. cgit comes with some pretty good ones, and I use its provided one for the source-filter, but I wrote my own about-filter to add support for scdoc.
Thatās about it for cgit itself, the rest is for nginx.
I assume you have a working nginx installation, and all we need to do is add a new subdomain `git.example.com`. If you donāt already have it set up, then thereās numerous tutorials online, go get it working and come back.
Something thatās good for nginx security but bad for trying to run stuff is that nginx has no built-in cgi support, and it only supports fastcgi, while cgit is a regular cgi script. Luckily, thereās a simple little thing called fcgiwrap to the rescue! Just install fcgiwrap (`sudo pacman -S fcgiwrap`), and start up its service (if youāre unfortunate enough to be using systemd on your server, thereās a `fcgiwrap.socket`). Thatās it, no more configuration required for fcgiwrap!
Before we even start getting nginx setup, weāre going to address a very common issue with cgit: cloning through http is /really fucking slow/. This is because cgit only supports the ādumbā http protocol (see above)! Luckily, git comes with a script called `git-http-backend` that supports smart http. In the nginx config weāll just match paths that are queried for by a smart git request and pass those to git-http-backend. If we didnāt do this, those requests would 404 and the client would fall back to cgitās dumb http.
Now, this is where people can get worried about security. Well, as long as you havenāt given out your ssh key, then your ssh is safe. cgitās dumb http is safe, and when we set up the git protocol daemon later on, it disables pushing unless you explicitly enable it on a per-repo basis (which you shouldnāt, thatās public!). The smart http is all we have to worry about, but itās fine just like this. It wonāt enable pushing unless you set up authentication in your nginx config, so unless you just authenticate people willy-nilly itās all read-only!
Now all you just need to make a new config in sites-available, for instance, mineās git.nytpu.com.conf, and then enable it once you get it all pasted in.
A final note, is that if you mess around with css, or even just change repo descriptions, youāre going to need to delete the cache files in order for your changes to show up immediately. For instance, I run `sudo rm /var/cache/cgit/*` (which is pretty dangerous, what if my history got corrupted and put a space after ācgit/ā?).
There! Look you lucky duck, you have your cgit all set up once you restart nginx! Before you start doing anything though, letās get a regular git protocol server set up and show you how to actually set up repos.
Just for fun, letās set up a git protocol server. However, my setup is literally word-for-word from the git book, all I changed is the path itās serving from. Just follow this link:
git book: 4.5 Git on the Server - Git Daemon
A few notes: make sure to punch open port 9418 outgoing on your server. Also, if youāre a clever boi and already have repos set up, it wonāt work quite yet, Iāll go over it in the next section.
Now we gotta work on getting repos set up on your server. First off is creating them, even if you have a local repo you want to push up, you need to create an empty repo on the server to push to.
It initializes a bare repo (i.e. no working copies of files), and cds into it. Bare repos traditionally have ā.gitā on the end to indicate itās a bare repo. Donāt worry, our cgit configuration set it up so the .git is stripped off when creating the repo name. It then touches āgit-daemon-export-okā which is what tells the git daemon we set up last section that itās okay to make this repo public. If you didnāt touch this, then the git daemon would be smart enough to think that maybe you accidentally put it in the public folder and wonāt serve it. Finally, it opens up the ādescriptionā file in the standard editor so you can set it. Now all you do is `git remote add origin user@host:pub/newrepo.git` and itās all set up!
Edit (2021-03-08): I forgot to say this when I first wrote this but you can actually have āprivateā and āpublicā repos. For instance, I serve my repos from ~/pub/, but I also have a ~/priv/ that I can still push/pull to over ssh. When Iām ready to make a repo public, I just move the repo from priv/ to pub/ and change my remote urlās path. As simple as that!
I host a few go repos, none of them are libraries but I am partway through writing an actual library that people wonāt^W^W^W/will/ want to use. The thing is, google has the /outright hubris/ to expect a language-agnostic, no, file agnostic (git works for anything, not just source code) version control system to include programming-languageāspecific meta tags! At least they didnāt continue going for hardcoding specific code forges. I guess since google is the centralization monopoly of the world they probably donāt quite understand that the entire point of git is that itās decentralized, but I mean, come on, you canāt be so sheltered as to not know you can self host git. They didnāt even include sourcehut, they basically said āif youāre not on github or gitlab, youāre SOL.ā
drew devault says a thing about how shitty pkg.go.dev is.
But thatās neither here nor there, they discarded the latter proposal, but you still need the go-specific tags. Luckily, the author of cgit also wrote a little script to automatically serve these for us on a new subdomain (mineās golang.nytpu.com). Unluckily, the dependencies are really egregious unless you already have them installed.
He gives /no/ documentation, and Iāve never once in my life used uwsgi before this, so Iāll share what I got working for your benefit.
First, clone golang-package server wherever you want (Iām using ā/usr/share/webapps/ā for the rest of this). Edit repos.txt and load up the name of your modules and a link to the repo in cgit.
my repos.txt. note that it must be tab separated.
Now we have to install uwsgi and the uwsgi python plugin. Weāre going to be using uwsgiās āemperorā mode because itās the easiest. Start out by checking that /etc/uwsgi/emperor.ini exists, and if not create it and use mine:
files/cgit/golang-package-server/emperor.ini
Then, create /etc/uwsgi/vassals/ and edit /etc/uwsgi/vassals/golang-package-server.ini with this:
Now you can start up the uwsgi emperor daemon using your preferred metod.
Finally, we need to set up our new nginx config:
all you should need to change is the socket path, if necessary.
And thatās it, you can go to golang.nytpu.com and get a list of hosted repos, import golang.nytpu.com/LIBRARY as normal, and if you go to golang.nytpu.com/LIBRARY youāll be redirected to git.nytpu.com/LIBRARY!
You are in a gemlog post. There is an appendix with strange gothic lettering here.
> examine appendix
The engravings translate to "This space intentionally left blank."
ā