Sadly it seems that Emacs Wiki is down. Currently Nic Ferrier is paying for the servers, so I sent him an email but haven’t heard back. So I’m thinking of resurrecting the site on my own servers.
I still remember the fiasco that started with 2014-12-18 Emacs Wiki Migration.
2014-12-18 Emacs Wiki Migration
☯
These are my notes. Perhaps they’re useful in case I have to restore another backup, or they might be useful to you if you want to fork Emacs Wiki.
First, restore the backups from Chile provided by zeus. Thanks, man!
rsync -az kensanata@theshire.emacs.cl:emacswiki.org . rsync -az kensanata@theshire.emacs.cl:emacswiki .
I want to run the script from my Mojolicious Toadfarm. I added the following lines to my Toadfarm setup:
# emacswiki.org
mount "$farm/emacswiki.pl" => {
"Host" => qr{^emacswiki\.org:8080$},
mount_point => '/wiki',
};
And this is the Mojolicious CGI Plugin wrapper, `emacswiki.pl`:
#! /usr/bin/env perl
use Mojolicious::Lite;
plugin CGI => {
support_semicolon_in_query_string => 1,
};
plugin CGI => {
route => '/',
script => '/home/alex/farm/wiki.pl', # not necessary
errlog => '/home/alex/farm/emacswiki.log',
run => \&OddMuse::DoWikiRequest,
before => sub {
no warnings;
$OddMuse::RunCGI = 0;
$OddMuse::DataDir = '/home/alex/emacswiki';
require '/home/alex/farm/wiki.pl';
},
};
app->start;
In order for this to work, I need an Apache site. I created `/etc/apache2/sites-available/500-emacswiki.org.conf` with the following:
<VirtualHost *:80>
ServerName emacswiki.org
ServerAlias www.emacswiki.org
Redirect permanent / https://emacswiki.org/
</VirtualHost>
<VirtualHost *:443>
ServerAdmin alex@emacswiki.org
ServerName emacswiki.org
ServerAlias www.emacswiki.org
DocumentRoot /home/alex/emacswiki.org
<Directory /home/alex/emacswiki.org>
Options ExecCGI Includes Indexes MultiViews SymLinksIfOwnerMatch
# legacy CGI scripts like info-ref.pl
AddHandler cgi-script .pl
AllowOverride All
Require all granted
</Directory>
SSLEngine on
SSLCertificateFile /etc/letsencrypt.sh/certs/emacswiki.org/cert.pem
SSLCertificateKeyFile /etc/letsencrypt.sh/certs/emacswiki.org/privkey.pem
SSLCertificateChainFile /etc/letsencrypt.sh/certs/emacswiki.org/chain.pem
SSLVerifyClient None
ProxyPass /wiki http://emacswiki.org:8080/wiki
ProxyPass /mojo http://emacswiki.org:8080/mojo
</VirtualHost>
I remove all the `*.pl` files except for `info-ref.pl` in the `emacswiki.org` directory.
Reloded the farm using `./farm reload`. Check the log file for `Mounting emacswiki with conditions`.
Activating the site using `sudo a2ensite 500-emacswiki.org`.
Check the config using `sudo apachectl configtest`. Oops! This is an obvious erorr, of course: `SSLCertificateFile: file '/etc/letsencrypt.sh/certs/emacswiki.org/cert.pem' does not exist or is empty`.
I need to get the SSL certificates, too.
I added `emacswiki.org www.emacswiki.org` to `/etc/letsencrypt.sh/domains.txt` and ran `/etc/letsencrypt.sh/letsencrypt.sh -c` but that doesn’t work. I guess it doesn’t work because the name `emacswiki.org` still points to the old server. I guess for the moment I’ll try to do without HTTPS.
So this is what I’ll be using instead for the site:
<VirtualHost *:80>
ServerName emacswiki.org
ServerAlias www.emacswiki.org
ServerAdmin alex@emacswiki.org
DocumentRoot /home/alex/emacswiki.org
<Directory /home/alex/emacswiki.org>
Options ExecCGI Includes Indexes MultiViews SymLinksIfOwnerMatch
# legacy CGI scripts like info-ref.pl
AddHandler cgi-script .pl
AllowOverride All
Require all granted
</Directory>
ProxyPass /emacs http://emacswiki.org:8080/wiki
ProxyPass /wiki http://emacswiki.org:8080/wiki
ProxyPass /mojo http://emacswiki.org:8080/mojo
</VirtualHost>
Now `sudo apachectl configtest` says `Syntax OK`.
Reloaded Apache using `sudo service apache2 reload`.
Added a line to my `/etc/hosts` file:
192.71.233.105 emacswiki.org
Testing `w3m http://emacswiki.org/wiki` and `w3m http://emacswiki.org/wiki` seems to work!
Better make the wiki read-only: `touch ~/emacswiki/noedit`.
Following links doesn’t work. `w3m http://www.emacswiki.org/emacs/HowToNavigate` tells me: `Can't load http://www.emacswiki.org/emacs/HowToNavigate`. The problem is that Apache has `www.emacswiki.org` as a server alias, but the Toadfarm only listens for `emacswiki.org`.
Change that:
# emacswiki.org
mount "$farm/emacswiki.pl" => {
"Host" => qr{^(www\.)?emacswiki\.org:8080$},
mount_point => '/wiki',
};
And reload: `./farm reload`.
That didn’t work. Hah, of course not. I need to add `www.emacswiki.org` to my `/etc/hosts`, of course!
Now it works.
OK, next problem: Why does `w3m http://www.emacswiki.org/` give me the directory listing? Surely I’m missing my `.htaccess` file. Is it not being read? The `/var/log/apache2/error.log` file has not suspicious. Well, it does mention something about the `emacswiki.org/rss` directory but I just deleted it. Are the permissions wrong? I did a `chmod g-w .htaccess` just to be sure and now it says:
-rw-r--r-- 1 alex alex 1955 May 29 2016 emacswiki.org/.htaccess
This looks correct to me.
In there, it says `DirectoryIndex emacs.pl`. Ah. That might be a problem because I removed that script. Changing that to `DirectoryIndex emacs` did the job!
☯
OK, so anybody who has access to their own `/etc/hosts` file can now access a read-only copy of the site.
Here’s what I have planned:
1. change the DNS entry ✓
2. see how the site explodes 🔥🔥🔥
3. add HTTPS
When I tried to add a News page, I noticed that I was unable to get the wiki back into writeable mode. I had to remove the `noedit` file I had created earlier using `rm ~/emacswiki/noedit`.
Then, when I tried to save, the wiki complained about some page that looked like spam not being readable and I figured that the page index must have been out of sync so I simply removed it using `rm ~/emacswiki/pageidx`.
And finally I recreated the lock using `touch ~/emacswiki/noedit`.
OK, now I’m waiting for the DNS change to spread and watching my Munin graphs.
Also, all the people with HTTPS bookmarks will get errors like the following: `Bad cert ident from emacswiki.org: dNSName=alexschroeder.ch rpg.alexschroeder.ch www.alexschroeder.ch : accept? (y/n)`. That’s because emacswiki.org is currently longer listening on port 443 and the default site is alexschroeder.ch. Oh well! In a few hours I’m hoping that Let’s Encrypt will allow me to regenerate certificates for Emacs Wiki and then we’ll move to HTTPS.
☯
Hours later, I checked again and HTTP access was working. So I ran `sudo /etc/letsencrypt.sh/letsencrypt.sh -c` to get the certificates and this time it worked. I reverted the changes to the site config file `/etc/apache2/sites-available/500-emacswiki.org.conf` and we’re no using this:
<VirtualHost *:80>
ServerName emacswiki.org
ServerAlias www.emacswiki.org
Redirect permanent / https://emacswiki.org/
</VirtualHost>
<VirtualHost *:443>
ServerAdmin alex@emacswiki.org
ServerName emacswiki.org
ServerAlias www.emacswiki.org
DocumentRoot /home/alex/emacswiki.org
<Directory /home/alex/emacswiki.org>
Options ExecCGI Includes Indexes MultiViews SymLinksIfOwnerMatch
# legacy CGI scripts like info-ref.pl
AddHandler cgi-script .pl
AllowOverride All
Require all granted
</Directory>
SSLEngine on
SSLCertificateFile /etc/letsencrypt.sh/certs/emacswiki.org/cert.pem
SSLCertificateKeyFile /etc/letsencrypt.sh/certs/emacswiki.org/privkey.pem
SSLCertificateChainFile /etc/letsencrypt.sh/certs/emacswiki.org/chain.pem
SSLVerifyClient None
ProxyPass /emacs http://emacswiki.org:8080/wiki
ProxyPass /wiki http://emacswiki.org:8080/wiki
ProxyPass /mojo http://emacswiki.org:8080/mojo
</VirtualHost>
Notice that both `/emacs` and `/wiki` will work. Is this a bad idea? `sudo apachectl configtest` says the changes are good and so I ran `sudo service apache2 reload`. Everthing seems to be working!
What about load? It’s definitely going up! 🙁
OK, time to read up on mod_cache. I *think* I want something like the following:
# Turn on caching
CacheSocache shmcb
CacheSocacheMaxSize 102400
<Location "/emacs">
CacheEnable socache
</Location>
Well, before diving into this, I think we should just monitor how load develops over the next few hours.
☯
A few hours later it would seem to me that there are no open issues so there is no need for extra caching.
And that also means, I can try and make the website editable again.
Let’s see, what else do we need to check?
1. does git work?
2. what about cronjobs?
As for git, this is simple. I created the page 2017-05-18 and I expect to see it on the emacsmirror master branch. Sadly, it isn’t there. Why not?
Let’s take a look:
alex@sibirocobombus:~/emacswiki/git$ git log
commit a08f867084896e9892d148f76a54976166cd75db
Author: Alex Schroeder <anybody@emacswiki.org>
Date: Thu May 18 13:56:01 2017 +0200
Backup site!
Oops! Apparently, the git repository wasn’t checked out. It makes sense, actually. But now I need to fix this. `git remote -v` shows no remotes. Let’s add it, and fetch data. This works because my public key is already part of the emacswirror org on GitHub.
git remote add origin git@github.com:emacsmirror/emacswiki.org.git git fetch git branch -m master temp git checkout master
At this point it should tell you `Branch master set up to track remote branch master from origin.`
git cherry-pick a08f86 git push git branch -D temp
OK. Time to test it! In order to be able to save, I now have to change the site URL in the config file back to HTTPS. It should read
my $root = "https://www.emacswiki.org";
I made the page edit and that seems to do the trick. `git log` in the git directory lists the new edit.
This brings me to the next part: cron jobs. Somebody has to push those commits, right?
1. I added `emacswiki` and `emacswiki.org` to the shell script that uses rsync to store daily backups in Chile. Thanks again, zeus!
2. I found an old `emacs-git-update` in my `bin` directory and added an appropriate entry to my crontab using `crontab -e`.
3. I found an old `maintain-emacswiki` in my `bin` directory, fixed it, and also added it to crontab. I definitely need to check the maintenance page a few times over the next few days.
4. I found an old `update-ell` in my `bin` directory and decided to check the XML file referenced. The timestamp says `Wed 24 Dec 2014 11:36:00 GMT` so I think it’s safe to say that this won’t be required anymore.
5. I did not find a copy of the `emacs-elisp-area` script. I checked the code in my config file and now I remember: this job used to call the expensive Elisp Area URLs and save the result to disk, and then URL rewriting made sure that the Elisp Area URLs called from the web would read those files instead. I just tried those links (”Alphabetical list of all Elisp libraries on this Wiki” with and without context, and “Chronological list of all Elisp libraries on this Wiki”) and it seems to work just fine. It takes a few seconds, but nothing terrible. I’ll say that this won’t be required anymore.
6. I found an old copy of `emacs-rss` in my `bin` directory. That one precomputes some resource intensive RSS feeds. I should definitely get those back into working condition. When I run it, the four files are generated, and they’re the four RSS feed advertised in the HTML of the wiki, so that’s great.
And that’s all the jobs I found in an old `crontab` file!
☯
Current status, one day later:
The only suspicious thing is the spike around 2:30 in the morning. But the explanation might be simple enough, looking at my `crontab`:
#m h dom mon dow command 02 5 * * * /home/alex/bin/maintain-campaignwiki 47 4,16 * * * /home/alex/bin/backup 28 4 * * * /home/alex/bin/subscriptions 14 3 * * * /home/alex/bin/emacs-git-update 32 2 * * * /home/alex/bin/maintain-emacswiki
At 2:32, the maintenance job runs. The `curl` output is available in the maintenance directory:
--2017-05-19 02:32:01-- https://www.emacswiki.org/emacs
Resolving www.emacswiki.org (www.emacswiki.org)... 192.71.233.105
Connecting to www.emacswiki.org (www.emacswiki.org)|192.71.233.105|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘/home/alex/emacswiki.org/maintenance/last.html’
0K .......... .......... .......... .......... .......... 43.1K
50K .......... .......... .......... .......... .......... 11.3K
100K .......... .......... .......... .......... .......... 3.66K
150K .......... .......... .......... .......... .......... 21.3K
200K .......... .......... .......... .......... .......... 35.7K
250K .......... .......... .......... .......... .......... 7.86K
300K .......... .......... .......... .......... .......... 12.4K
350K .......... .......... .......... .......... .......... 22.8K
400K .......... .......... .......... .......... .......... 19.3K
450K .......... .......... .......... .......... .......... 3.64K
500K .......... .......... .......... .......... .......... 5.91K
550K .......... .......... .......... .......... .......... 14.6K
600K .......... .......... .......... .......... .......... 10.3K
650K .......... .......... .......... .......... .......... 15.4K
700K .......... .......... .......... .......... .......... 20.5K
750K .......... .......... .......... .......... .......... 15.3K
800K .......... .......... .......... .......... .......... 15.3K
850K .......... .......... .......... .......... .......... 15.4K
900K .......... .......... .......... ...... 18.5K=86s
2017-05-19 02:34:03 (10.9 KB/s) - ‘/home/alex/emacswiki.org/maintenance/last.html’ saved [959329]
I’m guessing that these two minutes are causing the spike.
☯
When I did some testing with the CSS, I ran into problems. If you choose a different theme via the CSS page, it gets stored in a cookie. Cookies are specific to a site, and so cookies set on http://emacswiki.org, https://emacswiki.org, http://www.emacswiki.org and https://www.emacswiki.org are separate from each other. This will not do. I’ve now changed the Apache config file to create the appropriate redirections. At the same time, I wanted to clean up the https://www.emacswiki.org/wiki vs. https://www.emacswiki.org/emacs situation.
https://www.emacswiki.org/wiki
https://www.emacswiki.org/emacs
<VirtualHost *:80>
ServerName www.emacswiki.org
ServerAlias emacswiki.org
Redirect permanent / https://www.emacswiki.org/
</VirtualHost>
<VirtualHost *:443>
ServerName emacswiki.org
Redirect permanent / https://www.emacswiki.org/
SSLEngine on
SSLCertificateFile /etc/letsencrypt.sh/certs/emacswiki.org/cert.pem
SSLCertificateKeyFile /etc/letsencrypt.sh/certs/emacswiki.org/privkey.pem
SSLCertificateChainFile /etc/letsencrypt.sh/certs/emacswiki.org/chain.pem
SSLVerifyClient None
</VirtualHost>
<VirtualHost *:443>
ServerAdmin alex@emacswiki.org
ServerName www.emacswiki.org
DocumentRoot /home/alex/emacswiki.org
<Directory /home/alex/emacswiki.org>
Options ExecCGI Includes Indexes MultiViews SymLinksIfOwnerMatch
# legacy CGI scripts like info-ref.pl
AddHandler cgi-script .pl
AllowOverride All
Require all granted
</Directory>
SSLEngine on
SSLCertificateFile /etc/letsencrypt.sh/certs/emacswiki.org/cert.pem
SSLCertificateKeyFile /etc/letsencrypt.sh/certs/emacswiki.org/privkey.pem
SSLCertificateChainFile /etc/letsencrypt.sh/certs/emacswiki.org/chain.pem
SSLVerifyClient None
Redirect permanent /wiki https://www.emacswiki.org/emacs
ProxyPass /emacs http://emacswiki.org:8080/wiki
ProxyPass /mojo http://emacswiki.org:8080/mojo
</VirtualHost>
I wonder whether it’s important to prevent outside access to `http://emacswiki.org:8080/wiki`. I see no problem?
#Emacs
(Please contact me if you want to remove your comment.)
⁂
Heroic effort, and amazing notes! Thanks for Emacs Wiki!
– AaronHawley 2017-05-19 01:27 UTC
---
Thanks!
– Alex Schroeder 2017-05-19 07:04 UTC