I’m trying to run Oddmuse within a Perl web framework: Mojolicious using Mojolicious::Plugin::CGI. It won’t be as perfect as a true Mojolicious app, but it will still be much faster than a simple CGI script. When running as a CGI script, every request loads Perl and compiles all the modules – including the CGI module itself – and Oddmuse and the config files. When running under Mojolicious, we no longer load Perl and we compile Oddmuse just once. Oddmuse itself will keep loading the config file and all that, but it’s still much better than before. The Mojolicious app itself is then started by Toadfarm. And in order to force myself to test it, I’ve switched this wiki over to the new setup!
First, tell Apache to act as a reverse proxy and pass all `/wiki` requests to the new server. The two `ProxyPass` instructions at the bottom are the important bits:
<VirtualHost *:80>
ServerName alexschroeder.ch
ServerAlias www.alexschroeder.ch
Redirect permanent / https://alexschroeder.ch/
</VirtualHost>
<VirtualHost *:443>
ServerAdmin alex@alexschroeder.ch
<Directory />
Options None
AllowOverride None
Order Deny,Allow
Deny from all
</Directory>
ServerName alexschroeder.ch
ServerAlias www.alexschroeder.ch
ServerAlias rpg.alexschroeder.ch
DocumentRoot /home/alex/alexschroeder.ch
<Directory /home/alex/alexschroeder.ch>
Options ExecCGI Includes Indexes MultiViews SymLinksIfOwnerMatch
AddHandler cgi-script .pl
AllowOverride All
Order Allow,Deny
Allow from all
</Directory>
SSLEngine on
SSLCertificateFile /home/alex/ssl/alexschroeder.crt
SSLCertificateKeyFile /home/alex/ssl/alexschroeder.key
SSLCertificateChainFile /home/alex/ssl/GandiStandardSSLCA2.pem
SSLVerifyClient None
ProxyPass /wiki https://alexschroeder.ch:8080/wiki
ProxyPass /mojo https://alexschroeder.ch:8080/mojo
</VirtualHost>
Toadfarm setup in `~/farm/farm`:
#!/usr/bin/env perl
use Toadfarm -init;
logging {
combined => 1,
file => "farm.log",
level => "info",
};
1. other apps go here...
mount "$farm/alexschroeder.pl" => {
"Host" => qr{^(www.)?alexschroeder\.ch$},
mount_point => '/wiki',
};
plugin "Toadfarm::Plugin::AccessLog";
start; # needs to be at the last line
The above means that all requests to `htpp://alexschroeder.ch:8080/wiki` and `htpp://www.alexschroeder.ch:8080/wiki` will be handled by Oddmuse.
Mojolicious wrapper in `~/farm/alexschroeder.pl`:
#! /usr/bin/env perl
use Mojolicious::Lite;
plugin CGI => {
support_semicolon_in_query_string => 1,
};
plugin CGI => {
route => '/',
script => 'wiki.pl', # ~/farm/wiki.pl
env => {WikiDataDir => '/home/alex/alexschroeder'},
errlog => 'alexschroeder.log', # path to where STDERR from cgi script goes
};
app->start;
In this case, `wiki.pl` is simply the original Oddmuse script, unchanged. 😄
#Wikis #Oddmuse #Perl #Software
(Please contact me if you want to remove your comment.)
⁂
Cool! That’s interesting.
What about side effects? Like, if you forget to initialize some variable somewhere, what is going to happen? (Thinking about mod_perl and all stuff related to it)
Also, I miss one important thing from your report... What about page load time? How much faster is it right now, comparing to regular cgi script?
– AlexDaniel 2015-10-09 11:21 UTC
---
I did not measure page load time. I also think that it keeps forking to run the script, so globals start empty every time? Not sure about the details. I haven’t seen any side effects, yet. Let me know if you see anything!
– Alex Schroeder 2015-10-09 11:39 UTC
---
Could you be so kind to measure the load time?
– AlexDaniel 2015-10-09 11:50 UTC
---
I think I was under the *illusion* that it was working. *Now* it is working. Hah! I had to rename my old wrapper script to realize that my ProxyPass instructions had been wrong.
Things to remember:
Check the config file for weird stuff.
1. if ($ENV{SERVER_PORT} == 8080) {
1. $ScriptName = 'https://alexschroeder.ch:8080/wiki';
1. $FullUrl = 'https://alexschroeder.ch:8080/wiki';
1. }
Make sure the user running toadfarm can write alle the files in your data directory. In my case, the data directory belongs to `www-data.alex` but there were temp files that `alex` could not write.
– Alex Schroeder 2015-10-09 13:27 UTC
---
Hm. Current setup, I get more or less the same time when I look at network analysis with my browser when I’m loading `/wiki/About` and when I’m loading `/wiki2/About` with `wiki2.pl` being my old `wiki.pl` wrapper script. I get numbers between 200ms and 700ms. I do note that `/wiki/About` keeps getting me a `200 OK` response instead of a `304 NOT MODIFIED`. I’m not quite sure what the problem is.
– Alex Schroeder 2015-10-09 13:40 UTC
---
Oh, and the code shows that our script is being exec’d. I’m not sure about the headers. I think if our script prints HTTP headers, we should be fine.
– Alex Schroeder 2015-10-09 13:47 UTC
---
Well, at least it seems to work right now. I think I got confused about the various restarts required.
The missing `304 NOT MODIFIED` remains a mistery, however. I’ve tried `$q = new CGI; $q->nph(1)` in the config file, `$CGI::NPH = 1` in the config file, `CGI::nph(1)` in the config file, `use CGI qw/-utf8 -nph/` in the core script, and `print $q->header(-nph=>1, -status=>'304 NOT MODIFIED') and return if PageFresh();` in the core script, reloading my toadfarm between edits and found no change.
When I tried to `export MOJO_PLUGIN_CGI_DEBUG=1` before starting the toadfarm, I got a 503 error when visiting the site.
This is very annoying.
– AlexSchroeder 2015-10-09 14:32 UTC
---
OK, so the question is: How often is `304 NOT MODIFIED` actually used?
alex@kallobombus:~/farm$ sudo wc -l /var/log/apache2/access.log.1 259852 /var/log/apache2/access.log.1 alex@kallobombus:~/farm$ sudo grep ^oddmuse /var/log/apache2/access.log.1 | wc -l 32867 alex@kallobombus:~/farm$ sudo grep "^oddmuse.*GET /wiki.* 304 " /var/log/apache2/access.log.1 | grep -v feed | wc -l 270
Filtering out feed request we’re down to 8‰.
For my own site:
alex@kallobombus:~/farm$ sudo grep ^alexschroeder /var/log/apache2/access.log.1 | wc -l 79286 alex@kallobombus:~/farm$ sudo grep "^alexschroeder.*GET /wiki.* 304 " /var/log/apache2/access.log.1 | grep -v feed | wc -l 702
This reduces the percentage to 9‰.
So, if we redirect feed requests to files and use a cron job to produce some of them, the `304 NOT MODIFIED` response seems to be something we can do without, even if I think it ought to work.
– AlexSchroeder 2015-10-09 14:54 UTC
---
In other words, no noticeable performance boost? Do I read it right? But then why bother?
– AlexDaniel 2015-10-09 21:26 UTC
---
Hehe, indeed. The point is, however, that I want some sort of plan that will allow me to incrementally develop a solution. `Mojolicious::Plugin::CGI` does a lot of what I want. It integrates into Mojolicious and thus it can do logging, forking, and so on. It handles input and output. The only thing that’s missing is that it exec’s the script – loading Perl and compiling all the modules on every request.
I think I’ll try and modify the Plugin for my purposes. Then it’ll require Oddmuse once and just run `DoWikiRequest` on every request, mod_perl style.
– AlexSchroeder 2015-10-10 07:40 UTC
---
These are the changes I had to make to `Mojolicious::Plugin::CGI`: Add run option for a code reference and Make sure the status code is actually used and Add HTTP_IF_NONE_MATCH to environment.
Add run option for a code reference
Make sure the status code is actually used
Add HTTP_IF_NONE_MATCH to environment
This is the new server script I used:
#! /usr/bin/env perl
use lib '/Users/alex/src/mojolicious-plugin-cgi/lib';
use Mojolicious::Lite;
plugin CGI => {
support_semicolon_in_query_string => 1,
};
plugin CGI => {
route => '/wiki',
script => 'wiki.pl',
run => \&OddMuse::DoWikiRequest,
before => sub {
$OddMuse::RunCGI = 0;
$OddMuse::DataDir = '/tmp/oddmuse';
require 'wiki.pl' unless defined &OddMuse::DoWikiRequest;
},
env => {},
errlog => 'wiki.log', # path to where STDERR from cgi script goes
};
get '/' => sub {
my $self = shift;
$self->redirect_to('/wiki');
};
app->start;
– AlexSchroeder 2015-10-11 06:51 UTC
---
With the above setup now installed on alexschroeder.ch, I’m getting response times of 150–230ms when looking at `/wiki/About` (and a `304 Not Modified` response). The old CGI setup under `/wiki2/About` is giving response times of 250–720ms.
So, much better?
I’ve also started noticing the first signs of trouble: Edit page, save, look at it (looks good), edit page again and notice that it’s showing the old text, not the new text.
– AlexSchroeder 2015-10-11 10:28 UTC
---
When I tried it with this page, I saw my browser requesting the page using the following:
GET /wiki?action=edit;id=Comments_on_2015-10-09_Oddmuse_and_Mojolicious HTTP/1.1 ... If-None-Match: 1444395217%1e1%1eAlexSchroeder%1ekensanata%40gmail.com
Oddmuse replied with a `304 Not Modified` and the content was wrong. As I checked, however, the timestamp of the index file does in fact match:
alex@kallobombus:~/alexschroeder$ date -r pageidx +%s 1444395217
Looking at the log of recent changes, however:
alex@kallobombus:~/alexschroeder$ tail -n 1 rc.log 1444559479Comments_on_2015-10-09_Oddmuse_and_Mojolicious1With the above setup now installed on alexschroeder.ch, I'm getting response times of 150–230ms when looking at wikiAbout (and a 304 Not Modified response). So, much better? 127.0.0.1AlexSchroeder23en
I guess something about `TouchIndexFile` isn’t working.
sub TouchIndexFile {
my $ts = time;
utime $ts, $ts, $IndexFile;
$LastUpdate = $Now = $ts;
}
I’ve made sure that all the files belong to `alex.alex` instead of `www-data.alex`. And that seems to help. After saving:
alex@kallobombus:~/alexschroeder$ date -r pageidx +%s
1444560771
alex@kallobombus:~/alexschroeder$ tail -n 1 rc.log
1444560771Comments_on_2015-10-09_Oddmuse_and_MojoliciousWhen I tried it with this page, I saw my browser requesting the page using the following: {{{ GET wiki?action=edit;id=Commentson2015-10-09OddmuseandMojolicious HTTP1.1 … If-None-Match: 1444395217%1e1%1eAlexSchroeder%1ekensanata%40gmail.com }}} Oddmuse replied with a 304 Not Modified and the content…127.0.0.1AlexSchroeder24en
And the next request:
GET /wiki?action=edit;id=Comments_on_2015-10-09_Oddmuse_and_Mojolicious HTTP/1.1 ... If-None-Match: 1444395217%1e1%1eAlexSchroeder%1ekensanata%40gmail.com HTTP/1.1 200 OK ... Etag: 1444560771%1e1%1eAlexSchroeder%1ekensanata%40gmail.com ...
Yay?
– AlexSchroeder 2015-10-11 10:52 UTC
---
OK, added one final patch to make it work without having to use the `-nph` option for the CGI library and submitted a pull request. And I installed it for this site as well. “Eat your own dog food,” and all that.
– AlexSchroeder 2015-10-11 12:41 UTC
---
Wow, many more commits after talking to the maintainer.
– AlexSchroeder 2015-10-11 16:14 UTC