Shortly after my last post [1], I noticed more errors happening with the web server:
[Tue Jul 30 10:43:00.768556 2024] [cgid:error] [pid 1706:tid 4149532352] (12)Cannot allocate memory: AH01252: couldn't create child process: 12: boston.cgi [Tue Jul 30 10:43:00.768714 2024] [cgid:error] [pid 1869:tid 3908467632] [client 178.128.207.138:57024] End of script output before headers: boston.cgi [Tue Jul 30 10:43:01.137027 2024] [cgid:error] [pid 1706:tid 4149532352] (12)Cannot allocate memory: AH01252: couldn't create child process: 12: boston.cgi [Tue Jul 30 10:43:01.137173 2024] [cgid:error] [pid 1707:tid 3774184368] [client 178.128.207.138:57034] End of script output before headers: boston.cgi
I found one potential fix [2] for this—just set the default stack size of Apache to 512k (kilobyte) in size (just 512k—sheesh I'm old). That was simply adding ulimit -s 512 to the startup script, but that still wasn't the full answer:
[Tue Jul 30 22:37:33.042327 2024] [mpm_worker:alert] [pid 5599:tid 3924880304] (12)Cannot allocate memory: AH03142: ap_thread_create: unable to create worker thread [Tue Jul 30 22:37:35.045085 2024] [mpm_worker:alert] [pid 5672:tid 3924880304] (12)Cannot allocate memory: AH03142: ap_thread_create: unable to create worker thread [Tue Jul 30 22:38:38.108993 2024] [mpm_worker:alert] [pid 6091:tid 3924880304] (12)Cannot allocate memory: AH03142: ap_thread_create: unable to create worker thread [Tue Jul 30 22:38:40.112150 2024] [mpm_worker:alert] [pid 6161:tid 3924880304] (12)Cannot allocate memory: AH03142: ap_thread_create: unable to create worker thread [Tue Jul 30 22:38:42.114553 2024] [mpm_worker:alert] [pid 6229:tid 3924880304] (12)Cannot allocate memory: AH03142: ap_thread_create: unable to create worker thread [Tue Jul 30 22:38:44.117182 2024] [mpm_worker:alert] [pid 6304:tid 3924880304] (12)Cannot allocate memory: AH03142: ap_thread_create: unable to create worker thread
I ended up having a little bit of back-and-forth with the settings of the VM (Virtual Machine) that runs my server, alllowing a bit more memory than it was configured for. And since then, no more such errors in the error log. All that's now showing up is crap like:
[Sun Aug 04 15:58:49.248654 2024] [core:error] [pid 872:tid 4129409968] [client XXXXXXXXXXXXXXX:59190] AH10244: invalid URI path (/cgi-bin/.%2e/.%2e/.%2e/.%2e/.%2e/.%2e/.%2e/.%2e/.%2e/.%2e/bin/sh) [Sun Aug 04 15:58:51.854583 2024] [core:error] [pid 2444:tid 4120341424] [client XXXXXXXXXXXXXXX:59760] AH10244: invalid URI path (/cgi-bin/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/bin/sh)
and notifiations about secure certificate rotatations [3]. Another interesting thing I noticed is the accumulated CPU (Central Processing Unit) time over the past few days:
[spc]brevard:~>ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND ... root 868 0.0 0.1 26760 24652 ? Ss Aug01 0:12 /usr/local/apache2/bin/httpd apache 871 0.0 0.1 26188 20876 ? S Aug01 0:35 /usr/local/apache2/bin/httpd apache 872 0.0 0.2 64532 34184 ? Sl Aug01 5:09 /usr/local/apache2/bin/httpd apache 873 0.0 0.1 61060 32152 ? Sl Aug01 4:59 /usr/local/apache2/bin/httpd apache 875 0.0 0.1 62516 31460 ? Sl Aug01 5:01 /usr/local/apache2/bin/httpd apache 1035 0.0 0.1 61716 32148 ? Sl Aug01 5:03 /usr/local/apache2/bin/httpd apache 2444 0.0 0.1 61500 32160 ? Sl Aug01 4:45 /usr/local/apache2/bin/httpd ... gopher 1231 0.0 0.0 10708 9596 ttyp4 S+ Aug01 0:33 lua port70.lua /home/spc/gopher/config.lua spc 1232 0.1 0.0 13600 12140 ttyp5 S+ Aug01 6:51 lua GLV-1.12556.lua non-checkin/conf.lua ... [spc]brevard:~>
I've included the lines for my gopher [4] and Gemini [5] servers just for comparison. Apache [6] is accumulating a bit more CPU time than I expected, but when thinking about it, it's not so strange. Not only is it running TLS (Transport Layer Security) (like the Gemini sever) but it's also supports the HTTP (HyperText Transport Protocol)/2.0 protocol, which is a bit more involved than the simpler HTTP/1.x protocol.
[2] https://serverfault.com/questions/137769/apache2-is-not-starting-my-webserver/220452#220452
[4] gopher://gopher.conman.org:70/1