Hello all! I've been checking out the mailing list and reading about Gemini every now and then, and decided to try and host a Gemini server for my blog. One question I had is: Which server software is the most mature one? On the software list [1], there is no description about the state and feature set of any of the listed server software. [1]: gemini://gemini.circumlunar.space/software/ I only need to serve static files, as everything else happens on a static site generator before being uploaded. I'd also like a server which supports multiple host names (vhosts?), but that's mostly optional. (Browsing though the server software list, most of them seem to only accept a single host in their configuration.) Thanks in advance for any responses, and feel free to say no such thing exists. I am not averse to trying to build my own :)
It was thus said that the Great Ecmel Berk Canl?er once stated: > Hello all! > > I've been checking out the mailing list and reading about Gemini every > now and then, and decided to try and host a Gemini server for my blog. > > One question I had is: Which server software is the most mature one? What do you mean by "most mature"? By age, it's GLV-1.12556 (disclaimer: I wrote it). By feature set, it's a toss-up between GLV-1.12556: * support for any type of documents (from multiple directories) * CGI (both Gemini and HTTP specific, RFC-3875 compliant) * user directories * client certificate support * easy to extend with custom modules (if you know Lua) and Gemserv: * support for any type of document * CGI (Gemini only, not quite RFC-3875 compliant) * user directories * reverse proxy * virtual hosts Both have recent updates. By popularity, last time I checked, it was JetForce (which also has had recent updates). Of the three mentioned, JetForce is in Python, Gemserv in Rust, and GLV-1.12556 in Lua. The lack of virtual hosting in GLV-1.12556 is because I lack experience in dealing with server certificates with multple hosts, and because of that, I can't exactly test that feature. > On the software list [1], there is no description about the state and > feature set of any of the listed server software. > > [1]: gemini://gemini.circumlunar.space/software/ > > I only need to serve static files, as everything else happens on a > static site generator before being uploaded. If that's the case, then most of the servers would probably work, it then comes down to which language do you have available for compiling/running the server. -spc
I have found both of these servers very difficult to install. With gemserv, I tried debian's packaged rust, rustup, and rustup nightly. All failed for different interesting reasons. I could never get the dependencies sorted out for GLV-1.12556. Jetforce is very easy to install with pip, which is probably why it's the most used server currently. I'm using molly-brown currently, because I like how it handles user public_gemini directories. Full cgi support would be nice. Thanks for all the hard work all you software developers are doing! On Thu, May 14, 2020 at 04:50:41AM -0400, Sean Conner wrote: > It was thus said that the Great Ecmel Berk Canl?er once stated: > > Hello all! > > > > I've been checking out the mailing list and reading about Gemini every > > now and then, and decided to try and host a Gemini server for my blog. > > > > One question I had is: Which server software is the most mature one? > > What do you mean by "most mature"? By age, it's GLV-1.12556 (disclaimer: > I wrote it). By feature set, it's a toss-up between GLV-1.12556: > > * support for any type of documents (from multiple directories) > * CGI (both Gemini and HTTP specific, RFC-3875 compliant) > * user directories > * client certificate support > * easy to extend with custom modules (if you know Lua) > > and Gemserv: > > * support for any type of document > * CGI (Gemini only, not quite RFC-3875 compliant) > * user directories > * reverse proxy > * virtual hosts > > Both have recent updates. By popularity, last time I checked, it was > JetForce (which also has had recent updates). Of the three mentioned, > JetForce is in Python, Gemserv in Rust, and GLV-1.12556 in Lua. The lack of > virtual hosting in GLV-1.12556 is because I lack experience in dealing with > server certificates with multple hosts, and because of that, I can't exactly > test that feature. > > > On the software list [1], there is no description about the state and > > feature set of any of the listed server software. > > > > [1]: gemini://gemini.circumlunar.space/software/ > > > > I only need to serve static files, as everything else happens on a > > static site generator before being uploaded. > > If that's the case, then most of the servers would probably work, it then > comes down to which language do you have available for compiling/running the > server. > > -spc
On Thu, May 14, 2020 at 12:35:45PM +0000, tiwesdaeg at tilde.pink wrote: > I'm using molly-brown currently, because I like how it handles user > public_gemini directories. Full cgi support would be nice. I've been meaning to write a gemlog about this for ages, but never seem to have the time... GCI support in Molly Brown scares the pants off me. Go doesn't reliably support setuid() and this totally breaks all the standard pradigms of unix server design. In particular, I can find no way for MB to spawn a CGI process which doesn't run as the same user as MB itself, and therefore necessarily has the ability to read the TLS private key files and write to the log file, which is obviously a massive security hole. Because the MB process never runs as root, it's not possible to chroot the CGI processes somewhere they couldn't see those files, either. I'm frankly baffled that a language with Ken Thompson and Rob Pike as designers, created specifically to help Google engineers write server software, could turn out to be so terrible for...writing servers on unix. I can only imagine this has happened because nobody at Google would ever do something so terribly mundane and non-scalable as fork off an entire whole new process on the same machine to generate some content. Probably there'd be ten layers of containers and proxying and reverse-proxying and other Cloudian faff keeping the content-generating processes separated from the server processes so that uids were a quaint and irrelevant consideration. (if any Go wizards have recently joined the list and can correct me on any of the above I'd be thrilled) If you're just running your own server and you have control over the CGI code, then go for it and be careful. In a pubnix type environment where you are letting third parties upload stuff, you basically need to be extremely careful, use the configuration to restrict CGI to directories writable only by people you really trust or will actively verify the code of. This was the reason that I pondered a few months back on this list some kind of FastCGI-esque system for a Gemini server to communicate with a process it didn't spawn itself over a domain socket or similar. But maybe I should just give up on Go for servers (this exact same issue affects Shizaru, too.). Cheers, Solderpunk > Thanks for all the hard work all you software developers are doing! > > On Thu, May 14, 2020 at 04:50:41AM -0400, Sean Conner wrote: > > It was thus said that the Great Ecmel Berk Canl?er once stated: > > > Hello all! > > > > > > I've been checking out the mailing list and reading about Gemini every > > > now and then, and decided to try and host a Gemini server for my blog. > > > > > > One question I had is: Which server software is the most mature one? > > > > What do you mean by "most mature"? By age, it's GLV-1.12556 (disclaimer: > > I wrote it). By feature set, it's a toss-up between GLV-1.12556: > > > > * support for any type of documents (from multiple directories) > > * CGI (both Gemini and HTTP specific, RFC-3875 compliant) > > * user directories > > * client certificate support > > * easy to extend with custom modules (if you know Lua) > > > > and Gemserv: > > > > * support for any type of document > > * CGI (Gemini only, not quite RFC-3875 compliant) > > * user directories > > * reverse proxy > > * virtual hosts > > > > Both have recent updates. By popularity, last time I checked, it was > > JetForce (which also has had recent updates). Of the three mentioned, > > JetForce is in Python, Gemserv in Rust, and GLV-1.12556 in Lua. The lack of > > virtual hosting in GLV-1.12556 is because I lack experience in dealing with > > server certificates with multple hosts, and because of that, I can't exactly > > test that feature. > > > > > On the software list [1], there is no description about the state and > > > feature set of any of the listed server software. > > > > > > [1]: gemini://gemini.circumlunar.space/software/ > > > > > > I only need to serve static files, as everything else happens on a > > > static site generator before being uploaded. > > > > If that's the case, then most of the servers would probably work, it then > > comes down to which language do you have available for compiling/running the > > server. > > > > -spc
On 20/05/14 03:08PM, solderpunk wrote: > I'm frankly baffled that a language with Ken Thompson and Rob Pike as > designers, created specifically to help Google engineers write server > software, could turn out to be so terrible for...writing servers on > unix. I can only imagine this has happened because nobody at Google > would ever do something so terribly mundane and non-scalable as fork > off an entire whole new process on the same machine to generate some > content. Probably there'd be ten layers of containers and proxying and > reverse-proxying and other Cloudian faff keeping the content-generating > processes separated from the server processes so that uids were a quaint > and irrelevant consideration. > > (if any Go wizards have recently joined the list and can correct me on > any of the above I'd be thrilled) > As an FYI, there's is a proposed patch[0] to add proper setuid dropping for Golang. It's not through yet but is in review and testing stage. Hopefully it'll make it in for 1.15. Otherwise you're correct; all-thread setuid is done either through raw syscalls and process forking wizardry, syscalls, wrapper programs, etc. For my (also Golang) server SecretShop I've been handling it through the systemd unit file. A bit messy, but I'm also not explictly supporting multi-user deployments. -Steve [0] https://go-review.googlesource.com/c/go/+/210639
On Thu, May 14, 2020 at 04:50:41AM -0400, Sean Conner wrote: > What do you mean by "most mature"? By age, it's GLV-1.12556 (disclaimer: > I wrote it). By feature set, it's a toss-up between GLV-1.12556: > > * support for any type of documents (from multiple directories) > * CGI (both Gemini and HTTP specific, RFC-3875 compliant) > * user directories > * client certificate support > * easy to extend with custom modules (if you know Lua) > > and Gemserv: > > * support for any type of document > * CGI (Gemini only, not quite RFC-3875 compliant) > * user directories > * reverse proxy > * virtual hosts > > Both have recent updates. By popularity, last time I checked, it was > JetForce (which also has had recent updates). Goodness, I'm getting left right behind! Poor, neglected Molly Brown. :( Cheers, Solderpunk
On Thu, May 14, 2020 at 11:26:23AM -0400, Steve Ryan wrote: > As an FYI, there's is a proposed patch[0] to add proper setuid dropping > for Golang. It's not through yet but is in review and testing stage. > Hopefully it'll make it in for 1.15. Ah, good to know! Thanks very much for bringing this to my attention. I'll keep a close eye on this and put off doing anything else with dynamic content in Molly Brown until it's been accepted. > Otherwise you're correct; all-thread setuid is done either through raw > syscalls and process forking wizardry, syscalls, wrapper programs, etc. Yeah, I'd seen a few examples of this black magic online but it didn't seem like anything I wanted to put deep trust in. Cheers, Solderpunk
On Thu May 14, 2020 at 8:35 AM EDT, tiwesdaeg wrote: > > I have found both of these servers very difficult to install. With > gemserv, I tried debian's packaged rust, rustup, and rustup nightly. > All failed for different interesting reasons. Could you email information on what problems you were having? I threw a linux binary up on the project page to make it easier on people. > Jetforce is very easy to install with pip, which is probably why it's > the most used server currently. > > Thanks for all the hard work all you software developers are doing! Indeed thank you everyone! int 80h
It was thus said that the Great tiwesdaeg at tilde.pink once stated: > > I have found both of these servers very difficult to install. With > gemserv, I tried debian's packaged rust, rustup, and rustup nightly. > All failed for different interesting reasons. I could never get the > dependencies sorted out for GLV-1.12556. I admit it might be difficult to install GLV-1.12556. I missed listing one dependency (oops). But I feel the biggest hurdle is the requirement of LibreSSL, as not many people use it (shame, because it has what I feel is the easiest to use TLS implementation with its libtls). I'm not sure what to do about that, but thanks for trying it (or even looking at it). > Thanks for all the hard work all you software developers are doing! You're welcome. -spc
On Thu May 14, 2020 at 11:08 AM EDT, solderpunk wrote: > This was the reason that I pondered a few months back on this list some > kind of FastCGI-esque system for a Gemini server to communicate with a > process it didn't spawn itself over a domain socket or similar. But > maybe I should just give up on Go for servers (this exact same issue > affects Shizaru, too.). That's a good idea, CGI scares me. Are you planning to create a standard or should we just go off of regular FASTCGI? I've never used CGI outside of gemini so I'm not sure what people need. int 80h
It was thus said that the Great solderpunk once stated:
> On Thu, May 14, 2020 at 12:35:45PM +0000, tiwesdaeg at tilde.pink wrote:
>
> > I'm using molly-brown currently, because I like how it handles user
> > public_gemini directories. Full cgi support would be nice.
>
> I've been meaning to write a gemlog about this for ages, but never seem
> to have the time...
>
> GCI support in Molly Brown scares the pants off me. Go doesn't reliably
> support setuid() and this totally breaks all the standard pradigms of
> unix server design. In particular, I can find no way for MB to spawn a
> CGI process which doesn't run as the same user as MB itself, and
> therefore necessarily has the ability to read the TLS private key files
> and write to the log file, which is obviously a massive security hole.
> Because the MB process never runs as root, it's not possible to chroot
> the CGI processes somewhere they couldn't see those files, either
Does Go actually check the permissions of a file before calling exec() on
it? And refuse to run it if it's setuid? That sounds strange to me. Or do
you mean something like?
child = fork()
if (child == 0) /* child process */
{
setuid(CGIowner);
exec(CGIprogram);
}
And because I feel compelled to state this: setuid() only works for root.
And it is possible to chroot programs, but it's not generally done in
these days of dynamically linked programs because you need to copy in all
the shared objects into the chroot with the program. Hmmm ... I wonder if
you can't do a chroot with some LD_PRELOAD trickery ... might have to play
around with that.
> I'm frankly baffled that a language with Ken Thompson and Rob Pike as
> designers, created specifically to help Google engineers write server
> software, could turn out to be so terrible for...writing servers on
> unix.
Rob Pike doesn't trust programmers. Direct quote from him:
The key point here is our programmers are Googlers, they're not
researchers. They're typically, fairly young, fresh out of school,
probably learned Java, maybe learned C or C++, probably learned
Python. They're not capable of understanding a brilliant language
but we want to use them to build good software. So, the language
that we give them has to be easy for them to understand and easy to
adopt.
http://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2014/From-Parallel-to-Concurrent
> This was the reason that I pondered a few months back on this list some
> kind of FastCGI-esque system for a Gemini server to communicate with a
> process it didn't spawn itself over a domain socket or similar. But
> maybe I should just give up on Go for servers (this exact same issue
> affects Shizaru, too.).
FastCGI could be a way of doing it. I might just look into that for
GLV-1.12556 ...
-spc (There's always Rust ... )
It was thus said that the Great int 80h once stated: > On Thu May 14, 2020 at 11:08 AM EDT, solderpunk wrote: > > This was the reason that I pondered a few months back on this list some > > kind of FastCGI-esque system for a Gemini server to communicate with a > > process it didn't spawn itself over a domain socket or similar. But > > maybe I should just give up on Go for servers (this exact same issue > > affects Shizaru, too.). > > That's a good idea, CGI scares me. Are you planning to create a standard > or should we just go off of regular FASTCGI? I've never used CGI outside > of gemini so I'm not sure what people need. The specification of CGI is RFC-3875 [1] and there are a number of Gemini servers that have implemented it to some degree [2]. I found the specification for FastCGI [3] and it seems a bit more involved (with it being a protocol and all) but I think it could be used for Gemini, much like CGI can be used for Gemini. -spc [1] https://tools.ietf.org/html/rfc3875 [2] GLV-1.12556 can handle both Gemini specific CGI scripts and HTTP specific CGI scripts (by converting the return code from HTTP to Gemini). [3] https://fastcgi-archives.github.io/FastCGI_Specification.html
On Thu May 14, 2020 at 2:28 PM EDT, Sean Conner wrote: > The specification of CGI is RFC-3875 [1] and there are a number of > Gemini > servers that have implemented it to some degree [2]. I found the > specification for FastCGI [3] and it seems a bit more involved (with it > being a protocol and all) but I think it could be used for Gemini, much > like > CGI can be used for Gemini. > > -spc > > [1] https://tools.ietf.org/html/rfc3875 > > [2] GLV-1.12556 can handle both Gemini specific CGI scripts and HTTP > specific CGI scripts (by converting the return code from HTTP to > Gemini). > > [3] https://fastcgi-archives.github.io/FastCGI_Specification.html Thank you for the links. I see there's also SCGI which looks similar. I guess I have some reading to do int 80h
It was thus said that the Great Sean Conner once stated: > > And because I feel compelled to state this: setuid() only works for root. Let me further clarify this before the replies come pouring in---the simple answer is "setuid() only works for root". The complex answer is more nuanced and involves the "effective user id" concept. -spc
I finally got it to compile on debian using rustup stable. It's now the geminid for gemini://libraryoferis.org The big win here is that I was able to get CGI working perfectly to serve dynamic content using shell script, python, and sqlite3! Have you thought about support the .gmi file type as well? It took me forever to realize what was going on with index.gmi. I am looking at trying gemserv out on tilde.pink, but my current users are all using .gmi. I'm really happy with gemserv, thanks! On Thu, 14 May 2020 17:01:29 -0400 "int 80h" <int at 80h.dev> wrote: > On Thu May 14, 2020 at 8:35 AM EDT, tiwesdaeg wrote: > > > > I have found both of these servers very difficult to install. With > > gemserv, I tried debian's packaged rust, rustup, and rustup nightly. > > All failed for different interesting reasons. > > Could you email information on what problems you were having? I threw > a linux binary up on the project page to make it easier on people. > > > Jetforce is very easy to install with pip, which is probably why > > it's the most used server currently. > > > > Thanks for all the hard work all you software developers are > > doing! > > Indeed thank you everyone! > > int 80h
On Thu May 14, 2020 at 3:13 PM EDT, Tiwesdaeg Twohands wrote: > I finally got it to compile on debian using rustup stable. It's now the > geminid for gemini://libraryoferis.org The big win here is that I was > able to get CGI working perfectly to serve dynamic content using shell > script, python, and sqlite3! > > Have you thought about support the .gmi file type as well? It took me > forever to realize what was going on with index.gmi. I am looking at > trying gemserv out on tilde.pink, but my current users are all > using .gmi. > > I'm really happy with gemserv, thanks! Thank! You're the first person to run anything I've made. It now supports configurable index per vhost. int 80h
int 80h <int at 80h.dev> wrote: > Thank! You're the first person to run anything I've made. It now > supports configurable index per vhost. > Gemserv has filled my need for server software since a few days ago. Maybe I should upgrade to the new version when I wake up tomorrow? -- Katarina > -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://lists.orbitalfox.eu/archives/gemini/attachments/20200515/a62d 5477/attachment.htm>
On Fri May 15, 2020 at 12:27 AM EDT, Katarina Eriksson wrote: > Gemserv has filled my need for server software since a few days ago. > Maybe > I should upgrade to the new version when I wake up tomorrow? Oh, thank you! If you find any problems please let me know. I thought it was only used by me. int 80h
---