💾 Archived View for separateconcerns.com › 2015-06-18-love-hate-luajit.gmi captured on 2023-01-29 at 15:33:38. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2022-03-01)
-=-=-=-=-=-=-
published 2015-06-18
Clarification: The title is exaggerated. I have never hated LuaJIT, I just went back to using PUC Lua primarily.
I started using Lua in early 2007. I had already been programming for years, but I had reached a point where I had decided that I wanted to know exactly what was going on when my computer executed dynamic code. That meant understanding two critical pieces of the software stack: the OS and the interpreter.
I already had notions of how an interpreter works and I could tell the main functions of a Unix kernel, but really understanding software, for me, implied reading its source code. I quickly realized that I would not succeed if I started with Perl or Python and Linux, which were what I was using at the time, so I set my sights on Lua and Minix3.
Knowing software also implies using it, so Minix3 and Lua became my main OS and language for several months. To be honest, I was running Minix3 in a VM due to the lack of drivers for my hardware. I started to study their code, with the help of the book [1] for Minix3 and resources found online for Lua. I learnt a lot during that period.
Eventually, I went back to Linux for practical reasons, but I had been charmed by the down-to-Earth elegance of Lua. It became my favorite programming language. I was still a student though, so I mostly wrote small utilities, Web pages and game prototypes with it. Coursework was Java, Python and C, and for "serious" stuff at ResEl [2] I used Perl, Python or Bash.
Then in 2010 I went to Moodstocks [3] for my MSc Thesis. We were a very tiny startup then, just the founders and a couple of interns. They were a Ruby and C++ shop, so I learnt Ruby, but I didn't like it: too much magic, and the object-oriented interface to write bindings was a mess compared to the very clean Lua C API.
Eventually, Moodstocks hired me and I worked for them for over three years. I ended up writing most of their server-side code, I already gave details here [4]. A lot of that code leveraged LuaJIT. It was the perfect tool for the job: as flexible and dynamic as Ruby so it worked well as a scripting language, about as fast as C++ so it could be used to implement Computer Vision algorithms, and the FFI made it very easy to call into our existing C / C++ libraries.
Those are the features that make LuaJIT so attractive: its blazing speed and the FFI. They are why I could write Lua professionally for several years, and I could never thank Mike Pall enough for this. Moreover, LuaJIT has been playing a huge role in the growing popularity of Lua those last few years.
However, paradoxically, LuaJIT negates the very reason that made me pick Lua in the first place. And by that I mean: I do not deeply understand how LuaJIT works. I have tried, and I will certainly try again, but it is one or two orders of magnitude more complicated than PUC Lua.
Beyond the purist ideal of understanding the whole stack, this has practical implications. When you find a bug in LuaJIT, understanding it and fixing it is terribly complicated. The best I can do is usually to try to produce a small test case that reproduces the bug (even that is not always easy) and hope Mike Pall finds a fix. With PUC Lua I could probably fix it myself - but of course PUC Lua is so simple that I have never found a bug in it!
Another issue is that PUC Lua and LuaJIT are diverging. LuaJIT implements Lua 5.1. Lua 5.2 code can be made to work as long as it does not use `_ENV`, but code that leverages the new features in Lua 5.3 is not supported at all (although modules like lua-compat-5.3 [5] can probably help). On the other hand, more and more modules require the FFI, making them incompatible with PUC Lua (of course I am a culprit of that myself [6]). And to top it all, the interpreters work so differently that efficient code in LuaJIT is not necessarily good in PUC Lua, and vice versa.
These days, the purist, simplicity-loving part of me tends to win over the pragmatist in my (rare) non-work code, and my current target of choice is PUC Lua. At work, I write almost exclusively C and almost no Lua, and the little I do write runs on a machine [7] which does not support LuaJIT. The only reasons I still use LuaJIT are maintenance of luajit-msgpack-pure and my use of OpenResty, which for my purposes could as well be built with PUC Lua instead.
That being said, I like knowing that I could take my Lua code and, with a few tweaks and a change of interpreter, get a serious performance boost. So I still hope someone (if not Mike Pall) will make a version of LuaJIT that supports the Lua 5.3 interface. Otherwise, I guess I can still decide that learning more about JIT compilation is worth it and spend a few months diving into that LuaJIT codebase!
Oh, and by the way, with this whole systemd thing, I am semi-seriously considering giving Minix3 a second chance :)
1: http://www.minix3.org/doc/#book
4: http://blog.separateconcerns.com/2013-06-20-three-years-proprietary-projects.html
5: https://github.com/keplerproject/lua-compat-5.3