💾 Archived View for gemini.ctrl-c.club › ~nttp › writing › scripting-without-scripting.md captured on 2024-08-18 at 20:36:26.
-=-=-=-=-=-=-
# Scripting without scripting
14 February 2022
It's been commonly accepted wisdom for a while now that many if not most games and apps can be made better by the addition of scripting support.
Now, there are several ways to achieve that. The stereotypical approach is to embed Lua (substitute your favorite language here). If that's not an option for some reason, the adventurous programmer can implement a custom interpreter, or at least an ad-hoc parser for a glorified configuration file.
No, seriously, that's OK. Just be careful what your config file can tell the game to do. Because no matter how you do it, the whole point is making software do stuff you didn't think of beforehand without changing the source code. Like this:
["print", "2 + 3 =", ["+", 2, 3]]
Yes, that's JSON. I wrote it by hand, but let's pretend it was exported from a visual editor. See how it's structured? It's a series of nested lists, where the first element always says what to do, and the others become arguments / operands. This way you don't have to hunt around for the addition operator for example, and it becomes much easier to handle each list. Let's try it in Python:
def evals(form, scope):
name = form[0]
function = scope[name]
arguments = form[1:]
return function(arguments)
I called my function "evals" so it won't be mistaken for Python's own built-in. As for the arguments, a "form" is anything we want to evaluate (long story), and a "scope" contains everything we care about here, which for now is just a dictionary of functions:
scope = {
"print": lambda args: print(*args),
"+": lambda args: args[0] + args[1],
}
Now we can put two and two together (har har) and try to do what the list says:
script = ["print", "2 + 3 =", ["+", 2, 3]]
evals(script, scope)
Oops. It handled a literal string as intended, but not the nested list. What to do? We could have each function handle its arguments as needed, but that would be a lot of boilerplate. Or we could write a smarter evals:
def evals2(form, scope):
if type(form) == list:
name = form[0]
function = scope[name]
arguments = form[1:]
evaluated = [evals2(i, scope) for i in arguments]
return function(evaluated)
else:
return form
(I hope you know about list comprehensions and recursivity.) Now for take two:
evals2(script, scope)
How about that, it worked! We have a scripting engine!
Because see, this is what scripting is really about: not parsing, not syntax checking. Those are details. Well, maybe not when you have to deal with them. But the essence is right there: one short function that handles ordinary data.
Well, let's see what else we can do with it.
For one thing: branches! Just like in Python, it would be sometimes useful to do one thing or another depending on a condition, but not both. And a function won't cut it. Obviously.
scope["<"] = lambda args: args[0] < args[1]
scope["if"] = lambda args: args[1] if args[0] else args[2]
script = ["if", ["<", 2, 3], ["print", "yay!"], ["print", "nay..."]]
evals2(script, scope)
By the time our function can look at the condition, both branches were already evaluated. You have to handle this directly:
def evals3(form, scope):
if type(form) != list:
return form
elif form[0] == "if":
if evals(form[1], scope):
evals(form[2], scope)
else:
evals(form[3], scope)
else:
name = form[0]
function = scope[name]
arguments = form[1:]
evaluated = [evals3(i, scope) for i in arguments]
return function(evaluated)
evals3(script, scope)
Note how "if" looks like any function when you only read the JSON, but it's not. Guess you could say it's a special form.
You can probably figure out how to add a while loop, or a block statement. Gonna need them anyway. Really, that's how it works: it took an if to implement an if. There's a reason why all programming languages have these things. And you know what else they all have? Variables.
Never mind creating them. Maybe they're all set by the engine, from a separate configuration file. But how to read them in the first place? In Python you just say `print(a)` to get the value of "a". But our system is based on JSON. It only has strings, and we need strings to keep literal data, like messages to print.
So here's a trick: make up a special form that takes a string and looks up its value. Where? In the scope, of course.
def evals4(form, scope):
if type(form) != list:
return form
elif form[0] == "get":
name = form[1]
return scope[name]
elif form[0] == "if":
if evals(form[1], scope):
evals(form[2], scope)
else:
evals(form[3], scope)
else:
name = form[0]
function = scope[name]
arguments = form[1:]
evaluated = [evals4(i, scope) for i in arguments]
return function(evaluated)
scope["a"] = 2
scope["b"] = 3
script = ["print", "a + b =", ["+", ["get", "a"], ["get", "b"]]]
evals4(script, scope)
Why not a function? Because functions don't have access to the scope. Well, technically they do: it's declared right there in the same module. But that's not a given. What if you want to keep scripts isolated from each other? You could also pass the scope to each function when you call it, along with the list of arguments. Seemed easier this way, that's all.
Our JSON is getting verbose by now. We'll keep pretending it's not supposed to be manually edited, but this is why many people would rather say `(print "2 + 3 =" (+ $a $b))` and be done with it. A few more remarks:
- Our scripting engine mixes functions and variables in the same scope. A lot of languages do that, including Python, but it's not the only option.
- If there's no variable by that name, evals errors out. That's how Python works, too. It's a good idea. Catches a very common mistake very quickly.
Other languages are more lenient and return some sort of null value instead. Lua for example. We'll try that later. But first, any scripting system is much more powerful if its users can define functions: lists of actions they can put together once then call from several places.
We could do it exactly like that, too: modify evals (again!) to notice when it's pointed at a chunk of JSON instead of a function, and evaluate it in place, the same way arguments are handled. But instead I'm going to use some Python magic:
def say_hi():
print("Hi!")
class Greeting:
def __call__(self):
print("Hi!")
say_hi2 = Greeting()
say_hi()
say_hi2()
Thanks to the __call__ method, I could put say_hi2 in the scope with a suitable name, and evals would never know the difference. But the point here was to run user-supplied scripts:
class Function:
def __init__(self, code, scope):
self.code = code
self.scope = scope
def __call__(self, args):
return evals4(self.code, self.scope)
scope["say-hi"] = Function(["print", "Hi!"], scope)
evals4(["say-hi"], scope)
Note how our Function doesn't know its own name, but it needs to know about the scope, otherwise it can't even call evals. Conveniently, that also gives access to our own "print". But it doesn't use the argument list, even though it has to take one, or else evals would complain ("say-hi" ends up called from evals).
Where do we put arguments so a user-defined function can use them? In the scope, obviously, but not the same scope everyone else sees, because they'd make a mess. But it can't be just a new, blank scope either because then it wouldn't see the other functions anymore. So let's make a copy first:
class Function2:
def __init__(self, code, scope):
self.code = code
self.scope = scope
def __call__(self, args):
local_scope = dict(self.scope)
for index, value in enumerate(args):
local_scope[index] = value
return evals4(self.code, local_scope)
scope["double"] = Function2(["+", ["get", 0], ["get", 0]], scope)
evals4(["print", "Two times 5 is", ["double", 5]], scope)
I didn't want to mess with argument names, so I used the list index instead. That's easy enough to fix, but it would take even more code and explanations. And then, it gets the job done: our JSON "functions" can take arguments now, so they act just like one written in Python.
It's also kind of limited. Maybe you want each function to see the scope it was called from (a.k.a. dynamic scope). Or maybe you want functions split into groups, such that only functions from the same group can see each other, while all of them can see the global scope (a.k.a. namespaces). For fancy tricks like that, we need scopes all the way down. The question is how.
It would be nice if we could chain dictionaries, like with object prototypes in Javascript, but Python doesn't roll that way. We're going to need a new class. And that's going to require changing evals again, since it expects scopes to be dictionaries. Luckily we don't have to, because Python knows more magic tricks:
class Scope:
def __init__(self, parent=None):
self.names = {}
self.parent = parent
def __getitem__(self, key):
if key in self.names:
return self.names[key]
elif self.parent != None:
return self.parent[key]
else:
return None
It really is that simple. We just have to copy everything into our new scope:
scope2 = Scope()
scope2.names.update(scope)
evals4(["print", "Two times 5 is", ["double", 5]], scope2)
Even our Function2 class got tricked and never noticed the change. Let's make it use new-style scopes, too.
class Function3:
def __init__(self, code, scope):
self.code = code
self.scope = scope
def __call__(self, args):
local_scope = Scope(self.scope)
for index, value in enumerate(args):
local_scope.names[index] = value
return evals4(self.code, local_scope)
All we need to do now is use the new style of function object in the new scope:
scope2.names["double"] = Function3(
["+", ["get", 0], ["get", 0]], scope2)
evals4(["print", "Two times 5 is", ["double", 5]], scope2)
This works, and as a bonus it's as flexible as it gets. Let's see how much code it took to get here: 10 lines for Function3, 12 for Scope, and 17 for evals4. Total: around 40 lines of code, or less than a sheet of printer paper. Oh, start adding new functions to the scope, and more special forms, and it will quickly balloon to ten times the size. But it never gets more complicated, so you can stop worrying about interpretation, and focus on your task.
When it comes to scripting support, users are afraid of writing code, and programmers are afraid of parsing it. That's why I pretended this is about making a back-end for a visual language. Mostly however it was because once you get past the syntax, on the inside it works about the same for any language. Swap things around, make up your own operations and name them whatever; it's part of the fun. You might even learn why some things are always done the same way in most other languages.
Every user interface is a language, even if the user can only talk in mouse clicks. And now you know how to listen.