💾 Archived View for gemini.circumlunar.space › users › kraileth › neunix › eerie › 2019 › daemon_fre… captured on 2023-04-26 at 14:09:42. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2023-03-20)
-=-=-=-=-=-=-
Here I'm republishing an old blog post of mine originally from November 2019. The article has been slightly improved.
The previous part of this series left off with a running "baby daemon" example. It covered Python fundamentals, signal handling, logging as well as an init script to start the daemon.
Writing a daemon using FreeBSD and Python pt.1
The outcome of part 1 was a program that needed external help actually to be daemonized. I used FreeBSD's handy _daemon(8)_ utility to put the program into the background, to handle the pidfile, etc. Now we're making one step forward and try to achieve the same thing using just Python.
To do that, we need a module that is not part of Python's standard library. So you might need to first install the package _py36-daemon_ if you don't already have it on your system. Here's a small piece of code for you - but don't get fooled by the line count, there's actually a lot of things going on there (and of concepts to grasp):
#!/usr/local/bin/python3.6
# Imports #
import daemon, daemon.pidfile
import logging
import signal
import time
# Fuctions #
def handler_sigterm(signum, frame):
logging.debug("Exiting on SIGTERM")
exit(0)
def main_program():
signal.signal(signal.SIGTERM, handler_sigterm)
try:
logging.basicConfig(filename='/var/log/bdaemon.log', format='%(levelname)s:%(message)s', level=logging.DEBUG)
except:
print("Error: Could not create log file! Exiting...")
exit(1)
logging.info("Started!")
while True:
time.sleep(1)
# Main #
with daemon.DaemonContext(pidfile=daemon.pidfile.TimeoutPIDLockFile("/var/run/bdaemon.pid"), umask=0o002):
main_program()
I dropped some ballast from the previous version; e.g. overriding SIGINT was a nice thing to try out once, but it's not useful as we move on. Also that countdown is gone. Now the daemon continues running until it's signaled to terminate (thanks to what is called an "infinite loop").
We have two new imports here that we need for the daemonization. As you can see, it is possible to import multiple modules in one line. For readability reasons I wouldn't recommend it in general. I only do it when I import multiple modules that kind of belong together anyway. However in the coming examples I might just put everything together to save some lines.
The first more interesting thing with this version is that the main program was moved to a function called "main_program". We could have done that before if we really wanted to, but I did it now so the code doesn't take attention away from the primary beast of this example. Take a look at the line that starts with the _with_ keyword. Now that's a mouthful, isn't it? Let's break this one up into a couple of pieces so that it's easier to chew, shall we?
The value for umask is looking a bit strange. It contains an "o" among the numbers, so it has to be a string, doesn't it? But why is it written without quotes then? Well, it _is_ a number. Python uses the "0o" prefix to denote octal (the base-8 numbering system) numbers and 0x would mean hexadecimal (base-16) ones.
Remember that we talked about try/except before (for the logging)? You can expand on that. A _try_ block can not only have _except_ blocks, it can also have a _finally_ block. Statements in such a block are meant to be executed no matter the outcome of the try block. The classical example is that when you open a file, you _definitely_ want to close it again (everything else is a total mess and would make your program an exceptionally bad one).
Closing it when you are done is simple. But what if an exception is raised? Then the code path that properly closes the file might never be reached! You could close the file in every thinkable scenario - but that would be both tedious and error-prone. For that reasons there's another way to handle those cases: Close the file in the _finally_ block and you can be sure that it will be closed regardless of what happens in the _try_ or in any _except_ block.
Ok, but what does this have to do with our little daemon? A lot actually. That case of try/finally has been so common that Python provides a shortcut with so-called _context managers_. They are _objects_ that manage a resource for you like this: You request it, it is valid only inside one block (the _with_ one!) and when the block ends, the context manager takes care of properly cleaning up for you without having you add any extra code (or even without you knowing, if you just copy/paste code from the net without reading explanations like this).
So the _with_ statement in our code above lets Python handle the daemonization process while the main_program function is running. When it ends on the signal, Python cleans up everything and the process terminates - which is great for us. Accept that for now and live with the fact that you might not know just how it does that. We'll come back to things like that.
Ok, the one thing left to do here is making the required changes to the init script. We are no longer using the _daemon(8)_ utility, so we need to adjust it. Here it is the new one:
#!/bin/sh
. /etc/rc.subr
name=bdaemon
rcvar=bdaemon_enable
command="/root/bdaemon.py"
command_interpreter=/usr/local/bin/python3.6
pidfile="/var/run/${name}.pid"
load_rc_config $name
run_rc_command "$1"
Not too much changed here, but let's still go over what has. The command definition is pretty obvious: The program can now daemonize itself, so we call it directly. It doesn't take any arguments, which means we can drop _command_args_.
However we need to add _command_interpreter_ instead (one important thing that I had overlooked first), because the program will look like this in the process list:
/usr/local/bin/python3.6 /root/bdaemon.py
Without defining the interpreter, the init system would not recognize this process as being the correct one. Then we also need to point it to the to the _pidfile_, because in theory there could be multiple processes that match otherwise.
And that's it! Now we have a daemon process running on FreeBSD, written in pure Python.
This next part is a completely optional excursion for people who are pretty new to programming. We'll take a step back and discuss concepts like functions and arguments, modules, as well as namespaces. This should help you better understand what's happening here, if you like to know more. Feel free to save some time and skip the excursion if you are familiar with those things.
As you've seen, functions are defined in Python by using the _def_ keyword, the function name and - at the very least - an empty pair of parentheses. Inside the parentheses you could put one or more arguments if needed:
def greet(name):
print("Hi, " + name + "!")
greet("Alice")
greet("Bob")
Here we're passing a string to the function that it uses to greet that person. We can add a second argument like this:
def greet(name, phrase):
print("Hi, " + name + "! " + phrase)
greet("Alice", "Great to see you again!")
greet("Bob", "How are you doing?")
The arguments used here are called _positional arguments_, because it's decided by their position what goes where. Invert them when calling the function and the output will obviously be garbage as the strings are assigned to the wrong function variable. However it's also possible to refer to the variable by name, so that the order does no longer matter:
def greet(name, phrase):
print("Hi, " + name + "! " + phrase)
greet(phrase="Great to see you again!", name="Alice")
greet("Bob", "How are you doing?")
This is what is used to assign the values for the daemon context. Technically it's possible to mix the ways of calling (as done here), but that's a bit ugly.
We're not using it, yet, but it's good to know that it exists: There are also default values. Those mean that you can leave out some arguments when calling a function - if you are ok with the default value.
def greet(name, phrase = "Pleased to meet you."):
print("Hi, " + name + "! " + phrase)
greet(phrase="Great to see you again!", name="Alice")
greet("Bob", "How are you doing?")
greet("Carol")
And then there's something known as <em>function overloading</em>. We're not going into the details here, but you might want to know that you can have multiple functions with the same name but a different number of arguments (so that it's still possible to precisely identify which one needs to be called)!
When reading about Python it usually won't take too long before you come across the word _module_. But what's a module? Luckily that's rather easy to explain: It's a file with the _.py_ extension and with Python code in it. So if you've been following this daemon tutorial, you've been creating Python modules all the way!
Usually modules are what you might want to refer as to _libraries_ in other languages. You can import them and they provide you with additional functions. You can either use modules that come with Python by default (that collection of modules is known as the _standard library_, so don't get confused by the terminology there), additional third-party modules (there are probably millions) or modules that you wrote yourself.
It's fairly easy to do the latter. Let's pick up the previous example and put the following into a file called "greeter.py":
forgot_name = "Sorry, what was your name again?"
def greet(name, phrase = "Pleased to meet you."):
print("Hi, " + name + "! " + phrase)
Now you can do this in another Python program:
import greeter
greeter.greet("Carol")
print(greeter.forgot_name)
This shows that after importing we can use the "greet()" function in this program, even though it's defined elsewhere. We can also access variables used in the imported module ("greeter.forgot_name" in this case).
Ever wondered what that dot means (when it's not used in a filename)? You can think of it as a hierarchical separator. The standard Python functions (e.g. _print_) are available in the _global namespace_ and can thus be used directly. Others are in a different namespace and to use them, it's necessary to refer to that namespace as well as the function name so that Python understand what you want and finds the function. One example that we've used is _time.sleep()_.
Where does this additional namespace come from? Well, remember that we did "import time" at the top of the program? That created the _time_ namespace (and made the functions from the time module available there).
There's another way of importing; we could import either everything (using an asterisk (*) character, but that's considered poor coding) or just specific functions from one module into the global namespace:
from time import sleep sleep(2) exit(0)
This code will work because the "from MODULE import FUNCTION" statement in this example imported the sleep function so that it becomes available in the global namespace.
So why do we go through all the hassle to have multiple namespaces in the first place? Can't we just put everything in the global one? Sure, we could - and for more simple programs that's in fact an option. But consider the following case: Python provides the _open_ keyword. It's used to open a file and get a nice object back that makes accessing or manipulating data really easy. But then there's also _os.open_, which is not as friendly, but let's you use more advanced things since it uses the raw operating system functionality. See the problem?
If you import the functions from os into the global namespace, you have a name clash in the case of _open_. This is not an error, mind you. You can actually do that, but you should know what happens. The function imported later will _override_ the one that went by that name previously, effectively making the original one inaccessible. This is called "shadowing" of the original function.
To avoid problems like this it's often better to have your own separate namespace where you can be sure that no clashes happen.
In the next part we'll take a look at implementing IPC (inter-process communication) using _named pipes_ (a.k.a "fifos").