Migrating prosody internal storage to SQLite on OpenBSD

Comment on Mastodon

Introduction

As some may know, I'm an XMPP user, an instant messaging protocol which used to be known as Jabber. My server is running Prosody XMPP server on OpenBSD. Recently, I got more users on my server, and I wanted to improve performance a bit by switching from the internal storage to SQLite.

Actually, prosody comes with a tool to switch from a storage to another, but I found the documentation lacking and on OpenBSD the migration tool isn't packaged (yet?).

The switch to SQLite drastically reduced prosody CPU usage on my small server, and went pain free.

Prosody documentation: Prosody migrator

Setup

For the migration to be done, you will need a few prerequisites:

On OpenBSD, the migration tool can be retrieved by downloading the sources of prosody. If you have the ports tree available, just run `make extract` in `net/prosody` and cd into the newly extracted directory. The directory path can be retrieved using `make show=WRKSRC`.

The migration tool can be found in the subdirectory `tools/migration` of the sources, the program `gmake` is required to build the program (it's only replacing a few variables in it, so no worry about a complex setup).

In the migration directory, run `gmake`, you will obtain the migration tool `prosody-migrator.install` which is the program you will run for the migration to happen.

Prepare the configuration file

In the migration directory, you will find a file `migrator.cfg.lua.install`, this is a configuration file describing your current prosody deployment and what you want with the migration, it defaults to a conversion from "internal" to "sqlite" which is what most users will want in my opinion.

Make sure the variable `data_path` in the file refers to `/var/prosody` which is the default directory on OpenBSD, and check the hosts in the "input" part which describe the current storage. By default, the new storage will be in `/var/prosody/prosody.sqlite`.

Run the tool

Once you have the migrator and its configuration file, it's super easy to proceed:

storage = "sql"
sql = {
    driver = "SQLite3";
    database = "prosody.sqlite";
}

If you had an error at the migration step, check the logs carefully to check if you missed something, a bad path, maybe.

Conclusion

Prosody comes with a migration tool to switch from a storage backend to another, that's very handy when you didn't think about scaling the system correctly at first.

The migrator can also be used to migrate from the server ejabberd to prosody.

Thanks prx for your report about some missing steps!