💾 Archived View for theparanoidtimes.org › blog › 2024 › 06 › 07 › 0x0-local-instance captured on 2024-08-18 at 17:04:25. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2024-06-16)
-=-=-=-=-=-=-
Address
Unknown
0x0.st (the null pointer) [1] is a simple and straight-to-the-point file hosting service. Its wonderful author has shared the complete source code and even encourages people to host their instance. This is what we'll do in this post.
The service is written in Python [2] using Flask [3] and we will use Nginx [4] and Gunicorn [5] as our reverse proxy server and application server respectively. Everything will be run from a virtual environment [6], as it should be.
Let's dive right in!
We need to make the necessary preparations for our (Ubuntu or CentOS/Rocky) environment before configuring the application.
# update our repositories sudo apt update sudo apt upgrade # or sudo yum update -y
Then we deal with system packages. Make sure that the Python version is 3.10+.
sudo apt-get install python3-pip sudo apt-get install nginx sudo apt-get install sqlite # not needed for application run, only for peeking into DB # or sudo yum install python3.11 sudo yum install pip3.11 sudo yum install nginx sudo yum install sqlite # non needed for application run, only for peeking into DB
We only need one global pip package.
python3.11 -m pip install virtualenv
It is always good for services to have their user which should belong to a specific group that will express their intended behavior. We will create a new user and a group, but any sane existing group can be used.
sudo addgroup nullpointer sudo adduser nullpointer nullpointer sudo passwd nullpointer # set a password sudo adduser nullpointer sudo # or sudo groupadd nullpointer sudo useradd nullpointer -g nullpointer sudo passwd nullpointer # set a password usermod -aG wheel nullpointer
Log in as a new user to ensure it is set up properly. Now we are ready to set up the application.
First, determine where the service will be located and run from. For this post, we will use the newly created user's home directory.
cd /home/nullpointer curl https://git.0x0.st/mia/0x0/archive/master.zip --output 0x0.zip unzip 0x0.zip cd 0x0
Then set up a virtual environment and install dependencies.
python3.11 -m virtualenv .venv source .venv/bin/activate pip install -r requirements.txt pip install gunicorn
When done, these commands have installed all the necessary dependencies of the 0x0 service and Gunicorn server that we'll use to serve the application (to Nginx). We can try to run and test the application locally using Flask after we adjust the configuration.
cp instance/config.example.py instance/config.py
Since we will use Nginx the 0x0 configuration can remain mostly as is because it expects Nginx by default. We need to set the location of the SQLite database. Using your favorite editor change the value of 'SQLALCHEMY_DATABASE_URI' to the absolute path of the database file and the value of 'FHOST_STORAGE_PATH' to the absolute path of the directory where the uploaded files will go. Make sure that this directory exists. For this post, these will be:
SERVER_NAME = localhost # used by the terminal admin panel only, should be your hostname SQLALCHEMY_DATABASE_URI = 'sqlite:///' + '/home/nullpointer/0x0/database.sqlite' FHOST_STORAGE_PATH = '/home/nullpointer/0x0/up'
Before the first run, we need to perform database migrations by invoking a Flask Alembic [7] command.
export FLASK_APP=fhost flask db upgrade
Now, let's try to run it.
flask run
If 0x0 starts successfully it will be served on port 5000 on localhost only and we can test it with a curl request.
echo "Test1" > test1.txt curl -F'file=@test1.txt' http://localhost:5000
The server will return the URL of the file which may or may not work, but this is okay because we have more work to do. For now, we are satisfied with a regular response.
To continue without any side effects we should unset the 'FLASK_APP' variable as it was only used for testing.
unset FLASK_APP
The Flask server is used for development only and it is designed to sit behind a WSGI server in production environments. Gunicorn is the server of choice, and since we already installed it, we will move on to configuring it. First, we need a file that Gunicorn will reference and start the Flask application. Using your favorite editor create a file named 'wsgy.py' and add the following content:
from fhost import app if __name__ == '__main__': app.run(debug=False)
We can already try to start the server, but it will be beneficial if we introduce server configuration now. Create a new file named 'gunicorn.nullpointer.config.py' and add the following:
bind = '0.0.0.0:5000' worker_class = 'sync' workers = 2 loglevel = 'debug' accesslog = '/var/log/gunicorn/nullpointer.access.log' access_log_format = '%(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"' errorlog = '/var/log/gunicorn/nullpointer.error.log'
Make sure that the directory /var/log/gunicorn exists.
Just in case set permissions on both files.
chmod u+x wsgi.py chmod u+x gunicorn.nullpointer.config.py
Now we can test with Gunicorn
export FLASK_APP=fhost gunicorn -c gunicorn.nullpointer.config.py wsgi:app
This will start a server on the port 5000 which we can target with curl again.
echo "Test2" > test2.txt curl -F'file=@test2.txt' http://localhost:5000
The response will be the URL of the stored file that may or may not work, but that is still okay since we need Nginx to complete our stack. But, first, we should create the Gunicorn service and enable it in systemctl. To continue, kill the active server process and unset the 'FLASK_APP' environment variable (for the last time I promise). To have all our prerequisites (like the environment variables) defined in one place, and to make our service script simpler we will create a bash script that will be called by systemctl. All paths in it must be absolute. Create the file 'nullpointer-start.sh' with the contents:
#!/bin/bash export FLASK_APP=fhost export FLASK_ENV=production /home/nullpointer/0x0/.venv/bin/gunicorn -c /home/nullpointer/0x0/gunicorn.nullpointer.config.py wsgi:app
Set the permissions:
chmod +x nullpointer-start.sh
Let's create the service script as root user in '/etc/systemd/system' called 'nullpointer.service'.
[Unit] Description=nullpointer 0x0 file sharing service After=network.target [Service] User=nullpointer Group=nullpointer WorkingDirectory=/home/nullpointer/0x0 ExecStart=/home/nullpointer/0x0/nullpointer-start.sh SuccessExitStatus=143 TimeoutStopSec=30 Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target
If the script is okay we can enable and start the service.
sudo systemctl enable nullpointer sudo systemctl start nullpointer
We already installed Nginx in the first step, so now we will make the necessary preparations before introducing it to 0x0. We need to create the folder structure where we'll register all the sites available to Nginx.
sudo mkdir /etc/nginx/sites-available sudo mkdir /etc/nginx/sites-enabled
Register enabled sites by adding the following line in the Nginx configuration file '/etc/nginx/nginx.conf':
include /etc/nginx/sites-enabled/*;
Let's enable the service.
sudo systemctl enable nginx
Since we are running 0x0 and Nginx on the same machine we are going to use Unix sockets as a way of communication between them. First, let's configure Gunicorn to bind to a socket instead of HTTP. In the Gunicorn configuration file change the 'bind' line to:
bind = 'unix:/home/nullpointer/0x0/0x0.sock'
And restart the service:
sudo sustemctl restart nullpointer
Now, we can create a Nginx configuration for proxying to 0x0. The configuration convention is that all available sites that Nginx could serve are configured in the 'sites-available' directory. The ones that are serving, or in other terms are enabled are configured in 'sites-enabled'. To avoid multiple configurations, enabled sites' configurations are linked to the 'sites-enabled' directory. In '/etc/nginx/sites-available' create the 'nullpointer.conf' file with the contents:
server {
listen 80;
server_name nullpointer;
access_log /var/log/nginx/nullpointer.access.log;
error_log /var/log/nginx/nullpointer.error.log;
location / {
include proxy_params;
proxy_pass http://unix:/home/nullpointer/0x0/0x0.sock;
}
location //home/nullpointer/0x0/up {
root /;
internal;
}
}
With this configuration file, we specified the port Nginx will listen to, where will it place the log files, and that it will proxy all traffic to our Unix socket. When proxying Nginx should include parameters (headers) from the 'proxy_params' file. If that file does not exist in '/etc/nginx' directory, create it:
proxy_set_header Host $http_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme;
To make Nginx able to serve back the uploaded files using the URL provided by 0x0 we specified the '//home/nullpointer/0x0/up' location as internal. Because we used the absolute paths in both 0x0 and Nginx configurations, the only way I made them work together was by adding one more leading slash '/' in the location declaration. The null pointer service appends the leading slash in a header when responding with the file URL, so the Nginx location path must also have it so it can recognize it and fetch the file. I don't like this, because it looks like a hack, so I will update this post if I find a more elegant solution. For now, let's continue with our setup.
To enable our configuration let's link it to 'sites-enabled' directory.
sudo ln -s /etc/nginx/sites-available/nullpointer.conf /etc/nginx/sites-enabled
It is good to check if Nginx configuration is valid before starting the service.
nginx -t sudo systemctl nginx start sudo systemctl nginx status
If we did everything correctly everything should work! We can test again with curl.
echo "Test3" > test3.txt curl -F'file=@test3.txt' http://localhost
Now check the response URL.
curl http://localhost/E.txt # Test3
Nice!
The setup is complete and 0x0 now does everything it says on the box. However, there are a couple of things left to do. The null pointer comes with a Textual [8] CLI application for file and URL administration. All dependencies are already installed during the setup, so we need to run it.
python3.11 mod.py
If you need to peek inside the database, you can use the installed SQLite package.
sqlite3 database.sqlite
Null pointer service has a signature feature where all files have an implicit retention policy. When the time expires they are removed. This is done by calling the 'prune' Flask command. To avoid doing this manually we will set up a separate service with this task and enable a timer for it. We can do it in the same way we did for the 0x0 service. First, create a new script that will be invoked by the service - '0x0-prune.sh' (with added executable permissions).
#!/bin/bash export FLASK_APP=fhost /home/nullpointer/0x0/.venv/bin/flask prune.
Create a new service script in '/etc/systemd/system/0x0-prune.service'. We can just copy the one from the 0x0 directory and adjust it.
[Unit] Description=Prune 0x0 files After=remote-fs.target [Service] Type=oneshot User=nullpointer WorkingDirectory=/home/nullpointer/0x0 BindPaths=/home/nullpointer/0x0 ExecStart=/home/nullpointer/0x0-prune.sh ProtectProc=noaccess ProtectSystem=strict ProtectHome=tmpfs PrivateTmp=true PrivateUsers=true ProtectKernelLogs=true LockPersonality=true [Install] WantedBy=multi-user.target
In the same place, we define a timer 0x0-prune.timer.
[Unit] Description=Prune 0x0 files [Timer] OnCalendar=daily Unit=0x0-prune.service Persistent=true [Install] WanterBy=timers.target
Timer configuration can be anything, I configured it to execute daily. Now let's enable them.
sudo systemctl enable 0x0-prune sudo systemctl enable 0x0-prune.timer sudo systemctl start 0x0-prune.timer
Now we are safe from running out of disk memory.
Well, that is it! We fully configured a local instance of 0x0 service. It does support virus and NSFW scanning, but I will leave that as an exercise for the reader.
Happy local file sharing!
--
theparanoidtimes@posteo.net
Released under CC BY-SA
Made in 🇷🇸