💾 Archived View for jacksonchen666.com › posts › 2023-11-19 › 22-49-58 › index.gmi captured on 2024-05-10 at 10:35:24. Gemini links have been rewritten to link to archived content

View Raw

More Information

⬅️ Previous capture (2023-12-28)

-=-=-=-=-=-=-

Recovering From Syncthing Deleting My Data

2023-11-19 21:49:58Z

Syncthing-fork

So I decided to try Syncthing-fork to see if it was any better at battery life (spoilers: I have no idea).

To my surprise, it has much more tweaking options and stuff, plus more information.

However, I now wanted to migrate from Syncthing to Syncthing-fork because now I wanted to use Syncthing-fork.

The easiest way was to just export the config from Syncthing and import it into Syncthing-fork. Easy and done.

Data loss

I explore around the app, saw a folder with the "revert local changes" button, promptly pressed it and didn't think about it more.

Much later, I wanted to use my password manager to login to GitHub to find the code that defines the mode function in libqalculate (actually the specifics don't matter, I just wanted to use my password manager). I open my password manager app, and it says file not found.

So I went ahead and looked in my Sync folder (at the root of user facing internal storage). I found... nothing but .stfolder and .stversions.

Well, fuck. I've just lost data, and setting it all back up was going to be a pain.

Complaints

I file an issue about the data loss I had, then move on to some other thing.

The issue I filed

Data recovery

Later, I look into the .stversions folder. And I realize that my data is literally right there. It's all there.

So I made a copy of all my data in .stversions by doing an rsync from my phone to my computer in Termux. I think it took over 5 hours.

(In hindsight, I did not need to make a copy of 100GB of data, only the filenames. But hey, who knows what the future can hold.)

Anyways, looking at the filenames, I came up with this python program to change the name to remove the syncthing time marker:

import re

filenames = open("list").read()

result = re.sub(r"(.*)~\d{8}-\d{6}(.*)", r"\1\2", filenames, 0, re.MULTILINE)

if result:
    with open("list2", "w") as f:
        f.write(result)

(Code is under public domain/CC0-1.0 by the way)

The program gets file contents from a file called "list", does some regex (which should only remove the last date and time part in the filename added by Syncthing) and writes it to "list2".

Renaming the file was done with a shell script with a bunch of `mv` commands, not the python program.

After making the (hyperspecific to my setup and files) shell script, I put it on my phone and transferred it.

I then also had to restart my phone to fix Syncthing-fork not using the right private keys.

And my data is back, and Syncthing-fork is working.

Bye bye Syncthing (not fork)

After all that, I just deleted Syncthing (not the fork one) from my phone, because I already had another version (it was the fork version).

Post-mortem

So how did Syncthing just delete all my data? And keep it?

Well, here's how it went:

1. A receive only folder was set to the entire Sync folder

2. I revert local changes

3. It deletes (and saves a copy into .stversions)

4. I realize the issue (trying to use my password manager)

I have no idea how one of my receive only folders was set to the entire Sync folder, but I guess it was in that state.

public inbox (comments and discussions)

public inbox archives

(mailing list etiquette for public inbox)