OK, so I’m trying to run my own syncserver. It’s separate from the account server, so you still need a Firefox Account. If you do things in the wrong order your browser syncs immediately – to their own sync server, which is not what I wanted. 😭
Anyway, let’s start from the beginning.
First, check out the project, syncserver. It uses Python 2. Oh well!
The Sync Server software runs using python 2.7, and the build process requires make and virtualenv.
I looked at this blog post by Matthias Dietel.
apt install python-dev git-core python-virtualenv g++
Easy enough! Build the project:
cd ~/src git clone https://github.com/mozilla-services/syncserver.git cd syncserver make build make test
Edit the `syncserver.ini` file. I’ve made the following changes:
diff --git a/syncserver.ini b/syncserver.ini
index ccf1ae0..21d0586 100644
--- a/syncserver.ini
+++ b/syncserver.ini
@@ -1,6 +1,6 @@
[server:main]
use = egg:gunicorn
-host = 0.0.0.0
+host = localhost
port = 5000
workers = 1
timeout = 30
@@ -11,7 +11,7 @@ use = egg:syncserver
[syncserver]
# This must be edited to point to the public URL of your server,
# i.e. the URL as seen by Firefox.
-public_url = http://localhost:5000/
+public_url = https://alexschroeder.ch/sync
# By default, syncserver will accept identity assertions issued by
# any BrowserID issuer. The line below restricts it to accept assertions
@@ -20,7 +20,7 @@ public_url = http://localhost:5000/
identity_provider = https://accounts.firefox.com/
# This defines the database in which to store all server data.
-#sqluri = sqlite:////tmp/syncserver.db
+sqluri = sqlite:////home/alex/syncserver.db
#sqluri = pymysql://sample_user:sample_password@127.0.0.1/syncstorage
# This is a secret key used for signing authentication tokens.
@@ -34,7 +34,7 @@ identity_provider = https://accounts.firefox.com/
# Set this to "false" to disable new-user signups on the server.
# Only requests by existing accounts will be honoured.
-# allow_new_users = false
+allow_new_users = new
# Set this to "true" to work around a mismatch between public_url and
# the application URL as seen by python, which can happen in certain reverse-
@@ -55,4 +55,4 @@ force_wsgi_environ = false
# MySQL based syncstorage-rs 1.5 server hosted at http://localhost:8000/1.5
# "{node}/1.5/{uid}"
-# sync-1.5 = "http://localhost:8000/1.5/{uid}"
+sync-1.5 = "https://alexschroeder.ch/sync/token/{node}/sync/1.5/{uid}"
That is to say:
1. the server only listens on `localhost:5000`
2. the server can be reached at `https://alexschroeder.ch/sync`
3. it uses a SQLite db at `/home/alex/syncserver.db`
4. it allows new users to sync their stuff
5. and I had to change that endpoint at the end... 🤷
So now I need to set up my website. I use Apache and I already have proxy stuff set up. All I had to add was this:
ProxyPass /sync http://localhost:5000/
Thus, any request starting with `/sync` to my existing website gets sent to port 5000 on `localhost` which happens to be the URL where the sync server is running.
Next, open `about:config` in the browser and change `identity.sync.tokenserver.uri` to `https://alexschroeder.ch/sync/token/1.0/sync/1.5` – then I’m ready to open the browser preferences and sign in.
If everything worked, you should see your display name in the top spot of the hamburger menu. 🍔 😁
If you see a warning sign after a second or two, stuff isn’t working.
It all sounds so easy now, but it took me a while to get everything working, believe me. 😭
Now I’m seeing the following errors in the log output of `make serve`:
ERROR:syncserver:The public_url setting doesn't match the application url.
This will almost certainly cause authentication failures!
public_url setting is: https://alexschroeder.ch/sync
application url is: http://localhost:5000/sync
You can disable this check by setting the force_wsgi_environ
option in your config file, but do so at your own risk.
I have no idea what it means. I’m thinking it works anyway? I’m making an additional change to `syncserver.ini` just to be sure.
force_wsgi_environ = true
The instructions tell me that we only need this in certain reverse-proxy setups but this *is* a reverse-proxy setup so I think I should be fine?
Now I wonder how to leave the sync server running... Perhaps it’s OK to just do this every now and then?
Now that I have it running I wonder about the utility of it all:
1. there’s no need for me to mix my private Firefox and my work Firefox bookmarks and logins
2. I currently cannot make Firefox on my iPad and my iPhone use a different sync address
Oh Apple, the golden prison, where I’m safe, but also where I’m locked in. Sigh. 😔
But wait… This issue is closed: Add ability to set custom identity.sync.tokenserver.uri for self-hosted Sync #5006?
Add ability to set custom identity.sync.tokenserver.uri for self-hosted Sync #5006
One of the last comments on that issue, by user *fireglow*:
Just to chime in a little: Python 2.7, the flavor of Python the sync software is written in, will go End-of-Life at the end of this month, year, and decade. Mozilla already has indicated that there will be no rewrite for Python 3. I gather there’s a rewrite of these services in Rust in the works, at https://github.com/mozilla-services/syncstorage-rs It’s as of now unclear to me how all these parts will fit together in a way so us self-hosters will be able migrate over.
https://github.com/mozilla-services/syncstorage-rs
Oh wow.
I think I’m going to stop all of this. It’s making my head hurt. I checked the Rust rewrite project. Apparently you need to use either MySQL or Spanner as your DB. And with that, I’ve decided that not running this service and not syncing my stuff is probably for the best.
I undid all the changes, stopped the server, uninstalled the software, and deleted the Firefox account again.
#Administration #Firefox
(Please contact me if you want to remove your comment.)
⁂
Hi Alex,
You can set a custom sync server amd token server in Firefox ios.
To access it:
Note that I haven’t done this, as I don’t (yet) have a sync server.
– dgold 2020-02-16 19:25 UTC
---
I get the feeling that this doesn’t work on the iPad... I get a bunch of extra options but no place where I can set the sync server URL.
– Alex Schroeder 2020-02-16 19:28 UTC
---
I don’t know what to say to you. I set up my server in the way you described (except I didn’t set the last two options in the file)
I accessed the Advanced Settings in FFox on iPadOS. All works perfectly.
– dgold 2020-02-16 20:24 UTC
---
How weird. I just updated Firefox on the iPad and it does look different, but I still don’t know where to start. Do you feel like pasting a screenshot, or can you explain it like I’m super confused? Because I feel like I am!
1. open Firefox on the iPad
2. click hamburger menu
3. scroll down until you see “Version 22.0 (17157)
4. tap this section five time until the list of options changes
5. now what?
At the top, I see three items:
1. my email address and a note saying I need to enter my password (but since I deleted my account, clicking on this item just takes me to accounts.firefox.com where I could probably recreated my account and start sync immediately but with their servers)
2. error diagnosis: ask for an upgrade (translating from German)
3. error diagnosis: forgot sync-status (translating from German)
No other item in these settings seem related to sync. What am I not seeing?
– Alex Schroeder 2020-02-16 22:08 UTC
---
After you tap the section until the lost of options changes, there should be a new menu item at the top of the Preferences under “Firefox Account” called “Advanced Sync Settings”.
In that modal dialog you can set the address of your sync server, in the form https://example.com/sync/token/1.0/sync/1.5
https://example.com/sync/token/1.0/sync/1.5
I didn’t create a FFox account until after I’d created my sync server.
– dgold 2020-02-17 11:00 UTC
---
Oh wow, I just tried it on the iPhone and it worked. Tapped on the Version five times, it automatically scrolls to the top showing me the things I listed previously, but now I scroll back to the bottom and do it *again* and now I see new items at the top:
1. Sign in to Sync
2. Advanced Sync Settings
3. Debug: use stage servers
Hah, thanks for pointing me at it again. I might give this another try.
– Alex Schroeder 2020-02-17 11:40 UTC
---
Well, I think it’s working:
Firefox on iOS Advanced Sync Settings
Firefox on iOS Advanced Settings
On the iPad, the situation was different. I had to do the following:
1. log into sync even though I couldn’t change the sync server and Firefox synced with the default sync server
2. log out of sync
3. activate the advanced settings again
4. now you can change the sync server
5. log into sync *again*
🙄
I’m looking at Run your own Sync-1.5 Server right now and I’m wondering about the WSGI setup. Are you using it?
I guess I don’t need to use it as my setup is basically Apache + Gunicorn. 😀
Except have to turn on `force_wsgi_environ`. As far as I can see in the Apache documentation on Reverse Proxy Request Headers but then again, the nginx setup instructions mention `X-Forwarded-Proto` and `X-Real-IP`, both of which the Apache docs don’t mention. Gaaaaah! 😭
Setup for Monit:
check process syncserver matching syncserver
start program = "/bin/bash -c '(cd /home/alex/src/syncserver && exec /usr/bin/make serve 2&>1 syncserver.log)' &"
stop program = "/usr/bin/killall gunicorn"
if failed
host alexschroeder.ch
port 443
type tcpssl
protocol http
request "/sync"
for 5 cycles
then restart
if totalmem > 100 MB for 5 cycles then restart
if 3 restarts within 15 cycles then stop
– Alex Schroeder 2020-02-17 21:19 UTC
---
thank you very much for your post! i learned a lot here!
one question: does your setup still work on your iPad?
i have the following sync (token) server running: https://github.com/crazy-max/docker-firefox-syncserver
https://github.com/crazy-max/docker-firefox-syncserver
but i cant get past the following issue:
https://github.com/mozilla-mobile/firefox-ios/issues/6535
all the best, ma
– ma 2020-07-31 10:05 UTC
---
I don’t remember seeing any error messages about OAuth. In fact, I don’t remember seeing any errors at all. It was all silent fails.
– Alex 2020-07-31 13:03 UTC
---
It turns out that my Firefox on the laptop claims to logged in, but I can’t actually sync with other devices. They are listed, and I can send them my tabs, but nothing happens. And when I check my Firefox accounts, it claims that the phone and the tablet last synced “a year ago”. I know that the tablet and the phone seem to be synced.
This does not inspire confidence, at all.
I think what I really need to be doing is “sync” all my stuff using email. Just send myself some email if I want to keep a PDF or a bookmark or whatever.
Also, uninstall this shit.
I know that the tablet and the phone were synced because they started complaining immediately.
I really like the idea of exporting bookmarks as HTML files. To bad that on iOS, the browser does not allow that.
Deleted my Firefox account, too.
All of the above not entirely unrelated to my current frustration. 2021-07-28 Browsers.
– Alex 2021-07-28 12:49 UTC
---
Also, a Python 2.7 dependency.
– Alex 2021-11-18 10:44 UTC