💾 Archived View for thebackupbox.net › ~epoch › blog › veilid captured on 2024-12-17 at 10:11:33. Gemini links have been rewritten to link to archived content

-=-=-=-=-=-=-

poking at the thing

update post

a prequel

where I dump stuff while I'm poking

woke up this morning, booted my computer, and saw veilid-server had been failing to bootstrap.

so, ofc I bother someone on the internet with the problem before I actually look into it and get the "did you update it?" response.

ofc I had not.

I do that. and it still does the thing from before the update, and this is where I started poking.

I found that the bootstrap server is bootstrap.veilid.net. appropriate enough.

I dig the domain for the A and AAAA records, and ping to make sure they are all pingable.

$ dig -t ANY bootstrap.veilid.net +short | xargs -n1 ping -c3

They all were. are.

I went looking for the specific error message to see what the code around it was attempting.

grep found it.

./veilid-core/src/routing_table/tasks/bootstrap.rs:                            log_rtab!(warn "bootstrap server is not responding");

and it seemed to be not /just/ doing a lower layer network connectivity test, it was also checking that some crypto stuff was right.

if !nr.signed_node_info_has_valid_signature(RoutingDomain::PublicInternet) {

and there was another log_rtab that used debug priority I didn't see, but included a bit more information.

To get the veilid-server running in debug mode I ran it as root. and it error'd because root isn't the right user to run it as.

and su doesn't want to use the veilid user because it is disable or something.

so I used a program I made a while ago called "supersu" that lets me set whatever UID/GID/etc I want and run any program I want with them. (it isnt suid, so it isn't like anyone can use it for anything bad, except root)

./supersu `id -u veilid` `id -g veilid` `which veilid-server` --debug

The debug message just included the bootstrap's VLD0: thing.

which wasn't very helpful.

There was also another error message that was showing up a lot about not being able to get the address of my own node.

2023-08-21T22:42:13.304828Z DEBUG net: no external address detection peers of type UDP:IPV4
2023-08-21T22:42:13.304839Z DEBUG net: couldn't get external address 1 for UDP IPV4
2023-08-21T22:42:13.304852Z DEBUG net: === update_ipv6_protocol_dialinfo UDP tries_left=0 ===
2023-08-21T22:42:13.304860Z DEBUG net: no external address detection peers of type UDP:IPV6
2023-08-21T22:42:13.304863Z DEBUG net: couldn't get external address 1 for UDP IPV6

Assuming it was possible to manually set those values, I looked at the default config for what they'd likely be called using that same supersu line, but this time running veilid-server like:

veilid-server --dump-config

found network:protocol:{tcp,udp}:public_address and set them in my /etc/veilid-server/veilid-server.conf to appropriate values.

  network:
    protocol:
      tcp:
        public_address: 'lin.thebackupbox.net:5150'
      udp:
        public_address: 'lin.thebackupbox.net:5150'

I'm assuming I did that right, it didn't give errors about bad syntax or anything, but it didn't fix the bootstrap problem.

at this time I also forwarded both tcp and udp port 5150 to lin and made sure the port forward worked.

assuming it might've just been the bootstrap being setup wrong somehow, I remembered hearing something about no nodes in veilid being special.

so I tested to see if that applied to bootstrap nodes too.

it did not appear to.

used the --bootstrap flag for veilid-server to set the bootstrap node to some other random node I'd noticed as a peer the other day while I was still connecting.

cdc.ninja

veilid-server wasn't able to use that as a bootstrap, so I started looking into what it takes for something to be a bootstrap.

error messages had said something about txt_lookup so I started poking at TXT records for the bootstrap domain.

$ dig -t TXT bootstrap.veilid.net +short
"1,2"

that doesn't seem very useful. :/

I looked in the source again for more clues. found this:

        // Bootstrap TXT Record Format Version 0:
        // txt_version|envelope_support|node_ids|hostname|dialinfoshort*
        //
        // Split bootstrap node record by '|' and then lists by ','. Example:
        // 0|0|VLD0:7lxDEabK_qgjbe38RtBa3IZLrud84P6NhGP-pRTZzdQ|bootstrap-1.dev.veilid.net|T5150,U5150,W5150/ws

the bootstrap-1.dev.veilid.net made me assume the 1 and 2 were used inside of some record remplate and eventually guessed the template is just to use the comma separated values as subdomains.

$ dig -t TXT {1,2}.bootstrap.veilid.net +short
"0|0|VLD0:m5OY1uhPTq2VWhpYJASmzATsKTC7eZBQmyNs6tRJMmA|bootstrap-1.veilid.net|T5150,U5150,W5150/ws"
"0|0|VLD0:6-FfH7TPb70U-JntwjHS7XqTCMK0lhVqPQ17dJuwlBM|bootstrap-2.veilid.net|T5150,U5150,W5150/ws"

so I tried to make my own TXT records in that format for my own server so I could be a bootstrap myself.

the records are for lin.thebackupbox.net, and I probably did something wrong, but it might work as a bootstrap.