Naive question: how do you get the public key on the machine, without ssh-copy-id?
Yeah, that's the tricky part. You either copy the public key over first /then/ disable password authentication, or you use some other means of copying it... Nextcloud, Syncthing, USB, floppy disk...