Remote file storage with OpenBSD and rclone

Summary: If you’re looking for a way to keep your files on some remote server (or servers) but privacy issues and/or vendor lock-in drive you nuts, here’s an alternative: rclone. Since OpenBSD is my platform of choice, I’ll be using that in the following write up, but feel free to use your favourite OS.

Server (OpenBSD 7.0) configuration

If you have sshd running and sftp enabled, you’re good to go.

Let’s make dedicated account for this exercise. Don’t use passwords.

$ doas useradd -m rclone-test

I assume you have disk space monitoring and quotas in place, so I’m not covering those topics here.

Client configuration (MacOS Big Sur)

It should also work with Linux and BSDs, just use relevant package manager instead of homebrew.

SSH key

First of all, create separate SSH key for use with rclone:

$ ssh-keygen -m PEM -t ed25519 -f ~/.ssh/rclone-test.pem
Generating public/private ed25519 key pair.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /Users/tmarc/.ssh/rclone-test.pem
Your public key has been saved in /Users/tmarc/.ssh/rclone-test.pem.pub

Don’t forget to add the key to your ssh-agent!

$ ssh-add ~/.ssh/rclone-test.pem

And copy/append PUBLIC (!) key to the ~/.ssh/authorized_keys file of rclone-test account on the OpenBSD server.

Rclone installation and setup

If you use homebrew, then you can get rclone with:

$ brew install rclone

Now most people would go through lengthy process of answering questions of rclone config. We’ll just make our config on our own, it will be much faster. With your favourite editor open ~/.config/rclone/rclone.conf and fill it out using following template:

[openbsd-test]
type = sftp
host = YOUR_SERVER_FQDN
user = rclone-test
key_pem = -----BEGIN OPENSSH PRIVATE KEY-----<YOUR_PRIVATE_KEY_SEE_BELOW>-----END OPENSSH PRIVATE KEY-----\n
key_use_agent = true
use_insecure_cipher = true
key_file = ~/.ssh/rclone-test.pem
md5sum_command = md5 -r
sha1sum_command = sha1 -r

To fill the key_pem parameter use the handy shortcut from rclone’s docs to get the key in one line form, ie:

$ awk '{printf "%s\\n", $0}' ~/.ssh/rclone-test.pem

Try it out

Choose some directory to copy to remote server and launch:

rclone copy some_directory openbsd-test:

Files should be flying nicely:

Transferred:        2.679 MiB / 2.679 MiB, 100%, 146.606 KiB/s, ETA 0s
Transferred:           72 / 72, 100%
Elapsed time:        20.7s

Possible errors

ssh: unable to authenticate, attempted methods [none publickey]

Complete error message:

NewFs: couldn't connect SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

Logs are your friend and remind you that perhaps you forgot to add rclone-test user to AllowedUsers directive in your mega-hardened sshd configuration?

Error: NewFs: couldn’t initialise SFTP: packet too long

This error message is very disorienting. Check if the account on OpenBSD server has it’s shell disabled (eg. set to /sbin/nologin)

Some transfers seem to be very slow or stalled

When rclone usually works, but sometimes doesn’t and few small files give you wild transfers and ETA like:

Transferred:       22.915 KiB / 56.184 KiB, 41%, 14 B/s, ETA 40m21s

Again, logs are your friend: check if your sshd isn’t being bruteforced or DoSed, triggering it’s connection throttling.