Rclone as restic server for some 1000 hosts?

Help and Support
1

What is the problem you are having with rclone?

rclone can be used as a server for restic:

The problem is:

I am an admin at the computing centre of my university. We have 50 autarchic institutes with several 1000 hosts for which we offer central services like backup.

I can create restic/rclone accounts for each institute, but not for each of their hosts. WAY too many. I have no access to the institutes hosts. They have their own admins.

Is there an option for rclone server so each admin can create sub-accounts usable as restic repositories?

One restic repository per institute is not possible beacuse some restic commands use global locking and this prohibits independant cronjobs on the client hosts.

Which cloud storage system are you using? (eg Google Drive)

Local RAID (1 PB)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

restic

2

rclone serve restic can take a standard htpasswd file.

If you had a little external web app to manage that, or a script to joing multiple htaccess files together (one per department) you could do it like that.

I don't think you can do this without a little external software.

3

If you go with the htpasswd approach, I would probably keep the delegation outside rclone and make it boring: one small file/list per institute, then generate the combined htpasswd file from those fragments and reload the service. That avoids giving institute admins direct access to the whole auth file. Iā€™d also make the repository path include the institute and host name, and test that restic locking is only happening inside that final repo path, not at the shared institute prefix. It is a bit of glue code, but it seems safer than one huge shared repository per institute.

4

The main problem is still there: how can an institute admin add new repository (paths)?

Without that the rclone/restic server admin (that's me) getting involved?

The institute admins should be able to add new host repositories by themselves.

5

There is more than one way of doing this.

You probably want to be using this flag

The --private-repos flag can be used to limit users to repositories starting with a path of /<username>/.

Then you could make 50 usernames and passwords for the institutions. They could then manage their own subspace - you set the sub path using restic init and it doesn't need creating in advance or anything so the institution admins would be self sufficient.

This will mean anyone who has access to a user/pass will be able to read all the backups for the institution. This may/may not be acceptable.

You could go with the multiple htpasswd file approach also and get each institution to manage it and you write a small script to combine the htpassword files and restart the server.

Or you could go with a mixture of the two methods.

If you need more help your org might be interested in taking out a support contract which can help you get answers quicker and keeps the rclone project sustainable.

6

That's it: easy solution! :slight_smile: