Using unraid server as a backend

Help and Support
1

Hi,

First off, apologies if this has already been written about. Most of my searches are dominated by using rclone ON an unraid server. However, this post is specifically about using an unraid server as an rclone backend (so other servers can sync to it).

I am relatively new to unraid, however I have previously used rclone and would like to continue using it.

So far I have found two potential solutions to this problem:

  1. Use tailscale to connect to put the unraid server and the syncing server on the same "network". Then use an SMB share as the rclone backend.
  2. Use Minio on unraid server to setup an s3 like public endpoint for syncing. (Or maybe I can still use tailscale to make this non-public)

Option 1 seems to be the simplest approach from me, especially considering tailscale helps me avoid port forwarding and the like. But looking at the backend overview, it seems like SMB is missing some features that the backend I currently use (gdrive) has. I don't know how big a deal that is, but something I'm hesitant about since I'd love to just stick with my current rclone configuration and just switch backends.

Option 2 Seems like it would have all the features I'm used to, however I'm not sure if there are any other negative performance consequences of using Minio as an intermediary layer.

Keen to hear advice from anyone who has experience with setting up storage servers.

2

rclone does not offer smb as storage system. does not have rclone serve smb.

option 1 is the best:

  • unraid, like any linux server, has samba, sftp, etc...
  • the files are in their original state, can be accessed by any client/server/backup.tool

minio uses s3 as its storage system.
rclone has great support for s3.

3

Am I misunderstanding what this page is claiming?

I think the "samba, sftp, etc..." bit is what I'm asking about. As I mentioned in the original post, it doesn't look like he SMB support by rclone has features I'm typically used to on the gdrive one (e.g. modtime). I'm not sure how important that is and trying to understand what the best protocol is to use.

4

Apologies, I've also edited the original post since the formatting was messed up. Hopefully it's a little clearer.

5

rclone, as a client copy tool, works great over tailscale, works great with servers such as samba, minio, sftp, etc...
choose whatever server best fits your overall needs.

modtime is very important and even more so, for backups, checksums.
rclone supports that for smb/samba server, sftp server and minio server.

6

The link I provided indicates rclone does not support modtime for smb.

Again, this is what I'm looking for advice on since I'm not sure what the tradeoffs are.

7

ok, let's compare the ways rclone can access a smb/samba server.

  1. rclone's samba remote. as you noted, lacks many basic features, would not use.
rclone config smbserver:
[smbserver]
type = smb
host = XXX
user = XXX
pass = XXX

rclone copy file.ext smbserver:share -vv
DEBUG : rclone: Version "v1.64.0" starting with parameters ["rclone" "copy" "file.ext" "smbserver:share" "-vv"]
DEBUG : Creating backend with remote "file.ext"
DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
DEBUG : fs cache: adding new entry for parent of "file.ext", "/root"
DEBUG : Creating backend with remote "smbserver:share"
DEBUG : smb://username@server:445/share: Using root directory "share"
DEBUG : file.ext: Need to transfer - File not found at Destination
INFO  : file.ext: Copied (new)
  1. mounting the smb/samba share. supports modtime, checksums, --inplace, more...
mount.cifs -o user=username,pass=password //smbserver/share /mnt/share

rclone copy file.ext /mnt/share -vv
DEBUG : rclone: Version "v1.64.0" starting with parameters ["rclone" "copy" "file.ext" "/mnt/share" "-vv"]
DEBUG : Creating backend with remote "file.ext"
DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
DEBUG : fs cache: adding new entry for parent of "file.ext", "/root"
DEBUG : Creating backend with remote "/mnt/share"
DEBUG : file.ext: Need to transfer - File not found at Destination
DEBUG : file.ext: md5 = b026324c6904b2a9cb4b88d6d61c81d1 OK
DEBUG : file.ext.xibuvak3.partial: renamed to: file.ext
INFO  : file.ext: Copied (new)
8

These two options are a bit extreme - SMB has limited features and Minio is not trivial to setup/maintain properly (also resources intensive).

IMO the easiest would be to run SFTP server on Unraid linux box or use rclone serve webdav

9

Ah that's very interesting, I think I missed that nuance. Do you happen to know where the documentation talks about these protocols for mounting vs remote? I want to make sure I'm understanding what the differences are between my current setup and smb.

Webdav seems to not support the same things as sftp, so I guess sftp would be easier, good point! The only reason I landed on minio initially is because rclone seemed to support the most things for s3 like storage backends. But it seems like sftp may cover me for most things.

I think I may go with that option instead!

10

yes, it is very interesting.

as for understand the differences, the docs only go far, so i try to rely on my own simple testing.

for example, when you first posted this, i did a test
"it doesn't look like he SMB support by rclone has features I'm typically used to on the gdrive one (e.g. modtime)"

i found modtime support on smb remote. for windows smb server and for linux samba server.

rclone lsl smbserver:backup/zork
        2 2023-09-14 21:10:53.631747700 file.ext

rclone touch smbserver:backup/zork

rclone lsl smbserver:backup/zork
        2 2023-09-16 14:39:58.884043200 file.ext

as for checksums, i posted that test in previous post.

fwiw, based on my personal experience - for backups, over tailscale/vpn. sftp is not the way to go.
small packets, high latency, slow directory listings.
tho, rclone has very good support for sftp: modtime, checksum, --inplace, keyfile, etc...

11

Thanks! I massively appreciate your help here.

Would you recommend smb mount over sftp then? Or is there another option you prefer over all?

12

sftp on linux is rock stable, native, built-in and easy to setup.
so why not give that a try. might be all you need

in my case, i choose smb/samba. modern protocol, works well over vpn and supported everywhere.
rclone and most any copy/backup tool has great support, including veeam.

13

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.