How to mount an Rclone remote to a mounted SMB share?

Xenar · March 25, 2024, 8:32pm

What is the problem you are having with rclone?

Rclone is installed on a Linux VM
I have configured two remotes as a non-root user (uid=1000,gid=1000), one for Microsoft OneDrive and another for Google Drive
I want to mount automatically at startup these remotes on an already mounted share
The already mounted share is an SMB share on a TrueNAS CORE VM with two folders inside, one for OneDrive and the other for Google Drive
Here is my fstab configuration to mount this SMB share:

//192.168.X.X/rclone-share  /mnt/rclone-share  cifs  credentials=/home/non-root-user/.smbcredentials,uid=1000,gid=1000  0  0

---> I don't know what is the proper way to mount the two remotes on this already mounted share:
-> remote "onedrive" on /mnt/rclone-share/OneDrive
-> remote "googledrive" on /mnt/rclone-share/GoogleDrive

Run the command 'rclone version' and share the full output of the command.

rclone v1.66.0

os/version: debian 12.5 (64 bit)
os/kernel: 6.1.0-18-amd64 (x86_64)
os/type: linux
os/arch: amd64
go/version: go1.22.1
go/linking: static
go/tags: none

Are you on the latest version of rclone?
--> Yes

Which cloud storage system are you using?

Microsoft OneDrive
Google Drive

The command you were trying to run

Maybe something like this?
But I am not sure at all if it works and it will not be permanent. I want to make this working at startup.
And maybe with a delay to be sure the SMB share is mounted before mounting the remote on it.

rclone mount onedrive: /mnt/rclone-share/OneDrive --config /home/non-root-user/.config/rclone/rclone.conf --vfs-cache-mode writes  --allow-other --daemon
rclone mount googledrive: /mnt/rclone-share/GoogleDrive --config /home/non-root-user/.config/rclone/rclone.conf --vfs-cache-mode writes  --allow-other --daemon

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[googledrive]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive =

[onedrive]
type = onedrive
client_id = XXX
client_secret = XXX
token = XXX
drive_id = XXX
drive_type = personal

A log from the command that you were trying to run with the `-vv` flag

2024/03/26 00:28:13 DEBUG : rclone: Version "v1.66.0" starting with parameters ["rclone" "mount" "onedrive:" "/mnt/rclone-share/OneDrive" "--config" "/home/non-root-user/.config/rclone/rclone.conf" "--vfs-cache-mode" "writes" "--allow-other" "--daemon" "-vv"]
2024/03/26 00:28:13 DEBUG : Creating backend with remote "onedrive:"
2024/03/26 00:28:13 DEBUG : Using config file from "/home/non-root-user/.config/rclone/rclone.conf"
2024/03/26 00:28:13 DEBUG : onedrive: Loaded invalid token from config file - ignoring
2024/03/26 00:28:16 DEBUG : Saving config "token" in section "onedrive" of the config file
2024/03/26 00:28:16 DEBUG : onedrive: Saved new token in config file
2024/03/26 00:28:19 ERROR : Daemon timed out. Failed to terminate daemon pid 2902: os: process already finished
2024/03/26 00:28:19 Fatal error: daemon exited with error code 1

asdffdsa · March 25, 2024, 8:41pm

welcome to the forum,

for testing, to simply things about

pick a single remote to mount
run rclone mount on the command line. not using fstab, not using --daemon

Xenar · March 25, 2024, 8:49pm

Thank you for your welcome and your help.

2024/03/26 00:44:33 DEBUG : rclone: Version "v1.66.0" starting with parameters ["rclone" "mount" "onedrive:" "/home/non-root-user/onedrive_test" "--config" "/home/non-root-user/.config/rclone/rclone.conf" "--vfs-cache-mode" "writes" "-vv"]
2024/03/26 00:44:33 DEBUG : Creating backend with remote "onedrive:"
2024/03/26 00:44:33 DEBUG : Using config file from "/home/non-root-user/.config/rclone/rclone.conf"
2024/03/26 00:44:35 DEBUG : OneDrive root '': Next delta token is: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
2024/03/26 00:44:35 DEBUG : vfs cache: root is "/home/non-root-user/.cache/rclone"
2024/03/26 00:44:35 DEBUG : vfs cache: data root is "/home/non-root-user/.cache/rclone/vfs/onedrive"
2024/03/26 00:44:35 DEBUG : vfs cache: metadata root is "/home/non-root-user/.cache/rclone/vfsMeta/onedrive"
2024/03/26 00:44:35 DEBUG : Creating backend with remote "/home/non-root-user/.cache/rclone/vfs/onedrive/"
2024/03/26 00:44:35 DEBUG : fs cache: renaming cache item "/home/non-root-user/.cache/rclone/vfs/onedrive/" to be canonical "/home/non-root-user/.cache/rclone/vfs/onedrive"
2024/03/26 00:44:35 DEBUG : Creating backend with remote "/home/non-root-user/.cache/rclone/vfsMeta/onedrive/"
2024/03/26 00:44:35 DEBUG : fs cache: renaming cache item "/home/non-root-user/.cache/rclone/vfsMeta/onedrive/" to be canonical "/home/non-root-user/.cache/rclone/vfsMeta/onedrive"
2024/03/26 00:44:35 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2024/03/26 00:44:35 DEBUG : OneDrive root '': Mounting on "/home/non-root-user/onedrive_test"
2024/03/26 00:44:35 Fatal error: failed to mount FUSE fs: fusermount: exec: "fusermount3": executable file not found in $PATH

It seems I have to begin by an installation of a package for fuse I suppose.
It's really late where I live, I will test it tomorrow and report here.

asdffdsa · March 25, 2024, 8:54pm

i had the same issue with a synology nas.
the same issue has affected unix and older linux systems

if not able to install fusermount3, this worked for me.
ln -s /bin/fusermount /bin/fusermount3

Xenar · March 25, 2024, 9:05pm

I did install the "fuse3" package. The remote config seems to be working:

2024/03/26 00:59:21 DEBUG : rclone: Version "v1.66.0" starting with parameters ["rclone" "mount" "onedrive:" "/home/non-root-user/onedrive_test" "--config" "/home/non-root-user/.config/rclone/rclone.conf" "--vfs-cache-mode" "writes" "-vv"]
2024/03/26 00:59:21 DEBUG : Creating backend with remote "onedrive:"
2024/03/26 00:59:21 DEBUG : Using config file from "/home/non-root-user/.config/rclone/rclone.conf"
2024/03/26 00:59:23 DEBUG : OneDrive root '': Next delta token is: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
2024/03/26 00:59:23 DEBUG : vfs cache: root is "/home/non-root-user/.cache/rclone"
2024/03/26 00:59:23 DEBUG : vfs cache: data root is "/home/non-root-user/.cache/rclone/vfs/onedrive"
2024/03/26 00:59:23 DEBUG : vfs cache: metadata root is "/home/non-root-user/.cache/rclone/vfsMeta/onedrive"
2024/03/26 00:59:23 DEBUG : Creating backend with remote "/home/non-root-user/.cache/rclone/vfs/onedrive/"
2024/03/26 00:59:23 DEBUG : fs cache: renaming cache item "/home/non-root-user/.cache/rclone/vfs/onedrive/" to be canonical "/home/non-root-user/.cache/rclone/vfs/onedrive"
2024/03/26 00:59:23 DEBUG : Creating backend with remote "/home/non-root-user/.cache/rclone/vfsMeta/onedrive/"
2024/03/26 00:59:23 DEBUG : fs cache: renaming cache item "/home/non-root-user/.cache/rclone/vfsMeta/onedrive/" to be canonical "/home/non-root-user/.cache/rclone/vfsMeta/onedrive"
2024/03/26 00:59:23 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2024/03/26 00:59:23 DEBUG : OneDrive root '': Mounting on "/home/non-root-user/onedrive_test"
2024/03/26 00:59:23 DEBUG : : Root:
2024/03/26 00:59:23 DEBUG : : >Root: node=/, err=<nil>
2024/03/26 01:00:23 DEBUG : OneDrive root '': Checking for changes on remote
2024/03/26 01:00:23 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2024/03/26 01:01:23 DEBUG : OneDrive root '': Checking for changes on remote
2024/03/26 01:01:23 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
[...]

asdffdsa · March 25, 2024, 9:08pm

ok, great.

i guess the next step is figure out if you can rclone mount inside a smb share?
if not, there are workarounds.

Xenar · March 26, 2024, 5:06pm

I do not see any error:

2024/03/26 20:50:41 DEBUG : rclone: Version "v1.66.0" starting with parameters ["rclone" "mount" "onedrive:" "/mnt/rclone-share/OneDrive" "--config" "/home/non-root-user/.config/rclone/rclone.conf" "--vfs-cache-mode" "writes" "-vv"]
2024/03/26 20:50:41 DEBUG : Creating backend with remote "onedrive:"
2024/03/26 20:50:41 DEBUG : Using config file from "/home/non-root-user/.config/rclone/rclone.conf"
2024/03/26 20:50:41 DEBUG : onedrive: Loaded invalid token from config file - ignoring
2024/03/26 20:50:43 DEBUG : Saving config "token" in section "onedrive" of the config file
2024/03/26 20:50:43 DEBUG : onedrive: Saved new token in config file
2024/03/26 20:50:44 DEBUG : OneDrive root '': Next delta token is: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
2024/03/26 20:50:44 DEBUG : vfs cache: root is "/home/non-root-user/.cache/rclone"
2024/03/26 20:50:44 DEBUG : vfs cache: data root is "/home/non-root-user/.cache/rclone/vfs/onedrive"
2024/03/26 20:50:44 DEBUG : vfs cache: metadata root is "/home/non-root-user/.cache/rclone/vfsMeta/onedrive"
2024/03/26 20:50:44 DEBUG : Creating backend with remote "/home/non-root-user/.cache/rclone/vfs/onedrive/"
2024/03/26 20:50:44 DEBUG : fs cache: renaming cache item "/home/non-root-user/.cache/rclone/vfs/onedrive/" to be canonical "/home/non-root-user/.cache/rclone/vfs/onedrive"
2024/03/26 20:50:44 DEBUG : Creating backend with remote "/home/non-root-user/.cache/rclone/vfsMeta/onedrive/"
2024/03/26 20:50:44 DEBUG : fs cache: renaming cache item "/home/non-root-user/.cache/rclone/vfsMeta/onedrive/" to be canonical "/home/non-root-user/.cache/rclone/vfsMeta/onedrive"
2024/03/26 20:50:44 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2024/03/26 20:50:44 DEBUG : OneDrive root '': Mounting on "/mnt/rclone-share/OneDrive"
2024/03/26 20:50:44 DEBUG : : Root:
2024/03/26 20:50:44 DEBUG : : >Root: node=/, err=<nil>
2024/03/26 20:51:44 DEBUG : OneDrive root '': Checking for changes on remote
2024/03/26 20:51:44 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2024/03/26 20:52:44 DEBUG : OneDrive root '': Checking for changes on remote
2024/03/26 20:52:44 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
[...]

But I don't understand, the local mounted folder, first with the test directory and second with the mounted SMB share, are empty, so Rclone doesn't download any of the files on the remote corresponding cloud provider.
Is it only a one-way sync? From the local folder to the cloud provider?
Or am I missing some flag to implement?
Is there a way to make it a two-ways sync, from the cloud provider to the local mounted folder and reverse?

Once I understand and implement correctly this, do you know the proper way on Linux to run this command in the background, or as a service, and to automatically run it at startup?
Maybe with a little delay to make sure it is started after the SMB share is mounted.

Thanks again for your help, I really appreciate it.

asdffdsa · March 26, 2024, 5:20pm

the command you posted, is missing --allow-other

yes, but first, need to have mount commands working.

Xenar · March 27, 2024, 3:14am

But if I understand correctly, when I was mounting Rclone in the local test folder:

"/home/non-root-user/onedrive_test"

As it was only mounted by Rclone, I think I do not need the "--allow-other" flag, but it was also empty.
Is the two-ways sync the default mode for Rclone?

kapitainsky · March 27, 2024, 7:42am

Yes. There is nothing like one-way mount:) Unless you mean by this read only.

In case you do not see content of your remote than something is not working.

asdffdsa · March 27, 2024, 11:53am

does a simple command produce output
rclone lsd onedrive: -vv

what is two-ways sync?

Xenar · March 27, 2024, 7:38pm

Hey,

Thanks @kapitainsky and @asdffdsa for your help.

What I am calling two-ways sync is to synchronize the local storage with the content on the cloud storage, and reverse.
Seeing my local storage empty, I was wondering if by default, Rclone was maybe only synchronizing the cloud with the content on local storage.
But it is great if it is not the case!

Ok, so the command rclone lsd onedrive: -vv does list my root directories of this OneDrive account.

I tried to run my previous command in the test folder with --daemon and it is populating the folder with my OneDrvie directories and files.
rclone mount onedrive: /home/non-root-user/onedrive_test --config /home/non-root-user/.config/rclone/rclone.conf --vfs-cache-mode writes --daemon -vv

What I don't understand is, as soon as I unmount this folder, it becomes empty.
I suppose it was working before, but as I was not running the command with the --daemon flag, I could not see the content populating, because I had to interrupt the process with "Ctrl+C" to get access again to the CLI.

With the --daemon flag, I use the command below to unmount the mounting point:
fusermount3 -u /home/non-root-user/onedrive_test
Not sure if it is the right way to do this.

I tried with the TrueNAS SMB share /mnt/rclone-share/OneDrive with the exact same result.
It seems to be working without the --allow-other flag.

How can I make the files synchronized on the local folder persist?
And of course with my previous questions. ^^
I know and you're right, one step at a time!
I am new to Rclone and did not have much time at the moment to investigate, so your help is great, thanks again!

asdffdsa · March 27, 2024, 7:48pm

rclone mount is the same as any other mount.
the mount itself does to transfer files.
that is something you or a copy program has to do.

you can upload a file to cloud
you can download a file from the cloud to local.
you can use a copy tool, such as syncthing.

if you just want two-way sync, without using mount.
rclone bisync

well, the mount has to be live to access the files.
if you kill the mount, you kill access to the cloud.

Xenar · March 27, 2024, 7:54pm

Thanks for the explanation. And yes, this is exactly what I want to setup!
I will look into it.

Xenar · March 27, 2024, 7:59pm

Ok, I did look at it before, but at this time, I understood it was designed to sync two remotes between themselves. So I thought not a remote with a local folder.
But I will read it in detail.

asdffdsa · March 27, 2024, 8:02pm

correct.
in rclone lingo, local is a remote, same as onedrive:

all these command will sync files.
rclone bisync /path/to/local onedrive:
rclone sync remote: /path/to/local
rclone sync /path/to/local remote:

Xenar · March 28, 2024, 5:41pm

Ok, a dry run of bisync seems to be working (with --resync for the first time):

rclone bisync onedrive: /mnt/rclone-share/OneDrive --config /home/non-root-user/.config/rclone/rclone.conf --resync --verbose --dry-run
2024/03/28 21:29:42 NOTICE: bisync is IN BETA. Don't use in production!
2024/03/28 21:29:42 INFO  : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
2024/03/28 21:29:42 INFO  : Bisyncing with Comparison Settings:
{
        "Modtime": true,
        "Size": true,
        "Checksum": false,
        "HashType1": 0,
        "HashType2": 0,
        "NoSlowHash": false,
        "SlowHashSyncOnly": false,
        "SlowHashDetected": true,
        "DownloadHash": false
}
2024/03/28 21:29:42 INFO  : Synching Path1 "onedrive:/" with Path2 "/mnt/rclone-share/OneDrive/"
2024/03/28 21:29:42 INFO  : Copying Path2 files to Path1
2024/03/28 21:29:42 NOTICE: - Path2    Resync is copying files to         - Path1

And after the last line, it synchronize my files.
Not sure if it is better to add the --checksum flag or any other flag for the first bisync.

I think the next step is to run this command (without --resync) every X minutes, like 5 minutes for example, and make a service of it.

Maybe I can try something like this:
Sync Google Drive in Linux using rclone

But, is there a flag or something to implement, to avoid errors or problems when the remote cloud or the SMB share are unavailable?

asdffdsa · March 28, 2024, 5:45pm

if the source or destination is not available, then rclone will error out.

If any errors occur during the command execution, rclone will exit with a non-zero exit code.
This allows scripts to detect when rclone operations have failed.

for example?

Xenar · March 28, 2024, 6:06pm

For example, if I have a power outage or an Internet provider connection interruption at home, when some files are synchronizing.
Will the synchronization resumes correctly after? No risk of files corruption?

asdffdsa · March 28, 2024, 6:23pm

i have no experience with bisync, very complex beta contraption, with a unstable history.
need to be very careful about that.
tho recently, a lot of excellent work is being done.

unlike bisync, other simple commands, such as sync/copy/move do not maintain state or lock files.
very simple, each time rclone runs, it compares source and dest, decides what needs to be copied/moved/deleted...

Sync the source to the destination, changing the destination only. Doesn't transfer files that are identical on source and destination
so if rclone was in the middle of transferring a large file, and the internet goes down.
next time rclone runs, it will transfer the entire file.