Rclone mount shared cache?

Is it possible for multiple rclone mounts to share the same VFS cache?

Right now I have multiple mounts for each different services. I would like them all to use the same cache, but I'm wondering if this will cause some strange issues.

Additionally I have another question, what happens when a mount runs out of quota? Is there a simple command I can use in a bash script to restart the mount if it runs out of quota? (e.g. with anther user).

Not really as the other mount isn't aware of what's being put there as it would just confuse it.

You get an error writing to the mount like you would on a full disk.

For which backend? You can have another remote with another user or stop the mount and restart it with a new user. There is working being done with Google Drive to re-authenticate a running mount or you can even look at the new union remote as well as work is being done there and you could combine a few remotes as well. Really depends on what you are trying to do.

Ah sorry I should have been more clear, I'm currently running mergerfs using essentially the same setup you have (I used your scripts as a template).

So local and a gdrive merged together.

For union your suggesting a mount two different drives together in a union and have mergerfs then merge the union and local dir? That way if one gdrive fails I would atleast have read-only access to the other union drive right? (it seems that union only supports writing to one drive).

Something like this?
local+union (mergerfs) -> gdrive1+gdrive2 (union) -> gdrive

Is there a particular feature request or github issue I could follow regarding re-authenticating and existing mount?

I could write a shell script to do it, but I'm not quite sure how I would detect if the mount is dead or not in order to kill it and re-authenticate (is there an RC command for this)?

This post has the re-authenticated for Google Drive:

That's not my personal use case as other folks have talked about that in other posts. There is a new union remote update coming soon that has mergerfs like policies and such so that might be something to check out. There are a few posts on that as well. I personally use an unlimited drive and don't generate enough daily volume to ever come close to the 10TB download or per file quotas.

It is possible to start mounts via the rc in the latest beta, so you could run more than one mount in the same rclone process. These would share VFS caches for the same remote names I think.

The feature is a bit embryonic at the moment as it doesn't allow you to pass the vfs options yet but you could start one mount with the vfs options you want then start the futher ones via the rc.

mount/mount: Create a new mount point {#mount/mount}

rclone allows Linux, FreeBSD, macOS and Windows to mount any of
Rclone's cloud storage systems as a file system with FUSE.

If no mountType is provided, the priority is given as follows: 1. mount 2.cmount 3.mount2

This takes the following parameters

  • fs - a remote path to be mounted (required)
  • mountPoint: valid path on the local machine (required)
  • mountType: One of the values (mount, cmount, mount2) specifies the mount implementation to use

Eg

rclone rc mount/mount fs=mydrive: mountPoint=/home/<user>/mountPoint
rclone rc mount/mount fs=mydrive: mountPoint=/home/<user>/mountPoint mountType=mount

Authentication is required for this call.

mount/types: Show all possible mount types {#mount/types}

This shows all possible mount types and returns them as a list.

This takes no parameters and returns

  • mountTypes: list of mount types

The mount types are strings like "mount", "mount2", "cmount" and can
be passed to mount/mount as the mountType parameter.

Eg

rclone rc mount/types

Authentication is required for this call.

That is quite interesting. To be able to start consolidating these...

I was playing with this but it doesn't appear to share the same cache though.

If I start a mount and then run a "rclone rc vfs/refresh" against it results return immediate afterward. If I then do a rclone rc mount/mount using the same fs it does a full read without cache.

EDIT: Okay, i'm wrong. :slight_smile: running the rc mount/mount after the vfs/refresh seems to invalidate the cache. If we run things like this:

rclone mount drive: /drive
rclone rc mount/mount fs=drive: mountPoint=/drive2
rclone rc vfs/refresh

The refresh populates the cache for both and they seem to share it from what I can tell at least.

The VFSes are cached so it should pick them up if you reuse them. Should being the operative word! They may also be falling out of the cache after 5 minutes...

on ubuntu 20.04 lts

rclone rcd --rc-serve --rc-no-auth &
rclone rc mount/mount fs=Encrypt_TD1: mountPoint=/mnt/gmedia mountType=mount2 -o --allow-other --buffer-size 256M --log-level INFO --log-file /tmp/rclone_mount.log
ls -al /mnt/gmedia
total 0
drwxr-xr-x 1 root root 0 Dec 17 03:11 A_Movies_1080P
drwxr-xr-x 1 root root 0 Dec 17 03:11 A_Movies_4K
drwxr-xr-x 1 root root 0 Dec 17 03:11 A_Movies_ISO_1080P
drwxr-xr-x 1 root root 0 Dec 17 03:11 A_Movies_ISO_4K
drwxr-xr-x 1 root root 0 Apr 23 17:22 A_Movies_Mobile
drwxr-xr-x 1 root root 0 Dec 17 03:11 B_TV_Show
drwxr-xr-x 1 root root 0 Dec 17 03:11 B_TV_Show_Cartoon
drwxr-xr-x 1 root root 0 Dec 17 03:12 C_Cartoon_1080P
drwxr-xr-x 1 root root 0 Dec 17 03:12 C_Cartoon_4K
drwxr-xr-x 1 root root 0 Dec 17 03:12 C_Cartoon_ISO_1080P
drwxr-xr-x 1 root root 0 Dec 17 03:12 C_Cartoon_ISO_4K
drwxr-xr-x 1 root root 0 Dec 17 03:12 D_Upload_Temporary
drwxr-xr-x 1 root root 0 Dec 17 03:12 E_Other
drwxr-xr-x 1 root root 0 May 25 12:25 Non_Encrypt

samba don't show /mnt/gmedia
samba i config share /mnt

-o --uid 1000 --gid 1000 --dir-perms 0777 --file-perms 0777 --umask 0000

don't work

If you want to do multiple mounts from one rclone make your first mount use the --rc flag and add all the flags you want there.

You can mount subsequent ones with the rc.

You can't pass flags in to mount via the rc yet!

I don't understand.
I need example.

Thanks for the help.

Something like this

rclone --rc --rc-serve --rc-no-auth Encrypt_TD1: /mnt/gmedia mountType=mount2 --allow-other --buffer-size 256M --log-level INFO --log-file /tmp/rclone_mount.log --uid 1000 --gid 1000 --dir-perms 0777 --file-perms 0777 --umask 0000
rclone rc mount/mount fs=Encrypt_TD2: mountPoint=/mnt/gmedia2
1 Like

rclone --rc --rc-serve --rc-no-auth Encrypt_TD1: /mnt/gmedia mountType=mount2 --allow-other --buffer-size 256M --log-level INFO --log-file /tmp/rclone_mount.log --uid 1000 --gid 1000 --dir-perms 0777 --file-perms 0777 --umask 0000 --config /mnt/config/rclone/rclone.conf
Error: unknown command "Encrypt_TD1:" for "rclone"
Run 'rclone --help' for usage.
2020/05/28 13:54:08 Fatal error: unknown command "Encrypt_TD1:" for "rclone"

rclone -V
rclone v1.52.0-003-g764b90a5-beta

  • os/arch: linux/amd64
  • go version: go1.14.3

don't work T-T

That was just an example.

You need to replace Encrypt_TD1, Encrypt_TD2 with your remote names and the /mnt/gmedia, /mnt/gmedia2 mount points with whatever directories you want them mounted to.

remote name i use Encrypt_TD1:
and path mount i use /mnt/gmedia

rclone --rc --rc-serve --rc-no-auth Encrypt_TD1: /mnt/gmedia mountType=mount2 --allow-other --buffer-size 256M --log-level INFO --log-file /tmp/rclone_mount.log --uid 1000 --gid 1000 --dir-perms 0777 --file-perms 0777 --umask 0000 --config /mnt/config/rclone/rclone.conf
Error: unknown command "Encrypt_TD1:" for "rclone"
Run 'rclone --help' for usage.
2020/05/29 00:31:31 Fatal error: unknown command "Encrypt_TD1:" for "rclone"

rclone listremotes --config /mnt/config/rclone/rclone.conf
Encrypt_TD1:
embymanager28:
union:

ls -al /mnt
total 36
drwxrwxrwx 9 root root 4096 May 28 13:45 .
drwxr-xr-x 24 root root 4096 May 28 01:43 ..
drwxrwxrwx 3 neunghaha28 neunghaha28 4096 May 28 01:45 config
drwxrwxrwx 2 neunghaha28 neunghaha28 4096 May 28 02:05 gmedia

Sorry, my mistake.

Try these:

rclone --rc --rc-serve --rc-no-auth mount Encrypt_TD1: /mnt/gmedia --allow-other --buffer-size 256M --log-level INFO --log-file /tmp/rclone_mount.log --uid 1000 --gid 1000 --dir-perms 0777 --file-perms 0777 --umask 0000 --config /mnt/config/rclone/rclone.conf
rclone rc mount/mount fs=Encrypt_TD2: mountPoint=/mnt/gmedia2 mountType=mount2
2 Likes

Oh!!! working. :kissing_heart: :kissing_heart: :kissing_heart:
Thanks a lot for help.

root@vm1804:~# rclone --rc --rc-serve --rc-no-auth mount Encrypt_TD1: /mnt/gmedia --allow-other --buffer-size 256M --log-level INFO --log-file /tmp/rclone_mount.log --uid 1000 --gid 1000 --dir-perms 0777 --file-perms 0777 --umask 0000 --config /mnt/config/rclone/rclone.conf &
[1] 1923
root@vm1804:~# rclone rc mount/mount fs=Encrypt_TD2: mountPoint=/mnt/gmedia2 mountType=mount2
{}
root@vm1804:~# ls -al /mnt/gmedia
total 0
drwxrwxrwx 1 neunghaha28 neunghaha28 0 Dec 17 03:11 A_Movies_1080P
drwxrwxrwx 1 neunghaha28 neunghaha28 0 Dec 17 03:11 A_Movies_4K
drwxrwxrwx 1 neunghaha28 neunghaha28 0 Dec 17 03:11 A_Movies_ISO_1080P
drwxrwxrwx 1 neunghaha28 neunghaha28 0 Dec 17 03:11 A_Movies_ISO_4K
drwxrwxrwx 1 neunghaha28 neunghaha28 0 Apr 23 17:22 A_Movies_Mobile
drwxrwxrwx 1 neunghaha28 neunghaha28 0 Dec 17 03:11 B_TV_Show
drwxrwxrwx 1 neunghaha28 neunghaha28 0 Dec 17 03:11 B_TV_Show_Cartoon
drwxrwxrwx 1 neunghaha28 neunghaha28 0 Dec 17 03:12 C_Cartoon_1080P
drwxrwxrwx 1 neunghaha28 neunghaha28 0 Dec 17 03:12 C_Cartoon_4K
drwxrwxrwx 1 neunghaha28 neunghaha28 0 Dec 17 03:12 C_Cartoon_ISO_1080P
drwxrwxrwx 1 neunghaha28 neunghaha28 0 Dec 17 03:12 C_Cartoon_ISO_4K
drwxrwxrwx 1 neunghaha28 neunghaha28 0 Dec 17 03:12 D_Upload_Temporary
drwxrwxrwx 1 neunghaha28 neunghaha28 0 Dec 17 03:12 E_Other
drwxrwxrwx 1 neunghaha28 neunghaha28 0 May 25 12:25 Non_Encrypt
root@vm1804:~# ls -al /mnt/gmedia2
total 4
drwxrwxrwx 1 neunghaha28 neunghaha28 0 May 17 10:01 apk
drwxrwxrwx 1 neunghaha28 neunghaha28 0 May 14 15:18 automount
drwxrwxrwx 1 neunghaha28 neunghaha28 0 Jan 12 15:35 AutoRclone
-rwxrwxrwx 1 neunghaha28 neunghaha28 1222 May 22 03:32 autoscript.sh
drwxrwxrwx 1 neunghaha28 neunghaha28 0 May 14 15:13 emby
drwxrwxrwx 1 neunghaha28 neunghaha28 0 Apr 25 01:17 gdus
drwxrwxrwx 1 neunghaha28 neunghaha28 0 Mar 28 2019 Movie
drwxrwxrwx 1 neunghaha28 neunghaha28 0 May 4 2019 'Movie 4K'
drwxrwxrwx 1 neunghaha28 neunghaha28 0 Mar 29 2019 Music
drwxrwxrwx 1 neunghaha28 neunghaha28 0 Jul 3 2019 script
drwxrwxrwx 1 neunghaha28 neunghaha28 0 May 10 09:47 series
drwxrwxrwx 1 neunghaha28 neunghaha28 0 May 15 05:56 TEMP
root@vm1804:~#

The mount2 in the rc command should probably be replaced by mount instead, unless you specifically need that.

1 Like

I try mount2 problem.
mount2 disconnect auto. T-T
mount no problem.

Thanks again for the help. :heart_eyes: :heart_eyes: :heart_eyes:

I've just tried the rc mount command with: /usr/bin/rclone rc mount/mount fs=gmedia_enc2: mountPoint=/mnt/gmedia_2 mountType=mount

Unfortunately I've found a bug because I'm starting the original mount with --rc --rc-addr :5572 when I start another mount it tries to start another rclone instance with rc port 5572.... this sadly fails because the original mount has rc with port 5572.

Here's my systemd file for example:
ExecStart=/usr/bin/rclone mount gmedia_enc: /mnt/gmedia_1
--allow-other
--dir-cache-time 1000h
--log-level INFO
--log-file /opt/rclone/logs/rclone.log
--poll-interval 15s
--umask 002
--user-agent "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36"
--rc
--rc-no-auth
--rc-addr :5572
--vfs-read-chunk-size 32M
--vfs-cache-mode full
--vfs-cache-max-age 336h
--cache-dir /mnt/rclone_cache
ExecStop=/bin/fusermount -uz /mnt/gmedia_1
ExecStop=/bin/fusermount -uz /mnt/gmedia_2
ExecStop=/bin/fusermount -uz /mnt/gmedia_3
ExecStartPost=/usr/bin/rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5572 _async=true
ExecStartPost=/usr/bin/rclone rc mount/mount fs=gmedia_enc2: mountPoint=/mnt/gmedia_2 mountType=mount --rc-addr 127.0.0.1:5572
ExecStartPost=/usr/bin/rclone rc mount/mount fs=gmedia_enc3: mountPoint=/mnt/gmedia_3 mountType=mount --rc-addr 127.0.0.1:5572

I get the following logs:
2020/06/13 14:10:38 Failed to start remote control: start server failed: listen tcp :5572: bind: address already in use

You have something running already I'd surmise as those commands work fine.

rclone --rc --rc-serve --rc-no-auth mount GD: /home/felix/test -v --rc-addr :5573
2020/06/13 10:32:40 INFO  : Using --user felix --pass XXXX as authenticated user
2020/06/13 10:32:40 NOTICE: Serving remote control on http://[::]:5573/

GD:             1.2P  118T  1.0P  11% /home/felix/test
GD::            1.2P  118T  1.0P  11% /home/felix/test2
GD::            1.2P  118T  1.0P  11% /home/felix/test3
 2003  rclone rc mount/mount --rc-addr 127.0.0.1:5573 fs=GD:: mountPoint=/home/felix/test2 --rc-user felix --rc-pass felix
 2005  rclone rc mount/mount --rc-addr 127.0.0.1:5573 fs=GD:: mountPoint=/home/felix/test3 --rc-user felix --rc-pass felix

are what I tested with.