Rclone cache terminating the mount

What is the problem you are having with rclone?

Cache size is 10TB and rclone is not mounting. If I change the cache location to different location then works fine.

Run the command 'rclone version' and share the full output of the command.

ezio@ezio:~$ rclone --version
rclone v1.62.0-beta.6672.98fa93f6d

  • os/version: ubuntu 18.04 (64 bit)
  • os/kernel: 4.15.0-202-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.19.4
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)


[Unit]
Description=Rclone
Wants=network-online.target
After=network-online.target

[Service]
Type=notify
Environment=RCLONE_CONFIG=/home/ezio/.docker/appdata/rclone/rclone.conf
RestartSec=5
ExecStart=/usr/bin/rclone mount gdrive: /mnt/gdrive \
--dir-cache-time 9999h \
--log-file /home/ezio/.docker/appdata/rclone/rclone.log \
--log-level DEBUG \
--umask 002 \
--rc \
--rc-addr 127.0.0.1:5574 \
--rc-no-auth \
--cache-dir=/mnt/local/.rclone \
--vfs-cache-mode full \
--vfs-cache-max-size 10000G \
--vfs-fast-fingerprint \
--vfs-read-chunk-size 32M \
--vfs-write-back 1h \
--vfs-cache-max-age 9999h \
--vfs-read-ahead 1G \
--tpslimit 12 \
--tpslimit-burst 0 \
--bind 192.168.1.232 \
--disable-http2
ExecStop=/bin/fusermount -uz /mnt/gdrive
ExecStartPost=/usr/bin/rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5574 _async=true
Restart=on-failure
User=ezio
Group=ezio

[Install]
WantedBy=multi-user.target

The rclone config contents with secrets removed.

Paste config here

A log from the command with the -vv flag

Paste  log here

hi,
as i understand it, when rclone mount starts, it check all in the vfs file cache.
to see if a file needs to be uploaded and perform other housekeeping.

Yes that make sense but I left it for like 2 or 3 hours but drive was not mounted so I am not exactly sure how to fix this.

have you look into the debug file, might find some hints?

perhaps post the redacted output of
rclone show gdrive

might try --magic :wink:

Oops here we go

2023/01/20 19:02:06 INFO  : Starting transaction limiter: max 12 transactions/s with burst 1
2023/01/20 19:02:06 NOTICE: Serving remote control on http://127.0.0.1:5574/
2023/01/20 19:02:07 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2023/01/20 19:03:07 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2023/01/20 19:03:37 INFO  : Signal received: terminated
2023/01/20 19:03:37 INFO  : Exiting...
2023/01/20 19:03:37 INFO  : Starting transaction limiter: max 12 transactions/s with burst 1
2023/01/20 19:03:37 DEBUG : rclone: Version "v1.62.0-beta.6672.98fa93f6d" starting with parameters ["/usr/bin/rclone" "mount" "gdrive:" "/mnt/gdrive" "--dir-cache-time" "9999h" "--log-file" "/home/ezio/.docker/appdata/rclone/rclone.log" "--log-level" "DEBUG" "--umask" "002" "--rc" "--rc-addr" "127.0.0.1:5574" "--rc-no-auth" "--cache-dir=/mnt/local/.rclone" "--vfs-cache-mode" "full" "--vfs-cache-max-size" "10000G" "--vfs-fast-fingerprint" "--vfs-read-chunk-size" "32M" "--vfs-write-back" "1h" "--vfs-cache-max-age" "9999h" "--vfs-read-ahead" "1G" "--tpslimit" "12" "--tpslimit-burst" "0" "--bind" "192.168.1.232" "--disable-http2"]
2023/01/20 19:03:37 NOTICE: Serving remote control on http://127.0.0.1:5574/
2023/01/20 19:03:37 DEBUG : Creating backend with remote "gdrive:"
2023/01/20 19:03:37 DEBUG : Using config file from "/home/ezio/.docker/appdata/rclone/rclone.conf"
2023/01/20 19:03:37 DEBUG : vfs cache: root is "/mnt/local/.rclone"
2023/01/20 19:03:37 DEBUG : vfs cache: data root is "/mnt/local/.rclone/vfs/gdrive"
2023/01/20 19:03:37 DEBUG : vfs cache: metadata root is "/mnt/local/.rclone/vfsMeta/gdrive"
2023/01/20 19:03:37 DEBUG : Creating backend with remote "/mnt/local/.rclone/vfs/gdrive/"
2023/01/20 19:03:37 DEBUG : fs cache: renaming cache item "/mnt/local/.rclone/vfs/gdrive/" to be canonical "/mnt/local/.rclone/vfs/gdrive"
2023/01/20 19:03:37 DEBUG : Creating backend with remote "/mnt/local/.rclone/vfsMeta/gdrive/"
2023/01/20 19:03:37 DEBUG : fs cache: renaming cache item "/mnt/local/.rclone/vfsMeta/gdrive/" to be canonical "/mnt/local/.rclone/vfsMeta/gdrive"
2023/01/20 19:04:23 DEBUG : mydatafile: Added virtual directory entry vAddFile: "mydatafile"
2023/01/20 19:04:37 DEBUG : Google drive root '': Checking for changes on remote
2023/01/20 19:05:07 INFO  : Signal received: terminated
2023/01/20 19:05:07 INFO  : Exiting...
2023/01/20 19:05:12 INFO  : Starting transaction limiter: max 12 transactions/s with burst 1
2023/01/20 19:05:12 DEBUG : rclone: Version "v1.62.0-beta.6672.98fa93f6d" starting with parameters ["/usr/bin/rclone" "mount" "gdrive:" "/mnt/gdrive" "--dir-cache-time" "9999h" "--log-file" "/home/ezio/.docker/appdata/rclone/rclone.log" "--log-level" "DEBUG" "--umask" "002" "--rc" "--rc-addr" "127.0.0.1:5574" "--rc-no-auth" "--cache-dir=/mnt/local/.rclone" "--vfs-cache-mode" "full" "--vfs-cache-max-size" "10000G" "--vfs-fast-fingerprint" "--vfs-read-chunk-size" "32M" "--vfs-write-back" "1h" "--vfs-cache-max-age" "9999h" "--vfs-read-ahead" "1G" "--tpslimit" "12" "--tpslimit-burst" "0" "--bind" "192.168.1.232" "--disable-http2"]
2023/01/20 19:05:12 NOTICE: Serving remote control on http://127.0.0.1:5574/
2023/01/20 19:05:12 DEBUG : Creating backend with remote "gdrive:"
2023/01/20 19:05:12 DEBUG : Using config file from "/home/ezio/.docker/appdata/rclone/rclone.conf"
2023/01/20 19:05:13 DEBUG : vfs cache: root is "/mnt/local/.rclone"
2023/01/20 19:05:13 DEBUG : vfs cache: data root is "/mnt/local/.rclone/vfs/gdrive"
2023/01/20 19:05:13 DEBUG : vfs cache: metadata root is "/mnt/local/.rclone/vfsMeta/gdrive"
2023/01/20 19:05:13 DEBUG : Creating backend with remote "/mnt/local/.rclone/vfs/gdrive/"
2023/01/20 19:05:13 DEBUG : fs cache: renaming cache item "/mnt/local/.rclone/vfs/gdrive/" to be canonical "/mnt/local/.rclone/vfs/gdrive"
2023/01/20 19:05:13 DEBUG : Creating backend with remote "/mnt/local/.rclone/vfsMeta/gdrive/"
2023/01/20 19:05:13 DEBUG : fs cache: renaming cache item "/mnt/local/.rclone/vfsMeta/gdrive/" to be canonical "/mnt/local/.rclone/vfsMeta/gdrive"
2023/01/20 19:05:57 DEBUG : mydatafile: vfs cache: truncate to size=1430036235 (not needed as size correct)
2023/01/20 19:05:57 DEBUG : mydatafile: vfs cache: setting modification time to 2023-01-17 05:10:20.9792202 +0000 UTC
2023/01/20 19:05:57 INFO  : mydatafile: vfs cache: queuing for upload in 1h0m0s
2023/01/20 19:05:57 DEBUG : mydatafile: Added virtual directory entry vAddFile: "mydatafile"
2023/01/20 19:06:13 DEBUG : Google drive root '': Checking for changes on remote
2023/01/20 19:06:43 INFO  : Signal received: terminated
2023/01/20 19:06:43 INFO  : Exiting...
2023/01/20 19:06:48 INFO  : Starting transaction limiter: max 12 transactions/s with burst 1
2023/01/20 19:06:48 DEBUG : rclone: Version "v1.62.0-beta.6672.98fa93f6d" starting with parameters ["/usr/bin/rclone" "mount" "gdrive:" "/mnt/gdrive" "--dir-cache-time" "9999h" "--log-file" "/home/ezio/.docker/appdata/rclone/rclone.log" "--log-level" "DEBUG" "--umask" "002" "--rc" "--rc-addr" "127.0.0.1:5574" "--rc-no-auth" "--cache-dir=/mnt/local/.rclone" "--vfs-cache-mode" "full" "--vfs-cache-max-size" "10000G" "--vfs-fast-fingerprint" "--vfs-read-chunk-size" "32M" "--vfs-write-back" "1h" "--vfs-cache-max-age" "9999h" "--vfs-read-ahead" "1G" "--tpslimit" "12" "--tpslimit-burst" "0" "--bind" "192.168.1.232" "--disable-http2"]
2023/01/20 19:06:48 NOTICE: Serving remote control on http://127.0.0.1:5574/
2023/01/20 19:06:48 DEBUG : Creating backend with remote "gdrive:"
2023/01/20 19:06:48 DEBUG : Using config file from "/home/ezio/.docker/appdata/rclone/rclone.conf"
2023/01/20 19:06:48 DEBUG : vfs cache: root is "/mnt/local/.rclone"
2023/01/20 19:06:48 DEBUG : vfs cache: data root is "/mnt/local/.rclone/vfs/gdrive"
2023/01/20 19:06:48 DEBUG : vfs cache: metadata root is "/mnt/local/.rclone/vfsMeta/gdrive"
2023/01/20 19:06:48 DEBUG : Creating backend with remote "/mnt/local/.rclone/vfs/gdrive/"
2023/01/20 19:06:48 DEBUG : fs cache: renaming cache item "/mnt/local/.rclone/vfs/gdrive/" to be canonical "/mnt/local/.rclone/vfs/gdrive"
2023/01/20 19:06:48 DEBUG : Creating backend with remote "/mnt/local/.rclone/vfsMeta/gdrive/"
2023/01/20 19:06:48 DEBUG : fs cache: renaming cache item "/mnt/local/.rclone/vfsMeta/gdrive/" to be canonical "/mnt/local/.rclone/vfsMeta/gdrive"

[gdrive]
type = drive
client_id = 744fdsfdsfsfdssm6fbup1opfrc1.apps.googleusercontent.com
client_secret = 4TfdsfdsfdfsfYN3ut3EUFCz
scope = drive
token = {"access_token":"ya29.a0Aa4xrXfsfdsfsdwqOjH-lR6PEGXRgloJ","expiry":"2023-01-20T20:17:24.878046826Z"}
root_folder_id = 0ALdsfdfssdfA

looks like the vfs file cache was empty?
INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)

It might be a good idea to redact the id and secret if you still use it.

Above information is just made up. Thank you :slight_smile:

I just moved the cache to new location and restarted again. I know its not helpful for this topic but I tried lot of different ways but just couldnt figure out cache is stopping the mount.

Apologies for the logs above. I mistakenly copied the logs from the time when I moved to new location where nothing was in the queue.

Thanks again for looking into this for me.