Rclone cache terminating the mount

Help and Support
1

What is the problem you are having with rclone?

Cache size is 10TB and rclone is not mounting. If I change the cache location to different location then works fine.

Run the command 'rclone version' and share the full output of the command.

ezio@ezio:~$ rclone --version
rclone v1.62.0-beta.6672.98fa93f6d

  • os/version: ubuntu 18.04 (64 bit)
  • os/kernel: 4.15.0-202-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.19.4
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)


[Unit]
Description=Rclone
Wants=network-online.target
After=network-online.target

[Service]
Type=notify
Environment=RCLONE_CONFIG=/home/ezio/.docker/appdata/rclone/rclone.conf
RestartSec=5
ExecStart=/usr/bin/rclone mount gdrive: /mnt/gdrive \
--dir-cache-time 9999h \
--log-file /home/ezio/.docker/appdata/rclone/rclone.log \
--log-level DEBUG \
--umask 002 \
--rc \
--rc-addr 127.0.0.1:5574 \
--rc-no-auth \
--cache-dir=/mnt/local/.rclone \
--vfs-cache-mode full \
--vfs-cache-max-size 10000G \
--vfs-fast-fingerprint \
--vfs-read-chunk-size 32M \
--vfs-write-back 1h \
--vfs-cache-max-age 9999h \
--vfs-read-ahead 1G \
--tpslimit 12 \
--tpslimit-burst 0 \
--bind 192.168.1.232 \
--disable-http2
ExecStop=/bin/fusermount -uz /mnt/gdrive
ExecStartPost=/usr/bin/rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5574 _async=true
Restart=on-failure
User=ezio
Group=ezio

[Install]
WantedBy=multi-user.target

The rclone config contents with secrets removed.

Paste config here

A log from the command with the -vv flag

Paste  log here
2

hi,
as i understand it, when rclone mount starts, it check all in the vfs file cache.
to see if a file needs to be uploaded and perform other housekeeping.

3

Yes that make sense but I left it for like 2 or 3 hours but drive was not mounted so I am not exactly sure how to fix this.

4

have you look into the debug file, might find some hints?

perhaps post the redacted output of
rclone show gdrive

might try --magic :wink:

5

Oops here we go

2023/01/20 19:02:06 INFO  : Starting transaction limiter: max 12 transactions/s with burst 1
2023/01/20 19:02:06 NOTICE: Serving remote control on http://127.0.0.1:5574/
2023/01/20 19:02:07 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2023/01/20 19:03:07 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2023/01/20 19:03:37 INFO  : Signal received: terminated
2023/01/20 19:03:37 INFO  : Exiting...
2023/01/20 19:03:37 INFO  : Starting transaction limiter: max 12 transactions/s with burst 1
2023/01/20 19:03:37 DEBUG : rclone: Version "v1.62.0-beta.6672.98fa93f6d" starting with parameters ["/usr/bin/rclone" "mount" "gdrive:" "/mnt/gdrive" "--dir-cache-time" "9999h" "--log-file" "/home/ezio/.docker/appdata/rclone/rclone.log" "--log-level" "DEBUG" "--umask" "002" "--rc" "--rc-addr" "127.0.0.1:5574" "--rc-no-auth" "--cache-dir=/mnt/local/.rclone" "--vfs-cache-mode" "full" "--vfs-cache-max-size" "10000G" "--vfs-fast-fingerprint" "--vfs-read-chunk-size" "32M" "--vfs-write-back" "1h" "--vfs-cache-max-age" "9999h" "--vfs-read-ahead" "1G" "--tpslimit" "12" "--tpslimit-burst" "0" "--bind" "192.168.1.232" "--disable-http2"]
2023/01/20 19:03:37 NOTICE: Serving remote control on http://127.0.0.1:5574/
2023/01/20 19:03:37 DEBUG : Creating backend with remote "gdrive:"
2023/01/20 19:03:37 DEBUG : Using config file from "/home/ezio/.docker/appdata/rclone/rclone.conf"
2023/01/20 19:03:37 DEBUG : vfs cache: root is "/mnt/local/.rclone"
2023/01/20 19:03:37 DEBUG : vfs cache: data root is "/mnt/local/.rclone/vfs/gdrive"
2023/01/20 19:03:37 DEBUG : vfs cache: metadata root is "/mnt/local/.rclone/vfsMeta/gdrive"
2023/01/20 19:03:37 DEBUG : Creating backend with remote "/mnt/local/.rclone/vfs/gdrive/"
2023/01/20 19:03:37 DEBUG : fs cache: renaming cache item "/mnt/local/.rclone/vfs/gdrive/" to be canonical "/mnt/local/.rclone/vfs/gdrive"
2023/01/20 19:03:37 DEBUG : Creating backend with remote "/mnt/local/.rclone/vfsMeta/gdrive/"
2023/01/20 19:03:37 DEBUG : fs cache: renaming cache item "/mnt/local/.rclone/vfsMeta/gdrive/" to be canonical "/mnt/local/.rclone/vfsMeta/gdrive"
2023/01/20 19:04:23 DEBUG : mydatafile: Added virtual directory entry vAddFile: "mydatafile"
2023/01/20 19:04:37 DEBUG : Google drive root '': Checking for changes on remote
2023/01/20 19:05:07 INFO  : Signal received: terminated
2023/01/20 19:05:07 INFO  : Exiting...
2023/01/20 19:05:12 INFO  : Starting transaction limiter: max 12 transactions/s with burst 1
2023/01/20 19:05:12 DEBUG : rclone: Version "v1.62.0-beta.6672.98fa93f6d" starting with parameters ["/usr/bin/rclone" "mount" "gdrive:" "/mnt/gdrive" "--dir-cache-time" "9999h" "--log-file" "/home/ezio/.docker/appdata/rclone/rclone.log" "--log-level" "DEBUG" "--umask" "002" "--rc" "--rc-addr" "127.0.0.1:5574" "--rc-no-auth" "--cache-dir=/mnt/local/.rclone" "--vfs-cache-mode" "full" "--vfs-cache-max-size" "10000G" "--vfs-fast-fingerprint" "--vfs-read-chunk-size" "32M" "--vfs-write-back" "1h" "--vfs-cache-max-age" "9999h" "--vfs-read-ahead" "1G" "--tpslimit" "12" "--tpslimit-burst" "0" "--bind" "192.168.1.232" "--disable-http2"]
2023/01/20 19:05:12 NOTICE: Serving remote control on http://127.0.0.1:5574/
2023/01/20 19:05:12 DEBUG : Creating backend with remote "gdrive:"
2023/01/20 19:05:12 DEBUG : Using config file from "/home/ezio/.docker/appdata/rclone/rclone.conf"
2023/01/20 19:05:13 DEBUG : vfs cache: root is "/mnt/local/.rclone"
2023/01/20 19:05:13 DEBUG : vfs cache: data root is "/mnt/local/.rclone/vfs/gdrive"
2023/01/20 19:05:13 DEBUG : vfs cache: metadata root is "/mnt/local/.rclone/vfsMeta/gdrive"
2023/01/20 19:05:13 DEBUG : Creating backend with remote "/mnt/local/.rclone/vfs/gdrive/"
2023/01/20 19:05:13 DEBUG : fs cache: renaming cache item "/mnt/local/.rclone/vfs/gdrive/" to be canonical "/mnt/local/.rclone/vfs/gdrive"
2023/01/20 19:05:13 DEBUG : Creating backend with remote "/mnt/local/.rclone/vfsMeta/gdrive/"
2023/01/20 19:05:13 DEBUG : fs cache: renaming cache item "/mnt/local/.rclone/vfsMeta/gdrive/" to be canonical "/mnt/local/.rclone/vfsMeta/gdrive"
2023/01/20 19:05:57 DEBUG : mydatafile: vfs cache: truncate to size=1430036235 (not needed as size correct)
2023/01/20 19:05:57 DEBUG : mydatafile: vfs cache: setting modification time to 2023-01-17 05:10:20.9792202 +0000 UTC
2023/01/20 19:05:57 INFO  : mydatafile: vfs cache: queuing for upload in 1h0m0s
2023/01/20 19:05:57 DEBUG : mydatafile: Added virtual directory entry vAddFile: "mydatafile"
2023/01/20 19:06:13 DEBUG : Google drive root '': Checking for changes on remote
2023/01/20 19:06:43 INFO  : Signal received: terminated
2023/01/20 19:06:43 INFO  : Exiting...
2023/01/20 19:06:48 INFO  : Starting transaction limiter: max 12 transactions/s with burst 1
2023/01/20 19:06:48 DEBUG : rclone: Version "v1.62.0-beta.6672.98fa93f6d" starting with parameters ["/usr/bin/rclone" "mount" "gdrive:" "/mnt/gdrive" "--dir-cache-time" "9999h" "--log-file" "/home/ezio/.docker/appdata/rclone/rclone.log" "--log-level" "DEBUG" "--umask" "002" "--rc" "--rc-addr" "127.0.0.1:5574" "--rc-no-auth" "--cache-dir=/mnt/local/.rclone" "--vfs-cache-mode" "full" "--vfs-cache-max-size" "10000G" "--vfs-fast-fingerprint" "--vfs-read-chunk-size" "32M" "--vfs-write-back" "1h" "--vfs-cache-max-age" "9999h" "--vfs-read-ahead" "1G" "--tpslimit" "12" "--tpslimit-burst" "0" "--bind" "192.168.1.232" "--disable-http2"]
2023/01/20 19:06:48 NOTICE: Serving remote control on http://127.0.0.1:5574/
2023/01/20 19:06:48 DEBUG : Creating backend with remote "gdrive:"
2023/01/20 19:06:48 DEBUG : Using config file from "/home/ezio/.docker/appdata/rclone/rclone.conf"
2023/01/20 19:06:48 DEBUG : vfs cache: root is "/mnt/local/.rclone"
2023/01/20 19:06:48 DEBUG : vfs cache: data root is "/mnt/local/.rclone/vfs/gdrive"
2023/01/20 19:06:48 DEBUG : vfs cache: metadata root is "/mnt/local/.rclone/vfsMeta/gdrive"
2023/01/20 19:06:48 DEBUG : Creating backend with remote "/mnt/local/.rclone/vfs/gdrive/"
2023/01/20 19:06:48 DEBUG : fs cache: renaming cache item "/mnt/local/.rclone/vfs/gdrive/" to be canonical "/mnt/local/.rclone/vfs/gdrive"
2023/01/20 19:06:48 DEBUG : Creating backend with remote "/mnt/local/.rclone/vfsMeta/gdrive/"
2023/01/20 19:06:48 DEBUG : fs cache: renaming cache item "/mnt/local/.rclone/vfsMeta/gdrive/" to be canonical "/mnt/local/.rclone/vfsMeta/gdrive"
6

[gdrive]
type = drive
client_id = 744fdsfdsfsfdssm6fbup1opfrc1.apps.googleusercontent.com
client_secret = 4TfdsfdsfdfsfYN3ut3EUFCz
scope = drive
token = {"access_token":"ya29.a0Aa4xrXfsfdsfsdwqOjH-lR6PEGXRgloJ","expiry":"2023-01-20T20:17:24.878046826Z"}
root_folder_id = 0ALdsfdfssdfA

7

looks like the vfs file cache was empty?
INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)

8

It might be a good idea to redact the id and secret if you still use it.

9

Above information is just made up. Thank you :slight_smile:

10

I just moved the cache to new location and restarted again. I know its not helpful for this topic but I tried lot of different ways but just couldnt figure out cache is stopping the mount.

Apologies for the logs above. I mistakenly copied the logs from the time when I moved to new location where nothing was in the queue.

Thanks again for looking into this for me.

11

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.