What is the problem you are having with rclone?
I have setup rclone, with the following parameters:
--vfs-cache-mode full \
--vfs-cache-max-size 100G \
--vfs-read-ahead 512M \
--no-modtime \
--no-checksum \
--read-only \
--vfs-fast-fingerprint \
--cache-dir /rclone-cache \
--poll-interval 30s \
--dir-cache-time=1000h \
--timeout=10m \
--rc \
--rc-addr=localhost:5572
However am having a minor issue and wanted to know if this can be resolved via tuning some parameters?
My setup is a glusterfs mount, of which I am using the "local" option in rclone to point to it, my reason for using rclone is the effectiveness of the VFS cache making directory traversals near instant.
Now, it seems if i make a full cache using the following command:
rc vfs/refresh recursive=true --rc-addr localhost:5572 _async=true
everything works fine, however it none of the directories are cached it seems it takes about 15 - 20 seconds to start a file transfer process (these are "linux iso's" ranging from 1 - 10GB), and when doing this directly on the glusterfs mount it's instant as expected, and after rclone has cached the directories (such as in the command above) the transfer starts instantly as expected.
So here I sort of have two questions, 1. is my current setup decent in general for my use case? It seems to be hard to find a consensus that applies to my setup on the forums as most people are using remote data stores such as google drive which has various API limits, where I obviously don't have any limits in place. Will the polling also still work with a local glusterfs mount for cache invalidation?
and then 2. Is it possible to fix the issue described above, as make it so that if the directory is not cached the file won't take 15 to 20 seconds to load up?
Run the command 'rclone version' and share the full output of the command.
rclone v1.62.2
- os/version: ubuntu 20.04 (64 bit)
- os/kernel: 5.4.0-146-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.20.2
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
local
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
rsync --progress /file/on/rclone/mount/xyz.mkv /local/nvme/drive/
The rclone config contents with secrets removed.
rclone mount gluster:/mnt/data /mnt/rclone-data \
--vfs-cache-mode full \
--vfs-cache-max-size 100G \
--vfs-read-ahead 512M \
--no-modtime \
--no-checksum \
--read-only \
--vfs-fast-fingerprint \
--cache-dir /rclone-cache \
--poll-interval 30s \
--dir-cache-time=1000h \
--timeout=10m \
--rc \
--rc-addr=localhost:5572
A log from the command with the -vv
flag
Not a rclone command, it is rsync in this example