Rclone http remote

Help and Support
21 
--vfs-cache-max-age 50h --vfs-cache-max-size 500G

im using these 2 flags, i have a lot of spce still empty, so can i use only --vfs-cache-max-size 500G to use my full server space and delete --vfs-cache-max-age 50h this flag?

22

well, in that case, i agree, to use just --vfs-cache-max-size

1 Like
23

sorry, i gave you wrong advice

by default, --vfs-cache-max-age is just 1h0m0s
so instead, set that to the largest value you can, something like
--vfs-cache-max-age=1000d

24

i want to use full space of my server so i think i only can use --vfs-cache-max-size no?

25

soory, i think i was wrong, late night here where i am, getting tired.

by default, --vfs-cache-max-age is just 1h0m0s
so rclone will remove unused files after just one hour, which is not what you want.
i would use a high value, for example, --vfs-cache-max-age=1000d

26

thanks i got it now, we must have to use max age also,
Thanks a lot

27

I'll stick with sftp and try figure out whats happening with the speed.

28

So after adding --vfs-cache-mode=full speeds and playback has been fixed.
I do seem to have another issue though, changes on the remote files don't ever seem to be reflected on the rclone mount. eg. adding/removing or modifying files never show up on mount.
This is how I'm mounting the drive:

/usr/bin/rclone mount tfd:/home/xtinct101/data/media/ /home/xtinct101/data/media \
--allow-other \
--dir-cache-time 5000h \
--poll-interval 2s \
--log-file /home/xtinct101/scripts/logs/rclone.log \
--log-level NOTICE \
--umask 002 \
--cache-dir=/home/xtinct101/tmp/cache \
--rc \
--rc-addr 127.0.0.1:5574 \
--rc-no-auth \
--vfs-cache-mode full \
--vfs-fast-fingerprint \
--vfs-cache-max-size 150G \
--vfs-cache-max-age 5000h \
--vfs-cache-poll-interval 1m
29

You've said here - wait 5000h to see if a file has changed on the remote. Make this number smaller - say 5m so that rclone only caches the file listings for 5 minutes.

30

ahh...I see. I guess I misunderstood that setting.

From the documents I understood the --poll-interval supersedes --dir-cache-time.
Am I misunderstanding?

Using the --dir-cache-time flag, you can control how long a directory should be considered up to date and not refreshed from the backend. Changes made through the VFS will appear immediately or invalidate the cache.

--poll-interval duration    Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s)
31

hi, as per your debug log,
https://xxxx/: poll-interval is not supported by this remote

so you have two chocies.
--- reduce the value for --dir-cache-time
--- run a command to manually update the files listed in the mount to match the files listed by the http server
rclone rc vfs/refresh recursive=true -vv

32

I think that the debug you are referring to is using http as a backend, I'm now using sftp + vfs as per your recommendation. Is --poll-interval is not supported on sftp remote?

33

sftp is not a polling remote, the debug log would show that.

34

Got it.
Am I correct in reading that rclone serve sftp supports polling?
rclone serve sftp

35

if remote being served supports polling, rclone serve sftp should support polling.
so this will work rclone serve sftp gdrive

this is NOT going to work if the remote being served is http or sftp.
as discussed, these remotes do not support polling

2023/03/22 12:29:58 DEBUG : rclone: Version "v1.62.2" starting with parameters ["C:\\data\\rclone\\rclone.exe" "serve" "sftp" "gdrive01:" "--dir-cache-time=10000h" "--rc" "--log-level=DEBUG" "--log-file=.\\\\log.serve.gdrive01.txt" "--user=user" "--pass=pass" "--addr=:2022" "--poll-interval=10s"]
2023/03/22 12:29:58 NOTICE: Serving remote control on http://127.0.0.1:5572/
2023/03/22 12:29:58 DEBUG : Creating backend with remote "gdrive01:"
2023/03/22 12:29:58 DEBUG : Using config file from "C:\\data\\rclone\\rclone.conf"
2023/03/22 12:29:58 NOTICE: Loaded 0 authorized keys from "C:\\Users\\user01\\.ssh\\authorized_keys"
2023/03/22 12:29:58 DEBUG : Loaded private key from "C:\\Users\\user01\\AppData\\Local\\rclone\\serve-sftp\\id_rsa"
2023/03/22 12:29:58 DEBUG : Loaded private key from "C:\\Users\\user01\\AppData\\Local\\rclone\\serve-sftp\\id_ecdsa"
2023/03/22 12:29:58 DEBUG : Loaded private key from "C:\\Users\\user01\\AppData\\Local\\rclone\\serve-sftp\\id_ed25519"
2023/03/22 12:29:58 NOTICE: SFTP server listening on [::]:2022
2023/03/22 12:30:08 DEBUG : Google drive root '': Checking for changes on remote
2023/03/22 12:30:18 DEBUG : Google drive root '': Checking for changes on remote
2023/03/22 12:30:28 DEBUG : Google drive root '': Checking for changes on remote
2023/03/22 12:30:38 DEBUG : Google drive root '': Checking for changes on remote
2023/03/22 12:30:48 DEBUG : Google drive root '': Checking for changes on remote
2023/03/22 12:30:58 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Elapsed time:       1m0.1s
36

Yeah that makes sense. TY!

Out of these choices, which is the preferred method?

so you have two chocies.
--- reduce the value for --dir-cache-time
--- run a command to manually update the files listed in the mount to match the files listed by the http server
rclone rc vfs/refresh recursive=true -vv

37

no idea, depends on your particular use case.

38

thanks i got it now, we must have to use max age also.

.