Does --vfs-read-chunk-size work with --vfs-cache-mode=off

[asdffdsa (jojothehumanmonkey)]

What is the problem you are having with rclone?

does --vfs-read-chunk-size work with --vfs-cache-mode=off

confused about order of the VFS documentation.
chunked reading is mentioned before VFS - Virtual File System
VFS File Buffering is mentioned before VFS File Caching

which mount flags work with --vfs-cache-mode=off
--vfs-read-chunk-size ?
--dir-cache-time?
--buffer-size?
rc vfs/refresh?

What is your rclone version (output from rclone version)

rclone v1.54.1
- os/arch: windows/amd64
- go version: go1.15.8

Which OS you are using and how many bits (eg Windows 7, 64 bit)

WIN10.20H2.64bit

Which cloud storage system are you using? (eg Google Drive)

sftp

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount sftp: v: -read-only --no-checksum --rc --no-modtime --dir-cache-time=10000h --attr-timeout=10000h --poll-interval=0  --buffer-size=1G --vfs-read-chunk-size=10M --vfs-read-chunk-size-limit=0

The rclone config contents with secrets removed.

type = sftp
host = xxx.xxx.xxx.xxx
user = ecCsXa9Y3zHQGheKOcwS
port = 9999
pass = NZq-t8R9IacHmT03wgk21Vz6ri2jzb30CqPjNXKFSn0AtUzs
key_file_pass = Mhvkx_QrWprY9vMPF9PX0pB74sedzzIs
use_insecure_cipher = true
md5sum_command = md5sum
sha1sum_command = sha1sum

A log from the command with the -vv flag

if needed, will post

[Animosity022]

@asdffdsa

which mount flags work with --vfs-cache-mode=off
--vfs-read-chunk-size ?
--dir-cache-time?
--buffer-size?
rc vfs/refresh?

Those all work without a cache mode as those all predate it.

[asdffdsa (jojothehumanmonkey)]

rclone mount remote: s: --read-only --no-checksum --rc --no-modtime --dir-cache-time=10000h --attr-timeout=10000h --poll-interval=0 --vfs-read-chunk-size=300M --vfs-read-chunk-size-limit=0 --cache-dir=b:\rclone\cache

i am confused about something, not understanding the vfs-read
i have a 7GB, 4 hour movie, i can seek anywhere very quickly.

memory does not go above ~70MB and the cache-dir is empty
where are the chunks stored, as i cannot find them in memory or in the cache-dir?

[Animosity022]

Based on that command, nothing is stored anywhere other than in memory.

In the debug log, you'd see it seeking around the file.

Would depend I'd surmise on if it's direct playing or not as it might be transcoding so you'd have some local storage I'd imagine in Emby/Plex.

If direct playing, not sure other than that is a nice connection you have.

[asdffdsa (jojothehumanmonkey)]

--vfs-read-chunk-size and --vfs-read-chunk-size-limit does not work with --vfs-cache-mode=off
seem that is true

[Animosity022]

You can see it in the debug logs.

Mount command:

felix@gemini:~$ rclone mount gcrypt: /home/felix/test --vfs-read-chunk-size 512M --vfs-read-chunk-size-limit 1G -vv
2021/03/21 12:21:18 DEBUG : rclone: Version "v1.54.1" starting with parameters ["rclone" "mount" "gcrypt:" "/home/felix/test" "--vfs-read-chunk-size" "512M" "--vfs-read-chunk-size-limit" "1G" "-vv"]
2021/03/21 12:21:18 DEBUG : Creating backend with remote "gcrypt:"
2021/03/21 12:21:18 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2021/03/21 12:21:18 DEBUG : Creating backend with remote "GD:crypt"
2021/03/21 12:21:18 DEBUG : Encrypted drive 'gcrypt:': Mounting on "/home/felix/test"

and if you copy a file as an example. Note the chunksize.

2021/03/21 12:21:58 DEBUG : &{Movies/Unsane (2018)/Unsane (2018).mkv (r)}: Read: len=131072, offset=0
2021/03/21 12:21:58 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: ChunkedReader.openRange at 0 length 536870912
2021/03/21 12:21:59 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: ChunkedReader.Read at 0 length 4096 chunkOffset 0 chunkSize 536870912
2021/03/21 12:21:59 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: ChunkedReader.Read at 4096 length 8192 chunkOffset 0 chunkSize 536870912

Here's an example with a tiny chunksize:

felix@gemini:~$ rclone mount gcrypt: /home/felix/test --vfs-read-chunk-size 1M --vfs-read-chunk-size-limit 128M -vv
2021/03/21 12:25:19 DEBUG : rclone: Version "v1.54.1" starting with parameters ["rclone" "mount" "gcrypt:" "/home/felix/test" "--vfs-read-chunk-size" "1M" "--vfs-read-chunk-size-limit" "128M" "-vv"]
2021/03/21 12:25:19 DEBUG : Creating backend with remote "gcrypt:"
2021/03/21 12:25:19 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2021/03/21 12:25:19 DEBUG : Creating backend with remote "GD:crypt"
2021/03/21 12:25:19 DEBUG : Encrypted drive 'gcrypt:': Mounting on "/home/felix/test"
2021/03/21 12:25:19 DEBUG : : Root:
2021/03/21 12:25:19 DEBUG : : >Root: node=/, err=<nil>

and the chunksize when I open the file.

2021/03/21 12:30:17 DEBUG : Movies/Unsane (2018)/: Lookup: name="Unsane (2018).mkv"
2021/03/21 12:30:17 DEBUG : Movies/Unsane (2018)/: >Lookup: node=Movies/Unsane (2018)/Unsane (2018).mkv, err=<nil>
2021/03/21 12:30:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: Attr:
2021/03/21 12:30:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: >Attr: a=valid=1s ino=0 size=25938409160 mode=-rw-rw-r--, err=<nil>
2021/03/21 12:30:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: Open: flags=OpenReadOnly
2021/03/21 12:30:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: Open: flags=O_RDONLY
2021/03/21 12:30:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: >Open: fd=Movies/Unsane (2018)/Unsane (2018).mkv (r), err=<nil>
2021/03/21 12:30:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: >Open: fh=&{Movies/Unsane (2018)/Unsane (2018).mkv (r)}, err=<nil>
2021/03/21 12:30:17 DEBUG : &{Movies/Unsane (2018)/Unsane (2018).mkv (r)}: Read: len=131072, offset=0
2021/03/21 12:30:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: ChunkedReader.openRange at 0 length 1048576
2021/03/21 12:30:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: ChunkedReader.Read at 0 length 4096 chunkOffset 0 chunkSize 1048576
2021/03/21 12:30:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: ChunkedReader.Read at 4096 length 8192 chunkOffset 0 chunkSize 1048576
2021/03/21 12:30:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: ChunkedReader.Read at 12288 length 16384 chunkOffset 0 chunkSize 1048576
2021/03/21 12:30:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: ChunkedReader.Read at 28672 length 32768 chunkOffset 0 chunkSize 1048576
2021/03/21 12:30:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: ChunkedReader.Read at 61440 length 65536 chunkOffset 0 chunkSize 1048576
2021/03/21 12:30:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: ChunkedReader.Read at 126976 length 131072 chunkOffset 0 chunkSize 1048576
2021/03/21 12:30:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: ChunkedReader.Read at 258048 length 262144 chunkOffset 0 chunkSize 1048576

Starts at 1MB and bumps up as it continues to read the file to my 128MB limit.

2021/03/21 12:31:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: ChunkedReader.Read at 685764608 length 1048576 chunkOffset 670040064 chunkSize 134217728
2021/03/21 12:31:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: ChunkedReader.Read at 686813184 length 1048576 chunkOffset 670040064 chunkSize 134217728
2021/03/21 12:31:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: ChunkedReader.Read at 687861760 length 1048576 chunkOffset 670040064 chunkSize 134217728
2021/03/21 12:31:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: ChunkedReader.Read at 688910336 length 1048576 chunkOffset 670040064 chunkSize 134217728
2021/03/21 12:31:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: ChunkedReader.Read at 689958912 length 1048576 chunkOffset 670040064 chunkSize 134217728
2021/03/21 12:31:17 DEBUG : Movies/Unsane (2018)/Unsane (2018).mkv: ChunkedReader.Read at 691007488 length 1048576 chunkOffset 670040064 chunkSize 134217728