Behaviour on Windows: When playing a video or audio file, it would cache to RAM and vfs cache folder on disk would remain empty with 0 disk write activity. Behaviour on Linux (MX Linux): Even with --vfs-cache-mode set to minimal (same behaviour with writes), it writes to disk the whole as --vfs-cache-mode full would. Disk activity is really high and the cache folder contains all the played back files i.e the whole file is being cache. Desired behaviour: rclone will cache to RAM when using vfs-cache-mode writes. Note: On Windows, the video which is being played usually cached a few second ahead, whereas on Linux it keeps caching until the whole video is cached showing the behaviour of vfs-cache-mode full
Run the command 'rclone version' and share the full output of the command.
rclone v1.58.1
os/version: debian 11.3 (64 bit)
os/kernel: 5.10.0-14-amd64 (x86_64)
os/type: linux
os/arch: amd64
go/version: go1.17.9
go/linking: static
go/tags: none
Which cloud storage system are you using? (eg Google Drive)
Google Drive
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone mount GDrive: /home/Rafee/Documents/GDrive/ --vfs-cache-mode minimal --cache-dir /home/Rafee/Documents/cache
I'm not seeing anything wrong in the logs as that's what I would expect to happen.
The only thing that uses memory is buffer-size as that is how much of the file is kept in memory. Buffer is dumped if the file is closed.
That's not how the cache mode works. This only writes files locally to disk and uploads them. This is used for certain applications/programs that need a more compatible file system to write to. You'll see an error in the log saying you need this if the program is asking to write in a mode that isn't compatible with off.
Both OS'es do the same thing here. It's more dependent on how the file is being accessed. If the file is accessed sequentially, it continues to read ahead based on the buffer size until it's closed out. If you are using full, vfs-read-ahead comes into play as well.
I'm not sure if minimal uses sparse files as it's not called out as I've never used it or found a need for minimal.
Full cache mode definitely uses sparse files so you'll see a difference in what is being reported and what is actually used:
felix@gemini:/cache/TV$ du -sh *
499G vfs
3.5M vfsMeta
felix@gemini:/cache/TV$ du -sh * --apparent-size
2.4T vfs
376K vfsMeta
as an example as I'm using full so files are sparse which means only parts I've read are 'filled'. No idea how Windows shows that offhand as I'm primarily a Linux user.
My bad, I mixed up the terms. What I meant was on videos, when playing back a video, it'd buffer in RAM and that's all. It wouldn't cache anything in disk with vfs-cache-mode writes.
What I want to achieve is when playing back a video it'll only buffer it in RAM and not cache anything in disk at all. Do I need a separate flag for that ( like --buffer-size 1G)? Thanks!
I was trying to get same behaviour in Windows and the behaviour vfs-cache-mode writes should show which is
In this mode files opened for read only are still read directly from the remote, write only and read/write files are buffered to disk first.
since I was only reading the video file, yet it was buffering/caching the file to disk on Linux.
I managed to get the desired behaviour for playing back video vfs-cache-mode off and --buffer-size 1G, but I get errors when trying to upload/edit files.
If you are using cache-mode writes, that should not be the case if you reading. Would need to a debug log file and we can validate why as if the app opens it up in write mode even if it's reading only, that would cause a problem. Easy to see in the debug log.
That depends on how the application works and how it opens files. As you said, you'll see ERRORs in the logs if writes cache mode is required. That's nothing rclone can influence as that's application specific on how it asks for a file.
All I did was play back 10 seconds of a video (Brooklyn Nine Nine S01E10) of file size 2.2GB. Size on the cache folder doing the same thing in Windows, 0 MB
