Precache a directory or file in vfs cache

What is the problem you are having with rclone?

Is there a way to precache a certain directory while using the vfs cache mode full?

What is your rclone version (output from rclone version)

rclone v1.53.1

  • os/arch: windows/amd64
  • go version: go1.15

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10 64 bit

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The rclone config contents with secrets removed.

mount --config="C:\Users\<Username>\.config\rclone\rclone.conf" --allow-other --log-level INFO --log-file "F:\<redacted>\rclone.log" --rc --dir-cache-time 8760h --attr-timeout 8700h --poll-interval 20s --cache-dir="F:\<redacted>\gdrive_cache" --vfs-cache-mode full --vfs-cache-max-age 336h --vfs-cache-max-size 200G ud:"/<redacted>" "F:\<redacted>"

The thing I am trying to achieve is this: Given a file name, or folder name, is there a way to tell rclone to precache the file(s) in the given vfs cache directory, it is sort of like downloading the files, but it will follow the cache rules of rclone (like getting deleted after x hours, etc).

P.S. (Reason as to why I am trying to do this) The above files/folders mostly have video files which I am trying to precache, given my internet is not the best and sometimes gives up in the middle of direct streaming, I was trying to precache the video(s) that I wanna watch tonight so that it even works if my internet starts acting up.

There isn't currently. You could do this though

rclone md5sum --checkers 4  "F:\<redacted>\dir"

which is almost exactly what you want apart from calculating md5sums which takes very little CPU time on a modern CPU. Tune --checkers for max throughput. Add --bwlimit if you want it to be less agressive.

Just trying to understand what is happening here. So md5sum will try to calculate the sum of the file in my google drive mount, thereby causing rclone to go through the file because of which it will be stored in the vfs cache. Right?

Btw just tried on a single file and it does what it is supposed to thanks. <3

EDIT: Seems like --bwlimit flag does not work with md5sum, below is the log with -vv flag. But when I look in task manager, rclone is using the maximum speed my connection has to offer, around 40Mbps (5MBps)
Log:

PS C:\Users\<Username>> rclone md5sum -vv --bwlimit 1M "F:\<Path_To_File>\VideoName.mkv"
2020/10/13 20:20:32 DEBUG : rclone: Version "v1.53.1" starting with parameters ["E:\\dev\\bin\\rclone.exe" "md5sum" "-vv" "--bwlimit" "1M" "F:\\<Path_To_File>\\VideoName.mkv"]
2020/10/13 20:20:32 DEBUG : Creating backend with remote "F:\\<Path_To_File>\\VideoName.mkv"
2020/10/13 20:20:32 DEBUG : Using config file from "C:\\Users\\<Username>\\.config\\rclone\\rclone.conf"
2020/10/13 20:20:32 INFO  : Starting bandwidth limiter at 1MBytes/s
2020/10/13 20:20:32 DEBUG : fs cache: adding new entry for parent of "F:\\<Path_To_File>\\VideoName.mkv", "//?/F:/<Path_To_File>"
2020/10/13 20:20:32 DEBUG : Other_Files_In_The_Folder.mkv: Excluded
2020/10/13 20:20:32 DEBUG : Other_Files_In_The_Folder.mkv: Excluded
2020/10/13 20:20:32 DEBUG : Other_Files_In_The_Folder.mkv: Excluded
2020/10/13 20:20:32 DEBUG : Other_Files_In_The_Folder.mkv: Excluded
2020/10/13 20:20:32 DEBUG : Other_Files_In_The_Folder.mkv: Excluded
2020/10/13 20:20:32 DEBUG : Other_Files_In_The_Folder.mkv: Excluded
2020/10/13 20:20:32 DEBUG : Other_Files_In_The_Folder.mkv: Excluded
2020/10/13 20:20:32 DEBUG : Other_Files_In_The_Folder.mkv: Excluded
2020/10/13 20:20:32 DEBUG : Other_Files_In_The_Folder.mkv: Excluded

That is right.

I'd expect the bw usage to peak then average 1 MByte/s eventually.

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.