What would be a good value for --buffer-size with a big --vfs-read-ahead such as 4G?
(with 10G link and 32GB RAM)
The purpose of --buffer-size is for the network read-ahead buffer allowing the thread reading from the network to get a bit ahead of the thread reading the memory. With a large read ahead buffer I don't think buffer-size needs to be big - the default of 16M should be OK. You could try making it bigger 32M or 64M or even removing it. All those choices have consequences which are difficult to predict in advance!
Thanks for your advice. Will keep testing it with default --buffer-size
Will this features be merged to beta soon as i am looking for these features but is running in docker and don't want to build it by myself 
It’s already in beta.
It's in a separate branch, not in the master branch from where the beta builds go out.
So when would that branch be merged to master? looking forward for these functions.
I've merged it now - I just needed to complete the VFS torture test on it!
v1.52.2-285-g4d7f9130-beta on branch master (uploaded in 15-30 mins)
2 Likes
--vfs-read-ahead default size 16M or ?
Hello I am trying u use --vfs-read-ahead for plex but I dont know if is correct what I do....
I have use curl https://rclone.org/install.sh | sudo bash -s beta to upgrade my rclone on ubuntu on raspberry pi 4, now I have a version : v1.52.3-294-g61c7ea40-beta
I have this configuration:
[team1prueba]
type = drive
client_id = *******
client_secret = *********
scope = drive
root_folder_id = *********
token = ********
team_drive = *********
[team1pruebacache]
type = cache
remote = team1:/prueba
plex_url = http://xxx.xxx.x.xx:32400
plex_username = *********
plex_password = ********
chunk_size = 10M
info_age = 1d
chunk_total_size = 10G
plex_token = **********
I use this command:
rclone mount team1prueba:/ /home/ubuntu/team1prueba --allow-other
and
rclone mount team1pruebacache:/ /home/ubuntu/team1pruebacache/ --allow-other --vfs-read-ahead 5G
Plex takes data from team1pruebacache but i think it dosnt work, I think I do everoghit bad :)....
Thanks.
You need to remove the cache backend as this is not part of it. Point to your regular remote if you want to test this.
Hello this?.
[team1pruebacachevfs]
type = drive
client_id = **********
client_secret = *********
scope = drive
token = **********
team_drive = **********
plex_url = http://xxx.xxx.x.xx:32400
plex_username = ********
plex_password = ********
plex_token = *************
using:
rclone mount team1pruebacachevfs:/ /home/ubuntu/team1pruebacache2/ --allow-other --vfs-read-ahead 5G
I have test it but con rclone/.cache dir not show anyghint film play but dosnt doenload to cache
You can remove all of this.
You need to add this to your command to use the new feature.
Ok Use this comand:
rclone mount team1pruebacachevfs:/ /home/ubuntu/team1pruebacache2/ --allow-other --vfs-read-ahead 5G --vfs-cache-me full
[team1pruebacachevfs]
type = drive
client_id = *******
client_secret = *******
scope = drive
root_folder_id = *******
token = *****
team_drive = ********
it works , I have a bandwidth meter to show if ubuntu is downloading or not.
I am testing this:
1.- I click play and film start, bandwidth start 15MB
2.- I click stop play and bandwidth bo to 0, CORRECT.
3.- I click again play and fil start and bandwidth start 15MB CORRECT
4.- Film I think is downloading on cache because the space on my sd.
I see 2 folder:
/.cache/rclone/vfsMeta/team1pruebacachevfs/prueba -> size is 12 K
/.cache/rclone/vfs/team1pruebacachevfs/prueba -> size is 5.5G and increasing .......
But I see one strange thing.
Play film and I see this fil is downloading.
Now it has downloading...
I see the download on /.cache/rclone/vfs/team1pruebacachevfs/prueba is 42 GB
I go to minute 20 and it takes time to load.
If am on monute 30, and return to minute 1 the download of the film from internet stops and load from the cache only.
I dont know why the downloas speed is not constant... 13MB -- 1MB --- 7 MB --- 2 MB --- 12 MB----
I have open this same file on mobile and all cache stored on /.cache/rclone/vfs/team1pruebacachevfs/prueba was deleted!, I dont know why.... any idea???.
I will make more test and post it here.
Can I limit the cache stored on /.cache/rclone/vfs/team1pruebacachevfs/prueba ?? for example 20 GB.
And limit form example 5 hours?.
If my SD is full... what will happen????????.
Thanks.
You limit based on size as this is not chunks but sparse files. The file looks to take 10GB and if only uses a few MBs of the file, it only actually uses a few MBs. If you are streaming movie or tv, it would eventually have the whole file for a period of time as it's not like the cache backend which worked in chunks.
For limiting the size:
--vfs-cache-max-age duration Max age of objects in the cache. (default 1h0m0s)
--vfs-cache-max-size SizeSuffix Max total size of objects in the cache. (default off)
It would error out writing files and be generally bad.
It's best to make sure you have space and configure it properly.
Hello for example....
rclone mount team1pruebacache:/ /home/ubuntu/team1pruebacache/ --allow-other --vfs-read-ahead 5G -vfs-cache-mode full --vfs-cache-max-age 2D --vfs-cache-max-size 1G
?? its right?
And the --vfs-read-ahead 5G -> This 5G what does??
Lot Lot of thanks.
Ok it works fine but I see a little problem.
I moount with:
rclone mount team1pruebacachevfs:/ /home/ubuntu/team1pruebacache2/ --allow-other --vfs-read-ahead 5G --vfs-cache-mode full --vfs-cache-max-age 48h --vfs-cache-max-size 2G
I add on plex my folder, I go to :~/.cache/rclone/vfs/team1pruebacachevfs/prueba AND without playing anything the folder size is 31M, only with 2 films... I imagine is when plex index the files?, but... what happends when the original folder contain 1.000 archives?? I imagine the size will be big big big!!!!!.
Please keep in mind that this is still in beta so the docs for the various flags are spread across various posts in this thread and on the website. If you do want to try it out, make sure to go through this thread so you understand the various options and what they control.
This controls the maximum size of the cache. Irrespective of how many files are scanned, the cache will never grow beyond 2G (i.e 2 GB)
The vfs-cache-max-size should be set to at least the value of the vfs-read-ahead * number of files that you expect to be playing simultaneously. Setting it any lower won't be of any help and may even negatively impact the playback.
This controls how much rclone reads ahead and stores in the cache when a file is being read continuously from the mount.
Ok Ok, Excuse me please.
Lot of thanks. I am testing and its wonderfull!, incredible!.
Its curious,, I delete the folder /.cache/rclone/vfs/ but the size of the disc dosnt increase!, I am trying to find where are the GB used! , because if I kill the rclone process inmediately increase the size deleted on the cache folder. Its curious
I can only say ... The rclone designer and programmer is a genius, amazing program!!!