I use Rclone on my gdrive to mount in windows. I have a tool that reads the files on that mounted drive. The tool shows me how fast it reads the files. I see reading speed of 6-8 MB/s. Rclone is running from an SSD, therefor I assume this shouldnt be a bottleneck. I upload at 50MB/s (I read that this is API V3 limit to upload, but this is with rclone browser move command). The download link is 10gbit (upload also), so speed wise the bottleneck is not my fiberline. I wonder if my speeds are normal or I could somehow improve the reading speed. Often only parts of the files are read by my tool. So far I tried to change the --drive-chunk-size 64M --vfs-read-chunk-size 64M to 32M for both, 16M for both and even 8M for both. Nothing helped. Are there some flags that could make read speeds faster? The read files are only read one time btw, after some time other files will be read and so on.
I set up my own API btw and I see activity on it.
Thank you for your help!
What is your rclone version (output from rclone version)
rclone v1.53.3
os/arch: windows/amd64
go version: go1.15.5
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Windows 10 Enterprise 64 bit
Which cloud storage system are you using? (eg Google Drive)
Gdrive
The command you were trying to run (eg rclone copy /tmp remote:tmp)
mount --allow-other --drive-chunk-size 64M --vfs-read-chunk-size 64M gdrive_crypt: Z: --config "C:\mypath\rclone.conf"**
#### The rclone config contents with secrets removed.
I am not quite a 'pro' user of rclone, but for my gdrive setup I used some of the VFS file caching flags to speed up things alot. You should check them out: https://rclone.org/commands/rclone_mount/#vfs-file-caching
You can set the caching mode, the max. space to use and the max. time files should be cached. This decreases GDrive API hits to a minimum, so you should have less errors and limits from Google.
The tool is a solution a friend of mine coded for myself, that is why I dont want to go into details. It is basicly simple: It grabs my file from rclone mount and reads a part of it.
I actually did not try standard flags, since those line is what I, with very little Idea, got together.
You are right, with the units, i will clarify it a bit:
I have full duplex 10gbit fiber at home, with a 10gbit router and 10gbit on hand (speedtests proof that it is not a hardware thing on my side).
The 50MB/s is what rclone browser shows me, when I upload with 1 transfer at a time.
I simply try to speed up the download, that rclone mount does, when I access files to read.
I will try now to remove the 2 flags you posted
Edit: befor I change the arguments: My speed with rclone copy are 150 MBytes/s to 160 MBytes/s
Since I can only post 1 Screenshot at a time as a new user: Here you see the speedtest:
I use my routers integrated speedtest (but it still uses speedtest,net):
Good Point on your side (please check the 2nd screenshot I posted with speedtest too).
I know it, since locally it reads at 50MB/s + on its report.
When I point it to the mount, it reads at 8ish MB/s on its report.
I cant really meassure it otherwise.
My guess was, that rclone mount does not get the files as fast as a copy command - that is why I asked if there is something for mounting that could speed it up.
I removed the 2 named arguments - but nothing changed.
I dont expect the same speed for remote than local obviously, but more than 8 MB/s should be possible now? I mean copy a file proofs that it could be faster (at least with copy).
I know I removed it. Still no change, I also tried all different kind of vfs-cache-mode but nothing changed. If I do off, it even only reads at 3-4 MB/s.
I guess you are out of ideas?
I don't get the part with the chunk sizes 100%, would it be better to set it to higher values? I mean if the speed for one chunk tiny chunk is slow, might it improve for others?
i tend to run multiple mounts to the same remote, each with optimized settings.
one mount for streaming/scanning media for jellyfin
one mount for editing small files like word documents or notepad++.
one mount if i want to net share a mount over a lan.
so it depends on what you are using the mount for.
do you have another computer you can test with, other than the windows machine?
I actually have one other windows machine where I could test.
I already had that idea and was able to mount the drive there.
But if you mean a Linux machine - no I dont have a Linux machine.
The files I read are between 50 and 100gb - but there is only a small portion of that file that will be read.
I wont share it over lan or something. I dont know if that really helps?
That is exactly what it does:
It reads small parts of big files.
Lets say its like the following: A movie file of 4 hours, my tool says "read minute 10 to 11, 145 to 146".
That is what it does. I guess one could compare it to your jellyfin mount, except that it doesnt have to read the entire mediafile in one go.
Is 6-8 MB/s reading on my pattern, which is small portions of big files, normal speed for rclone mount?
If no, my problem is, how to speed that up in rclone.
