Work around for vfs mount download bandwidth limitations?

Help and Support
1

What is the problem you are having with rclone?

Is there any workaround for this issue? (Support multi-threaded downloads when downloading a file to the cache · Issue #4760 · rclone/rclone · GitHub) currently, I'm only getting 1/6 (45mbps) of potential max download bandwidth (270 mbps) and that causes seekable buffer to fill way slower than it could on a video. Apparently seeking through random parts creates another rclone "downloader", making it utilize more bandwidth but it's not that conventional, and I don't know whether it actually helps or not. I've checked and rclone copy limited to single stream thread results in the same bandwidth I get when I stream a video, as observed with the issue above.
I've tried different mount settings but I couldn't increase bandwidth usage no matter what, is there any way to do it?

What is your rclone version (output from rclone version)

Latest

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10 20H2

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

mount --vfs-cache-mode full --use-mmap --vfs-read-ahead 1T --cache-dir C:\Cache --config C:\Users\Administrator\.config\rclone\rclone.conf --vfs-cache-max-size 900G --dir-cache-time 1000h --vfs-cache-max-age 24h --buffer-size 0M
2

I'm hoping to get this feature done for 1.56.

I'm not sure this feature will work very well with google drive though.

If you want to see then try using rclone copy to copy the file direct from drive to a local disk and see what bandwidth you get there. It will use 4 download streams so you can have some idea as to whether it would help.

3

It definitely does, setting --multi-thread-streams to 8 fully saturates my download bandwidth of 270 mbps, where as only 1 (which is the case when manually copying or accessing a file directly from a mount to local disk) only uses 45mbps, the same could be observed when seeking a .mkv file from a mount which spawns another rclone downloader and increases bandwidth usage.

This feature could be a game changer to people who use a rclone mount as their media streaming/Plex backend (which is a way bigger use case to this feature) since this would enable users who are bottlenecked by poor single thread stream performance, causing buffering or not being able to play 4K/big file size content at all (even though they have sufficient bandwidth to stream 4K) to be able to fully utilize their bandwidth and experience an overall way smoother and faster buffer playback. Being able to mitigate this issue using rclone would be awesome since a lot of people are affected by it.

Better internet streaming using multiple parallel connections (or threads). - Feature Suggestions - Plex Forum, this thread explains it better than I do.

4

Thanks for confirming that.

There will be a bit of performance tuning to do as it won't work exactly like multithread streams do - this splits the dowloads into 4 and downloads your file like this

AAAAAAAAAABBBBBBBBBBCCCCCCCCCCDDDDDDDDDD

Whereas this new feature will download your file like this

ABCDABCDABCDABCDABCDABCDABCDABCDABCDABCD

It will buffer the chunks in memory, so exactly how big they will affect performance and memory use. I would guess that making the chunks 4-16 MB will be best for performance.

5

Will it have a customizable int to set how many threads? Also, is there any way I could beta test it when available?

6

Yes it will be customizable.

Here is the issue

I'll post betas there for testing so subscribe to that.

7

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.