Dedicate rclone threads to download vs upload

Help and Support
1

What is the problem you are having with rclone?

Right now, I'm uploading quite a lot of files (both in size and quantity). When I'm doing this, downloads are extremely slow or not doing anything (I believe because all threads are currently working towards uploading files).

I'd like to prioritize and always dedicate download threads over upload threads.

What is your rclone version (output from rclone version)

1.51.0

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10, 64bit

Which cloud storage system are you using? (eg Google Drive)

Google Drive (Business)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount gcrypt: G: --allow-other --cache-db-purge --buffer-size 16M --use-mmap --dir-cache-time 72h --drive-chunk-size 16M  --timeout 1h  --vfs-cache-mode writes --vfs-read-chunk-size 16M --vfs-read-chunk-size-limit 2G -o ThreadCount=16
2

Are you maxing out your upload while that is happening? Rclone does pretty well in terms of doing both at the same time in general.

What your rclone.conf look like?

3

This will be because your uplink is full which means that the acks for the downloads aren't getting back to the source.

One solution for this is a bit of QoS at the router.

The other will be using --bwlimit - though I suspect you are going to need different upload and download rates which is currently in development planning.

4

This is my rclone.conf

[gdrive]
type = drive
client_id = <<omitted>>.apps.googleusercontent.com
client_secret = <<omitted>>
scope = drive
token = <<omitted>>
root_folder_id = <<omitted>>

[gcache]
type = cache
remote = gdrive:/Media

[gcrypt]
type = crypt
remote = gdrive:/Media/crypt
filename_encryption = standard
directory_name_encryption = true
password = <<omitted>>
password2 = <<omitted>>

Hmm I see, I have a net limiting program on my computer I can use to limit down/upload rates, I'll try applying an upload limit to rclone and see if that helps

5

You'd want to separate out the uploads probably with rclone move/copy and bwlimit them. You'll have a tough time on the same process trying to limit out a thread for upload without impacting the downloads too.

6

So I tried limiting the bandwidth; my max up speed is 2MB/s, and only once I limited it to 1.6MB/s did downloads start working again (via streaming on Plex).

Hmm I do like the convenience of being able to move things over via File Explorer though. I'm assuming rclone move/copy is just a command that performs the move once?

What if I have two mounts, one that is read-only and I hook that up to Plex, and the other has write capability for uploads? Are there any issues with multiple mounts from a performance perspective or API rate limiting perspective?

7

I think that would work. You'll do the same number of API operations just spread over 2 mounts I would have thought.

8

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.