Feature Request - Dynamically change --transfers

Feature
1

I looked thru the supported remote control commands and didn't see this.
Please forgive me if this is already available and I missed it.

Just as rc core/bwlimit can dynamically change the bandwidth limit, I would like to be able to dynamically change the number of concurrent transfers (--transfers) on a running rclone copy process.
Increasing the number of transfers would cause new transfers to start immediately.
Decreasing would just stop new ones from starting until the --transfer setting is reached.

What do you think? Worthwhile?

2

There is API already to change --transfers but it won't change the number of transfers on a running sync.

In fact that is reasonably tricky since rclone starts lots of threads to run the transfers in parallel.

So I think it is a nice idea, but probably too difficult to implement easily.

3

If it's difficult, I'd say leave it and focus on more important things. It would be cool to have for easier testing and fine-tuning, but once that is done I don't see much session to change it during a session.

Somewhat related: How about making the VFS cache upload transfers # adjustable instead? That seems more useful. Currently I know of no way to make it anything but 4 - which may not be ideal for all use-cases.

4

I don't think the VFS cache transfers are limited at all at the moment... it will do as many as is necessary.

5

On downloads yes, but not uploads right? I thought that was basically hard-coded to 4.
How would it determine "as many as necessary" ? From the status of the pacer? Pretty sure I've never gotten more than 4 even on very large long-running files.

I'm specifically talking about the background process that the VFS write-cache uses to copy files over to Cloud once they have entered the cache via mount.

6

The VFS cache will upload as many files as you have open...

It doesn't yet do them in the background, though that is something I'd like to do for v1.51

7

Then there is some other limiter in another part of the chain, because from your description it sounds like if I copied over 100 files to the mount it would start up to 100 concurrent transfers (as quick as the API/pacer would allow).
That definitely does not happen.

8

That probably depends on how you are copying the files? Explorer will only copy a few at once I think.

9

Hmm... will test through cache on the webUI and with a copier than can do proper multi-threading then. You might be right.

10

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.