Bandwith limit doesn't work when it specifyed in remote control mode

Help and Support
1

What is the problem you are having with rclone?

Bandwith limit doesn't work when it specifyed in remote control mode (rclone rc ..... --bwlimit 1M)
Interesting that flag --bwlimit-file 1M works as expected

Run the command 'rclone version' and share the full output of the command.

rclone v1.65.0

  • os/version: darwin 14.1.2 (64 bit)
  • os/kernel: 23.1.0 (x86_64)
  • os/type: darwin
  • os/arch: amd64
  • go/version: go1.21.4
  • go/linking: dynamic
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

S3 --> S3

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone rc sync/copy srcFs=SOURCE:buck1 dstFs=DEST:buck1 --bwlimit 1M

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[DEST]
type = s3
access_key_id = XXX
endpoint = http://10.79.128.34
provider = Ceph
secret_access_key = XXX

[SOURCE]
type = s3
provider = Ceph
secret_access_key = XXX
access_key_id = XXX
endpoint = http://10.79.128.32

A log from the command that you were trying to run with the -vv flag

2023/12/05 12:38:08 INFO  : Starting bandwidth limiter at 1Mi Byte/s
2023/12/05 12:38:08 DEBUG : rclone: Version "v1.65.0" starting with parameters ["rclone" "rc" "sync/copy" "srcFs=SOURCE:buck1" "dstFs=DEST:buck1" "--rc-no-auth" "--bwlimit" "1M" "-vv"]
2

Try to use options/set to set BwLimit - (you can get all available options by running rclone rc options/get):

# by default it is unlimited
$ rclone rc options/get | grep -i BwLimit
		"BwLimit": "",

$ rclone rc options/set --json '{"main": {"BwLimit": "1Mi"}}'

# now it is set to 1Mi
$ rclone rc options/get | grep -i BwLimit
		"BwLimit": "1Mi",

and then you can run you copy command:

rclone rc sync/copy srcFs=SOURCE:buck1 dstFs=DEST:buck1
3

I have also tried set it up via config params:
rclone rc sync/copy srcFs=SOURCE:buck1 dstFs=DEST:buck1 --rc-no-auth _config='{"BwLimit":"1M"}' -vv
This way also doesn't work

rclone rc options/set - As I know this command set bandwith to rcd instead of only one job. It means that another commands will inherit bandwith limits.

I am looking way to setup all params and run job in one command. Idea is to run several commandas with different options like BwLimit, Transfers ....

4

It should unless something is broken.

This is not instant though, bandwidth is only checked like every minute so you might have to wait a minute to see limits to kick in. Maybe this is why you consider it not working...

5

I have waited more than 2h, so I totally sure that it doesn't work. In example below, I decreased the count of transfers to 1 and used 'BwLimitFile' option just to simulate BwLimit. And it works, so I suggest that in some reason BwLimit doesnt work with rc command

rclone rc sync/copy srcFs=SOURCE:buck1 dstFs=DEST:buck1 _config='{"BwLimitFile":"1M", "Transfers": 1}'

6

I'm not sure bwlimit works as a per command setting - the _config='{"BwLimitFile":"1M", "Transfers": 1}'

I think you'll have to set it globally with options/set.

Currently in rclone there is only one global bandwidth limit, there isn't one per command.

7

To be honest, I thought so, after 3 days of experiments)) I was just very surprised why BwLimitFile works per command but BwLimit doesn't.

It would be nice to add to the documentation which options we can use in commands with rc and which ones only globally. In any case thank you for support!

8

That is because there is one bandwidth limiter per file, rather than a gobal one, so every time rclone makes a limiter for the file it uses the config you supplied.

Yes I agree.

The RC is a bit underdocumented and there are some things like this which don't work as expected which could be fixed.

9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.