Docker Volume Plugin - bwlimit timetable

Help and Support
1

Titel:

Hi fellow rclone-users,

I am trying to use the Docker Volume Plugin with a timetable.
I would like to set the following: 08:00,400k:off 23:00,off
This should limit the upload from 08:00 to 23:00 and leave it unlimited the rest of the day and in the morning.

To do this i first disabled the rclone plugin:
docker plugin disable rclone

Then i edited my compose file, adding the bwlimit-line:

  vol_rc_crypt_Landing:
    driver: rclone
    driver_opts:
      remote: 'gcrypt:<deleted>'
      allow_other: 'true'
      allow_non_empty: 'true'
      vfs_cache_mode: full
      poll_interval: 0
      bwlimit: "08:00,400k:off 23:00,off"

This gave me the following error:
ERROR: create compose_stash_vol_rc_crypt_Landing: VolumeDriver.Create: unsupported backend option "bwlimit"

Then I tried various variations of the following args, all with the same error:
The command consists of 3 stages. First the set args, then a read args. The last step is the plugin enable. I included the output from the read args (As the list of parameters) and the plugin enable (as "Output:").

No special quotes

docker plugin set rclone args="--allow-other --buffer-size=50G --drive-impersonate=xxx@some.domain --bwlimit=08:00,400k:off 23:00,off"; docker plugin inspect rclone | jq '.[0].Settings.Args'
docker plugin enable rclone

  "--allow-other",
  "--buffer-size=50G",
  "--drive-impersonate=xxx@some.domain",
  "--bwlimit=08:00,400k:off",
  "23:00,off"

Output: Error response from daemon: dial unix /run/docker/plugins/92ca36642e202fea0eb143c5db96e2f1d43914325fc25363b909b27503aa9ac8/rclone.sock: connect: no such file or directory

Single quotes around --bwlimit Parameter and values

docker plugin set rclone args="--allow-other --buffer-size=50G --drive-impersonate=xxx@some.domain '--bwlimit=08:00,400k:off 23:00,off'"; docker plugin inspect rclone | jq '.[0].Settings.Args'
docker plugin enable rclone

  "--allow-other",
  "--buffer-size=50G",
  "--drive-impersonate=xxx@some.domain",
  "'--bwlimit=08:00,400k:off",
  "23:00,off'"

Output: Error response from daemon: dial unix /run/docker/plugins/92ca36642e202fea0eb143c5db96e2f1d43914325fc25363b909b27503aa9ac8/rclone.sock: connect: no such file or directory

Single quotes around values only

docker plugin set rclone args="--allow-other --buffer-size=50G --drive-impersonate=xxx@some.domain --bwlimit='08:00,400k:off 23:00,off'"; docker plugin inspect rclone | jq '.[0].Settings.Args'
docker plugin enable rclone

  "--allow-other",
  "--buffer-size=50G",
  "--drive-impersonate=xxx@some.domain",
  "--bwlimit='08:00,400k:off",
  "23:00,off'"

Output: Error response from daemon: dial unix /run/docker/plugins/92ca36642e202fea0eb143c5db96e2f1d43914325fc25363b909b27503aa9ac8/rclone.sock: connect: no such file or directory

Single quotes around all args, double quotes around values

docker plugin set rclone args='--allow-other --buffer-size=50G --drive-impersonate=xxx@some.domain --bwlimit="08:00,400k:off 23:00,off"'; docker plugin inspect rclone | jq '.[0].Settings.Args'
docker plugin enable rclone

  "--allow-other",
  "--buffer-size=50G",
  "--drive-impersonate=xxx@some.domain",
  "--bwlimit=\"08:00,400k:off",
  "23:00,off\""


Output: Error response from daemon: dial unix /run/docker/plugins/92ca36642e202fea0eb143c5db96e2f1d43914325fc25363b909b27503aa9ac8/rclone.sock: connect: no such file or directory

Escaped double quotes around --bwlimit and value

docker plugin set rclone args="--allow-other --buffer-size=50G --drive-impersonate=xxx@some.domain \"--bwlimit=08:00,400k:off 23:00,off\""; docker plugin inspect rclone | jq '.[0].Settings.Args'
docker plugin enable rclone

  "--allow-other",
  "--buffer-size=50G",
  "--drive-impersonate=xxx@some.domain",
  "\"--bwlimit=08:00,400k:off",
  "23:00,off\""

Output: Error response from daemon: dial unix /run/docker/plugins/92ca36642e202fea0eb143c5db96e2f1d43914325fc25363b909b27503aa9ac8/rclone.sock: connect: no such file or directory

Just an escape-Slash

docker plugin set rclone args="--allow-other --buffer-size=50G --drive-impersonate=xxx@some.domain --bwlimit=08:00,400k:off\ 23:00,off"; docker plugin inspect rclone | jq '.[0].Settings.Args'
docker plugin enable rclone

  "--allow-other",
  "--buffer-size=50G",
  "--drive-impersonate=xxx@some.domain",
  "--bwlimit=08:00,400k:off\\",
  "23:00,off"

Output: Error response from daemon: dial unix /run/docker/plugins/92ca36642e202fea0eb143c5db96e2f1d43914325fc25363b909b27503aa9ac8/rclone.sock: connect: no such file or directory

Two escape-Slashes

docker plugin set rclone args="--allow-other --buffer-size=50G --drive-impersonate=xxx@some.domain --bwlimit=08:00,400k:off\\ 23:00,off"; docker plugin inspect rclone | jq '.[0].Settings.Args'
docker plugin enable rclone

  "--allow-other",
  "--buffer-size=50G",
  "--drive-impersonate=xxx@some.domain",
  "--bwlimit=08:00,400k:off\\",
  "23:00,off"

Output: Error response from daemon: dial unix /run/docker/plugins/92ca36642e202fea0eb143c5db96e2f1d43914325fc25363b909b27503aa9ac8/rclone.sock: connect: no such file or directory

Just to make sure i also tried the simple bwlimit-format and it worked as expected:

docker plugin set rclone args="--allow-other --buffer-size=50G --drive-impersonate=xxx@some.domain --bwlimit=400k:off"; docker plugin inspect rclone | jq '.[0].Settings.Args'
docker plugin enable rclone

Output: <Everything is fine>

And now for the usual info:

I am trying to limit the upload to gSuite.

Sadly i am unable to get logs when the plugin is disabled. I tried these commands to access the logs:

myID="$(docker plugin inspect -f {{.Id}} rclone)"
cd /run/docker/plugins/$myID
cat < init-stdout
# or
cat < init-stderr

My /var/lib/docker-plugins/rclone/config/rclone.conf looks like this (with some more entries for SFTP or other cloud drives):

[gsuite]
type = drive
scope = drive
service_account_file = /data/config/<deleted>.json
client_id = <deleted>
client_secret = <deleted>

[gcrypt]
type = crypt
remote = gsuite:<deleted>
filename_encryption = <deleted>
directory_name_encryption = <deleted>
password = <deleted>
password2 = <deleted>

I am using the latest tag.

2

I sadly forgot the actual question: How do you set a timetabled --bwlimit with the docker volume plugin?

3

Could my problems with the docker volume --bwlimit timetable be considered a bug?
Would anybody here be against me filing a bug report?

4

Setting arguments with spaces is impossible becuase this line in the docker plugin set source code sucks miserably:

p.PluginObj.Settings.Args = strings.Split(s.value, " ")

It splits the args by space and does not allow for any value escaping like quotes or backslashes.
I mentioned this shortcoming in the plugin docs.
We could submit a bug report on the docker tracker, but after Google and RedHat/IBM took away all key people from Docker Inc, their issue tracker is approaching 4K and counts so the chances they fix anything but critical bugs are slim.
However there is a workaround using rclone's ability to take any flag from environment. If you run rclone manually you could

export RCLONE_BWLIMIT="08:00,400k:off 23:00,off"
export RCLONE_BWLIMIT_FILE="oh bla dee oh bla dah"

The docker plugin runs plain rclone inside a plugin container. We are required to describe every possible environment variable for plugin in advance in a plugin config file.
If you add the two new variables there and rebuild the plugin, you will be able to

docker plugin set RCLONE_BWLIMIT="08:00,400k:off 23:00,off"

You are welcome to submit a pull request or just an issue.

5

Wow, for some reason i didn't even consider a docker limitation.
I will try to modify my local plugin and if that is successfull i will try my hand at a pull request.

Thanks for your help and the incredible insight!

6

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.