Full bandwidth usage with rclone docker

Help and Support
1

Rclone within a docker container with bandwidth schedule progressively uses the full bandwidth available.
I also tried this script but with the same result: for a certain amount of time everything runs smoothly then for no apparent reason rclone uses more and more bandwidth to finally use 100% of it.

rclone v1.53.1

  • os/arch: linux/amd64
  • go version: go1.15

Docker container runs on a Unraid server (version 6.8.2).
rclone mount Google drives

I am using the official rclone/rclone container.

I am using 3 different containers:

  1. Personal Google Drive
  2. Shared Google Drive
  3. Sync process to sync data from Google drive folders locally
1.

Container parameter: --cap-add SYS_ADMIN --security-opt apparmor:unconfined

Container post argument:

mount gdrive: /mnt/gdrive --config /rclone/config/rclone.conf --allow-other --size-only --uid=1000 --gid=1000 --umask 002 --dir-cache-time 2m --buffer-size 64M --log-level INFO --log-file /rclone/logs/rclone-mount-gdrive.log --timeout 1h --vfs-cache-max-age 72h --vfs-read-chunk-size 64M --vfs-read-chunk-size-limit 5G --drive-chunk-size 128M --buffer-size 64M --user-agent='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36' --rc --rc-addr 127.0.0.1:5572

Volumes:
/dev/fuse:/dev/fuse
/etc/passwd:/etc/passwd
/etc/group:/etc/group

2.

Container parameter: --cap-add SYS_ADMIN --security-opt apparmor:unconfined

Container post argument:

mount tdrive: /mnt/tdrive --config /rclone/config/rclone.conf --allow-other --size-only --uid=1000 --gid=1000 --umask 002 --dir-cache-time 2m --buffer-size 64M --log-level INFO --log-file /rclone/logs/rclone-mount-gdrive.log --timeout 1h --vfs-cache-max-age 72h --vfs-read-chunk-size 64M --vfs-read-chunk-size-limit 5G --drive-chunk-size 128M --buffer-size 64M --user-agent='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36' --rc --rc-addr 127.0.0.1:5572

Volumes:

/dev/fuse:/dev/fuse
/etc/passwd:/etc/passwd
/etc/group:/etc/group

3.

Conatiner parameter: --user $(id -u):$(id -g) --device /dev/fuse --cap-add SYS_ADMIN --security-opt apparmor:unconfined

Container post argument:

copy tdrive:/remote_dir /local_dir --bwlimit "00:30,10M 08:00,256K 17:00,100K" --tpslimit 8 --tpslimit-burst 5 --drive-chunk-size 256M --fast-list --drive-acknowledge-abuse --log-file /rclone/logs/rclone-sync-remote_dir.log 

Rclone config file

[gdrive]
type = drive
client_id = XXXXXXXXXXXXXXXXXXXXXXXXX.apps.googleusercontent.com
client_secret = YYYYYYYYYYYYYYYYYYYYYYYY
scope = drive
token = {"access_token":"ZZZZZZZZZZZZZZZZZZZZZZ","expiry":"2020-09-14T12:48:18.575356238+02:00"}
root_folder_id = ROOT_ID

[tdrive]
type = drive
client_id = XXXXXXXXXXXXXXXXXXXXXXXXX.apps.googleusercontent.com
client_secret = YYYYYYYYYYYYYYYYYYYYYYYY
scope = drive
token = {"access_token":"ZZZZZZZZZZZZZZZZZZZZZZ","expiry":"2020-09-14T18:05:01.369531163+02:00"}
team_drive = ROOT_ID
2

hello and welcome to the forum

and the results of a speedtest are?

3

If your question is: am I still able to use the internet, the answer is yes.
Here is a print screen of the last test I conducted

image
image1039×214 40.9 KB
.

4

the max speed is approx 9MBps.
and
the bwlimit is 10MBps

so in effect, rclone is told to use more then 100% of available bandwdith

5

I understand your point, the fact is that even if I don't start the 3rd container (the one that copy data) rclone eats all my bandwidth.

6

so without the third container, the problem persists.
you did not mention that.

so of the two remaining containers, which is the problem(s)?

7

The problem seems not linked to one or the other container. I got the same symptoms by running a script to start rclone from the command line. To be sure I was using the right configuration I used the script from BinsonBuzz. The script is here: https://github.com/BinsonBuzz/unraid_rclone_mount
I only used the mount option from the script.

8

ok. just trying to understand,

so the containers have nothing to do with the problem?

since you are not running the script, simply using the rclone mount settings only then that script have nothing to do with the problem?

does this problem happen at the same time each day?

is that all correct?

9

Hi @asdffdsa,
Sorry for the delay.
To be honest I am trying to identify the root cause of the problem and I cannot answer with certainty.
I cannot confirm if the problem is linked to the container or to a the simple script.
What is for sure is that by simply mounting a rclone mount, through a script or a container, without copy or sync activities, cause a progressive bandwidth consumption of my Internet usage.
No the problem does not happen at the same time. There is no trend or logic in the consumption.

10

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.