# Rclone saturating network bandwidth

**URL:** <https://forum.rclone.org/t/rclone-saturating-network-bandwidth/19772>\
**Category:** Help and Support\
**Created:** [October 13, 2020, 6:27pm UTC](https://forum.rclone.org/t/rclone-saturating-network-bandwidth/19772 "2020-10-13T18:27:39Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![RyanH](https://forum.rclone.org/user_avatar/forum.rclone.org/ryanh/32/4577_2.png) [@RyanH](https://forum.rclone.org/u/RyanH)\
**Post date:** [October 13, 2020, 6:27pm UTC](https://forum.rclone.org/t/rclone-saturating-network-bandwidth/19772/1 "2020-10-13T18:27:39Z")

</div>

**#### What is the problem you are having with rclone?**  
When running rclone, the entire network is brought to a complete crawl including the device that rclone is on. The WiFi network and connected clients are most affected.

There are no other applications running performing uploads at the same time. The total available upload speed is 400Mbps.

The currently solution has been to reduce the bandwidth consumption to ~6 or 7MB which is equivalent to circa 50Mbps.

**#### What is your rclone version (output from `rclone version`)**  
rclone v1.53.1

**#### Which OS you are using and how many bits (eg Windows 7, 64 bit)**  
Microsoft Windows 10 Professional version 1909 Build 18636.1110

**#### Which cloud storage system are you using? (eg Google Drive)**  
Microsoft OneDrive

#### The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

`rclone.exe --stats 10s --progress --bwlimit 7M --checksum --log-file=rclone_log.txt --log-level DEBUG -copy /path/to/files/on/local/hard/drive remote:/ --backup-dir=remote:/path`

---

<div class="post-metadata">

**Author:** ![Animosity022](https://forum.rclone.org/user_avatar/forum.rclone.org/animosity022/32/451_2.png) [@Animosity022](https://forum.rclone.org/u/Animosity022)\
**Post date:** [October 13, 2020, 6:29pm UTC](https://forum.rclone.org/t/rclone-saturating-network-bandwidth/19772/2 "2020-10-13T18:29:55Z")

</div>

I'm not sure what the question is? You can limit the bandwidth if that's the problem to reduce saturating your network.

rclone works very well 🙂

---

<div class="post-metadata">

**Author:** ![RyanH](https://forum.rclone.org/user_avatar/forum.rclone.org/ryanh/32/4577_2.png) [@RyanH](https://forum.rclone.org/u/RyanH)\
**Post date:** [October 13, 2020, 6:33pm UTC](https://forum.rclone.org/t/rclone-saturating-network-bandwidth/19772/3 "2020-10-13T18:33:28Z")

</div>

@Animosity022, yes the current bandwidth for rclone is limited to 6MB. However at 10MB for example, it consumes the entire bandwidth even though the upload is 400Mbps.

---

<div class="post-metadata">

**Author:** ![Animosity022](https://forum.rclone.org/user_avatar/forum.rclone.org/animosity022/32/451_2.png) [@Animosity022](https://forum.rclone.org/u/Animosity022)\
**Post date:** [October 13, 2020, 6:34pm UTC](https://forum.rclone.org/t/rclone-saturating-network-bandwidth/19772/4 "2020-10-13T18:34:44Z")

</div>

Can you post a log and share some screenshots so we can look at the issue?

---

<div class="post-metadata">

**Author:** ![RyanH](https://forum.rclone.org/user_avatar/forum.rclone.org/ryanh/32/4577_2.png) [@RyanH](https://forum.rclone.org/u/RyanH)\
**Post date:** [October 13, 2020, 6:37pm UTC](https://forum.rclone.org/t/rclone-saturating-network-bandwidth/19772/5 "2020-10-13T18:37:11Z")

</div>

@Animosity022, sure if you can let me know the screenshots that would be helpful. What log information is needed as it's limited to events such as multipart start, SHA validation, etc?

There is no network related information at this point unless I am missing a command flag.

---

<div class="post-metadata">

**Author:** ![Animosity022](https://forum.rclone.org/user_avatar/forum.rclone.org/animosity022/32/451_2.png) [@Animosity022](https://forum.rclone.org/u/Animosity022)\
**Post date:** [October 13, 2020, 6:38pm UTC](https://forum.rclone.org/t/rclone-saturating-network-bandwidth/19772/6 "2020-10-13T18:38:14Z")

</div>

Run the command with debug log.  
Share whatever tool you are using to measure bandwidth so the log can be correlated to the log.

---

<div class="post-metadata">

**Author:** ![RyanH](https://forum.rclone.org/user_avatar/forum.rclone.org/ryanh/32/4577_2.png) [@RyanH](https://forum.rclone.org/u/RyanH)\
**Post date:** [October 13, 2020, 6:41pm UTC](https://forum.rclone.org/t/rclone-saturating-network-bandwidth/19772/7 "2020-10-13T18:41:44Z")

</div>

@Animosity022, with or without the flag `--bwlimit`? If without, it'll have to be tested late at night as during the daytime, the network is actively used.

---

<div class="post-metadata">

**Author:** ![Animosity022](https://forum.rclone.org/user_avatar/forum.rclone.org/animosity022/32/451_2.png) [@Animosity022](https://forum.rclone.org/u/Animosity022)\
**Post date:** [October 13, 2020, 6:50pm UTC](https://forum.rclone.org/t/rclone-saturating-network-bandwidth/19772/8 "2020-10-13T18:50:41Z")

</div>

Without the bwlimit flag, it'll consume what it can.

For during day, I'd surmise you want to use bwlimit.

bwlimit only works for actual file copy operations and such.

Few issues back to finer grain controls:

> <https://github.com/rclone/rclone/issues/4537>
>
> Yet I'm here again to ask for improve in bandwidth controls, after a 24 hours outage caused by just let rclone...

> <https://github.com/rclone/rclone/issues/2055>
>
> At the moment we have one bandwidth limiter on the output. but we have no bandwidth limiter before the async buffer...

---

<div class="post-metadata">

**Author:** ![RyanH](https://forum.rclone.org/user_avatar/forum.rclone.org/ryanh/32/4577_2.png) [@RyanH](https://forum.rclone.org/u/RyanH)\
**Post date:** [October 13, 2020, 7:06pm UTC](https://forum.rclone.org/t/rclone-saturating-network-bandwidth/19772/9 "2020-10-13T19:06:41Z")

</div>

Thanks @Animosity022. When you say that `bwlimit only works for actual file copy operations and such`, do yo mean copy, sync or move (bidirectional) and not checksum?

---

<div class="post-metadata">

**Author:** ![Animosity022](https://forum.rclone.org/user_avatar/forum.rclone.org/animosity022/32/451_2.png) [@Animosity022](https://forum.rclone.org/u/Animosity022)\
**Post date:** [October 13, 2020, 7:19pm UTC](https://forum.rclone.org/t/rclone-saturating-network-bandwidth/19772/10 "2020-10-13T19:19:05Z")

</div>

I'm assuming we are talking about your command.

The checksum is calculated locally on your file.  
The remote (cloud provider) calculates on the checksum on their side.

There isn't any data transferred to calculate the checksum as the data is transferred when moving/copying the file.

---

<div class="post-metadata">

**Author:** ![RyanH](https://forum.rclone.org/user_avatar/forum.rclone.org/ryanh/32/4577_2.png) [@RyanH](https://forum.rclone.org/u/RyanH)\
**Post date:** [October 13, 2020, 7:30pm UTC](https://forum.rclone.org/t/rclone-saturating-network-bandwidth/19772/11 "2020-10-13T19:30:44Z")

</div>

If the checksums are calculated locally (source and destination), how are these compared? Is there no data transferred e.g. listing all directories and files?

---

<div class="post-metadata">

**Author:** ![Animosity022](https://forum.rclone.org/user_avatar/forum.rclone.org/animosity022/32/451_2.png) [@Animosity022](https://forum.rclone.org/u/Animosity022)\
**Post date:** [October 13, 2020, 7:36pm UTC](https://forum.rclone.org/t/rclone-saturating-network-bandwidth/19772/12 "2020-10-13T19:36:03Z")

</div>

Take one file as an example.

```auto
felix@guardian:~$ rclone copy /etc/hosts GD: -vv --checksum
2020/10/13 15:34:12 DEBUG : rclone: Version "v1.52.3" starting with parameters ["rclone" "copy" "/etc/hosts" "GD:" "-vv" "--checksum"]
2020/10/13 15:34:12 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2020/10/13 15:34:12 DEBUG : fs cache: adding new entry for parent of "/etc/hosts", "/etc"
2020/10/13 15:34:12 DEBUG : hosts: Need to transfer - File not found at Destination
2020/10/13 15:34:13 DEBUG : hosts: MD5 = 3e3007aa5490459a1658d5d31be3a594 OK
2020/10/13 15:34:13 INFO : hosts: Copied (new)
2020/10/13 15:34:13 INFO :
Transferred: 455 / 455 Bytes, 100%, 338 Bytes/s, ETA 0s
Transferred: 1 / 1, 100%
Elapsed time: 1.3s

2020/10/13 15:34:13 DEBUG : 4 go routines active

```

What happens is the file has a checksum calculated locally. The file is copied. The cloud calculates the checksum on the file. Rclone has both values and compares and they are the same. Once the command is done, you'd repeat as checksums are not stored locally so if you checked again, it would calculate locally and get the sum from the provider since it's part of the metadata.

```auto
felix@guardian:~$ rclone copy /etc/hosts GD: -vv --checksum
2020/10/13 15:35:52 DEBUG : rclone: Version "v1.52.3" starting with parameters ["rclone" "copy" "/etc/hosts" "GD:" "-vv" "--checksum"]
2020/10/13 15:35:52 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2020/10/13 15:35:52 DEBUG : fs cache: adding new entry for parent of "/etc/hosts", "/etc"
2020/10/13 15:35:52 DEBUG : hosts: MD5 = 3e3007aa5490459a1658d5d31be3a594 OK
2020/10/13 15:35:52 DEBUG : hosts: Size and MD5 of src and dst objects identical
2020/10/13 15:35:52 DEBUG : hosts: Unchanged skipping
2020/10/13 15:35:52 INFO :
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks: 1 / 1, 100%
Elapsed time: 0.0s

2020/10/13 15:35:52 DEBUG : 4 go routines active

```

---

<div class="post-metadata">

**Author:** ![system](https://forum.rclone.org/uploads/default/original/2X/3/39ad3235f6ffec76ccd0c81616bd55697b4c300d.png) [@system](https://forum.rclone.org/u/system)\
**Post date:** [December 13, 2020, 3:36pm UTC](https://forum.rclone.org/t/rclone-saturating-network-bandwidth/19772/13 "2020-12-13T15:36:10Z")

</div>

Closed on December 13, 2020, 3:36pm UTC

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
