Rclone upload kills my download speed

What is the problem you are having with rclone?

Maybe thats a dumb question, but: When I sync files to my gsuite, rclone should simply upload files, right? But after a couple of minutes running rclone, my download speed drops very hard. I can't even google something (Google will load about a minute). After closing the rclone cmd window, everything works well again.

That's quite annoying because I always have to restart rclone when I notice that my website loading speed is slow.

What is your rclone version (output from rclone version)

rclone v1.50.2
os/arch: windows/amd64
go version: go1.13.4

I tried rclone beta some days ago. Did not solve my problem.

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Win 10, 64 bit

Which cloud storage system are you using? (eg Google Drive)

gsuite

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync U:\ secret: -v -P --retries 1440 --retries-sleep=60s --transfers=10 --exclude *.mrimg

Please note that trying to use a --bwlimit or less --transfers did not solve my problem...

U:\ is a network share. I added some retries because it's possible that the network share is not online at the moment.

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

Many thousands of "Size and modification time the same"

What's your upload and what did you set bwlimit to?

This is because rclone is filling up your upload pipe. This means acknowledgements for downloads slow down and everything slows down.

You can tweak QoS settings in routers to fix this, but you might find the easiest fix is to set a --bwlimit in rclone to slightly less than your upload bandwidth. Find your upload bandwidth by running rclone with the -v or -P flags - it will show upload bandwidth in MByte/s or kByte/s. Use a bit less that this (say 10%) as the parameter to --bwlimit.

Thanks for the answers! But I already tried that. My upload is about 4M. I tried to use --bwlimit 2M, it did not help. I will try it again today.

Rclone uses MByte/s as input (that is why I suggested reading it from rclone's stats) so if your upload is 4 Mbit/s then you want to divide this by 8 so 0.5M or 500k less a bit, say 450k as a parameter to --bwlimit, eg --bwlimit 450k.

Yes I did it with the help of rclones stats. Rclone normally uploads with about 4 MByte/s. With --bwlimit 2M rclone did not exceed 2 MByte/s.

As I said in my last answer I tried it again today. I have experimented with different bandwith limits. But always, after some time, my internet connection gets slowed down drastically.

I fear it's a random windows bug... Very strange behaviour. Or maybe something is wrong with my antivirus. I will check if it works better without it.

Edit: Antivirus doesn't seem to be the problem.

What's your actual upload rated at?

You need to use lower if you are maxing it out.

Rclone normally uploads with about 4 MByte/s.

Without bwlimit rclone uploads with about 4 MByte/s.

What's your internet rated at though? 4MB is ~32Mb/s.

You might have an underperforming router? Where did that come from - the ISP? There have been quite a few reports of rclone overloading ISP provided routers...

You mean what my internet provider offers me? I have a 100k line. That would be a maximum download rate of 100 Mb/s. But it's not a synchronous line, so the upload rate is less. Ookla Speedtest says my actual upload rate is 36 Mb/s. 36 / 8 = 4,5 MB/s. That matches with

Rclone normally uploads with about 4 MByte/s.

It's a self-bought router, not from the ISP.

You'd need to limit your bandwidth to something that makes it work. There's nothing in rclone that is causing the issue as it's going to use what it can consume.

Keep lowering --bwlimit until it works is my advice.

Also you could set some QoS on the router to prioritize small packets - that will help a lot if you can.

I tried many bandwith limits, even ridiculous slow ones, nothing works...

I also tried to write a script which starts and kills rclone in a 10 minute loop, but that's a very bad "solution".

Today I tried to run rclone on a linux (Ubuntu server) VM. The host is my windows system.
It did not help, even though I also tried some --bwlimits again there.

I am out of ideas. The connections dl speed always gets killed, I can't do something else (Which needs internet access) while running rclone sync (Full speed uploads are no problem with other programs, it only happens with rclone).

And unfortunately I can't configure the router. But it should work without configuring routers anyway.

:frowning:

Can you be specific with what you are using and what command you are running and share the debug log "-vv" with the full command?

hello,

  1. your windows computer, wifi or wired? if wifi, 2.4 GHz and 5.0GHz?
  2. have you tried to upload from your local windows computer to cloud, not from network share?
  3. use this program, create the file on local computer and test rclone. http://www.mynikko.com/dummy/
  4. use this program, test the speed to the network share? https://totusoft.com/lanspeed
  5. what is the make and model of the router, are you using latest firmware?
  1. Wifi, 5 GHz
  2. Yes. Same problem.
  3. Thanks
  4. Speed:
    Unbenannt
    I dont't think the network share speed is the problem. I am and was able to transfer terrabytes of files with windows explorer without issues.
  5. It's a FRITZ!Box 7490, newest firmware. But I don't think it's a router problem:

Over the last days I found out that the problem does not always happen. Sometimes I can run rclone all day long (Even without bandwith limit). Very strange.

When you say Windows explorer, I'm going to assume you mean all internal traffic which wouldn't stress your router's outbound connection?

Can you be specific in what you are running and share a debug log with -vv on the command?

I've moved over 110TB via rclone at full gigabit speed but I do traffic shaping and ensure my downloads have upload capacity.

You can either bwlimit rclone or have someone troubleshoot your network as it's a bit beyond the scope of rclone support.

You should check out:

https://en.avm.de/service/fritzbox/fritzbox-7583/knowledge-base/publication/show/228_Prioritizing-internet-access-for-important-network-applications-and-devices/

As it talks about how to setup QoS on your router as that's really going to be what you are looking for.

  1. you mentioned that your purchased the router yourself, so you should be able to configure it.
  2. that router is for dsl connection, correct?
    if so, i have setup a couple of dozen dsl over the years and they are always a problem. basically, trying to pass high speed internet data packets over a telephone copper wires designed for voice. the dsl connections i have setup were for voip phone systems and often requires tweaking the modem and or router. also that router does not seem to have good wifi, perhaps test using a network cable from computer to router?