I know it's not rclone's fault but my broadband's router which is a trashy mitrastar my ISP installed.
When I'm transferring stuff to my teams drive account, there's a chance my router will end up losing internet connection, having to reboot it for it to regain uplink.
Now, I'm trying to sync two google drive repos with "rclone sync gdrive1 gdrive2" and, after an hour or two, my router loses that network interface.
I'm using a raspberry pi 2 with arch linux to do all the syncing.
It's weird, it's not rclone's fault - I just wanted to ask if there could be a logical explaination for this and if there's any way to avoid this issue from rclone's side. I don't know if it's transfer rate, packet size or what could be the issue here but it's annoying, and my ISP is not going to help at all...
Most common causes of routers crapping out:
- Large amount of concurrent connections (but rclone does not use many, unlike for example a torrent client)
- Overheating from crappy build-quality, bought on by high throughput (which rclone definitely does when actively transferring)
- Bad firmware (bugs). If so then I'd expect it to be more random, but of course it's not that surprising if a bug has a higher change of happening under heavy load than otherwise... Upgrading your firmware may help - assuming there is any newer firmware available that is.
This may not be "fixable", it sounds like a bad router to me (I used to work at an ISP as a tech so I've seen my fair share of ho-hum routers).
I'd just ask your broadband provider to give you an exchange (and maybe a different model if they can offer one). Say that it craps out on you when you use it heavily for a while and that this is not acceptable for the service you are paying for.
You are not the first person to report rclone crashing their router!
Rclone works the internet connection hard with lots of concurrent connections.
You might find you can work around it by setting a
--bwlimit to slightly less than your maximum upload speed. So if you upload at 10 MB/s normally (as measured with rclone) then try
I normally recommend trying to upgrade the firmware of your router at this point.
Thanks for the replies. Got news:
The router is on its latest version.
I kind of mitigated the issue by a combination of factors:
- --bwlimit 10M
- --drive-chunk-size 64M
- --transfers 2
It's been running for around 8 hours and still didn't crash.
Will report if it holds further, since it's got stuff for upload for almost a week
BTW I managed to get my ISP router to run a lot more reliably by taking the case off which stopped it overheating! (I use a linux box as a router now-a-days though).
Lots of poorly constructed routers can be stabilized with just a little cooling yes. A lot of the cheaper stuff is horribly constructed and the chips can run at 80-90C under load. Some of worst ones don't even have basic ventilation holes anymore... so of course they can't take a sustained load...
I'm not talking about strapping a fan to it - but just something as simple as removing unnecessary plastic covers and putting it in a place that has some air-movement helps. If you want to get a little more fancy then it's not hard to open them up and thermal-glue on a better heat-sink on the CPU. I might be a bit weird, but I often end up doing this on most my routers - but then I hack cheaper routers with better firmware and then stabilize the hardware cause im cheap
If it's a router that you don't mind making modifications to - a really effective and easy mod is just to add some more air-holes to it near the CPU location. Probably not a great idea to put holes in your ISPs router if they still technically own it though
Yea, running it at slightly less than max capacity will significantly reduce the load - so that is definitely recommended. If it tries to run faster than the connection can really handle then it has to do a significant amount of wasted work re-sending packets that got dropped.
So to illustrate, dropping your max-bandwidth down by 5% below actual max might result in 15'ish % less work the router has to work though, and that could be the difference between stability or crash over time
Just be aware of TCP overhead when you set a limit.
Your connection has a gross speed, which is the technical speed.
But then TCP also needs some control data, OS your practical net speed will be less.
That overhead is typically 10-15%
Your ISP may tell you the connection speed in either gross or net numbers - depending on the consumer protection laws in your country.
In short: do a practical speed test (speedtest.net) and use that as a guidepost for what your max is.
I'll try to open it, but I'm afraid I will void the warranty.
Now it seems more stable.
It did get locked some hours ago, but I didn't reboot it before the modifications, so I'm giving it another chance.
Thanks for the suggestions
Yeah, that didn't go well: My router crashed again.
Fun thing is: It's the network port it's connected to, not the whole router: So, If I unplug the LAN cable and plug it to another port, it does get connection. That first port remains unusable until the router restarts.
I have 600 Symmetric Mbps: https://www.speedtest.net/result/8727513690 - I'm uploading @10 Mbps...
I'll start requesting a firmware upgrade and reporting the issue to my ISP (although I already have a firmware from june).
In the meantime, is there anything - software-wise that I could tweak even further?
you should be to use a router of your own choice.
i have done with 50+ times over the years for my clients.
often i will install a voip phone system and the cheap isp router cannot handle the udp sip packets.
verizon fios quantum router is a good example.
you can ask the isp to turn the router into straight-thru bridge mode.
basically use the isp device as a modem and plug a good router into that 'modem'
it takes about one minute to do via the web interface of the isp device.
Ok, here's my question.
If the ISP router is having a hard time dealing witth those many connections, wouldn't it have the same issues when acting as passthrough?
At the end of the day, the number of connections is still the same.
that is a good question but my answer would be 'no, it is not the same.
as you mentioned, your isp router is a cheap one.
a router has to route packets, routing takes processing power to make lots of decisions.
if the cpu in the router is overloaded and/or starts to get hot, the cpu processing speed is throttled down, reducing cpu speed, slower performance and can and will drop packets.
for each packet,
- decide what to do based on the the routing table.
- decide to allow/block/drop the packet based on the firewall rules.
- the packets flowing back and forth are tcp packets, not lightweight udp, so the router has to examine each and every packet, look at the header, compute checksums, tcp packets might arrive out of order and the router has to track each packet, if a packet is missing, the router has to ask the source to resend the packet again.
- then there is overhead with the https protocol, its own checksums.
- packets that are dropped because the router is overloaded have to be sent again, creates a nasty loop.
- and so on...
when the router is dumbed down into bridge mode, like any modem, the packets just flow back and forth like a water pipe.
Is this relevant? Could this mean there's network congestion before the crash?
your router could have buggy firmware, could be damaged, any number of problems.
network congestion should not cause a router to hard crash, as that would mean less packets are flowing back and forth.
i suggest that you ask your isp, there could be something wrong with the basic connection, not the router.
speed tests are not at all useful to diagnose problems.
i have seen this many times.
ask them to run an extended test, for a couple of days.
next time the router crashes, call the isp and ask them to look at the connection
call the isp, ask for a new router
Yea, that is probably the simplest first-step to do. I MAY not solve the issue, but assuming your ISP can give you a free replacement for the one you have then there's decent chance that just fixes it.
It's literally the first thing any network tech will do if a customer complains about spontaneous reboots or crashes on the router.
And if not, then as asdffdsa says, if the router can be easily bridged then you are free to use a router of your own choosing. A bridged router will do almost nothing in terms of logic and load. It will just de-shell the protocol and pass it on. A trivial task...
The problem is that many default ISP routers are NOT trivial to bridge heh.. (but some are), and they usually won't help you do it either. In those cases a simple fix turns into a major headache.
I'll try that.
In my experience, the ISP will try to blame it somewhere else and avoid replacing it (it's this crappy).
From here on, I will see if it's better choice to either change ISP or look for a decent router.
Thanks for your help!
In some cases it's not even required to bridge the ISP router.
If the protocol used is a fairly common one (like let's say PPPoE) and your ISP is willing to tell you your login details (basically the same info that comes auto-configured in the ISP router), then you can actually plug your own rotuer directly into the connection - configure the WAN interface and rely only on that, and shelve the ISP's router.
But this is a little bit more involved than just bridging, because you will need to understand a little bit of network basics to not be confused by the tech details + be capable of checking that the equipment you use is in fact compatible with the connection you are trying to use it with. Just randomly pulling some router off a store shelf is unlikely to end well... It's not really that complicated all in all, but everything is hard and confusing if you have never done it before
I just figured I would at least mention this.
My ISP is giving me sh*t tring to blame it on the software.
I'm thinking about troubleshooting it myself. I already have netdata pulling a lot of statistics on the rpi but I can't seem to see anything weird.
Other way to go might be trying to dump packets and investigate that way and see if the router is getting a boo-boo from too many connections or anything alike.
I can save you some time and say that won't find anything wrong with the traffic - simply because it is irrelevant.
A router should not crash no matter what the traffic is - otherwise it will be a fundamentally flawed system from the onset. It would be somewhat understandable if there were an insane amount of connections because that's realistically going to crap out anything system if they are enough of them - but this is not what rclone does. For rclone it's not even that - it's just basic high throughput (a fair bit less demanding and mostly a congestion and heat issue for a router - excepting any major firmware bugs).
The router apparently can't handle you using your full capacity for an extended period. Unfortunately that is not that uncommon on cheap routers Those are very often just built (in terms of build-quality) to be capable of the small burst-loads of a typical user, not a sustained max load, and some of them will crap out under loads that should technically be well within their specs. Even among the non-crap ones, there's going to be a some percentage of defective/flawed ones that have sub-par stability. These are after all produced en-masse and not stress-tested properly, so you inevitably get a few "monday products" in the bunch. This is exactly why swapping the rotuer is usually the first thing you try (easy potential fix for an engineer if you know the rotuer model line is not inherently crap and ought to be working).
You unfortunately made a strategic blunder in mentioning rclone here. You should have just played dumb and said the router was randomly crapping out rather than elaborate too much. Probably should have warned you about that heh... but decent chance you can just call again and talk to a different guy and try again. Hope they didn't log the conversation in too much detail
A lot of ISPs really suck at this kind of service aspect towards their customers. It really is unfortunate. My own workplace started out being focused on solving problems and having happy customers - and then turned into a corporate middlemanagement hell over time where the only concern was to get "number of calls" on the board regardless of if you actually solved the problem (as that was the only thing the bosses at the top referenced as goal statistics). Real shame
The simplest fix is the one mentioned earlier where you cap rclone to use 90-95% of max throughput. If you have not already done so, do that. This will eliminate congestion and make the routers job much less intense (much more work saved than the 5-10% lost in speed). If the problem is just poor build quality there's a decent chance this will significantly increase stability - and perhaps even remove the problem entirely.