At first, I used the default Client ID, but then I read that it's better to have your own and I think that I managed to set it up correctly. I'm not 100% sure, because the instructions are a little outdated in the sense that Google renamed/changed things around. However, I set up the client ID etc. and input the data into rclone, and it works, except that it didn't affect the upload speed in the least. I think that if I set it up wrong, then it wouldn't work at all.
[deleted] I have AirVPN, but obviously I turned it off for the test.
100 Mbps is per ISP specification. Currently, depending on speedtest, I get 100-120 Mbps.
Ok, I don't really suspect [redacted] would have issues (but then again, my knowledge of the local infrastructure is also limited). The second test might give me a clue to go on.
How do you upload 5 files? Should I just point rclone to a folder with the 5 files as a source? Should I also add --drive-chunk-size 128M or just --transfers 5?
Your first test was on a single file.
Seeing how it fares with multiple active connections will give me some hints on where the problem may lie (for example on an oversaturated local link it would likely improve the speed a good bit as you'd "steal" a larger share of the total capacity from other connections).
If you have a VPN then it would absolutely be worth trying that too - because if you get drastically different results via VPN then be can basically say for certain that it's a routing issue (in which case we need to log some server IPs and inform your ISP about the issue).
Yep, so it's basically the same as I experienced with FileZilla Pro, Air Explorer and the like some time ago. If we throw several files at rclone, it does utilise full upload speed.
Stop bragging, you! Your upload is really awesome. Unfortunately, where I live they only give you asymmetric fiber. With an upload speed like yours, cloud storage really makes sense. For me, it's still a couple of days to even get one of my 3TB up there...
I obviously can't claim to know based on such limited data - but from experience (including working as a network engineer professionally), I'd say this is likely not a problem in rclone/settings based on what I see here.
My best guess is that this is either a routing or piping issue in the WAN (i'd guess more likely the latter).
In plain english this means a spesific route that is being chosen for the data to go from you - via your ISP - to the server (wherever it is - maybe Germany or Netherlands) may be overcrowded or improperly provisioned.
This could be temporary (due to Corona changes, as we know this is causing issues right now), or it could be permanent. If it temporary you should note varying and shifting speeds, and probably better speeds late at night. If it is permanent you should contact the ISP - preferably providing a tracert to the affected server so they can verify the problem and fix it by routing traffic differently or provisioning more resources to that link.
@ncw Can you suggest what the easiest way to get hold of the server-ip for a gdrive in use is? Is this picked up in any of he logs?
At that speed you are going to start to run into some Google infrastructure limitations pr. connection.
You should be able to easily double that or more if you copy 3-4 files at once.
(entirely unrelated to the issue the OP has...)
It's interesting, because I get the same bottleneck with FileZilla Pro and Mountain Duck (i.e. around 2.3MB/s), but Air Explorer can get 4.7MB/s per file.
Is there any way to bypass this at all? Perhaps break up the file and upload it in parallel (just using rclone)? Honestly, I just signed a contract with my ISP for 2 years, so I don't expect them to be very accommodating at this time...
Oh absolutely - you can zip or rar files into multiple archive files (which still open as a single unit).
In winrar for example you simply select that in the options, and can set it as default too.
Or if you prefer an automatic solution - rclone has a "chunker" backend that can split files for you as you desire - an a completely transparent way (so they appear as single files in the remote and function normally).
But I would not discount the ISP doing their job out of hand (assuming that it's not just a temporary thing from Corona). ISP admins are humans too - and they miss things, or automatic systems fail to detect problems. You'd be surprised at how many problems rely on user-reporting to be fixed quickly.
This is often a problem that a higher-level admin can fix remotely, so don't just assume the problem is "too big" for them to consider helping you. The worst that can happen is that nothing happens and you wasted a few minutes writing them the email
I wouldn't bother contacting them without concrete data though. For that we need a tracert and maybe some simple pinglogs (I can help you gather that and also analyze the results) - but I'd need to know from NCW how to grab the relevant IP first, as that's not something I've had a need to do yet.
It might be using multipart upload. It's not really possible for me to comment when I don't know the internal mechanisms of the software. Most likely it is due to it leveraging more than one concurrent connection compared to the others.
Your upload is really awesome. Unfortunately, where I live they only give you asymmetric fiber. With an upload speed like yours, cloud storage really makes sense. For me, it's still a couple of days to even get one of my 3TB up there...