Can you upload to an rclone mount without caching?

At first, I used the default Client ID, but then I read that it's better to have your own and I think that I managed to set it up correctly. I'm not 100% sure, because the instructions are a little outdated in the sense that Google renamed/changed things around. However, I set up the client ID etc. and input the data into rclone, and it works, except that it didn't affect the upload speed in the least. I think that if I set it up wrong, then it wouldn't work at all.

[deleted] I have AirVPN, but obviously I turned it off for the test.

100 Mbps is per ISP specification. Currently, depending on speedtest, I get 100-120 Mbps.


jojo dumb, ma smart.

Edit advanced config? (y/n)

Ok, I don't really suspect [redacted] would have issues (but then again, my knowledge of the local infrastructure is also limited). The second test might give me a clue to go on.

select n
(unless you have a really good reason to do otherwise)

jojo still dumb...

Configure this as a team drive?

That depends - are you trying to set up a teamdrive? :stuck_out_tongue:
Teamdrives are now known as "shared drives" as of recently.

If you are testing on a free private account that comes with your gmail - the answer is "no" (not a teamdrive).

If you should need a larger drive to run tests on compared to the standard 15GB free, then let me know. I'll set you up with something temporary.

jojo thanks ma...

How do you upload 5 files? Should I just point rclone to a folder with the 5 files as a source? Should I also add --drive-chunk-size 128M or just --transfers 5?


preferably use both.

Your first test was on a single file.
Seeing how it fares with multiple active connections will give me some hints on where the problem may lie (for example on an oversaturated local link it would likely improve the speed a good bit as you'd "steal" a larger share of the total capacity from other connections).

If you have a VPN then it would absolutely be worth trying that too - because if you get drastically different results via VPN then be can basically say for certain that it's a routing issue (in which case we need to log some server IPs and inform your ISP about the issue).

Yep, so it's basically the same as I experienced with FileZilla Pro, Air Explorer and the like some time ago. If we throw several files at rclone, it does utilise full upload speed.

C:\Users\an531\Desktop\Rclone\rclone-v1.51.0-windows-amd64>rclone copy "C:\Users\an531\Desktop\test3" gdrive:\ -P --drive-chunk-size 128M --transfers 5
Transferred:      176.199M / 396.643 MBytes, 44%, 12.022 MBytes/s, ETA 18s
Transferred:            0 / 5, 0%
Elapsed time:        14.6s
 *                                     test1.rar: 42% /79.329M, 2.323M/s, 19s
 *                                     test2.rar: 45% /79.329M, 2.462M/s, 17s
 *                                     test3.rar: 45% /79.329M, 2.455M/s, 17s
 *                                     test4.rar: 44% /79.329M, 2.386M/s, 18s
 *                                     test5.rar: 44% /79.329M, 2.453M/s, 18s

Do you know why this is happening and if there is any way to fix this for large individual files?

2020/03/24 18:23:02 INFO :
Transferred: 2G / 2 GBytes, 100%, 53.929 MBytes/s, ETA 0s
Transferred: 1 / 1, 100%
Elapsed time: 37.9s

2020/03/24 18:23:02 DEBUG : 5 go routines active
2020/03/24 18:23:02 DEBUG : rclone: Version "v1.51.0" finishing with parameters ["c:\data\rclone\scripts\rclone.exe" "sync" "C:\data\rclone\scripts\rr\other\2GB" "gdrive:testtest\" "--drive-chunk-size=256M" "--transfers=10" "--log-file=C:\data\rclone\scripts\rr\other\test\log2GB.txt" "--log-level=DEBUG" "--progress"]

Stop bragging, you! :slight_smile: Your upload is really awesome. Unfortunately, where I live they only give you asymmetric fiber. With an upload speed like yours, cloud storage really makes sense. For me, it's still a couple of days to even get one of my 3TB up there...

I obviously can't claim to know based on such limited data - but from experience (including working as a network engineer professionally), I'd say this is likely not a problem in rclone/settings based on what I see here.

My best guess is that this is either a routing or piping issue in the WAN (i'd guess more likely the latter).
In plain english this means a spesific route that is being chosen for the data to go from you - via your ISP - to the server (wherever it is - maybe Germany or Netherlands) may be overcrowded or improperly provisioned.

This could be temporary (due to Corona changes, as we know this is causing issues right now), or it could be permanent. If it temporary you should note varying and shifting speeds, and probably better speeds late at night. If it is permanent you should contact the ISP - preferably providing a tracert to the affected server so they can verify the problem and fix it by routing traffic differently or provisioning more resources to that link.

@ncw Can you suggest what the easiest way to get hold of the server-ip for a gdrive in use is? Is this picked up in any of he logs?

1 Like

At that speed you are going to start to run into some Google infrastructure limitations pr. connection.
You should be able to easily double that or more if you copy 3-4 files at once.
(entirely unrelated to the issue the OP has...)

that could be the problem,

i drink Guinness Extra Stout

It's not a great time for that brand-name I think lol...


1 Like

It's interesting, because I get the same bottleneck with FileZilla Pro and Mountain Duck (i.e. around 2.3MB/s), but Air Explorer can get 4.7MB/s per file.

Is there any way to bypass this at all? Perhaps break up the file and upload it in parallel (just using rclone)? Honestly, I just signed a contract with my ISP for 2 years, so I don't expect them to be very accommodating at this time...

Done. let me know if I missed anything.

Oh absolutely - you can zip or rar files into multiple archive files (which still open as a single unit).
In winrar for example you simply select that in the options, and can set it as default too.

Or if you prefer an automatic solution - rclone has a "chunker" backend that can split files for you as you desire - an a completely transparent way (so they appear as single files in the remote and function normally).

But I would not discount the ISP doing their job out of hand (assuming that it's not just a temporary thing from Corona). ISP admins are humans too - and they miss things, or automatic systems fail to detect problems. You'd be surprised at how many problems rely on user-reporting to be fixed quickly.
This is often a problem that a higher-level admin can fix remotely, so don't just assume the problem is "too big" for them to consider helping you. The worst that can happen is that nothing happens and you wasted a few minutes writing them the email :slight_smile:

I wouldn't bother contacting them without concrete data though. For that we need a tracert and maybe some simple pinglogs (I can help you gather that and also analyze the results) - but I'd need to know from NCW how to grab the relevant IP first, as that's not something I've had a need to do yet.

1 Like

once again, you are correct

C:\data\rclone\scripts\rr\other\test>c:\data\rclone\scripts\rclone.exe sync C:\data\rclone\scripts\rr\other\2GB gdrive:testtest\ --drive-chunk-size=256M --transfers=10 --log-file=C:\data\rclone\scripts\rr\other\test\log2GB.txt --log-level=DEBUG --progress
Transferred:            6G / 6 GBytes, 100%, 97.852 MBytes/s, ETA 0s
Transferred:            3 / 3, 100%
Elapsed time:       1m2.7s

It might be using multipart upload. It's not really possible for me to comment when I don't know the internal mechanisms of the software. Most likely it is due to it leveraging more than one concurrent connection compared to the others.