Very Slow Upload to Dropbox Remote

Help and Support
1

What is the problem you are having with rclone?

Been using Google Drive for years (with Animosity022 settings) and it has been perfect. Just been sent the email about my Gdrive account exceeded and thought that I would start on Dropbox. I am getting very slow upload speeds (approx 150-250 kBytes/s). I have a hetzner server with gigabit down/up. Google gets excellent speeds.

My mount parameters are like this:-
rclone mount --alllow-other --dir-cache-time 9999h --log-level INFO --log-file /opt/rclone/logs/rclonecrypt_tv.log --cache-dir=/cache/TV --vfs-cache-mode full --vfs-cache-max-size 1000G --vfs-cache-max-age 9999h --tpslimit 12 --tpslimit-burst 0 crypt_tv: /home/XXX/crypt_tv

I hope someone can help. Would like to keep using an 'unlimited' cloud storage if I can without having to have to resort to self-hosting.
Also, my download speeds are around 17-20 MB/sec.

Run the command 'rclone version' and share the full output of the command.

rclone v1.55.1

  • os/type: linux
  • os/arch: amd64
  • go/version: go1.16.3
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Dropbox Business

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy file.mkv  crypt_tv: -P

The rclone config contents with secrets removed.

[cloud]
type = drive
client_id = XXXX
client_secret = XXXX
scope = drive
token = XXXX

[crypt_tv]
type = crypt
remote = dropbox:/TV/
filename_encryption = standard
directory_name_encryption = true
password = XXXX
password2 = XXXX

[crypt_movies]
type = crypt
remote = dropbox:/Movies
filename_encryption = standard
directory_name_encryption = true
password = XXXX
password2 = XXXX

[dropbox]
type = dropbox
client_id = XXXX
client_secret = XXXX
token = XXXX

A log from the command with the -vv flag

rclone copy XXX.mkv  crypt_tv: -vv -P
2023/06/03 01:46:49 DEBUG : Using config file from "/home/XXX/.config/rclone/rclone.conf"
2023/06/03 01:46:49 DEBUG : rclone: Version "v1.55.1" starting with parameters ["rclone" "copy" "XXXX" "crypt_tv:" "-vv" "-P"]
2023/06/03 01:46:49 DEBUG : Creating backend with remote "XXXX"
2023/06/03 01:46:49 DEBUG : fs cache: adding new entry for parent of "XXXX", "/home/XXX/Downloads/XXXX/XXXX"
2023/06/03 01:46:49 DEBUG : Creating backend with remote "crypt_tv:"
2023/06/03 01:46:49 DEBUG : Creating backend with remote "dropbox:/TV/3aihp6p10fpfo95osu11aj9i48/3e3mj5tf9rqlqsbc7gdrl9tr8g"
2023/06/03 01:46:49 DEBUG : Dropbox root '': Using root namespace "4207327649"
2023/06/03 01:46:50 DEBUG : fs cache: renaming cache item "dropbox:/TV/3aihp6p10fpfo95osu11aj9i48/3e3mj5tf9rqlqsbc7gdrl9tr8g" to be canonical "dropbox:TV/3aihp6p10fpfo95osu11aj9i48/3e3mj5tf9rqlqsbc7gdrl9tr8g"
2023/06/03 01:46:50 DEBUG : fs cache: switching user supplied name "dropbox:/TV/3aihp6p10fpfo95osu11aj9i48/3e3mj5tf9rqlqsbc7gdrl9tr8g" for canonical name "dropbox:TV/3aihp6p10fpfo95osu11aj9i48/3e3mj5tf9rqlqsbc7gdrl9tr8g"
2023-06-03 01:46:50 DEBUG : XXX.mkv: Need to transfer - File not found at Destination
2023-06-03 01:46:50 DEBUG : crcdrkt91ue5vugurtnvm5cdc5brihg2ro5c7bc419cn7a5ed9ouab3ln0fbs94ofl16sqdkc94qbcr1hjuo8c538odud047857bj3o: Uploading chunk 1/46
Transferred:        2.344M / 2.124 GBytes, 0%, 121.071 kBytes/s, ETA 5h6m14s
Transferred:            0 / 1, 0%
Elapsed time:        20.7s
Transferring:
 * XXXX.mkv:  0% /2.124G, 119.332k/s, 5h10m42s^C
2

I have just updated (rclone selfupdate) and my rclone is now on version:-
rclone v1.62.2

  • os/version: ubuntu 20.04 (64 bit)
  • os/kernel: 5.4.0-150-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.20.2
  • go/linking: static
  • go/tags: none

still getting slow speeds:-

Transferred:        2.354 MiB / 2.124 GBytes, 0%, 176.034 kBytes/s, ETA 3h30m48s
Transferred:            0 / 1, 0%
Elapsed time:        21.2s
Transferring:
 * XXXX.mkv:  0% /2.124G, 176.034k/s, 3h30m48s
3

Did you create your own app ID?

4

Hi, yep I have created my own app ID and have added the client_id (app key) and client_secret (app secret) to my rclone config of dropbox remote.

5

Try changing the chunk size: --dropbox-chunk-size 150M or a different value depending on your connection speed

6

Ok, if I change (or in my case, add) --dropbox-chunk-size 150M it still gives same upload speed with rclone copy or move command.
However, if I just drag from my Ubuntu file explorer to the crypt_tv mount....I get speeds of 45MB/sec (and the file upload completed all ok)

7

you can ignore it. This is only cache at work. File has been copied to the local cache and upload is happening in the background.

Your:

rclone copy file.mkv  crypt_tv: -P

is the right way to test bare bone speed.

8

Thank you. I can see the files in the cache. If i don't add --vfs-write-back 1h (or any other time), what would the default write back time be?

9
10

Brilliant...Thank You!

11

Hi Dean I'm having the exact same issue. I was migrating fine for the past week, then since yesterday speeds have gone to a crawl, barely 200KiB/s. I raised a ticket with Hetzner asking if they have been throttling Dropbox traffic explicitly and they claimed not. Find it hard to believe, as same setup and Dropbox app IDs etc work fine from my local machine. Been driving me nuts!

Did using a mount fix the issue for you?

Edit: just realised you already were using the mount sorry! Have you found any way to increase the speeds?

12

Hi, what i found is that I updated to latest release of rclone, and then applied Animosity022's settings and can now drag and drop on Hetzner (ubuntu 20.04) and got speeds up to 100MB/s!!! If i move things through Sonarr or Radarr i get high speeds too.
But just using rclone CLI, i get slow speeds (as previous message this was for the cache and then cache uploads all ok.
Before I updated rclone and applied correct settings (with fuse3 too), my web uploads in dropbox were same slow speed. After....much better with web upload in dropbox too.

13

Oooffff what a pain in the bum. I'm running Debian right now so might have to get the rest of my stuff off it and try Ubuntu instead if that really works! Hetzner must be throttling the traffic. I don't believe otherwise is possible :confused: as my Dropbox app details and same rclone.conf work fine on multiple other machines :joy:

14

it seems that few people these days are pumping 100s of TB towards Dropbox. And a lot of the use Hetzner VPS. Hetzner might do not care - they give you shared 1GB Cx - but Dropbox might.

1 Like
15

Hmm I did wonder if it's possible Dropbox are limiting Hetzner IPs like Google have been doing recently :confused: didn't see any history of them doing that before but it is as you say very high traffic now.

1 Like
16

It is very difficult to asses - as things might be changing all the time.

17

Single file upload it slow, I can't get more than 32MB/s (from a server in the Netherlands), but for multiple file upload using something like --dropbox-chunk-size 150M --transfers 20 I can get up to 300MB/s

18

Thanks! Unfortunately I'm already trying with doing 12 transfers at once, which was saturating the connection earlier in the week. Now it's 200kb/sec up to maybe 1mb if very lucky (!) for the lot.

This worked really well before yesterday:

rclone sync "gdrive:Media/Movies" "dropboxcrypt:/Movies" --checkers 12 --tpslimit 12 --transfers 12 --fast-list --dropbox-batch-size 1000 --dropbox-batch-timeout 10s --dropbox-batch-mode sync --bwlimit 70M:70M --progress

Tried with the chunk size 150m also but no difference.

My server is in FSN. Perhaps I'm with all the other busy guys.

19

yeah you have not changed anything so something changes in environment you are running it

20

Yeah, since Friday I've only been getting 200KB/sec. There must be throttling from either Hetzner or Dropbox.