Copying Help + Read/Write Speeds

Help and Support
1

Please show the effort you've put in to solving the problem and please be specific -- people are volunteering their time to help you! Low effort posts are not likely to get good answers! DO NOT REDACT any information except passwords/keys/personal info. You should use 3 backticks to begin and end your paste to make it readable. Or use a service such as https://pastebin.com or https://gist.github.com/ -->

What is the problem you are having with rclone?

Hi, I'm a total newbie to rclone and am a video person not an IT person. I am using rclone to copy a ton of footage (~80TBs) off of my team's Dropbox and to an on-premises DAM. I'm using the copy command to go in large chunks, but am getting inconsistent read/write speeds. They go from 100mb/s all the way down to 1mb/s or lower. In the command, the Q: is our on-prem storage destination. Being new, I'm sure there's a better way to get write this command. Any help or guidance is greatly appreciated, thanks!

Run the command 'rclone version' and share the full output of the command.

rclone v1.65.1
  • os/version: Microsoft Windows 10 Pro 22H2 (64 bit)
  • os/kernel: 10.0.19045.3803 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.21.5
  • go/linking: static
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Dropbox

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Paste command here

rclone copy Dropbox:/"FOOTAGE\21-22 FOOTAGE ARCHIVE\CLEM" Q:"FOOTAGE HOLD\FOOTAGE\21-22 FOOTAGE ARCHIVE\CLEM" -P

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[Dropbox]
type = dropbox
token = XXX
### Double check the config for sensitive info before posting publicly

A log from the command that you were trying to run with the -vv flag

I can't paste the log because it's currently in the middle of the copy command.
2

welcome to the forum,

  • need to look at a debug log by adding -vv to the command.
    not the terminal output from --P

  • create your own client id
    https://rclone.org/dropbox/#get-your-own-dropbox-app-id

  • use --tpslimit=12 --tpslimit-burst=0

  • fwiw, should use full paths for local/smb, not relative
    "Q:\FOOTAGE HOLD\FOOTAGE\21-22 FOOTAGE ARCHIVE\CLEM"

1 Like
3

Thanks! I created my own client id and added the ==tpslimit=12 --tpslimit-burst=0 flags.

I'm getting more consistent r/w speeds. They're hovering around 35-40 mb/s. I guess that's better than bouncing all the way down to 2.

4

Good, there are a few other tweaks that can done.
Would be helpful, if you post the results of a internet speed test.

that is megabits per second which would be a very slow connection.
I think you meant MB/s - MegaBytes per second?

5

Make sure to run (to bind it to your token):

rclone config reconnect Dropbox:

after inserting client_id/secret to your config file.

Or recreate your remote from scratch.

1 Like
6

It's actually displaying MiB/s so a mebibit? Not exactly sure about the conversion. I've seen various conversions online to MB/s.

My speed test returned:

844.18Mbsp down
627.65Mbsp up

7

rclone use bytes, not bits.

MiB, which, for our purposes, is equivalent to MB.
so when posting, try to use MiB or MB.

1 Like
8

Ah gotcha, makes sense. Then yes, I am getting roughly 35-40MiB/s. I did not reconnect the config after establishing my clientid so I will do that next.

9

The key was what @asdffdsa suggested - to add tpslimit flags.

Making sure that your own client_id is really used will help even further by isolating your limits from other rclone users. It wont necessarily make anything faster:) but definitely more stable for long runs.

1 Like
10

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.