Rclone copy Issue between Onedrive to Dropbox

What is the problem you are having with rclone?

I need to copy almost 1TB from Onedrive to Dropbox but the transfer fails and I received the following error:

Too many requests. Trying again in 300 seconds.
2022/06/21 12:29:29 DEBUG : pacer: low level retry 1/10 (error 429 Too Many Requests: )
2022/06/21 12:29:29 DEBUG : pacer: Rate limited, increasing sleep to 5m0s
2022/06/21 12:29:29 DEBUG : Couldn't decode error response: EOF
2022/06/21 12:29:29 DEBUG : Too many requests. Trying again in 1226 seconds.
2022/06/21 12:29:29 DEBUG : pacer: low level retry 2/10 (error 429 Too Many Requests: )
2022/06/21 12:29:29 DEBUG : pacer: Rate limited, increasing sleep to 20m26s

Run the command 'rclone version' and share the full output of the command.

rclone v1.58.1

  • os/version: debian 11.3 (64 bit)
  • os/kernel: 5.10.0-8-amd64 (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.17.9
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

One Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync ode: dbe: --tpslimit=5 --checkers=2 --progress --vv

The rclone config contents with secrets removed.

Paste config here

A log from the command with the -vv flag

022-06-21 12:43:14 DEBUG : pacer: Rate limited, increasing sleep to 1h16m6s
2022-06-21 12:43:14 DEBUG : Couldn't decode error response: EOF
2022-06-21 12:43:14 DEBUG : Too many requests. Trying again in 4566 seconds.
2022-06-21 12:43:14 DEBUG : pacer: low level retry 1/10 (error 429 Too Many Requests: )
2022-06-21 12:43:14 DEBUG : Couldn't decode error response: EOF
2022-06-21 12:43:14 DEBUG : Too many requests. Trying again in 4565 seconds.
2022-06-21 12:43:14 DEBUG : pacer: low level retry 1/10 (error 429 Too Many Requests: )
2022-06-21 12:43:14 DEBUG : pacer: Rate limited, increasing sleep to 1h16m5s
2022-06-21 12:43:15 DEBUG : Couldn't decode error response: EOF
2022-06-21 12:43:15 DEBUG : Too many requests. Trying again in 4565 seconds.
2022-06-21 12:43:15 DEBUG : pacer: low level retry 1/10 (error 429 Too Many

hi,
the following info is missing, please post:
--- the redacted config file.
--- top 30 lines of a debug log.

Hi Jean Paul,

You are most likely being throttled by OneDrive.

I suggest you read these posts:

good chance the OP never created client id/secret for the remotes.

What it means OP?

that is you, my fellow rcloner, the Original Poster.

Oh =) I have crypt mounts. The client ID is for the main remote or for the crypt one? Or both?

the main remotes.

and the Client ID has expiration date?

it uses a token that expires, usually in one hour.
rclone will refresh the token as needed.

I don't see a positive effect of a personal clientID/secret for OneDrive and don't use it anymore.

I don't know about Dropbox, but don't think it will make any difference since OneDrive seems to be the bottleneck.

My simple and easy recommendation for OneDrive is to use default tuning parameters and have a lot of patience with large transfers; and then restart the command if no progress at all in 24 hours.

So I would remove --tpslimit and --checkers (unless needed by Dropbox) and then be prepared for a sync that probably takes several days and sometimes pauses in long periods (minutes/hours).

Finally, I suggest you try to confirm that the throttling errors are indeed from OneDrive and not from Dropbox. You can do this by syncing a few folders from OneDrive to your local disk after the above throttling has started. That should result in the same behavior/errors - if not then try syncing from local disk to DropBox.