Dropbox rate limiting for upload

Help and Support
21

This is the list of commands I tried:
rclone copy bluray_backup dropbox:blueray_backup --progress --tpslimit=10 --dropbox-chunk-size=140M
rclone copy bluray_backup dropbox:blueray_backup --progress --tpslimit=10 --dropbox-batch-mode=off
rclone copy bluray_backup dropbox:blueray_backup --progress --tpslimit=10 --dropbox-batch-mode=sync
rclone copy bluray_backup dropbox:blueray_backup --progress --tpslimit=10 --dropbox-batch-mode=async
rclone copy bluray_backup dropbox:blueray_backup --progress --tpslimit=10 --transfers=15
rclone copy bluray_backup dropbox:blueray_backup --progress --tpslimit=10 --transfers=30
rclone copy bluray_backup dropbox:blueray_backup --progress --tpslimit=10 --transfers=50

All of them have the same behaviour -- the upload goes to certain level like 60-70% and then files upload resets to 0. I tried to keep the upload as long as practical. Tried keeping the upload for a two day -- as a result no files upload with success.

22

the crypted file names makes it hard to see the problem
pick one file and documented the problem.

23

so upload just 1 file? --transfers=1?

24

you need to document that, pick a single file where that happens.

25

I just have posted a log file when 1 file from 4 parallel uploads just got reset to 0% and started the upload of the file again.

26

what is the exact file name?

27

can't say that anymore unfortunately. But the log file is complete.

28

from the log, it's logical to assume it is this one: ve1a6s8i85nkupnho67par21tdmhpgkupdb2ekoo44914tbaincg

The chuck has been almost uploaded:
2022-03-18 16:00:54 DEBUG : ve1a6s8i85nkupnho67par21tdmhpgkupdb2ekoo44914tbaincg: Uploading chunk 57/57

but then it says start again:
2022-03-18 16:01:24 DEBUG : Dropbox root 'chunk/bluray_backup': Committed sync batch length 1 starting with: chunk/bluray_backup/ve1a6s8i85nkupnho67par21tdmhpgkupdb2ekoo44914tbaincg

Does this ring any bell?

29

looks ok to me, where do you see rclone re-uploading twice?

there might be a confusion about dropbox batch mode.
so as a test, copy a single file from local to dropbox_src: and take a look at the debug log.

30

Thanks will try that

31

But generally speaking does this behaviour look like a Dropbox's rate limiting or some other potential issue?

32

so we agree that rclone did not attempt to re-upload ve1a6s8i85nkupnho67par21tdmhpgkupdb2ekoo44914tbaincg, correct?

33

Hold on. Are you challenging my observations? What's the point of this support forum?

34

well, you stated this topic by writing

i asked for an example, and you replied

and based on your debug log, i do not see rclone trying to re-upload that file.

i guess i am not helpful to you, i will drop off now, good luck!

35

I posted the command, config, and the log file (what else is missing for rclone-experienced folks to opine on the issue?). So you "didn't see" the file being re-uploaded and you straight concluded I am making this up?

36

i am just a volunteer here in the forum.
based on that debug log, i do not see rclone re-uploading the file you claimed.
perhaps i am wrong.

so again, i will drop off now, no need to reply, good luck!
no worries, i am sure another volunteer rcloner will stop by soon.

1 Like
37

Thanks for help anyways.

38

After spending some time and trying to catch the error, I think there is what happens before the upload resets.

Any idea why this happens?

What is the error "read: connection reset by peer"?

2022-03-19 21:43:12 DEBUG : pacer: low level retry 1/10 (error Post "https://content.dropboxapi.com/2/files/upload_session/append_v2": read tcp 5.0.0.9:50970->162.125.65.14:443: read: connection reset by peer)
2022-03-19 21:43:16 DEBUG : pacer: low level retry 2/10 (error incorrect_offset/...)
2022-03-19 21:43:18 DEBUG : pacer: low level retry 3/10 (error incorrect_offset/..)
2022-03-19 21:43:33 DEBUG : pacer: low level retry 4/10 (error incorrect_offset/.)
2022-03-19 21:43:45 DEBUG : pacer: low level retry 5/10 (error incorrect_offset/...)
2022-03-19 21:43:50 DEBUG : pacer: low level retry 6/10 (error incorrect_offset/...)
2022-03-19 21:43:55 DEBUG : pacer: low level retry 7/10 (error incorrect_offset/)
2022-03-19 21:44:00 DEBUG : pacer: low level retry 8/10 (error incorrect_offset/.)
2022-03-19 21:44:06 DEBUG : pacer: low level retry 9/10 (error incorrect_offset/)
2022-03-19 21:44:12 DEBUG : pacer: low level retry 10/10 (error incorrect_offset/..)
2022-03-19 21:44:13 DEBUG : paq9nbm8j2vp3raqd5kblp2uqgo6v8auh7hm230ci5tkdtkmj750: Received error: upload failed: incorrect_offset/.. - low level retry 1/10

39

is it possible to see the exact error returned by POST request to append_v2?

40

The issue seems to be similar to that one of 5 years ago:

but append_v2 returns "connection reset by peer" compared to the issue under the link. Behaviour is exactly the same, files reset to 0% uploaded during progression of upload and the upload never finishes. Any idea please?