sq52236
March 17, 2022, 9:09pm
1
I am trying to upload files of different sizes 2Gb-10Gb, using rclone copy. The upload usually goes to 60-70% and then resets to 0, i.e. the random files start to upload from the beginning.
rclone v1.57.0
os/version: darwin 12.2.1 (64 bit)
os/kernel: 21.3.0 (x86_64)
os/type: darwin
os/arch: amd64
go/version: go1.17.2
go/linking: dynamic
go/tags: cmount
Dropbox Business with API keys
rclone copy /tmp remote:tmp)
rclone copy dropbox: --progress --tplimit=10
just usual config to dropbox with API keys. Nothing special.
-vv flag
Paste log here
1 Like
asdffdsa
March 17, 2022, 9:27pm
2
hello,
yes, answer the all questions in the help and support template.
the rclone debug log would show that.
sq52236
March 17, 2022, 9:40pm
3
Hi, thanks for a quick response. What is help and support template?
I have reached out to the Dropbox support. After 2 days of back and forth they suggested to reach out to rclone devs which is obviously a way to divert from actual limits conversations.
I will post results of running with --vv
asdffdsa
March 17, 2022, 9:44pm
4
the list of questions, that you were asked to post the full answers to.
--- the exact command is missing?
have you read and implemented this ?
sq52236
March 19, 2022, 6:06pm
5
Hi, apologies, I am aiming to update the required information, but can't edit the original message anymore. I will remove the original topic and will recreate a new one
asdffdsa
March 19, 2022, 6:09pm
6
no need to start a new topic. just do as i asked up above
make sure you did this this ?
post this information
sq52236
March 19, 2022, 6:19pm
7
Command used:
Config:
Debug log.
2022-03-18 15:32:54 DEBUG : nd5hfqcej1u19nl5pg18s21pa98qi2rdae3o7t4sr3jor0le8ab0: Uploading chunk 27/57https://content.dropboxapi.com/2/files/upload_session/append_v2 ": read tcp 5.0.0.9:50904->162.125.65.14:443: read: connection reset by peer)https://content.dropboxapi.com/2/files/upload_session/append_v2 ": write tcp 5.0.0.9:50910->162.125.65.14:443: i/o timeout)
sq52236
March 19, 2022, 6:20pm
8
The annoying part of it is that dropbox resets the upload at 60-70% and the file upload starts from the beginning. Do you have any suggestions please? I also tried with rclone 1.58 -- same results. I suspect it's rate limiting as shown in the log.
asdffdsa
March 19, 2022, 6:29pm
11
looks like a possible network issue.
sq52236
March 19, 2022, 6:31pm
12
interesting... How can I diagnose that for sure?
asdffdsa
March 19, 2022, 6:31pm
13
post the entire rclone debug log.
sq52236
March 19, 2022, 6:50pm
14
Apologies, I have got a chunker sitting on top of the dropbox connection. Here is config:
[dropbox_src]
[dropbox]
Log:
2022/03/18 15:17:53 INFO : Starting transaction limiter: max 10 transactions/s with burst 1https://content.dropboxapi.com/2/files/upload_session/append_v2 ": write tcp 5.0.0.9:50908->162.125.65.14:443: i/o timeout)https://content.dropboxapi.com/2/files/upload_session/append_v2 ": read tcp 5.0.0.9:50904->162.125.65.14:443: read: connection reset by peer)https://content.dropboxapi.com/2/files/upload_session/append_v2 ": write tcp 5.0.0.9:50910->162.125.65.14:443: i/o timeout)https://content.dropboxapi.com/2/files/upload_session/append_v2 ": read tcp 5.0.0.9:50920->162.125.65.14:443: read: connection reset by peer)https://content.dropboxapi.com/2/files/upload_session/append_v2 ": write tcp 5.0.0.9:50926->162.125.65.14:443: i/o timeout)https://content.dropboxapi.com/2/files/upload_session/append_v2 ": write tcp 5.0.0.9:50924->162.125.65.14:443: i/o timeout)
sq52236
March 19, 2022, 6:51pm
15
last interrupt command is me cancelling the copy when rclone started uploading the file from the beginning.
asdffdsa
March 19, 2022, 6:53pm
16
i have suggested twice, that you implement the suggestions documented here --transfers=32
sq52236
March 19, 2022, 6:57pm
17
I tried with --dropbox-batch-mode=off; --dropbox-batch-mode=sync; --dropbox-batch-mode=async and all of them give the same result currently. The upload goes to 60-70% and then the file upload resets to 0 and starts from the beginning.
sq52236
March 19, 2022, 6:57pm
18
also tried with large --transfers going up to 50. Same result.
sq52236
March 19, 2022, 6:58pm
19
tried to run with --dropbox-chunk-size=140M -- same result unfortunately.
asdffdsa
March 19, 2022, 6:59pm
20
need to post the exact command you tried.
post the debug log showing that and which file is being retried.