Jottacloud transfer speed limit (when you are over 5TB) does not work for some specific files and that causes the specific transfer to fail

What is the problem you are having with rclone?

Some specific files are uploaded at full speed even though their upload speed should be limited by Jottacloud because I am over 5TB (over 6TB currently so it should be up to ~1.85 MB/s max per transfer). That "speedy" upload causes these faulty transfers to fail with "Failed to copy: HTTP error 420 (420 status code 420)".

What is your rclone version (output from rclone version)

rclone v1.52.0

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10, 64 bit

Which cloud storage system are you using? (eg Google Drive)

Jottacloud, Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync gdrive:path/to/encrypted/folder jotta:path/to/encrypted/folder -P --fast-list

or

rclone copy gdrive:path/to/encrypted/folder jotta:path/to/encrypted/folder -P --fast-list

or

rclone copy C:\path\to\non-uploadable\files jotta:path/to/encrypted/folder -P --fast-list

also fails without --fast-list.

The rclone config contents with secrets removed.

I don't think they matter in this case as I use bare-bone settings for those specific remotes. It even fails on direct copy from local HDD.

[jotta]
type = jottacloud
tokenURL = https://id.jottacloud.com/auth/realms/jottacloud/protocol/openid-connect/token
token =
configVersion = 1

[gdrive]
type = drive
scope = drive
token =
root_folder_id = root

A log from the command with the -vv flag

Some redacted log. It was not easy to redact that many things. I just wanted to leave as little personal information as possible :). As you understand, I also redacted encrypted file paths.

2020/06/xx xx:xx:xx DEBUG : jottacloud root 'path/to/encrypted/folder': Waiting for transfers to finish
2020/06/xx xx:xx:xx DEBUG : path/to/encrypted/folder/ENCRYPTED_FILE1: Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13608: use of closed network connection - low level retry 1/10
2020/06/xx xx:xx:xx DEBUG : path/to/encrypted/folder/ENCRYPTED_FILE3: Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13617: use of closed network connection - low level retry 1/10
2020/06/xx xx:xx:xx DEBUG : path/to/encrypted/folder/ENCRYPTED_FILE1: Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13659: use of closed network connection - low level retry 2/10
2020/06/xx xx:xx:xx DEBUG : path/to/encrypted/folder/ENCRYPTED_FILE3: Received error: Post "https://068-up.jotta.cloud/files/v1/upload/[!REDACTED!]": write tcp 192.168.XXX.XXX:13679->185.179.128.XXX:443: wsasend: An existing connection was forcibly closed by the remote host. - low level retry 2/10
2020/06/xx xx:xx:xx DEBUG : path/to/encrypted/folder/ENCRYPTED_FILE2: Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13680: use of closed network connection - low level retry 1/10
2020/06/xx xx:xx:xx DEBUG : path/to/encrypted/folder/ENCRYPTED_FILE1: Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13682: use of closed network connection - low level retry 3/10
2020/06/xx xx:xx:xx DEBUG : path/to/encrypted/folder/ENCRYPTED_FILE1: Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13717: use of closed network connection - low level retry 4/10
2020/06/xx xx:xx:xx DEBUG : path/to/encrypted/folder/ENCRYPTED_FILE3: Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13725: use of closed network connection - low level retry 3/10
2020/06/xx xx:xx:xx DEBUG : path/to/encrypted/folder/ENCRYPTED_FILE1: Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13760: use of closed network connection - low level retry 5/10
2020/06/xx xx:xx:xx ERROR : path/to/encrypted/folder/ENCRYPTED_FILE3: Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":\"[!REDACTED (12 numbers)!]\"}"
2020/06/xx xx:xx:xx DEBUG : path/to/encrypted/folder/ENCRYPTED_FILE2: Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13807: use of closed network connection - low level retry 2/10
2020/06/xx xx:xx:xx ERROR : path/to/encrypted/folder/ENCRYPTED_FILE1: Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":\"[!REDACTED (12 numbers)!]\"}"
2020/06/xx xx:xx:xx DEBUG : path/to/encrypted/folder/ENCRYPTED_FILE4: Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13853: use of closed network connection - low level retry 1/10

The thing that could influence this behaviour is that files are rclone encrypted (file contents and their filename). I copy them directly because you have MD5 readily available from GDrive as Jottacloud requires it and files matches completely. If you copy them from crypt to crypt backbone, they are re-encrypted and their raw content and MD5s are completely different compared to Gdrive one due to encryption peculiarities. I don't want that. If you rename the file to something different (like test_file.zip) it most likely will be uploaded with Jottacloud's speed limiting in effect without problems. However if you use some encrypted names, they may fail on some specific names. However, the problem is not only filename but the file contents to. If you create a random file (same size, same filename, different contents), it will upload correctly (most likely) with speed limit. In my last case I had about 10/40 of those "speedy" files (this was most extreme case which caused me to write here). Usually they were very rare (less than 1/100 or so). Started to occur recently (less than a month ago). I could solve them by forcefully limiting bandwidth and using one transfer:

rclone sync gdrive:path/to/encrypted/folder jotta:path/to/encrypted/folder -P --fast-list --bwlimit 1.8M --transfers 1

But that is very slow method. And even that may not work if the speed is close to the limit.

I think that as circumvention solution there could be some user flag which forces the specific upload transfer (not overall bandwidth as with "--bwlimit") to be at user specified speed. Another circumvention solution would be a flag which would upload files with different name (safe one) and then rename the file to the correct one after the upload. That could be used as a finishing touch for all "speedy" files which were left not uploaded.

Currently I am failing the upload even with "--bwlimit 1.5M --transfers 1" :frowning:. 6 files left.

Edit:
It even were failing with "--bwlimit 1M --transfers 1". So I uploaded files directly to Jottacloud's crypt backend from HDD. Those files were "abiding" Jotta's limits (they should as the raw content is completely different). However even then some files thrown errors (2020/06/XX XX:XX:XX ERROR : Folder/File.ext: put: error uploading: error reading error out of body: set tcp 192.168.XXX.XXX:XXXX: use of closed network connection) which caused to restart particular upload transfer (one transfer were restarted not even once). Then I checksum synced Jotta with Gdrive so both files on both remotes are identical.

Maybe it is possible to somehow easily remove failing to upload files from Gdrive (give all failing to upload files in txt list for removal). Then upload them again so they have different raw data content which would probably be "compatible" with Jotta's speed limiter.

The error message here makes me think that rclone is doing something wrong here or jottacloud and rclone have got out of sync somehow.

The previous errors look like some kind of networking problem.

These are the root cause of the problem I think. They could be caused by lots of things. Maybe jottacloud, maybe your ISP, maybe your router or OS.

The problem is likely solely with Jottacloud and/or rclone. I tested even without router. Totally same behaviour. Same transfers to Google Drive don't fail at all. Some encrypted filename+encrypted content combination causes that behaviour. You could rename the file (for e.g. I tried with TEMP_FILE) and it will most likely will be uploaded perfectly on the first try even though it literally failed to upload 50+ times before with original name. And then rename it to original name. File transfer will always fail if upload speed is higher than supposed to be. Some files manage to be uploaded after many tries. Some probably will never be uploaded (that one with 50+ tries - it was the last one from one upload batch, failed to upload with --bwlimit 0.5M too).

I would like that you make a new flag (e.g.: --jottacloud-try-renamed-file-upload) as a temporary fix which would try re-upload failed transfers with different name (e.g.: TEMP_NAME01). I think it should be quite easy to implement it. It would be good if it would also be possible to specify your own name. Then after upload, rename to the original file name and check if it was renamed properly. For proper solution you probably need test files which fails to upload (I need to make those) and Jottacloud subscription (you must be over 5-6 TB so speed limiting would be in effect). Such failing to upload files are now very common (can happen with every bigger (10 or more) upload) so it shouldn't be very hard to create them. At least with this, I won't waste so much of the bandwidth. With the last upload, out of ~35GB about 29GB was uploaded and in total 50GB of the bandwidth was consumed. You will see that in the log bellow.

So I made a full one fresh and substantial upload session --vv log up until it stops. It was way more difficult to redact it. Used regex this time. I am not programmer so it was complete trial&error experience playing in https://regex101.com/ as a guide. But I need better regex skills so I could rename mass files more effectively. So it was good practice. I hope I haven't missed something or made incorrect edits. If I would continue with leftover files, some will probably be uploaded after many tries, some will never be uploaded. Almost all of them be uploaded without speed limits (until it will upload correctly). Lots of bandwidth will be wasted. I hope the log is not too long for you (1886 lines - couldn't post it all). It makes reply box very laggy :smile:.

I had to trim it to post it (32000 characters limit). I put lines from the beginning as much as possible and added the status lines from the end.

rclone copy gdrive:ENCRYPTED_FOLDER jotta:ENCRYPTED_FOLDER -P --fast-list --transfers 8 --track-renames --order-by size,desc -vv
2020/06/XX XX:XX:XX DEBUG : rclone: Version "v1.52.0" starting with parameters ["rclone" "copy" "gdrive:ENCRYPTED_FOLDER" "jotta:ENCRYPTED_FOLDER" "-P" "--fast-list" "--transfers" "8" "--track-renames" "--order-by" "size,desc" "-vv"]
2020/06/XX XX:XX:XX DEBUG : Using config file from "[!REDACTED!]\\.config\\rclone\\rclone.conf"
2020/06/XX XX:XX:XX DEBUG : jottacloud root 'ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT': Token expired but no uploads in progress - doing nothing
2020/06/XX XX:XX:XX DEBUG : jotta: Loaded invalid token from config file - ignoring
2020/06/XX XX:XX:XX DEBUG : Keeping previous permissions for config file: -rw-rw-rw-
2020/06/XX XX:XX:XX DEBUG : jotta: Saved new token in config file
2020/06/XX XX:XX:XX ERROR : jottacloud root 'ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT': Ignoring --track-renames as it doesn't work with copy or move, only sync
2020/06/XX XX:XX:XX DEBUG : jottacloud root 'ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT': Waiting for checks to finish
2020/06/XX XX:XX:XX DEBUG : jottacloud root 'ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT': Waiting for transfers to finish
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:9664: use of closed network connection - low level retry 1/10
2020/06/XX XX:XX:XX ERROR : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":"000000000000"}"
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FILE.EXT: MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FILE.EXT: Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:10947: use of closed network connection - low level retry 1/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:10967: use of closed network connection - low level retry 2/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:10972: use of closed network connection - low level retry 3/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:10974: use of closed network connection - low level retry 4/10
2020/06/XX XX:XX:XX ERROR : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":"000000000000"}"
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX ERROR : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":"000000000000"}"
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX ERROR : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Failed to copy: HTTP error 421 (421 Misdirected Request) returned body: "{\"code\":421,\"message\":null,\"cause\":\"\",\"error_id\":\"CorruptUploadOpenApiException\",\"x-id\":"000000000000"}"
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11447: use of closed network connection - low level retry 1/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11464: use of closed network connection - low level retry 2/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11469: use of closed network connection - low level retry 3/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11488: use of closed network connection - low level retry 4/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11495: use of closed network connection - low level retry 5/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11497: use of closed network connection - low level retry 6/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11505: use of closed network connection - low level retry 7/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11509: use of closed network connection - low level retry 8/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11511: use of closed network connection - low level retry 9/10
2020/06/XX XX:XX:XX ERROR : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Failed to copy: error reading error out of body: set tcp 192.168.XXX.XXX:11516: use of closed network connection
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11614: use of closed network connection - low level retry 1/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11628: use of closed network connection - low level retry 2/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11638: use of closed network connection - low level retry 3/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11651: use of closed network connection - low level retry 4/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11659: use of closed network connection - low level retry 5/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11681: use of closed network connection - low level retry 6/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11684: use of closed network connection - low level retry 7/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11759: use of closed network connection - low level retry 8/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11739: use of closed network connection - low level retry 1/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11773: use of closed network connection - low level retry 9/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11765: use of closed network connection - low level retry 2/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11779: use of closed network connection - low level retry 3/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11783: use of closed network connection - low level retry 4/10
2020/06/XX XX:XX:XX ERROR : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":"000000000000"}"
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11755: use of closed network connection - low level retry 1/10
2020/06/XX XX:XX:XX ERROR : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Failed to copy: error reading error out of body: set tcp 192.168.XXX.XXX:11786: use of closed network connection
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11790: use of closed network connection - low level retry 5/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: Post "https://[!REDACTED!]-up.jotta.cloud/files/v1/upload/[!REDACTED!]": write tcp 192.168.XXX.XXX:11796->185.179.XXX.XXX:443: wsasend: An existing connection was forcibly closed by the remote host. - low level retry 6/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11799: use of closed network connection - low level retry 7/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11797: use of closed network connection - low level retry 2/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11803: use of closed network connection - low level retry 8/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11812: use of closed network connection - low level retry 3/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11811: use of closed network connection - low level retry 9/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11793: use of closed network connection - low level retry 1/10
2020/06/XX XX:XX:XX ERROR : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Failed to copy: error reading error out of body: set tcp 192.168.XXX.XXX:11820: use of closed network connection
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11822: use of closed network connection - low level retry 4/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11827: use of closed network connection - low level retry 2/10
2020/06/XX XX:XX:XX ERROR : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":"000000000000"}"
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11844: use of closed network connection - low level retry 3/10
2020/06/XX XX:XX:XX ERROR : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":"000000000000"}"
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11868: use of closed network connection - low level retry 4/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11877: use of closed network connection - low level retry 5/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11879: use of closed network connection - low level retry 6/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11890: use of closed network connection - low level retry 7/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11894: use of closed network connection - low level retry 8/10
2020/06/XX XX:XX:XX ERROR : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":"000000000000"}"
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:11905: use of closed network connection - low level retry 1/10
2020/06/XX XX:XX:XX ERROR : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":"000000000000"}"
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:12164: use of closed network connection - low level retry 1/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:12199: use of closed network connection - low level retry 2/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:12205: use of closed network connection - low level retry 3/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: read tcp 192.168.XXX.XXX:12229->185.179.XXX.XXX:443: wsarecv: An existing connection was forcibly closed by the remote host. - low level retry 4/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:12233: use of closed network connection - low level retry 5/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:12260: use of closed network connection - low level retry 6/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:12268: use of closed network connection - low level retry 7/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:12274: use of closed network connection - low level retry 8/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:12279: use of closed network connection - low level retry 9/10
2020/06/XX XX:XX:XX ERROR : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Failed to copy: error reading error out of body: set tcp 192.168.XXX.XXX:12282: use of closed network connection
2020/06/XX XX:XX:XX ERROR : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":"000000000000"}"
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:12290: use of closed network connection - low level retry 1/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: Post "https://[!REDACTED!]-up.jotta.cloud/files/v1/upload/[!REDACTED!]": write tcp 192.168.XXX.XXX:12313->185.179.XXX.XXX:443: wsasend: An existing connection was forcibly closed by the remote host. - low level retry 2/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:12322: use of closed network connection - low level retry 3/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:12337: use of closed network connection - low level retry 4/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:12340: use of closed network connection - low level retry 5/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: Post "https://[!REDACTED!]-up.jotta.cloud/files/v1/upload/[!REDACTED!]": write tcp 192.168.XXX.XXX:12342->185.179.XXX.XXX:443: wsasend: An existing connection was forcibly closed by the remote host. - low level retry 6/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:12364: use of closed network connection - low level retry 7/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:12376: use of closed network connection - low level retry 8/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:12380: use of closed network connection - low level retry 9/10
2020/06/XX XX:XX:XX ERROR : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Failed to copy: error reading error out of body: set tcp 192.168.XXX.XXX:12386: use of closed network connection
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13067: use of closed network connection - low level retry 1/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13069: use of closed network connection - low level retry 2/10
2020/06/XX XX:XX:XX ERROR : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":"000000000000"}"
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13082: use of closed network connection - low level retry 1/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13113: use of closed network connection - low level retry 2/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13119: use of closed network connection - low level retry 3/10
2020/06/XX XX:XX:XX ERROR : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":"000000000000"}"
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13328: use of closed network connection - low level retry 1/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13352: use of closed network connection - low level retry 2/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13356: use of closed network connection - low level retry 3/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13369: use of closed network connection - low level retry 4/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13381: use of closed network connection - low level retry 5/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Received error: error reading error out of body: set tcp 192.168.XXX.XXX:13384: use of closed network connection - low level retry 6/10
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)
2020/06/XX XX:XX:XX DEBUG : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT MD5 = ffffffffffffffffffffffffffffffff OK
2020/06/XX XX:XX:XX INFO  : ENCRYPTED_FOLDER/ENCRYPTED_FILE.EXT Copied (new)

[!SKIPPED!]

Transferred:       50.845G / 50.845 GBytes, 100%, 17.354 MBytes/s, ETA 0s
Errors:                30 (retrying may help)
Checks:               453 / 453, 100%
Transferred:          236 / 236, 100%
Elapsed time:      50m0.0s
2020/06/XX XX:XX:XX INFO  :
Transferred:       50.845G / 50.845 GBytes, 100%, 17.354 MBytes/s, ETA 0s
Errors:                30 (retrying may help)
Checks:               453 / 453, 100%
Transferred:          236 / 236, 100%
Elapsed time:      50m0.0s

2020/06/XX XX:XX:XX DEBUG : 370 go routines active
2020/06/XX XX:XX:XX Failed to copy with 30 errors: last error was: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":"000000000000"}"

Tried to upload leftovers (~6GB). Out of 30, 13 was uploaded. ~25GB of bandwidth was used.

Transferred:       24.931G / 24.931 GBytes, 100%, X MBytes/s, ETA 0s
Errors:                17 (retrying may help)
Checks:               724 / 724, 100%
Transferred:           13 / 13, 100%
Elapsed time:     XmXs

2020/06/XX XX:XX:XX DEBUG : 230 go routines active
2020/06/XX XX:XX:XX Failed to copy with 17 errors: last error was: error reading error out of body: set tcp 192.168.XXX.XXX:4568: use of closed network connection

I've hidden overall upload speed because I don't want to reveal my upload speed :smiley:. It is quite close to it.

Edit:
Another session:

Transferred:       16.353G / 16.353 GBytes, 100%, X MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Checks:               761 / 761, 100%
Transferred:            7 / 7, 100%
Elapsed time:      XmXs

2020/06/XX XX:XX:XX DEBUG : 137 go routines active
2020/06/XX XX:XX:XX Failed to copy with 10 errors: last error was: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":\"000000000000\"}"

Edit2:
Last session. This one finally managed to upload all leftovers.

Transferred:        6.943G / 6.943 GBytes, 100%, X MBytes/s, ETA 0s
Checks:               513 / 513, 100%
Transferred:           10 / 10, 100%
Elapsed time:      XmXs

2020/06/XX XX:XX:XX DEBUG : 86 go routines active

Well you got there in the end!

The "use of closed network" error is probably Jottacloud closing the connection when it is overloaded I would guess. Did you try slowing down your upload with --bwlimit?

However it looks like rclone isn't doing the retry properly or Jottacloud is preventing it - that is what the IncompleteUploadOpenApiException error means I think.

To fix this I need an unedited log run with -vv --dump headers. You can gzip it and email it (or a link) to nick@craig-wood.com if you want.

Yes, but at what steep price. It was almost 2/3 of bandwidth totally wasted: to upload ~35GB, 99GB was used (198GB actually as it was copied between two cloud storages).

Anyway. When I tried uploading the next day. All errors were gone even with 15+ transfers. I don't know if that was fixed by Jottacloud themselves as they could have noticed such bandwidth wastage. Or it was caused by me as I copied main folder in Jottacloud web interface to remove all older file version history (the files in copied folder do not contain the past file versions). Then removed older folder (which I copied) as it was unnecessary duplicate. I did that to remove those older file versions so it would take less space as I noticed increase in total Jottacloud space usage above factual one. With Jottacloud you can't search for exact files which has older file versions (you should know which files where re-uploaded with different content). So I used the last folder I worked on as the most likely candidate which could have files with older file versions due to repeated same file uploads.

So this bug may be somehow related with older file version history. However the uploads to different folders now works well too (not the folder which I re-copied). Currently only one transfer has failed to upload (Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":\"000000000000\"}") but it managed to re-upload itself on 2/3 attempt (3rd upload have passed). I hope it is not a sign of possible return for this problem. At least it means that it is not completely eradicated and may occur again in the future when specific conditions are met. I believe that those conditions are: rclone encrypted file+filename combination and maybe they are copied to folder which has files with file version history.

Also --bwlimit didn't help at all (it helped for the first cases though). I mentioned that several times in my messages. I even used --transfers 1 to further limit "speedy" upload speed impact to fit Jottacloud upload speed limits so that would be even likelier for the transfer to pass. You probably weren't very attentive reading them. Jottacloud would force close all transfers if they were uploaded at full speed. About half or so of file would be uploaded before connection is closed. It also seems that for the same file it would close the transfer at the same completion percentage (e.g.: ~31% for the file which failed for 50+ times).

I have a question. Does the "HDD/remote: -> Jottacloud: (crypt)" uploaded file on second try is different (raw content/MD5 doesn't match) if the first one fails? I expect the answer to be true. That would explain why nobody except me complain about that Jottacloud problem. Even if they have some failed transfers they would most likely be uploaded correctly on the second try due to different file content.

Indeed!

I'm glad it seems to have improved. Maybe it is something to do with the older versions. If you can work out exactly what then I can have a chance of fixing it.

Yes it will be different as the crypt backend will choose a difference nonce.

I think that must be the cause of the problem.

Looking at the code rclone will try to resume the transfer which is clearly incorrect for crypt...

Can you please make a new issue on github about it and I'll get the Jottacloud maintainer to have a look - thank you :slight_smile:

Sorry, but I don't understand the problem :thinking:. I'm not programmer. My rclone logs were from direct copy/sync from one remote to the other (HDD/gdrive: to jotta:), so I could have identical files/MD5s. Both of those remotes have rclone encrypted files and folders. Resume of the transfer (if that is possible) could be expected in those cases as the raw files are identical, or not? Also, I remember that by copying from HDD to jottacrypt: (crypt) backend, I also were getting some transfer errors. However all files were still uploaded on consequentive tries because different encryption nonce were used (as you've confirmed). I don't know if there were any resume behaviour for that case as there should be no resumes due to different raw file content/MD5s as different nonce would be used. Jottacloud would probalby discard such incorectly resumed files due to MD5 mismatch.

I see. I think it is worth making an issue about it - that whole resume the transfer thing needs looking at I think.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.