Error-id: Incomplete Upload Open Api Exception

Id0 · April 13, 2020, 12:12am

I'm facing an issue when copying files to a crypt bucket with Jottacloud seen below. FILE_BIG files are several GB and give an error after some seconds, while FILE_SMALL are less than 10MB and fine. I think this is related to that I've changed TMPDIR since /tmp is too small. I created a tmp folder at /mnt/usb/tmp, so echo $TMPDIR gives /mnt/usb/tmp, but the folder has been empty every time I checked it during copy. Wouldn't a permission issue have been reported in the log if that was the case?

Let me know if anything is needed.

OS: 4.19.69-rockchip armv7l GNU/Linux

DEBUG LOG

TIMESTAMP DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone" "copy" "PATH_SOURCE" "PATH_DEST" "--size-only" "--transfers=3" "--checkers=2" "-vvvv" "-P"]
TIMESTAMP DEBUG : Using config file from "/home/USERNAME/.config/rclone/rclone.conf"
TIMESTAMP INFO  : Encrypted drive 'crypt:FOLDER/': Waiting for checks to finish
TIMESTAMP INFO  : Encrypted drive 'crypt:FOLDER/': Waiting for transfers to finish
TIMESTAMP ERROR : FILE_BIG_1: Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":\"X-ID\"}"
TIMESTAMP INFO  : FILE_SMALL_1: Copied (new)
TIMESTAMP ERROR : FILE_BIG_2: Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":\"X-ID\"}"
TIMESTAMP INFO  : FILE_SMALL_2: Copied (new)
TIMESTAMP ERROR : FILE_BIG_3: Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":\"X-ID\"}"

Id0 · April 13, 2020, 1:40pm

I've narrowed the setup down to rclone copy of a single file to a crypt volume at Jottacloud. The issue doesn't seem to be permission related even though I can't see any files getting added to the tmp folder. I tried to set a new tmp directory with root and got a permission error: TIMESTAMP ERROR : FILE: Failed to copy: failed to calculate MD5: open TMP_FOLDER/rclone-*md5-*: permission denied.

I guess this is related to that I'm using a low-powered arm server which is having a hard time with 100% CPU consumption while uploading. Are there any tips for adjustments I can do to get this working for a low-powered device? Example limit multi-upload streams (if that is being used) or something? I guess I can limit the network speed to try to see how that behaves.

Would implementing Cache/Chunking make it better? Example: Jottacloud -> Cache -> Crypt

DEBUG LOG

TIMESTAMP DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone" "copy" "PATH/FILE" "crypt:FOLDER/" "--exclude" ".*" "--size-only" "--transfers=1" "--checkers=2" "-vv" "-P"]
TIMESTAMP DEBUG : Using config file from "/home/USER/.config/rclone/rclone.conf"
TIMESTAMP DEBUG : FILE: Bad size for decrypt: file is too short to be encrypted
TIMESTAMP DEBUG : FILE: Bad size for decrypt: file is too short to be encrypted
TIMESTAMP DEBUG : FILE: Bad size for decrypt: file is too short to be encrypted
TIMESTAMP DEBUG : FILE: Sizes differ (src 3277329349 vs dst 0)
TIMESTAMP ERROR : FILE: Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":\"X-ID\"}"
TIMESTAMP ERROR : Attempt 1/3 failed with 1 errors and: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":\"X-ID\"}"
TIMESTAMP DEBUG : FILE: Bad size for decrypt: file is too short to be encrypted
TIMESTAMP DEBUG : FILE: Bad size for decrypt: file is too short to be encrypted
TIMESTAMP DEBUG : FILE: Bad size for decrypt: file is too short to be encrypted
TIMESTAMP DEBUG : FILE: Sizes differ (src 3277329349 vs dst 0)
TIMESTAMP ERROR : FILE: Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":\"X-ID\"}"
TIMESTAMP ERROR : Attempt 2/3 failed with 1 errors and: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":\"X-ID\"}"
TIMESTAMP DEBUG : FILE: Bad size for decrypt: file is too short to be encrypted
TIMESTAMP DEBUG : FILE: Bad size for decrypt: file is too short to be encrypted
TIMESTAMP DEBUG : FILE: Bad size for decrypt: file is too short to be encrypted
TIMESTAMP DEBUG : FILE: Sizes differ (src 3277329349 vs dst 0)
TIMESTAMP ERROR : FILE: Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":\"X-ID\"}"
TIMESTAMP ERROR : Attempt 3/3 failed with 1 errors and: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":\"X-ID\"}"
Transferred:       99.719M / 99.719 MBytes, 100%, 165.808 kBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Elapsed time:     10m15.8s
TIMESTAMP Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":\"X-ID\"}"

ncw · April 14, 2020, 11:14am

Jottacloud requires a temp directory for certain uploads - it has to calculate the checksum of the file before upload.

However if you try this beta I think it will fix the problem as it is uploading from crypt that you are having the problem with

https://beta.rclone.org/branch/v1.51.0-154-ga283c655-fix-crypt-src-hash-beta/

Id0 · April 14, 2020, 1:05pm

Great update. However, I'm stil facing the same issue and I skipped the task by CTRL + C before the third run had completed. Reading from the seventh line: Computing MD5 hash of encrypted source. Is the source encrypted? I thought the MD5 hash was calculated from the raw source (not encrypted). Concerning the error in the log. I'll do some more testing on a PC running Ubuntu to see if I still have the issue or if it's related to my low-powered Arm device.

DEBUG LOG

rclone copy FILE crypt:FOLDER --exclude '.*' --size-only --transfers=1 --checkers=2 -vv -P
2020/04/14 14:37:45 DEBUG : rclone: Version "v1.51.0-154-ga283c655-fix-crypt-src-hash-beta" starting with parameters ["rclone" "copy" "FILE" "crypt:FOLDER/" "--exclude" ".*" "--size-only" "--transfers=1" "--checkers=2" "-vv" "-P"]
2020/04/14 14:37:45 DEBUG : Using config file from "/home/USER/.config/rclone/rclone.conf"
2020/04/14 14:37:45 DEBUG : JC: Loaded invalid token from config file - ignoring
2020/04/14 14:37:46 DEBUG : JC: Saved new token in config file
2020-04-14 14:37:47 DEBUG : FILE_5GB: Need to transfer - File not found at Destination
2020-04-14 14:37:47 DEBUG : FILE_5GB: Computing MD5 hash of encrypted source
2020-04-14 14:42:11 ERROR : FILE_5GB: Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":\"X-ID\"}"
2020-04-14 14:42:11 ERROR : Attempt 1/3 failed with 1 errors and: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":\"X-ID\"}"
2020-04-14 14:42:11 DEBUG : FILE_5GB: Bad size for decrypt: file is too short to be encrypted
2020-04-14 14:42:11 DEBUG : FILE_5GB: Bad size for decrypt: file is too short to be encrypted
2020-04-14 14:42:11 DEBUG : FILE_5GB: Bad size for decrypt: file is too short to be encrypted
2020-04-14 14:42:11 DEBUG : FILE_5GB: Sizes differ (src 5068943484 vs dst 0)
2020-04-14 14:42:11 DEBUG : FILE_5GB: Computing MD5 hash of encrypted source
2020-04-14 14:47:20 ERROR : FILE_5GB: Failed to copy: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":\"X-ID\"}"
2020-04-14 14:47:20 ERROR : Attempt 2/3 failed with 1 errors and: HTTP error 420 (420 status code 420) returned body: "{\"code\":420,\"message\":null,\"cause\":\"\",\"error_id\":\"IncompleteUploadOpenApiException\",\"x-id\":\"X-ID\"}"
2020-04-14 14:47:20 DEBUG : FILE_5GB: Bad size for decrypt: file is too short to be encrypted
2020-04-14 14:47:20 DEBUG : FILE_5GB: Bad size for decrypt: file is too short to be encrypted
2020-04-14 14:47:20 DEBUG : FILE_5GB: Bad size for decrypt: file is too short to be encrypted
2020-04-14 14:47:20 DEBUG : FILE_5GB: Sizes differ (src 5068943484 vs dst 0)
2020-04-14 14:47:20 DEBUG : FILE_5GB: Computing MD5 hash of encrypted source
Transferred:   	   57.577M / 4.777 GBytes, 1%, 764.720 kBytes/s, ETA 1h47m19s
Transferred:            0 / 1, 0%
Elapsed time:     10m27.4s

ncw · April 15, 2020, 11:35am

Jottacloud needs the MD5 hash of the encrypted data since that is what we store there. It never sees the unencrypted data or hash.

The log looks cleaner now, but it appears the fundamental problem remains. I'd say this is this error

@buengese do you have an idea what this means? It looks like the upload failed because we stopped providing data?

Id0 · April 16, 2020, 9:11pm

That give sense since my server is a Tinkerboard Quad core 1.8 GHz (ARM Cortex-A17), so it might be that it can't handle the load. I've no issue on either MacOS, Windows or AMD Ubuntu. If anyone have an ARMv7 device, like a Raspberry Pi 3 or lower, and can test it that would be great.

system · June 16, 2020, 5:11pm

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.