Rclone + opendrive failed to transfer file from cache to remote: failed to create file: Incorrect chunk offset

[adamtheturtle (Adam Dangoor)]

I see that someone reported this earlier (Rclone fails to upload large files to OpenDrive intermittently (Incorrect chunk offset/Invalid upload file size)) and got "Let's call it a glitch at OpenDrive and move on! These things happen at cloud providers.".

However, I'm opening this as I see this all the time.

I'm using the rclone Docker Volume Plugin and see the following logs:

Oct 04 14:55:43 ns518990 dockerd[24727]: time="2022-10-04T14:55:43Z" level=error msg="2022/10/04 14:55:43 ERROR : : vfs cache: failed to upload try #6, will retry in 5m0s: vfs cache: failed to transfer file from cache to remote: failed to create file: Incorrect chunk offset: uploaded=41943040, chunk_offset=52428800 (Error 400)" plugin=cab963f65004a7bef6725eb6fe417e386f5345f154ceccd385c6e9a6771b4b1a

[ncw (Nick Craig-Wood)]

What probably caused this is that rclone uploaded a chunk but there was some confusion between opendrive and rclone as to whether that chunk really arrived. Maybe a race condition at opendrive, or a networking glitch. The result was that for the next chunk rclone and opendrive are out of sync, hence the error.

Can you reproduce the problem at will? If so a log with -vv would be very helpful. I suspect it will show the trouble started the block before the ERROR. Maybe rclone is retrying blocks incorrectly - that is a possibility - the log should show though.

[adamtheturtle (Adam Dangoor)]

@ncw Thank you for the quick response.

I'm not sure about "at will" but the error is repeatedly happening so I think I'd know if it is fixed somehow.

(How) Can I get the -vv log with the rclone Docker Volume Plugin?

[adamtheturtle (Adam Dangoor)]

@adamtheturtle

(How) Can I get the -vv log with the rclone Docker Volume Plugin?

I found Docker plugin fail to start/reinstall - #5 by ivandeex

docker plugin disable rclone # just to be sure, probably unneeded
docker plugin set rclone args="-vv" # bump up verbosity
docker plugin enable rclone # this will fail but also report in

[adamtheturtle (Adam Dangoor)]

A selection of some of the things I can see in the log:

• vfs cache: failed to upload try #1, will retry in 10s: vfs cache: failed to transfer file file from cache to remote: failed to create file: File update error. Temp file doesn't exist. Please restart task.
• Failed to copy: failed to create file: File update error. Temp file doesn't exist. Please restart task. (Error 400)
• vfs cache: failed to upload try #6, will retry in 5m0s: vfs cache: failed to transfer file from cache to remote: failed to create file: Incorrect chunk offset: uploaded=41943040, chunk_offset=52428800 (Error 400)

[ncw (Nick Craig-Wood)]

@adamtheturtle

Temp file doesn't exist. Please restart task.

This error popped up in this thread too

100+ percents of copying on last release

Ok. I have to wait until my files have been uploaded and then I will start a new upload with --vv. Where should I send a log?

That's probably the root cause. Can you ask opendrive support about the error Temp file doesn't exist. Please restart task. - I suspect it is a problem with one of their backends.

[adamtheturtle (Adam Dangoor)]

@ncw Thank you again for helping on this.

I will ask OpenDrive about this.

FWIW, one difference between me and that other post (I believe) is that my OpenDrive Activity Log shows file/chunk deletions (not purposefully prompted by me):

[ncw (Nick Craig-Wood)]

@adamtheturtle

FWIW, one difference between me and that other post (I believe) is that my OpenDrive Activity Log shows file/chunk deletions (not purposefully prompted by me):

That is proably rclone deleting or trying to delete the failed uploads