Retries on error not working correctly with --immutable and --ignore-existing

Hi,

It seems there is a bug with the error handling in rclone when existing files are not to be changed (either with --immutable or --ignore-existing).
I am using rclone to sync to OneDrive, and with this there is relatively regular (around one every ~20 files) a problem with setting the time of the files after the transfer. I’m not sure if this problem is with rclone or OneDrive, but rclone should handle this fine with the retries.
However, with --immutable turned on, on the next try rclone sees that the file already exists and refuses to change the time:

$ rclone copy -v --immutable /source OneDrive:/backup
2018/01/29 15:20:00 INFO  : One drive root 'backup': Modify window is 1s
[...]
2018/01/29 15:23:21 INFO  : data/e5/e5c366a22d0c425b63a3c9a66c0950470caa2548c9e8aa2f6a528237f29398cd: Copied (new)
2018/01/29 15:23:23 ERROR : data/ef/ef97c17a82bb340e36f79f4592cb2dbc7daa0a8c3a134319adba942766ca011e: Failed to copy: resourceModified: ETag does not match current item`s value
2018/01/29 15:23:25 ERROR : data/ee/ee602dcb0a20f29ee9dcb35b41fab03666f260e8fd0e9d203442bf15d4e7b4c7: Failed to copy: resourceModified: ETag does not match current item`s value
2018/01/29 15:23:26 INFO  : data/ed/ed2f4f74b6b5f3bb73adf638ad93e2a3988e7d20bb841bbdbea6332b465026bf: Copied (new)
[...]
2018/01/29 15:23:28 ERROR : Attempt 1/3 failed with 2 errors and: resourceModified: ETag does not match current item`s value
2018/01/29 15:23:29 ERROR : data/ee/ee602dcb0a20f29ee9dcb35b41fab03666f260e8fd0e9d203442bf15d4e7b4c7: Timestamp mismatch between immutable objects
2018/01/29 15:23:29 ERROR : data/ee/ee602dcb0a20f29ee9dcb35b41fab03666f260e8fd0e9d203442bf15d4e7b4c7: Source and destination exist but do not match: immutable file modified
2018/01/29 15:23:29 ERROR : data/ef/ef97c17a82bb340e36f79f4592cb2dbc7daa0a8c3a134319adba942766ca011e: Timestamp mismatch between immutable objects
2018/01/29 15:23:29 ERROR : data/ef/ef97c17a82bb340e36f79f4592cb2dbc7daa0a8c3a134319adba942766ca011e: Source and destination exist but do not match: immutable file modified
2018/01/29 15:23:32 INFO  : One drive root 'backup': Waiting for checks to finish
2018/01/29 15:23:32 INFO  : One drive root 'backup': Waiting for transfers to finish
2018/01/29 15:23:32 ERROR : Attempt 2/3 failed with 0 errors and: immutable file modified
2018/01/29 15:23:37 ERROR : data/ef/ef97c17a82bb340e36f79f4592cb2dbc7daa0a8c3a134319adba942766ca011e: Timestamp mismatch between immutable objects
2018/01/29 15:23:37 ERROR : data/ef/ef97c17a82bb340e36f79f4592cb2dbc7daa0a8c3a134319adba942766ca011e: Source and destination exist but do not match: immutable file modified
2018/01/29 15:23:37 ERROR : data/ee/ee602dcb0a20f29ee9dcb35b41fab03666f260e8fd0e9d203442bf15d4e7b4c7: Timestamp mismatch between immutable objects
2018/01/29 15:23:37 ERROR : data/ee/ee602dcb0a20f29ee9dcb35b41fab03666f260e8fd0e9d203442bf15d4e7b4c7: Source and destination exist but do not match: immutable file modified
2018/01/29 15:23:38 INFO  : One drive root 'backup': Waiting for checks to finish
2018/01/29 15:23:38 INFO  : One drive root 'backup': Waiting for transfers to finish
2018/01/29 15:23:38 ERROR : Attempt 3/3 failed with 0 errors and: immutable file modified
2018/01/29 15:23:38 Failed to copy: immutable file modified

With --ignore-existing the files with the wrong time are just ignored on the next try.

I think rclone should just not check if the file exists on retry as it was already decided in in earlier run to transfer the file.

Cheers,
Acros

It would be nice to get to the bottom of this. Can you make please make a new issue on github about this with a log with -vv of it happening please?

If you can figure out what makes it more likely that would be most useful (eg size of file)

Hmm…

At the moment a retry holds no state from the previous try, so this might be difficult fo fix…

Can you please make another new issue on github about this? I don’t guarantee we can fix it easily but we can have a think about it!

The best place to fix your particular issue will be to fix the onedrive problem (or at least make it do a low-level retry).