Avoiding Bisync 'times are equal' conflicts with newer?

[nielash (Niel)]

I think what would be really helpful here is to focus on the absolute simplest case -- the one where you are seeing unexpected conflicts even though:

• Only one machine is in use, and the other has been off for days
• --recover is not in use
• Graceful Shutdown is not in use
• We are certain the previous run finished successfully and the machine has not slept or otherwise interrupted a run since then.

I think it is also useful to back up and consider what exactly a conflict is:

In bisync, a "conflict" is a file that is new or changed on both sides (relative to the prior run) AND is not currently identical on both sides.

In our "simple" scenario here, the two sides are idrive and local. We know why the file is changing on local (because you have edited it during the normal course of working). What we don't know is why the file has changed on idrive (or, more accurately: why bisync thinks the file has changed on idrive, relative to the previous sync.) I think that is the main question we need to solve.

Since idrive is cloud storage, not a machine, nothing should really be changing there unless you have told it to. So the next logical question is whether the file's content is really changing, or whether the change is just an illusion (i.e. the file on idrive is not truly changing, and bisync just thinks it is for some reason.) If the file is really changing, you need to figure out what is causing the change (like possibly some other app that is syncing changes to idrive, or edits made manually in the browser UI). If idrive gives you an event audit log for the file, that could be useful to determine this.

On the other hand, if the change is an illusion, then we need to determine why bisync thinks the file is changing when it is really not. It would help to know what specifically the change is (for example, is idrive reporting a modtime that is newer than the one we expect based on the last sync? If so, is idrive wrong, or is our expectation wrong?) The debug log can help you figure this out, as all changes should show the "old" and "new" values (with "old" being our expectation based on the last run.)

As idrive uses s3, we also need to consider the s3-specific quirks related to modtime. (For example, rule out --use-server-modtime as a possible factor.) If possible, it would be really helpful to confirm whether we can reproduce this bug with a non-s3-based backend, to determine whether the problem is specific to s3 in some way. (For example, I bisync with drive and the only conflicts I ever see are the ones I expect to see.) Likewise, it would be helpful to remove hasher from the equation, if possible -- the goal is to narrow this down to the bare minimum repro case.

@saltysyncs

The best time to test this is when I am developing but I also want to make sure I do not lose any work, so if I add --conflict-resolve path2 and as a precaution conflict-loser idrive-sync-hasher:/conflicts/ will this mean that the conflict files I see now will populate that directory, or will setting path2 as the resolve option mean they are never created in the first place?

The only valid values for --conflict-loser are num, pathname, and delete, so the above example would give you a syntax error. I think maybe you are thinking of --conflict-suffix, but this still only changes the leaf -- it does not move the file to a new directory. If I understand your goal correctly, you could accomplish something similar by using --backup-dir1 and --backup-dir2 in combination with --conflict-loser delete. But to be honest, I think you would be better off doing your testing in a new environment with dummy files, instead of in production with files you care about. I totally get the convenience factor since your existing setup appears to produce the bug easily -- but I think at some point the needs of your production environment will inevitably be at odds with the debugging goal of stripping this down to the fewest possible settings. And, ultimately we do need to find a way to reproduce this outside of your exact existing setup (because if we can't, then we haven't ruled out that something about your setup may be causing it.)

@saltysyncs

I will settle for always choosing the local file to resolve the issue with locally generated conflicts, but ideally it would work something like this:

That's an interesting idea. To rephrase it, I think what you're suggesting (correct me if I'm wrong) is a new option that usually acts like --conflict-resolve newer when possible, but falls back to --conflict-resolve path2 as a tie-breaker when a winner cannot be determined by looking at modtime. I think this is worth considering -- feel free to submit a Feature Request for it on GitHub, if you would like.