Feature Request: make `--track-renames-strategy modtime` revert back in the case of ambiguity

Feature
1

As I continue my quest to make sure I understand rclone's behaviors, I have been playing with --track-renames-strategy modtime (since I will be syncing to an encrypted remote). I noticed in testing that in the face of ambiguity (more than one file with the same size and modtime), rclone will pick one (the first?).

Personally, in the face of ambiguity, I would prefer that rclone just decide it can't track that rename and do a new transfer + delete. It will alleviate some of the unsafeness of this strategy.

Would it be possible to add a flag to do that? Actually, I would say the default should be to not pick one and instead print a warning and then not track that rename.

(If spent some time analyzing my files and found that I had a lot of files with the exact size and modtime. Turns out they were from a macOS sparsediskbundle. Some common operations on them set all bands to the same modtime and by construction, all bands have the same size. Furthermore, it can get really nasty if they get messed up).

Thanks!

2

Had you run rclone dedupe to get rid of all the duplicates? I'm interested in this too, since I plan to keep a crypted remote in sync with my non encrypted remote, all server-side

3

:frowning:

That seems like a sensible thing to do to me.

It would be a reasonably easy change - this bit of code would need to count the number of matches and if > 1 bomb out.

Fancy having a go?

4

Aw man, I suspect that would be your answer. I really need to learn Go just so I can do this kind of thing.

Maybe I'll take a look this weekend if the baby naps well and long enough to try :slight_smile:

Otherwise, if anyone else wants to do it, feel free.

I am guessing, based on what I gather from this code and the descriptions as it comes out, that dst.ModTime is some kind of mapping. But here would be my best guess:

if s.trackRenamesStrategy.modTime() {
    i = -1
    c = 0
    srcModTime := src.ModTime(s.ctx)
    for j, dst := range dsts {
        dstModTime := dst.ModTime(s.ctx)
        dt := dstModTime.Sub(srcModTime)
        if dt < s.modifyWindow && dt > -s.modifyWindow {
            i = j
            c++
        }
    }
    // If nothing matched or more than one then return nil
    if (i < 0 || c > 1) {
        return nil
    }
}

(Again, I do not know golag, but I am not sure what is returned otherwise, it breaks but that's it)

5

Btw, if you want a immediate solution, you can use touch on unix to change the modification time of your files.

You could also use that to set the same modification time for identical files, for testing

6

This was actually my plan. Before the initial backup, i was going to go through and find all same size and modtime files and then randomly perturb them (in python). That would seed my initial backup with unique files.

The problem is, it does not stop it from happening again. Sure, I could run all kinds of scripts to figure out what is new and what has matching times and then again change the mtime but that is also very, very hacky! I am certainly not opposed to hacks but I would prefer to avoid it.

7

Very good. I'd score you 9 out of 10 for that!

You need to initialize the c variable c := 0 and then I think it will work.

I'd probably put a log message saying that it was refusing to guess as there were multiple matches.

8

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.