Having an issue while syncing two local copies

Help and Support
1

I have two external HDDs which I want to contain the same data. I use one as a master image and then every so often I mirror it to the other HDD using rclone. All of the data copied over fine originally but when I try to do a sync of the master to the backup it updates the modification time of the backup files, as they’re all out by 1 second. This isn’t so bad, but it isn’t actually updating the times, and so if I run the same sync command again straight away it will go through and “update” all of the modification times again, but again without actually changing anything.

I have ran the command using the “-vv” parameter and I can see it thinks the times are different and that it updates. I saved them to a file but they’re on my other PC so haven’t copied them here - I can provide them if needed.

If this is a bug, is there any way I can get around it?

Thanks

2

Strange… Which OS are you using? And which file systems are you using? I suspect your external filesystem might be FAT32

Anyway, you want this flag

  --modify-window duration            Max time diff to be considered the same (default 1ns)

Probably setting it to 2s will do the trick!

3

I am running the sync using Windows 7. I have managed to get the two logs I created and put them here:

https://pastebin.com/DFcNbqme
https://pastebin.com/HuBDuPSU

You can see in the command that I ran the same thing just a minute or two apart and it did the exact same thing both times.

Thanks for providing that parameter. I’ll give it a go. Would be good to not have to do it though so I can be certain that the two directories are exactly the same.

4

Unfortunately not all filing systems have the same time resolution. Rclone uses some heuristics to guess what to use, but it doesn’t always get it right. There isn’t an interface to read it from the filesystem and the only way rclone could tell would be to write files to the filesystem which isn’t desirable. Hence the --modify-window flag, which I copied from rsync which has exactly the same problem.

If you want you can use the --checksum flag which will ignore the timestamps and check the actual contents of the files.

5

OK, understood. I also saw that Microsoft’s robocopy has a flag to allow a 2s difference when doing a sync and so it does seem to be a fairly common solution.

Thanks for your help. Much appreciated.

1 Like
6

FWIW, FAT filesystems only store timestamps to 2s accuracy.

See also What Does Modify Window 1ms Mean?

1 Like