Modtime vs checksum transfering from Unencrypted Google to Encrypted Google


#1

Doing a copy or sync from Unencrypted Google to Encrypted Google I’m getting so unexpected results:

Normal:
Checks: 84996
Transferred: 75104
Elapsed time: 30m20.7s

–checksum:
Checks: 84997
Transferred: 24834
Elapsed time: 29m52.2s

I believe this is due to the same files have different modtimes on the source and destination. That’s fine I just use checksum. But after running a full sync with --checksum I expected rclone to update the modtimes on the destination so I don’t need to use checksum anymore, but it did not.


#2

Dry run with and without checksum.

rclone copy UNENCRYPTED:/Miscellaneous/ ENCRYPTED:/Miscellaneous/ –dry-run --checksum
Transferred: 0 Bytes (0 Bytes/s)
Errors: 0
Checks: 2918
Transferred: 0
Elapsed time: 10.5s

rclone copy UNENCRYPTED:/Miscellaneous/ ENCRYPTED:/Miscellaneous/ –dry-run
Transferred: 0 Bytes (0 Bytes/s)
Errors: 0
Checks: 2918
Transferred: 418
Elapsed time: 10.1s

Running a full sync with checksum.

rclone copy UNENCRYPTED:/Miscellaneous/ ENCRYPTED:/Miscellaneous/ –checksum
Transferred: 0 Bytes (0 Bytes/s)
Errors: 0
Checks: 2918
Transferred: 0
Elapsed time: 9.8s

Here I expected both folders to be 100% synced.
But running the command again without checksum it still wants to copy thoose files.

rclone copy UNENCRYPTED:/Miscellaneous/ ENCRYPTED:/Miscellaneous/
Transferred: 0 Bytes (0 Bytes/s)
Errors: 0
Checks: 2918
Transferred: 418
Elapsed time: 9.9s


#3

You should always get the same number regardless of what checking mechanism you use.

I suspect duplicates on drive…

Have you run rclone dedupe on the drive?


#4

Yes, no dupe issues. It’s definitive not updating the modtime when finding a match with checksum doing copy or sync. Running this modified version (Only update mod-time without copying file) fixed my differentiating modtimes for now but I think the official one should do it as well.


#5

Sorry, I misunderstood what you were saying - I thought you were worried about the different number of checks in your first posting.

As you noted running with --checksum doesn’t update mod-times - it only checks the checksum and ignores the mod times completely. This is because reading mod times on some remotes is expensive (takes another round trip) so you don’t want to do it by default.

I think what you want is https://github.com/ncw/rclone/issues/1505 - perhaps you’d like to add your input there?


#6

That is excactly what I’m was looking for. +1 :slight_smile: