Taking a look at recent log files I noticed something strange. Some old files, which had already been synced, are being reported as “copied (new)”. But not even Dropbox shows this new “copy”.
The files are not modified in the source folder, and Dropbox does not inform a new version.
Today log:
2018/04/03 22:26:24 INFO : 04.Biblioteca/BOW TIE/gerenciamento_bowtie.pdf: Copied (new)
What happens if you try rclone check from the source to the dest?
Can you try rclone ls the remote a few times and see if it gives the same results every time? It is as though there are files missing from the directory listing.
I finally got some time to verify the check output to identify the “4 differences”, and see what I found:
Line 6: 2018/04/04 20:11:19 ERROR : .dropbox: File not in Dropbox root '[storage at Dropbox]'
Line 7: 2018/04/04 20:11:19 ERROR : desktop.ini: File not in Dropbox root '[storage at Dropbox]'
Line 8790: 2018/04/04 20:19:25 ERROR : [file1]: DropboxHash differ
Line 10728: 2018/04/04 20:21:45 ERROR : [file2]: DropboxHash differ
.dropbox and desktop.ini are excluded by the --filter-from parameter, and I actually forgot to put it in this specific check script.
These two files above have not been changed recently, the first one had the last change on March 8 and the other in 2017:
The modification dates/times of the files in Dropbox are exactly the same:
What may have happened to change the hash? How can I check this?
And the main doubt: at the beginning of this post I had 106 “differences”. Running sync script several times reduced to only 2. But what is the reason for these “differences”?
From my point of view as a user, all are correct and equal. It is an Excel spreadsheet. The original file opens perfectly, the file downloaded from the backup opens identically (even in the same cell) and this same file opens in Dropbox via the web interface.
As a user, I would not even notice the different hashes.
Yet the mysteries remain: how can they be different if the modification times are the same? Could it be that Dropbox did something? Why only in this file, and a few others?
Anyway, it does not seem to be a problem with Rclone, it’s some exogenous factor.
Rclone is working properly, because the files have the same timestamp but different hashes, and Rclone is not synchronizing them because I don’t use the --checksum option.
I wonder if I should take this option as default …
Well, not exactly, because in this #2218 issue the modification times are different, and in my case (from the start with that “2014” file) the modification times are the same.
But I keep my opinion that in my case it’s not a problem with Rclone, but some external factor.
I’ll leave the job running normally (daily) and in a few days I’ll run a check to see if any new hash issues have come up.
So as I said above, I left the script running daily in these last days.
Today I checked again and:
Line 8804: 2018/04/09 20:12:34 ERROR : [file1]: DropboxHash differ
Line 10751: 2018/04/09 20:16:48 ERROR : [file2]: DropboxHash differ
(The same two files)
This time I’ll do it differently: I’ll delete these 2 files above in Dropbox and see what happens in the next executions of sync. If there were no more errors, then there was some problem with these files (some external cause, Dropbox, etc) and life goes on.
If, on the other hand, new hash errors appear in other files, then I’ll start to get worried.
Note: the files have the same modtime and open perfectly.
Well, unfortunately the hash problem came up in a new file (and did not appear any more for the other two above):
Line 973: 2018/04/13 12:43:58 ERROR : [file 3]: DropboxHash differ
But I think I managed to identify something: all problems occurred with files that are in a folder shared with another Dropbox user. I’ve identified that this new [file 3] with hash issues was recently modified by this user.
What I do not understand is:
Why, after all synchronized and with rclone sync running multiple times, with modtimes and content the same in all locations, there is still a hash difference.
Why other files (a lot) that this other user changed do not present the same problem?
The following is a schematic of the configuration, to clarify:
Anyway, the problem does not disturb my backup (rclone sync), since the new versions are being copied correctly.
The problem is that it generates “false positives” in the check command, so I can not get consistent results for an automated backup verification script.
I’m sure that if I delete the file in the backup folder and reupload it, the hashes will become equal, as the other two previous files.
