Rclone confusing Calibre via Sugarsync in a weird way

Looking at this:

*.db files are not supported

And you said:

It let me think how it worked before...

Maybe *.db files were never synced before? You only synced books' files and every windows Calibre maintained its own database?

Another thing:

Have a look at timestamps. Windows one is later than Linux meaning that db file has changed since it was synced. Maybe something is accessing it and modifying?

Doh!!!! :thinking:
What a weird weird thing... why explictly files with the .db extension???

No, that is not the case. Albeit on Windows, the workflow has always been the same:

  1. upload everything to Sugarsync (well, it's automatic)
  2. In the new machine: sync the library directory (all files)
  3. Open calibre and let it point to the library directory.

The same I did here, even though I used rclone to sync the library directory.
By the way, metadata.db is the "core" of the calibre library: if you remove it from the directory, calibre would not even recognize there is a library there.

Furthermore, if that was the case, as the Windows version in the new machine was able to build the book list, so the Linux version should...

Yeah, I noticed that one, but since on Sugarsync web there is no time, and rclone also doesn't retrieve it from Sugarsync, there is no conclusive proof.
Actually, no, there is no "something": calibre manages that file exclusively, and touches it (modifies and saves it) every time you close the program. I know it for a fact because you can see it being uploaded to the cloud each time you close Calibre. But then, it is left alone.

But when you retrieve it in Linux, it has that different?, wrong?, weird? time stamp.
I didn't point it out to you because I read in rclone documentation that timestamps were not treated correctly when working with Sugarsync in general, so we couldn't use that to debug metadata.db specifically

Going to open a support ticket to Sugarsync, then... let's see what they say.
BTW, each single book directory has a (much smaller) metadata.db file... and the ones that I checked showed the same size of their windows counterpart, so not ALL the .db files are managed incorrectly...

Alright kapitainsky, I think we can close the topic.
I feel a little ashamed writing this, and so so sorry for having wasted your time but... not only I didn't notice that Sugarsync disclaimer on their page but... you know what?

I tried to download metadata.db via web from SS account webpage on my Windows machine and... that way also the file is broken!!! :face_with_peeking_eye:

So:

  • You upload the file from Windows to cloud (via their sync agent) -> Ok

  • You download the file from the cloud to Windows - and supposedly, to Mac - via their sync agent -> Ok

  • You download the file from the web to either Win or Linux -> KO, you get served a shorter file, with the right format, it is a sqlite data file, but truncated at a certain point.

The reason rclone is showing the same issue a web browser has, is probably because internally it is using the same methods to access files, some http get request probably... and thus it gets served the same file the browser received.

I am really so sorry to have wasted your time, to me it was somewhat funny to have debugged it, but I could have though earlier to run these tests, and you could have probably spent better this day I took you. Sorry!!! :worried:

No worries. I still think it would be nice to get to the bottom of this:)

Oh, so much! Anyway, it has nothing to do with rclone (directly).
I'll keep you posted when I receive an answer from SugarSync support.

1 Like

For your info, I have got a solution... to the mistery, at least.

Ssync support tried to help, but was not finding a solution, nor the error tbh.
So, in order to help I "touched" the file in my windows pc, unpaused the SSync client and let it update the cloud, which it said it did...

But then I checked online and metadata.db had an 11 days old timestamp!
We never realized that, but it turned out that the file in the cloud was always dated 19/11.
It became apparent this afternoon when the timestamp on my local pc was, of course, 27/11.

So, I made a backup copy of the local file in another directory.
I then removed the file from the cloud (which in turn, as expected, removed the "newer" counterpart in my pc) and restored the local file.
This time it got synchronized to SugarSync cloud correctly and it is also synced correctly by rclone.

So: no bug on rclone, not that bug on Sugarsync, but possibly an even worse bug there: an old version of a file got "stuck" in the cloud, the client was saying it had been synchronized while it was not.
This is really bad because if you trust the cloud to be in sync and use it for backup purposes, or to move data to another pc, you risk losing data...

Anyway, mistery solved, at least!

2 Likes