Icloud to local: corrupted on transfer: sizes differ

[ncw (Nick Craig-Wood)]

Great investigation @jwink3101

@jwink3101

My guess: The API is reporting to rclone the size of the accumulated files stored in a zip file but, as expected, when you download the file, which is just a zip under the hood, you get the correct zipped up file.

OK that is interesting.

This is reminscent of the trouble we had with heic image files - the backend reports one size but the files are actually another.

Copy jpg & heic files recopied due to Sizes differ

What is the problem you are having with rclone? Copying from Onedrive there are a number of jpg and heic files that recopy everytime due to the sizes being different. These files havnt been changed and some go back 3 years. This is only happening on 30 odd files. Re-running rclone command minutes after the last run completes and it reports the sizes still differ. Run the command 'rclone version' and share the full output of the command. Docker rclone\rclone:latest (Not sure how to find the …

@jwink3101

I see six places in the log that look like binary but start with PK. Apple Numbers uses a zip-based format (like so many tools) so I extracted that chunk in a quick Python script to pull them. They are all 63330 byes (but I would take that with a grain of salt. I could be added with my rough script to pull it and/or missing the end). But even with that grain of salt, they are on the order of 63753. (Encoding as gzip wouldn't start with PK, right? It would be \x1F\x8B\x08)

Yes you are correct the magic for .zip files is "PK\x03\x04" whereas the magic for gzip files is "\x1f\x8b" as you said.

My first thought was maybe this is a content-encoding: gzip mixup, but it doesn't look like it is from that.

If you can make a nice repeatable test then you can put that into a github issue and we can ask @lostb1t to take a look (they aren't on the forum).