Several OneDrive files are detected as different size than they are in OneDrive remote, causing redundant upload of files which are identical with remove. Not all files, just some of them.
Run the command 'rclone version' and share the full output of the command.
rclone v1.60.1-DEV
os/version: ubuntu 24.10 (64 bit)
os/kernel: 6.11.0-18-generic (x86_64)
os/type: linux
os/arch: amd64
go/version: go1.22.2
go/linking: dynamic
go/tags: none
Which cloud storage system are you using? (eg Google Drive)
OneDrive
The command you were trying to run (eg rclone copy /tmp remote:tmp)
[tonionedrive]
type = onedrive
token = redacted
drive_id = redacted
drive_type = personal
A log from the command with the -vv flag
2025/03/16 09:18:41 DEBUG : rclone: Version "v1.60.1-DEV" starting with parameters ["rclone" "copy" "-vv" "--dry-run" "--log-file" "rclone_sync_after_1.log" "OneDrive/" "tonionedrive:"]
2025/03/16 09:18:41 DEBUG : Creating backend with remote "OneDrive/"
2025/03/16 09:18:41 DEBUG : Using config file from "/home/toni/.config/rclone/rclone.conf"
2025/03/16 09:18:41 DEBUG : fs cache: renaming cache item "OneDrive/" to be canonical "/home/toni/OneDrive"
2025/03/16 09:18:41 DEBUG : Creating backend with remote "tonionedrive:"
[...]
2025/03/16 09:18:52 DEBUG : Autot/Corolla/2024-06-06/20240606_220055230_iOS.heic: Sizes differ (src 2387308 vs dst 5674061)
2025/03/16 09:18:52 NOTICE: Autot/Corolla/2024-06-06/20240606_220055230_iOS.heic: Skipped copy as --dry-run is set (size 2.277Mi)
2025/03/16 09:18:52 DEBUG : Autot/Corolla/2024-06-06/20240606_220108175_iOS.heic: Sizes differ (src 3121996 vs dst 7938028)
2025/03/16 09:18:52 NOTICE: Autot/Corolla/2024-06-06/20240606_220108175_iOS.heic: Skipped copy as --dry-run is set (size 2.977Mi)
2025/03/16 09:18:52 DEBUG : Autot/Corolla/2024-06-06/20240606_220121250_iOS.heic: Sizes differ (src 3076952 vs dst 6705171)
2025/03/16 09:18:52 NOTICE: Autot/Corolla/2024-06-06/20240606_220121250_iOS.heic: Skipped copy as --dry-run is set (size 2.934Mi)
2025/03/16 09:18:52 DEBUG : Autot/Corolla/2024-06-06/20240606_220135515_iOS.heic: Sizes differ (src 2542623 vs dst 6315989)
2025/03/16 09:18:52 NOTICE: Autot/Corolla/2024-06-06/20240606_220135515_iOS.heic: Skipped copy as --dry-run is set (size 2.425Mi)
[...]
2025/03/16 09:20:27 DEBUG : OneDrive root '': Waiting for transfers to finish
2025/03/16 09:20:27 NOTICE:
Transferred: 4.982 GiB / 4.982 GiB, 100%, 117.352 MiB/s, ETA 0s
Checks: 92388 / 92388, 100%
Transferred: 2458 / 2458, 100%
Elapsed time: 1m45.9s
2025/03/16 09:20:27 DEBUG : 5 go routines active
[09:26] toni@purkki:~$ rclone lsl tonionedrive:Autot/Corolla
5674061 2024-06-06 15:04:10.183000000 2024-06-06/20240606_220055230_iOS.heic
7938028 2024-06-06 15:04:12.823000000 2024-06-06/20240606_220108175_iOS.heic
6705171 2024-06-06 15:04:12.446000000 2024-06-06/20240606_220121250_iOS.heic
6315989 2024-06-06 15:04:11.570000000 2024-06-06/20240606_220135515_iOS.heic
When downloading the file from OneDrive to my computer they are the exact same size, and the local size detection is correct, it's the OneDrive remote detection which is incorrect:
Downloads % ls -l
total 9328
-rw-r--r--@ 1 toni staff 2387308 Mar 16 09:26 20240606_220055230_iOS_local.heic
-rw-r--r--@ 1 toni staff 2387308 Mar 16 09:25 20240606_220055230_iOS_onedrive.heic
Downloads %
Thank you so much for the prompt reply! I put in a little donation if that matters. rclone is fantastic software!
I'll do the updating soon and try again.
I was aware of the live photos topic (should have mentioned it) and tried debugging whether it applies to my issue but am not sure how to see. All of the images where this happens are indeed .heif or .heic files, but how do I see if they're specifically the type of live photos mentioned in the issue? Should I check the size of the file in the OneDrive web interface somewhere, and compare to downloaded size? Or what?
With the same command, I also have what appears to be a completely different issue. Some files don't seem to be detected at all, even though rclone lsl shows them.
log file:
2025/03/16 09:18:43 DEBUG : 2022 taxes/Verotuspaatos (pdf)_11.09.2023.pdf: Unchanged skipping
2025/03/16 09:18:43 NOTICE: 2022 taxes/20230131_010631553_iOS.heic: Skipped copy as --dry-run is set (size 3.388Mi)
2025/03/16 09:18:43 NOTICE: 2022 taxes/20230131_010817403_iOS.heic: Skipped copy as --dry-run is set (size 2.903Mi)
2025/03/16 09:18:43 DEBUG : 2022 taxes/Your TurboTax Easy Extension has been approved.pdf: Size and modification time the same (differ by 0s, within tolerance 1s)
[16:28] toni@purkki:~$ rclone lsl tonionedrive:"2022 taxes/" | grep 20230131
6499139 2023-01-30 17:08:45.643000000 20230131_010631553_iOS.heic
6590543 2023-01-30 17:08:45.646000000 20230131_010817403_iOS.heic
[16:28] toni@purkki:~$
So for some reason rclone seems to not detect the file at all, but lsl shows them correctly. How can I debug?
So as you can see from the log, rclone wants to upload the file and doesn't even compare it to the similar file that's in the remote. There's no message saying "comparing 20230131_010631553_iOS.heic to remote, sizes differ", it just straight away says it's trying to copy it. I assume this indicates rclone doesn't see the remote file.
However, when I run rclone lsl, it clearly shows the file in the listing.
There's no indication why it wants to copy 20230131_010631553_iOS.heic, even though it exists in the destination.
On the other hand given that it looks like everything else works but iOS images files you could assume that OneDrive has some issue here - maybe there is more than live photos one mentioned in docs.
Unfortunately OneDrive is not a dumb storage but some service Microsoft offers. They mess up with content sometimes and it is beyond rclone control.
I suggest you use crypt remote to limit their chances. But even this might not be enough for Microsoft to stop changing your things - example here.
Overall if you need dumb storage than buy dumb storage service somewhere. If you buy MS Office/AI/everything else offering then enjoy it but do not expect it to do something else (even if as simple as storing your files)
