Backend iCloud Photos not listing all files (difference between cold listing and parallel fetch count)

ilGaspa · September 20, 2026, 10:03am

What is the problem you are having with rclone?

I have a medium sized library of 10.000 photos and a couple hundred albums. Lately, when listing the album contents using rclone, large chunks of files are missing, seemingly at random, with no specific reason. The files are always the same ones but there's nothing common in them. The files are available when using the icloud web gui, correctly sync between devices, are not corrupted and can be downloaded from the web or from the devices.

The logs I'm attaching are restricted to one of the folders, the correct file count should be 68 and the cold listing is correct.. however parallel fetch only retrieves 66 files.

The symptom is always the same: cold listing correctly counts the files, however parallel fetch misses some. Extended to the whole library, 15% of the files are missing.

2026/09/20 11:56:00 NOTICE: iclouddrive photos: parallel cold listing 68 photos in 1 partitions (1 workers)
2026/09/20 11:56:01 NOTICE: iclouddrive photos: parallel fetch complete, 66 photos

The files are not from bursts or animated photos: the missing ones are standard files like the ones shown, with no reasons to be excluded for.

Already attempted to:

Reducing the number of checkers, transfers, and tps
Clearing the cache and re-authenticating as well
Using a different build of rclone on a different machine
Using the latest beta version

RcloneView, which uses Rclone as a base, correctly shows all the files in the same album, so I have no clue about what could be wrong in my standalone rclone usage.

I don't know if this could be a bug or not, and if with rclone or with Apple APIs. It's probably something related to my specific photo library structure. I would really appreciate if someone could point me towards a solution.

RCloneView correctly lists all the files, so I suspect there's a setting or workaround I'm missing.

Thanks in advance! Of course if there's anything I can do to debug on my side just point me towards what I should do. Thanks!

Run the command 'rclone version' and share the full output of the command.

The following is from my main server, but the same happens on generic Windows installations.

[core-ssh rclone]$ ./rclone version
rclone v1.75.1
- os/version: alpine 3.24.1 (64 bit)
- os/kernel: 6.18.52-haos (aarch64)
- os/type: linux
- os/arch: arm64 (ARMv8 compatible)
- go/version: go1.26.8
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

iCloud Photos or iCloud Drive in photos service mode

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rm -rf  ~/.cache/rclone/
./rclone tree --iclouddrive-service photos -vvvvv --checkers 1 --transfers 1 --max-backlog 100000 "iclouddrive:/PrimarySync/2006/Cena di Natale RibesTecnologie (2006)" > out.txt 2>&1

The rclone config contents with secrets removed.

Nothing special here save my account details that I have to remove

[iclouddrive]
type = iclouddrive
apple_id = 
password = 
_auth_session = 
cookies = 
trust_token =

A log from the command with the `-vv` flag

2026/09/20 11:55:43 DEBUG : rclone: Version "v1.75.1" starting with parameters ["./rclone" "tree" "--iclouddrive-service" "photos" "-vvvvv" "--checkers" "1" "--transfers" "1" "--max-backlog" "100000" "iclouddrive:/PrimarySync/2006/Cena di Natale RibesTecnologie (2006)"]
2026/09/20 11:55:43 DEBUG : Creating backend with remote "iclouddrive:/PrimarySync/2006/Cena di Natale RibesTecnologie (2006)"
2026/09/20 11:55:43 DEBUG : Using config file from "/homeassistant/utils/rclone/rclone.conf"
2026/09/20 11:55:43 DEBUG : iclouddrive: detected overridden config - adding "{Ub7a0}" suffix to name
2026/09/20 11:55:44 DEBUG : iclouddrive: valid session, no need to reauth
2026/09/20 11:55:46 DEBUG : iclouddrive photos: refreshed 1 in-memory libraries from API
2026/09/20 11:55:59 DEBUG : iclouddrive photos: refreshed 1 in-memory libraries from API
2026/09/20 11:55:59 DEBUG : fs cache: renaming cache item "iclouddrive:/PrimarySync/2006/Cena di Natale RibesTecnologie (2006)" to be canonical "iclouddrive{Ub7a0}:PrimarySync/2006/Cena di Natale RibesTecnologie (2006)"
2026/09/20 11:55:59 DEBUG : iclouddrive photos: refreshed 1 in-memory libraries from API
2026/09/20 11:56:00 NOTICE: iclouddrive photos: parallel cold listing 68 photos in 1 partitions (1 workers)
2026/09/20 11:56:01 NOTICE: iclouddrive photos: parallel fetch complete, 66 photos
2026/09/20 11:56:01 DEBUG : Stat: filePath="/"
2026/09/20 11:56:01 DEBUG : >Stat: fi=, err=<nil>
2026/09/20 11:56:01 DEBUG : ReadDir: dir=/
2026/09/20 11:56:01 DEBUG : >ReadDir: names=[20061124_215106.jpg 20061124_215141.jpg 20061124_215258.jpg 20061124_215325.jpg 20061124_215446.jpg 20061124_215459.jpg 20061124_215521.jpg 20061124_215533.jpg 20061124_215603.jpg 20061124_215633.jpg 20061124_215648.jpg 20061124_215705.jpg 20061124_215832.jpg 20061124_215910.jpg 20061124_215926.jpg 20061124_220017.jpg 20061124_220110.jpg 20061124_220158.jpg 20061124_220445.jpg 20061124_220506.jpg 20061124_220519.jpg 20061124_220540.jpg 20061124_223435.jpg 20061124_223507.jpg 20061124_223619.jpg 20061124_223720.jpg 20061124_223840.jpg 20061124_223926.jpg 20061124_224134.jpg 20061124_224631.jpg 20061124_224650.jpg 20061124_224657.jpg 20061124_224704.jpg 20061124_224746.jpg 20061124_224928.jpg 20061124_225731.jpg 20061124_225802.jpg 20061124_230231.jpg 20061124_230717.jpg 20061124_230847.jpg 20061124_230930.jpg 20061124_231025.jpg 20061124_231209.jpg 20061124_231249.jpg 20061124_231343.jpg 20061124_231411.jpg 20061124_231438.jpg 20061124_231849.jpg 20061124_232109.jpg 20061124_232130.jpg 20061124_232147.jpg 20061124_232859.jpg 20061124_232945.jpg 20061124_233010.jpg 20061124_233029.jpg 20061124_233043.jpg 20061124_233055.jpg 20061124_233115.jpg 20061124_233220.jpg 20061124_233403.jpg 20061124_233455.jpg 20061124_233613.jpg 20061124_234636.jpg 20061124_234750.jpg 20061125_000848.jpg 20061125_000918.jpg], err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_215106.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_215106.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_215141.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_215141.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_215258.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_215258.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_215325.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_215325.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_215446.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_215446.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_215459.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_215459.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_215521.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_215521.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_215533.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_215533.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_215603.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_215603.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_215633.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_215633.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_215648.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_215648.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_215705.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_215705.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_215832.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_215832.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_215910.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_215910.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_215926.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_215926.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_220017.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_220017.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_220110.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_220110.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_220158.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_220158.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_220445.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_220445.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_220506.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_220506.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_220519.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_220519.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_220540.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_220540.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_223435.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_223435.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_223507.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_223507.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_223619.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_223619.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_223720.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_223720.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_223840.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_223840.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_223926.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_223926.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_224134.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_224134.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_224631.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_224631.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_224650.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_224650.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_224657.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_224657.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_224704.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_224704.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_224746.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_224746.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_224928.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_224928.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_225731.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_225731.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_225802.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_225802.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_230231.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_230231.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_230717.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_230717.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_230847.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_230847.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_230930.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_230930.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_231025.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_231025.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_231209.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_231209.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_231249.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_231249.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_231343.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_231343.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_231411.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_231411.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_231438.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_231438.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_231849.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_231849.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_232109.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_232109.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_232130.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_232130.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_232147.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_232147.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_232859.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_232859.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_232945.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_232945.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_233010.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_233010.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_233029.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_233029.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_233043.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_233043.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_233055.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_233055.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_233115.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_233115.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_233220.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_233220.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_233403.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_233403.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_233455.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_233455.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_233613.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_233613.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_234636.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_234636.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061124_234750.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061124_234750.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061125_000848.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061125_000848.jpg, err=<nil>
2026/09/20 11:56:01 DEBUG : Stat: filePath="/20061125_000918.jpg"
2026/09/20 11:56:01 DEBUG : >Stat: fi=20061125_000918.jpg, err=<nil>
/
├── 20061124_215106.jpg
├── 20061124_215141.jpg
├── 20061124_215258.jpg
├── 20061124_215325.jpg
├── 20061124_215446.jpg
├── 20061124_215459.jpg
├── 20061124_215521.jpg
├── 20061124_215533.jpg
├── 20061124_215603.jpg
├── 20061124_215633.jpg
├── 20061124_215648.jpg
├── 20061124_215705.jpg
├── 20061124_215832.jpg
├── 20061124_215910.jpg
├── 20061124_215926.jpg
├── 20061124_220017.jpg
├── 20061124_220110.jpg
├── 20061124_220158.jpg
├── 20061124_220445.jpg
├── 20061124_220506.jpg
├── 20061124_220519.jpg
├── 20061124_220540.jpg
├── 20061124_223435.jpg
├── 20061124_223507.jpg
├── 20061124_223619.jpg
├── 20061124_223720.jpg
├── 20061124_223840.jpg
├── 20061124_223926.jpg
├── 20061124_224134.jpg
├── 20061124_224631.jpg
├── 20061124_224650.jpg
├── 20061124_224657.jpg
├── 20061124_224704.jpg
├── 20061124_224746.jpg
├── 20061124_224928.jpg
├── 20061124_225731.jpg
├── 20061124_225802.jpg
├── 20061124_230231.jpg
├── 20061124_230717.jpg
├── 20061124_230847.jpg
├── 20061124_230930.jpg
├── 20061124_231025.jpg
├── 20061124_231209.jpg
├── 20061124_231249.jpg
├── 20061124_231343.jpg
├── 20061124_231411.jpg
├── 20061124_231438.jpg
├── 20061124_231849.jpg
├── 20061124_232109.jpg
├── 20061124_232130.jpg
├── 20061124_232147.jpg
├── 20061124_232859.jpg
├── 20061124_232945.jpg
├── 20061124_233010.jpg
├── 20061124_233029.jpg
├── 20061124_233043.jpg
├── 20061124_233055.jpg
├── 20061124_233115.jpg
├── 20061124_233220.jpg
├── 20061124_233403.jpg
├── 20061124_233455.jpg
├── 20061124_233613.jpg
├── 20061124_234636.jpg
├── 20061124_234750.jpg
├── 20061125_000848.jpg
└── 20061125_000918.jpg

0 directories, 66 files
2026/09/20 11:56:01 DEBUG : 7 go routines active

nite_route · September 20, 2026, 10:14am

cold listing counting right and parallel fetch dropping a couple usually means one of the parallel startRank pages came back short, not that apple deleted those files.

stuff that has helped me:
rm -rf ~/.cache/rclone/iclouddrive-photos/
rclone config reconnect YOURREMOTE
then list that album once with -vv and watch the parallel fetch NOTICE line.

RcloneView showing everything is a clue their wrapper is on a warmer/cached path. checkers/transfers knobs probably wont fix a listing gap. if wipe+reconnect still misses the same IDs on parallel fetch, worth a github issue with those 68 vs 66 NOTICE lines.

ilGaspa · September 20, 2026, 12:05pm

Thanks! I tried again by combining everything in one run:

deleted the cache folder
reconnected the remote
launched the command with also tpslimit 1

No luck, there's still that difference between cold and parallel, and still on the same files.

2026/09/20 14:04:48 NOTICE: iclouddrive photos: parallel cold listing 68 photos in 1 partitions (1 workers)
2026/09/20 14:04:50 NOTICE: iclouddrive photos: parallel fetch complete, 66 photos

I'll open an issue on github and post the link here.

EDIT: link to the issue on github

https://github.com/rclone/rclone/issues/9955