Rclone copy not copying every file?

[samsepiol59]

2019/06/15 15:15:14 INFO  : rInstaloader_backup/jaime_king/jaime_king - 2019-06-14_22-59-24 - BytTNGcAUAd.mp4: received cache expiry notification
2019/06/15 15:15:14 DEBUG : rInstaloader_backup/jaime_king: cache: expired rInstaloader_backup/jaime_king
2019/06/15 15:15:14 DEBUG : rInstaloader_backup/jaime_king: cache: expired rInstaloader_backup
2019/06/15 15:15:14 DEBUG : rInstaloader_backup/jaime_king: cache: expired
2019/06/15 15:15:14 DEBUG : rInstaloader_backup/jaime_king/jaime_king - 2019-06-14_22-59-24 - BytTNGcAUAd.mp4: notify: expired 'rInstaloader_backup/jaime_king'

I also keep getting these kind of debugs in my rclone cache mount screen... could they be related to the missing files problem?

I've uploaded 250k+ (small) files to gdrive in 16 hours with rclone copy, but when I check the source folder on my box it says there are 260k+ files, I'm missing around 10k files somewhere could these messages be related to that??

[ncw (Nick Craig-Wood)]

I've uploaded 250k+ (small) files to gdrive in 16 hours with rclone copy, but when I check the source folder on my box it says there are 260k+ files, I'm missing around 10k files somewhere could these messages be related to that??

Probably not...

Run rclone check to see what files are missing, then run the rclone copy again (it won't upload the files already there) to upload the missing ones.

[samsepiol59]

I've tried doing the rclone copy again, but nothing new uploaded! So I'm doing the checking now, with the --one-way flag

[samsepiol59]

@ncw This is the result of rclone check /home/samsepiol/Downloads/instaloader/ gdrive:rInstaloader_backup --user-agent="Mozilla/5.0" --one-way. Still doesn't explain the ~10k difference in files though... I'm at a loss!

2019/06/15 15:45:57 NOTICE: bluthcompanyofficial/bluthcompanyofficial - 2016-04-10_00-15-31 - BD_8_YvtB4L.jpg: Duplicate object found in destination - ignoring
2019/06/15 15:54:03 NOTICE: gigihadid/gigihadid - 2019-02-05_18-14-01 - Btgn-nVH3lX_9.jpg: Duplicate object found in destination - ignoring
2019/06/15 16:06:01 NOTICE: lilyaldridge/lilyaldridge - 2019-04-28_12-15-30 - BwzIIj5nhJT_3.jpg: Duplicate object found in destination - ignoring
2019/06/15 16:06:17 NOTICE: lilyjcollins/lilyjcollins - 2014-01-26_00-12-29 - jnIuzHMLjl.jpg: Duplicate object found in destination - ignoring
2019/06/15 16:31:47 NOTICE: Google drive root 'rInstaloader_backup': 0 differences found```

[samsepiol59]

I don't understand... I'm getting different numbers at every check!

[calisro (Rob)]

It is showing duplicates... Run rclone dedupe to fix the duplicates and then you can resync.