Google drive rclone sync crypt: gdrive: ( two different google accounts )

Suspected Bug
1

Something seems a bit off when syncing from one google drive crypt: to another unecrypted remote.
It seems same files are being replaced multiple times and since crypt does not support rclone check Iam not sure how to ensure consistency between those 2 drives.

By something off it looks like sync is replacing already synced files
Command: /usr/bin/rclone sync gdrivecrypt:/series gdajki:/series -v --transfers=30 --checkers=30 --stats 30s --log-file=/sync/logs/sync.log

p.s. I will try to dedupe gdrive: and gdajki: to see if that will help

2

I think this is probably the same as https://github.com/ncw/rclone/issues/1431 - any help tracking down exactly what is going on much appreciated!

3

Will dedupe work normally on gdrive: where all folder/files are encrypted eg overwriting same file/folder or renaming it will always give same encrypted name ?

4

Let see rclone dedupe gdrive: -v --dedupe-mode newest ( only crypt folder is on that drive. )
So far i have tons of notices about dupes being deleted, hopefully i wont find out that tons of my content is gone.

5

@ncw i cant figure it out, I was trannsfering for a while with -vv to spot if there are some weird errors, but looked normal.
I dedupe both drives ( the one having crypt files removed tons of duplicates ) atm still syncing but rclone size report is beyond weird eg destination being much bigger then source.

[14:22:54] ajki_email@instance-1:/sync$ /sync/rsize
Total objects: 11436
Total size: 37981.348 GBytes (40782162197433 Bytes)
05.06.2017 14:25:40 rclone size gdajki:/movies finished in 690 seconds
Total objects: 8473
Total size: 33884.617 GBytes (36383329956651 Bytes)
05.06.2017 14:25:40 rclone size gdrivecrypt:/movies finished in 786 seconds
Total objects: 29601
Total size: 31595.271 GBytes (33925163774209 Bytes)
05.06.2017 14:25:40 rclone size gdajki:/series finished in 260 seconds
Total objects: 29574
Total size: 31554.202 GBytes (33881066241208 Bytes)
05.06.2017 14:25:40 rclone size gdrivecrypt:/series finished in 263 seconds
05.06.2017 14:25:40 ############# FINISHED IN 1999 SECONDS #############

6

Well it seems sync is finishing now and not copying anymore files ( i need to pause all new uploads for 24h to make sure )

I deduped both drives until there was no duplicates removals.
/usr/bin/rclone dedupe gdrive: -v --dedupe-mode newest --log-file=$LOGFILE
06.06.2017 04:59:08 rclone dedupe gdrive finished in 1103 seconds
/usr/bin/rclone dedupe gdajki: -v --dedupe-mode newest --log-file=$LOGFILE
06.06.2017 04:59:08 rclone dedupe gdajki finished in 655 seconds

Current rclone size

Total objects: 8262
Total size: 33159.372 GBytes (35604604682861 Bytes)
06.06.2017 04:35:26 rclone size gdajki:/movies finished in 653 seconds
Total objects: 8226
Total size: 33018.743 GBytes (35453605169706 Bytes)
06.06.2017 04:35:26 rclone size gdrivecrypt:/movies finished in 677 seconds
Total objects: 29583
Total size: 31554.202 GBytes (33881066640115 Bytes)
06.06.2017 04:35:26 rclone size gdajki:/series finished in 160 seconds
Total objects: 29583
Total size: 31554.202 GBytes (33881066640115 Bytes)
06.06.2017 04:35:26 rclone size gdrivecrypt:/series finished in 529 seconds
06.06.2017 04:35:26 ############# FINISHED IN 2019 SECONDS #############

For some reason movies still do not match and there is nothing weird in logs.

p.s. I switched plex to new unencrypted drive now and will keep syncing it and see what will happen.