I'm glad to report that a really long run for rclone has finished here a few days ago:
rclone -v --low-level-retries=1000 copy egd: egdl:
2018/05/29 17:21:27 INFO :
Transferred: 14886.291 GBytes (717.001 kBytes/s)
Elapsed time: 6047h20m2.4s
2018/05/29 17:21:49 INFO : Encrypted drive 'egdl:': Waiting for checks to finish
2018/05/29 17:21:49 INFO : Encrypted drive 'egdl:': Waiting for transfers to finish
2018/05/29 17:21:49 ERROR : Attempt 3/3 failed with 15 errors and: failed to open source object: bad response: 500: 500 Internal Server Error
2018/05/29 17:21:50 Failed to copy: failed to open source object: bad response: 500: 500 Internal Server Error
As you can see above, it has taken more than 6047/24= 251 days (ie, over 8 months) to run; it was a copy of one of my entire Google Drive remotes to a different one.
Just 15 errors on this entire set of over 34M files is what I' d call an accomplishment, given Google Drive vagaries. Worth of note is also the fact that I've monitored that rclone process during the entire time it ran, and it always kept a reasonable memory footprint (as reported by ps auwx VSIZ and RSS fields), so I'd bet rclone has no memory leaks.
Thanks again to @ncw for this amazing tool! rclone is certainly one of the programs I use most, right after the Linux kernel, screen, bash and vi!
I can say simply, that we run rclone on over 1200 clients PC’s to backup their inventory images and keep them synced to our GCS Bucket. Everyone uploads daily, sending a config and a key to the folder while it runs, and then deleting it afterwards.
rclone helped so much, I cant thank the developers everyone enough for this fine piece of code. It’s always in my toolbox when the time comes.