I’ve been running rclone sync (v1.36) to backup a large set of files, some of which are 20GB+. For the last day or two, it’s been churning on the same 4 files and making no apparent progress. The log file looks like this:
2017/03/21 11:35:18 INFO : Starting bandwidth limiter at 10MBytes/s
2017/03/21 11:35:18 INFO : Encrypted amazon drive root '': Modify window not supported
2017/03/21 11:36:18 NOTICE: Scheduled bandwidth change. Limit set to 10MBytes/s
2017/03/21 11:36:18 INFO :
Transferred: 594.375 MBytes (9.857 MBytes/s)
Elapsed time: 1m0.3s
* <large file 1>: 0% done, 2.095 MBytes/s, ETA: 1h41m59s
* <large file 2>: 0% done, 2.460 MBytes/s, ETA: 1h45m28s
* <large file 3>: 1% done, 2.664 MBytes/s, ETA: 1h11m57s
* <large file 4>: 1% done, 2.663 MBytes/s, ETA: 1h36m59s
These lines are repeated every 60 seconds; the rates are always non-zero but the ETA never goes down and the completion percentage never goes up. What am I doing wrong?
Watching htop in the VM in which I have rclone running, it appears all the rclone processes close and reopen every 60-90 seconds… Not sure if this is expected behavior or if it’s related to the issue above.
I think I got this fixed. The issue was with the systemd unit file I was using to start rclone. I originally had the service defined with type=forking, but that caused the rclone processes to exit after about 60 seconds. I also had restart=always, so they would all start back up again, but begin the upload from scratch. Thus, only files that could be uploaded completely in ~60 seconds would transfer.
I changed the unit file so type=simple and it appears to work fine. Here is the full file (/etc/systemd/system/rclone-backup.service):
Description=rclone ACD data backup
ExecStart=/usr/sbin/rclone sync /media/britlib alexandria: --exclude "acd.log" --bwlimit "08:30,10M 00:30,off" --log-level INFO --log-file /media/britlib/Incoming/Notes/acd/acd.log