Hi there,
We are doing a large sync to Google Drive, and I am wondering if it is possible to get an accurate estimate of the amount of files left to transfer / time remaining? To try and improve things, I added:
--fast-list
which improved things significantly, both in terms of speed and giving a better initial indication of the amount of data needed to be transferred, and while it works well on most transfers, on the large sync we are doing now (~27TB), the amount of data left to transfer still slowly grows over time, making it very difficult to estimate how much longer the transfer will take.
For example, here is a sample of the output of rclone during the sync:
Transferred: 530.056G / 729.765 GBytes, 73%, 8.532 MBytes/s, ETA 6h39m29s
Errors: 19 (retrying may help)
Checks: 1481621 / 1481621, 100%
Transferred: 24135 / 34147, 71%
Elapsed time: 17h40m19.7s
Then two minutes later:
Transferred: 531.109G / 729.768 GBytes, 73%, 8.535 MBytes/s, ETA 6h37m14s
Errors: 19 (retrying may help)
Checks: 1481621 / 1481621, 100%
Transferred: 24143 / 34155, 71%
Elapsed time: 17h42m2.2s
Then some hours later:
Transferred: 1.152T / 1.279 TBytes, 90%, 8.151 MBytes/s, ETA 4h31m56s
Errors: 20 (retrying may help)
Checks: 1557484 / 1557484, 100%
Transferred: 76375 / 86387, 88%
Elapsed time: 41h9m18.2s
If you look closely, you can see under ‘Transferred:’, the total amount of files to be transferred has gone up. Approximately every minute a few files are added to this total.
Or is this behaviour because this sync is from a filesystem that is currently in use, and rclone is updating to reflect the new files being added to the filesystem?