I have a personal onedrive setup with an encrypted frontend.
I'm doing an initial copy of my backups to it. And it's mostly working. But partway through...
[OneDriveBackups]
type = crypt
remote = OneDriveSweharris2:Backups
filename_encryption = off
directory_name_encryption = false
password = hahahah
password2 = hahahah
[OneDriveSweharris2]
type = onedrive
token = thatdbetelling
drive_id = anumber
drive_type = personal
Now I'm doing a simple rclone sync . OneDriveBackups: and it's going pretty well.
But I noticed the occasional error:
2020/01/23 18:11:42 INFO :
Transferred: 377.624G / 600.533 GBytes, 63%, 30.980 MBytes/s, ETA 2h2m47s
Errors: 1 (retrying may help)
Checks: 0 / 0, -
Transferred: 1015 / 1150, 88%
Elapsed time: 3h28m1.6s
Transferring:
* penfold/20200105._FastData.0.gz_ak: 63% /5G, 8.982M/s, 3m25s
* penfold/20200105._FastData.0.gz_al: 59% /5G, 9.425M/s, 3m40s
* penfold/20200105._FastData.0.gz_am: 30% /5G, 8.870M/s, 6m42s
* penfold/20200105._FastData.0.gz_an: 26% /5G, 9.311M/s, 6m46s
2020/01/23 18:12:16 ERROR : penfold/20200105._FastData.0.gz_am: Failed to copy: invalidRange: fragmentOverlap: The uploaded fragment overlaps with data that has already been received.
2020/01/23 18:12:20 ERROR : penfold/20200105._FastData.0.gz_ak: Failed to copy: invalidRange: fragmentOverlap: The uploaded fragment overlaps with data that has already been received.
2020/01/23 18:12:42 INFO :
Transferred: 379.124G / 595.475 GBytes, 64%, 30.955 MBytes/s, ETA 1h59m17s
Errors: 3 (retrying may help)
Checks: 0 / 0, -
Transferred: 1015 / 1148, 88%
Elapsed time: 3h29m1.6s
Now this has been running for 3.5hrs and only 3 errors (looks like I missed the first one)
The same thing happened a little later:
2020/01/23 19:45:52 ERROR : penfold/20200119._FastData.0.gz_al: Failed to copy: invalidRange: fragmentOverlap: The uploaded fragment overlaps with data that has already been received.
2020/01/23 19:45:54 ERROR : penfold/20200119._FastData.0.gz_am: Failed to copy: invalidRange: fragmentOverlap: The uploaded fragment overlaps with data that has already been received.
Now rclone automatically retried:
2020/01/23 20:03:01 ERROR : Encrypted drive 'OneDriveBackups:': not deleting files as there were IO errors
2020/01/23 20:03:01 ERROR : Encrypted drive 'OneDriveBackups:': not deleting directories as there were IO errors
2020/01/23 20:03:01 ERROR : Attempt 1/3 failed with 5 errors and: invalidRange: fragmentOverlap: The uploaded fragment overlaps with data that has already been received.
2020/01/23 20:03:03 INFO : Encrypted drive 'OneDriveBackups:': Waiting for checks to finish
2020/01/23 20:03:03 INFO : Encrypted drive 'OneDriveBackups:': Waiting for transfers to finish
2020/01/23 20:03:42 INFO :
Transferred: 587.472G / 610.857 GBytes, 96%, 31.329 MBytes/s, ETA 12m44s
Errors: 0
Checks: 1146 / 1146, 100%
Transferred: 1146 / 1151, 100%
Elapsed time: 5h20m1.6s
Transferring:
* pugwash/20190625.root.0.gz_aa: 7% /4.569G, 8.547M/s, 8m27s
* penfold/20200105._FastData.0.gz_ak: 4% /5G, 5.650M/s, 14m25s
* penfold/20200105._FastData.0.gz_am: 6% /5G, 8.337M/s, 9m34s
* penfold/20200119._FastData.0.gz_al: 6% /5G, 8.103M/s, 9m53s
And this time the 5 uploaded.
2020/01/23 20:23:25 INFO : Waiting for deletions to finish
2020/01/23 20:23:25 ERROR : Attempt 2/3 succeeded
2020/01/23 20:23:25 INFO :
Transferred: 610.863G / 610.863 GBytes, 100%, 30.685 MBytes/s, ETA 0s
Errors: 0
Checks: 1146 / 1146, 100%
Transferred: 1151 / 1151, 100%
Elapsed time: 5h39m45.2s
Thu Jan 23 20:23:25 EST 2020 Ended OneDriveBackups:
So it doesn't look like the problem is easily replicable, which is gonna make it hard to provide debug level output!
rclone v1.46
- os/arch: linux/amd64
- go version: go1.11.5
I know that's a little old but the changelog doesn't mention anything related to this.
It's possible https://github.com/rclone/rclone/issues/3131 is related, and this might help narrow down that it's the onedrive backend and not the webdav frontend.
And it's clearly not specific to a file 'cos the 5 failes one re-uploaded without issue .
