GDRIVE 410 error

rclone.mac · December 3, 2018, 1:27pm

It looks like rclone copy lost the uploaded data and started over. Just curious if anyone knows of the most likely cause for a 410 error?

See output around 410 error

[02:20:37] xyz@macpro /Users/xyz/.config/rclone (0)

rclone --rc --rc-addr :5NNN -v copy /Volumes/Temp_4TB/bkup_20180523 tvol3:_disk_archive/bkup_20180523_xyz
2018/12/03 02:21:07 NOTICE: Serving remote control on http://[::]:5NNN/
2018/12/03 02:21:12 INFO : Encrypted drive ‘tvol3:_disk_archive/bkup_20180523_xyz’: Waiting for checks to finish
2018/12/03 02:21:12 INFO : Encrypted drive ‘tvol3:_disk_archive/bkup_20180523_xyz’: Waiting for transfers to finish
2018/12/03 02:21:29 INFO : .DS_Store: Copied (new)
2018/12/03 02:22:12 INFO :
Transferred: 1.273 GBytes (20.069 MBytes/s)
Errors: 0
Checks: 0
Transferred: 1
Elapsed time: 1m4.9s
Transferring:

                    bkup_20180523_xyz.dmg:  0% /838.190G, 29.690M/s, 8h1m5s

2018/12/03 02:23:12 INFO :
Transferred: 3.035 GBytes (24.870 MBytes/s)
Errors: 0
Checks: 0
Transferred: 1
Elapsed time: 2m4.9s
Transferring:

                    bkup_20180523_xyz.dmg:  0% /838.190G, 29.493M/s, 8h3m16s

[… deleted updates … ]

2018/12/03 08:31:13 INFO :
Transferred: 512.103 GBytes (23.616 MBytes/s)
Errors: 0
Checks: 0
Transferred: 1
Elapsed time: 6h10m4.9s
Transferring:

                    bkup_20180523_xyz.dmg: 61% /838.190G, 18.770M/s, 4h56m29s

2018/12/03 08:32:13 INFO :
Transferred: 513.242 GBytes (23.605 MBytes/s)
Errors: 0
Checks: 0
Transferred: 1
Elapsed time: 6h11m4.9s
Transferring:

                    bkup_20180523_xyz.dmg: 61% /838.190G, 19.497M/s, 4h44m26s

2018/12/03 08:33:13 INFO :
Transferred: 514.398 GBytes (23.594 MBytes/s)
Errors: 0
Checks: 0
Transferred: 1
Elapsed time: 6h12m4.9s
Transferring:

                    bkup_20180523_xyz.dmg: 61% /838.190G, 19.307M/s, 4h46m13s

2018/12/03 08:34:13 INFO :
Transferred: 515.523 GBytes (23.583 MBytes/s)
Errors: 0
Checks: 0
Transferred: 1
Elapsed time: 6h13m4.9s
Transferring:

                    bkup_20180523_xyz.dmg: 61% /838.190G, 19.223M/s, 4h46m28s

2018/12/03 08:35:13 INFO :
Transferred: 516.621 GBytes (23.570 MBytes/s)
Errors: 0
Checks: 0
Transferred: 1
Elapsed time: 6h14m4.9s
Transferring:

                    bkup_20180523_xyz.dmg: 61% /838.190G, 18.445M/s, 4h57m32s

2018/12/03 08:36:13 INFO :
Transferred: 517.633 GBytes (23.553 MBytes/s)
Errors: 0
Checks: 0
Transferred: 1
Elapsed time: 6h15m4.9s
Transferring:

                    bkup_20180523_xyz.dmg: 61% /838.190G, 17.129M/s, 5h19m23s

2018/12/03 08:37:13 INFO :
Transferred: 518.641 GBytes (23.536 MBytes/s)
Errors: 0
Checks: 0
Transferred: 1
Elapsed time: 6h16m4.9s
Transferring:

                    bkup_20180523_xyz.dmg: 61% /838.190G, 16.997M/s, 5h20m51s

2018/12/03 08:38:13 INFO :
Transferred: 519.344 GBytes (23.505 MBytes/s)
Errors: 0
Checks: 0
Transferred: 1
Elapsed time: 6h17m4.9s
Transferring:

                    bkup_20180523_xyz.dmg: 61% /838.190G, 5.370M/s, 16h53m24s

2018/12/03 08:39:13 INFO :
Transferred: 519.344 GBytes (23.443 MBytes/s)
Errors: 0
Checks: 0
Transferred: 1
Elapsed time: 6h18m4.9s
Transferring:

                    bkup_20180523_xyz.dmg: 61% /838.190G, 100.560k/s, 923h32m22s

2018/12/03 08:40:13 INFO :
Transferred: 519.727 GBytes (23.399 MBytes/s)
Errors: 0
Checks: 0
Transferred: 1
Elapsed time: 6h19m4.9s
Transferring:

                    bkup_20180523_xyz.dmg: 62% /838.190G, 13.818M/s, 6h33m20s

2018/12/03 08:41:13 INFO :
Transferred: 520.779 GBytes (23.384 MBytes/s)
Errors: 0
Checks: 0
Transferred: 1
Elapsed time: 6h20m4.9s
Transferring:

                    bkup_20180523_xyz.dmg: 62% /838.190G, 17.311M/s, 5h12m55s

2018/12/03 09:32:46 INFO :
Transferred: 539.461 GBytes (21.330 MBytes/s)
Errors: 1
Checks: 0
Transferred: 1
Elapsed time: 7h11m37.7s
Transferring:

bkup_20180523_xyz.dmg

2018/12/03 09:07:01 ERROR : bkup_20180523_xyz.dmg: Failed to copy: googleapi: got HTTP response code 410 with body: Service Temporarily Unavailable
2018/12/03 09:32:46 ERROR : Attempt 1/3 failed with 1 errors and: googleapi: got HTTP response code 410 with body: Service Temporarily Unavailable
2018/12/03 09:33:00 INFO : Encrypted drive ‘tvol3:_disk_archive/bkup_20180523_xyz’: Waiting for checks to finish
2018/12/03 09:33:00 INFO : Encrypted drive ‘tvol3:_disk_archive/bkup_20180523_xyz’: Waiting for transfers to finish
2018/12/03 09:33:13 INFO :
Transferred: 539.470 GBytes (21.308 MBytes/s)
Errors: 0
Checks: 1
Transferred: 1
Elapsed time: 7h12m4.9s
Transferring:

                    bkup_20180523_xyz.dmg:  0% /838.190G, 1.438M/s, 165h48m59s

2018/12/03 09:34:13 INFO :
Transferred: 540.813 GBytes (21.312 MBytes/s)
Errors: 0
Checks: 1
Transferred: 1
Elapsed time: 7h13m4.9s
Transferring:

                    bkup_20180523_xyz.dmg:  0% /838.190G, 25.645M/s, 9h16m54s

2018/12/03 09:35:13 INFO :
Transferred: 542.414 GBytes (21.326 MBytes/s)
Errors: 0
Checks: 1
Transferred: 1
Elapsed time: 7h14m4.9s
Transferring:

                    bkup_20180523_xyz.dmg:  0% /838.190G, 27.009M/s, 8h47m47s

2018/12/03 09:36:13 INFO :
Transferred: 544.017 GBytes (21.340 MBytes/s)
Errors: 0
Checks: 1
Transferred: 1
Elapsed time: 7h15m4.9s
Transferring:

                    bkup_20180523_xyz.dmg:  0% /838.190G, 26.696M/s, 8h52m56s

2018/12/03 09:37:13 INFO :
Transferred: 545.625 GBytes (21.354 MBytes/s)
Errors: 0
Checks: 1
Transferred: 1
Elapsed time: 7h16m4.9s
Transferring:

                    bkup_20180523_xyz.dmg:  0% /838.190G, 27.220M/s, 8h41m40s

[… deleted text …]

2018/12/03 12:34:13 INFO :
Transferred: 814.977 GBytes (22.687 MBytes/s)
Errors: 0
Checks: 1
Transferred: 1
Elapsed time: 10h13m4.9s
Transferring:

                    bkup_20180523_xyz.dmg: 32% /838.190G, 22.423M/s, 7h8m15s

2018/12/03 12:35:13 INFO :
Transferred: 816.362 GBytes (22.688 MBytes/s)
Errors: 0
Checks: 1
Transferred: 1
Elapsed time: 10h14m4.9s
Transferring:

                    bkup_20180523_xyz.dmg: 33% /838.190G, 23.978M/s, 6h39m30s

ncw · December 4, 2018, 10:33am

410 is not one of the errors that rclone will retry - it assumes it is a fatal error. I’ve not seen one of those before in fact!

According to the standard 410 should be treated as a fatal error: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/410

It isn’t mentioned in the docs either: https://developers.google.com/drive/api/v3/handle-errors

It would be easy enough to add it to the list of errors rclone will retry, but I’m not sure that would be the right thing to do.

rclone.mac · December 4, 2018, 6:50pm

I don’t believe you have to do anything. The rclone copy during which this error occurred completed successfully. Although, when I checked the amount of data it moved, it appears that the error caused whatever had been uploaded at that point to disappear. So it was effectively starting over on the copy.

ncw · December 4, 2018, 8:37pm

Provided this doesn’t happen very often then starting again is the correct thing to do. If you are happy with that then I won’t investigate further!

rclone.mac · December 4, 2018, 10:13pm

This has happened only once and I’ve pushed more than 40TB to Google drives. It seems to be rare enough that you shouldn’t bother researching at this time.