Error when uploading multiple files after 30 mins to google drive

Vento · April 30, 2020, 12:40pm

What is the problem you are having with rclone?

After 30 minutes passes of uploading multiple files in one go to google drive through rclone browser I get the same error.

What is your rclone version (output from `rclone version`)

rclone v1.51.0

os/arch: windows/amd64
go version: go1.13.7

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10 Pro, 64 Bit

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

C:\rclone\rclone.exe --config C:/Users/raghd/.config/rclone/rclone.conf copy --update --verbose --transfers 45 --checkers 50 --contimeout 60s --timeout 300s --retries 3 --low-level-retries 10 --stats 1s --stats-file-name-length 0 --fast-list H:\1Media\Shows\Plex Versions\Original Quality\Hunter x Hunter (2011)\test\New folder gdrive:Plex/Shows/Hunter x Hunter (2011)/Season 2

The Error I keep getting:

2020/04/30 13:48:39 INFO  :
Transferred:   	  844.867M / 2.086 GBytes, 40%, 377.022 kBytes/s, ETA 58m27s
Errors:                 2 (retrying may help)
Transferred:            0 / 3, 0%
Elapsed time:     38m14.6s
Transferring:
* Hunter x Hunter - 02x17 - Ging`s Friends x And x True Friends.mp4: 37% /620.850M, 4.896k/s, 22h39m42s
* Hunter x Hunter - 02x18 - Reunion x And x Understanding.mp4: 11% /691.962M, 2.080k/s, 83h35m12s
* Hunter x Hunter - 02x19 - Unease x And x Sighting.mp4: 42% /506.849M, 5.055k/s, 16h19m5s

2020/04/30 13:48:40 ERROR : Hunter x Hunter - 02x19 - Unease x And x Sighting.mp4: Failed to copy: Post https://www.googleapis.com/upload/drive/v3/files?alt=json&fields=id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink&supportsAllDrives=true&uploadType=resumable&upload_id=AAANsUmbLUzS2-YB-eKXNAZwBrueAkRQvsWUud3rqsCThNQUTCUK0mPEbKEZmfqAzb6zoEt0FfdK71zeEEYkEti-kcA: dial tcp: lookup www.googleapis.com: no such host
2020/04/30 13:48:40 ERROR : Hunter x Hunter - 02x18 - Reunion x And x Understanding.mp4: Failed to copy: Post https://www.googleapis.com/upload/drive/v3/files?alt=json&fields=id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink&supportsAllDrives=true&uploadType=resumable&upload_id=AAANsUlEUhQJ3L1ipDObhXsNsxPkfTyeuRRdY6MjJjBGJCBpL_1kaQT2GbydnoYFTBkJUckNbcmEWACcegVLvTQsME8: dial tcp: lookup www.googleapis.com: no such host
2020/04/30 13:48:40 ERROR : Hunter x Hunter - 02x17 - Ging`s Friends x And x True Friends.mp4: Failed to copy: Post https://www.googleapis.com/upload/drive/v3/files?alt=json&fields=id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink&supportsAllDrives=true&uploadType=resumable&upload_id=AAANsUkI_HhiMTK1OGHqIEHMttulCmV37VFdJs7x55vKqxCKIvj-EFAe5DGkvPGMMlBNeVsS-j2DP4va21e1PAy99qM: dial tcp: lookup www.googleapis.com: no such host
2020/04/30 13:48:40 ERROR : Attempt 1/3 failed with 5 errors and: Post https://www.googleapis.com/upload/drive/v3/files?alt=json&fields=id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink&supportsAllDrives=true&uploadType=resumable&upload_id=AAANsUkI_HhiMTK1OGHqIEHMttulCmV37VFdJs7x55vKqxCKIvj-EFAe5DGkvPGMMlBNeVsS-j2DP4va21e1PAy99qM: dial tcp: lookup www.googleapis.com: no such host
2020/04/30 13:48:40 ERROR : : error reading destination directory: couldn't list directory: Get https://www.googleapis.com/drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%2717wVH_PbbWzZPnK9sAIXUW9g2Ik3ex-zE%27+in+parents%29&supportsAllDrives=true: dial tcp: lookup www.googleapis.com: no such host
2020/04/30 13:48:40 INFO  : Google drive root 'Plex/Shows/Hunter x Hunter (2011)/Season 2/New folder': Waiting for checks to finish
2020/04/30 13:48:40 INFO  : Google drive root 'Plex/Shows/Hunter x Hunter (2011)/Season 2/New folder': Waiting for transfers to finish
2020/04/30 13:48:40 ERROR : Attempt 2/3 failed with 1 errors and: couldn't list directory: Get https://www.googleapis.com/drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%2717wVH_PbbWzZPnK9sAIXUW9g2Ik3ex-zE%27+in+parents%29&supportsAllDrives=true: dial tcp: lookup www.googleapis.com: no such host
2020/04/30 13:48:40 ERROR : : error reading destination directory: couldn't list directory: Get https://www.googleapis.com/drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%2717wVH_PbbWzZPnK9sAIXUW9g2Ik3ex-zE%27+in+parents%29&supportsAllDrives=true: dial tcp: lookup www.googleapis.com: no such host
2020/04/30 13:48:40 INFO  : Google drive root 'Plex/Shows/Hunter x Hunter (2011)/Season 2/New folder': Waiting for checks to finish
2020/04/30 13:48:40 INFO  : Google drive root 'Plex/Shows/Hunter x Hunter (2011)/Season 2/New folder': Waiting for transfers to finish
2020/04/30 13:48:40 ERROR : Attempt 3/3 failed with 1 errors and: couldn't list directory: Get https://www.googleapis.com/drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%2717wVH_PbbWzZPnK9sAIXUW9g2Ik3ex-zE%27+in+parents%29&supportsAllDrives=true: dial tcp: lookup www.googleapis.com: no such host
2020/04/30 13:48:40 Failed to copy: couldn't list directory: Get https://www.googleapis.com/drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%2717wVH_PbbWzZPnK9sAIXUW9g2Ik3ex-zE%27+in+parents%29&supportsAllDrives=true: dial tcp: lookup www.googleapis.com: no such host

Please help, Thanks

calisro · April 30, 2020, 12:55pm

Well you are running with 45 transfers and 50 checkers . I'd guess you're swamping something on your network. Additionally that many transfers/checkers will run into Google api limits as well. Reduce the number.

Why do you get it that high?

Vento · April 30, 2020, 1:14pm

Honestly I don't know how the transfers/checkers thing exactly works, maybe I should change it to 4 transfers and 8 checkers (default numbers). I'll update you if everything works out. Thanks

asdffdsa · April 30, 2020, 1:41pm

hello,
you should re-examine all your flags and figure out if you need them.

often, less flags is better then using more flags.

--retries 3 can be removed as the default is 3.
--low-level-retries 10 can be removed as the default is 10
--contimeout 60s can be removed as the default is 1m0s

system · June 30, 2020, 9:41am

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.