Rclone copyurl is not reliable

[axzxc1236]

I am currently running a command like this on my VPS

rclone -P --retries 9999 --retries-sleep=60s copyurl “https://(url to file)” Gcrypt:“folder”

And get error messages looks like these

2019-02-11 xx:xx:xx ERROR : (filename): post request put error: Post https://www.googleapis.com/upload/drive/v3/files/(many random characters): stream error: stream ID 1: INTERNAL ERROR
2019-02-11 xx:xx:xx ERROR : Attempt 1/9999 failed with 0 errors and: Post https://www.googleapis.com/upload/drive/v3/files/(many random characters): stream error: stream ID 1: INTERNAL ERROR
2019-02-11 xx:xx:xx ERROR : Attempt 2/9999 succeed

Note that although it says succeed, it does not succeed at all, and it doesn’t retry any further, sometimes it fails after download 4GiB, sometimes fails at 13GiB or fails at 200MiB.

The file I am copying to Gcrypt (remote of encrypted Google Drive) has size of 61GB, so download the whole file and then copy to Gcrypt isn’t an option.

Is there anyway to make it runs more stable? (I’m running rclone 1.45 right now)

[ncw (Nick Craig-Wood)]

2019-02-11 xx:xx:xx ERROR : (filename): post request put error: Post https://www.googleapis.com/upload/drive/v3/files/(many random characters): stream error: stream ID 1: INTERNAL ERROR

This looks like an error with the upload. Anything with an INTERNAL ERROR is likely a drive problem rather than an rclone problem

Does it always give that error?

Note that although it says succeed, it does not succeed at all, and it doesn’t retry any further, sometimes it fails after download 4GiB, sometimes fails at 13GiB or fails at 200MiB.

That sounds like a bug in rclone.

Can you post a complete log with -vv please? Feel free to obscure the filenames.

I’m running rclone 1.45 right now

1.46 is out now which could help!