Large upload to YandeDisk fails at 50%, retries, fails again

What is the problem you are having with rclone?
Large file upload to YandexDisk fails at 50%, eventually, retries then fail again
File in question is 10 GBytes in size
What is your rclone version (eg output from rclone -V)
v1.42-099-g751bfd45
Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows Server 2016

Which cloud storage system are you using? (eg Google Drive)

YandexDisk

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone -vv --dump responses --log-file C:/rclone.log --timeout 1800s --retries 1 --low-level-retries 1 copy E:\bckp\ yd:juno

most of all - I get those error

net/HTTP timeout awaiting response headers

A log from the command with the -vv flag is in attach.

2018/08/14 13:25:44 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2018/08/14 13:25:44 DEBUG : HTTP RESPONSE (req 0xc0420d8900)
2018/08/14 13:25:44 DEBUG : HTTP/1.1 504 Gateway Time-out
Content-Length: 182
Connection: keep-alive
Content-Type: text/html
Date: Tue, 14 Aug 2018 10:25:44 GMT
Keep-Alive: timeout=120
Server: nginx/1.8.1

<html>
<head><title>504 Gateway Time-out</title></head>
<body bgcolor="white">
<center><h1>504 Gateway Time-out</h1></center>
<hr><center>nginx/1.8.1</center>
</body>
</html>
2018/08/14 13:25:44 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2018/08/14 13:25:44 ERROR : juno_14-08-2018.7z.002: Failed to copy: upload error [504]: <html>
<head><title>504 Gateway Time-out</title></head>
<body bgcolor="white">
<center><h1>504 Gateway Time-out</h1></center>
<hr><center>nginx/1.8.1</center>
</body>
</html>
2018/08/14 13:25:46 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2018/08/14 13:25:46 DEBUG : HTTP RESPONSE (req 0xc042712300)
2018/08/14 13:25:46 DEBUG : HTTP/1.1 504 Gateway Time-out
Content-Length: 182
Connection: keep-alive
Content-Type: text/html
Date: Tue, 14 Aug 2018 10:25:46 GMT
Keep-Alive: timeout=120
Server: nginx/1.8.1

whole log

1 Like

Hmm, Gateway timeout is the Yandex server saying it can’t communicate with other Yandex parts.

Likely it is busy processing the 10GB file somehow and it timed out.

Does rclone print

Removing failed copy

or

Failed to remove failed copy

Can you try uploading the file with --retries 1 and see if it really does get uploaded?

no, rclone does not print about failed copy, it just trying to upload file again.
I tried to upload with --retries 1 key - and files does get uploaded at half of times.
Now i’m trying to use opendrive - and i got very similar problem (only files < 4Gb can be uploaded correctly)

Could this be your ISP closing the connections? That isn’t unheard of…

my server is in tier3 data-center, and has 5gb\s connection speed :frowning:
I have already wrote report to yandex support - they told, that this is bug of yandex-disk and they about to start fixing it

OK

Ah ha! Let me know what happens - thanks.