Rclone copy stucks at 99%

Suspected Bug
1

I have configured two remotes: my ftp and yonder.disk. For some files with size around 1Gb (e.g., 1,34Gb, 780Mb) copy from ftp to yonder.disk stops at 99% and whole process is stuck. But the point is - files are shown at Yandex.Disk and they are completed (e.g., I can open an archive which had only 99% transfer or play a video).

What could be a reason and how I can workaround it?

29
Screen Shot 2018-01-12 at 15.26.29.png1050×350 38.9 KB

and speed goes down and down

command I used
rclone copy --log-level=DEBUG --retries 1 --buffer-size 500M wdmycloud: yandexdisk:wdmycloud

tried without any additional parameter too

2

Can you try to see whether uploading files to Yandex from your computer shows the same problem? Or downloading files from your FTP to your computer? We can narrow down what is happening.

3

the thing is: I run this command with limit for file size <= 500MB and it works perfect. But when it comes to bigger files I receive such stuck.

from the image I can download files from ftp to my laptop and from laptop to yandex.disk
Even more.
When transfer stocks at 99% I can download this file from Yandex.Disk and it’s not corrupted. I downloaded zip archive and extract everything form it. The same with video files, they transfer is completed. However rclone can’t finish transfer for some reason.

4

I wonder if something has timedout the connection for some reason…

Try running with -vv --dump responses and see if that makes things clearer. Post some log demonstrating the problem if you can.

5

@ncw Will is save dump if I interrupt the script?
And where I will find the dump file later?

6

It will output the file to stderr in the log which you can capture with 2>logfile or --log-file file.log, or my favourite

rclone .... 2>&1 | tee -a file.log
7

@ncw thanks, going to try

8

@ncw
here’s a link to log file

9

@ncw
I found only two types of errors:

  1. that some folder are already exists which isn’t a case

2018/01/14 16:59:46 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2018/01/14 16:59:46 DEBUG : HTTP RESPONSE (req 0xc420457800)
2018/01/14 16:59:46 DEBUG : Error: EOF
2018/01/14 16:59:46 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

But maybe you’ll get better understanding of this log

10

That appears to be causing the problem...

Instead of the upload finishing cleaning, rclone is getting an EOF error and hence things the upload hasn't finished...

I'll investigate further - can you please make a new issue on github about this so I don't forget and we can continue the discussion there - thanks.

11

@ncw
interesting. Especially that it does not appear on small files.

I closed first issue: https://github.com/ncw/rclone/issues/1972

This one is new with this error in details: https://github.com/ncw/rclone/issues/1976

12

Thanks for doing that - I’ll respond on the issue in due course!

13

@ncw
great, will be waiting. Thanks a lot, your tool is really useful. Will be great if this issue will be fixed.