Now when I copy files to/from remote (OpenDrive) files are copying over 100%. Please take look to the picture crlone-01|690x491. Now 340 Gb has transferred from 238. 143% total, 364% max of one file. The last release I used 1.49.2 has no this bug.
What is your rclone version (output from rclone version)
1.49.4
Which OS you are using and how many bits (eg Windows 7, 64 bit)
QTS 4.4.1.1081 64bit (Qnap Linux)
Which cloud storage system are you using? (eg Google Drive)
OpenDrive
The command you were trying to run (eg rclone copy /tmp remote:tmp)
I've seen something similar, usually with very small files, where it can often go to 130-140% for a split-second before finishing. (not exactly the same as OP with his large files which I have not experienced).
I just assumed it was some artifact of the progress-calculation algorithm due to overhead or packet-sizes or something, so I've just been ignoring it since everything seems to be working fine otherwise.
I've fairly certain this phenomenon has not been tied to any spesific version as I remember seeing it several times when testing IOPS performance over several months.
Again, this may not be directly the same issue as the OP experienced, but I figured I'd mention it for being related.
Retries... Last week I did about 2 Tb with no these retries, the same source and remote using 1.49.2 (but with annoying "file already closed" errors).
Now I have 258% total and the process is still on despite all the files are on remote.
I hope it will end some time or other...
As NCW said, if you turn on -vv you should be able to see if this is the case (and what is causing the retries).
Because if you are getting more than 50% retries (to reach 200% or more) then that is not normal in any case and it should be investigated. Feel free to post the debug log here and we will help analyze what the underlying problem is.
To turn on -vv I need to start a new process. I still hope the current transfer will succeed so will be waiting a bit more... until 300% for example.
Also I'm thinking to get back to good old 1.49.2.
It might be worth trying 1.49.5 as people definitely did have problems with 1.49.4. There are very few code changes in 1.49.x as they are a patch release only, however I accidentally compiled 1.49.4 with go1.13.x instead of go1.12.x like the other 1.49.x
Just Ctrl-C'd. Checked all the files transferred. Everything is ok. Have no idea what Rclone did last 12 hours since it uploaded 100% of the last file. It's definitely a bug.
There's no 1.49.5 Qnap package yet so I'll go to 1.94.3. It has no "file already closed" issue and I hope it has no this 100%+ bug. Will test.
Just started a new upload with -vv. With no issues yet. Will let you know if something go the same way.
Ok. Tested a similar upload with 1.49.3. Almost the same issue. Two files: 33 Mb and 42 Gb. I manually stopped transfer on 57 Gb. The last -vv line showed:
Debug output, or "very verbose" if you prefer to think that way.
It makes rclone spam your output with every little thing it does. Way too much to absorb real-time - but crucial to figure out what is happening under the hood if you suspect problems.
Still on the subject of retries on opendrive want to report 1.50.1 I installed today don't fix this bug.
Please take a look to the screenshots. As you can see 5 files was uploaded successfully but they are still uploading over 100%. The only way to stop them is to Ctrl'C Rclone. So I have to wait until the last of 16 files will have been uploaded and then stop. It's very sad because all this time I will waste traffic and time to upload already uploaded files. And I don't know how to fix it
They are not actually failed. They were checked ok. Also I downloaded several and unzipped them. They are absolutely ok. But RCLONE for some strange reason didn’t receive an successful upload message or smth about.
As a follow-up... I’ve been monitoring RCLONE upload to OpenDrive for two months and wanna mention:
this issues raise only with large files over 10G.
on 1.49.2 this issue raises the error File already closed. Stopped the uploading and added the next file to a queue. As I said before the file was ok despite the error.
now this issue (seems like) not stops the uploading but continues it over and over. I’m sure it’s much worse. In previous version the uploading of other new files was not stopped as in the new one.
I stopped using 1.49.2 because of another issue I can’t live with (OpenDrive tries to fix it now). 1.50.1 has no this one but has the overuploading bug. I can live with it but not so good and it’s a bug. Maybe it’s time to fix it?
