What does it mean when progress shows a file is transfering at > 100%

Help and Support
1

What is the problem you are having with rclone?

I would like to know what it means when the file progress indicates a file is transferring at greater than 100%

What is your rclone version (output from rclone version)

rclone v1.50.2

  • os/arch: linux/amd64
  • go version: go1.13.6

Which OS you are using and how many bits (eg Windows 7, 64 bit)

KDE Neon 5.20 (Ubuntu)

Which cloud storage system are you using? (eg Google Drive)

pCloud

The command you were trying to run (eg rclone copy /tmp remote:tmp)

/usr/bin/rclone sync --delete-after --verbose --transfers 2 --checkers 8 --bwlimit 80 --contimeout 60s --timeout 300s --retries 3 --low-level-retries 10 -vv --max-size 1G --stats 1s --stats-file-name-length 0 --fast-list /path/ remote:path

A log from the command with the -vv flag

2020/11/06 13:12:37 INFO  :
Transferred:   	    3.732G /6.953 GBytes, 48%, 97.684 kBytes/s, ETA 1d9h27m26s
Errors:                 0
Checks:              1206 / 1206, 100%
Transferred:           21 / 641, 2%
Elapsed time:  11h7m40.6s
Transferring:
* path/file.ext:**127%** /356.062M, 46.684k/s, -

127%/356.062M

  • The size displayed after the / is correct.

I have bolded the part I am wondering about. The file says it is 127% transferred, it continues to transfer. I have seen file transferring at up to 160%. After I terminate the transfer if I look at the remote, it looks like around 75% of the file has actually transferred. There are no errors I can see.

I am on a very slow connection so when I see this I interpret it as an problem and terminate the transfer and start over. I do usually loose several hours of work however so this is not a great solution.

Has anyone seen this scenario? How does it happen and what does it mean? Also why does rclone now show the actual number of bytes transferred rather than a percentage of the total?

100/300M rather than
30%/300M

2

that version is very old

so update and test again.

3

I have considered that however I am hesitant to upgrade because this is the latest version from my package manager which has been tested with my OS. Do you know who maintains the Ubuntu repo for rclone, is it Ubuntu or is it someone external?

I scanned the change log here:https://rclone.org/changelog/ and didn't see anything specific mentions of this particular issue.

4

You should not use any package manager for rclone as they are all maintained by other people and you really don't know what you are getting.

You should only download from rclone.org.

5

not sure that is true, who does the testing and what kind of tests?

6

Yes I understand your point of view however I believe I am following the best practice for software on Linux. Here is an opposing point of view from the one of the oldest and most stable Linux distributions and the one that my OS is based on:

On Debian installing software from random websites is a bad habit. It's always better to use software from the official Debian repositories if at all possible. The packages in the Debian repositories are known to work well and install properly. Only using software from the Debian repositories is also much safer than installing from random websites which could bundle malware and other security risks.

There are a lot of advantages to using the linux model of a build in package manager system versus the Window model of downloading apps from multiple sources.

I and not 100% against installing binaries directly from the rclone website. I did read the change log between my version and the latest version and I didn't see anything specific to this issue.

But aside from the software installation debate, has anyone actually seen or understand how rclone generate the messages that a file is transferring at > 100%, what it means and what I should do about it?

7

Is rclone.org a random website?

A random 3rd party person is compiling from rclone.org and maintaining it on their repository.

Many times and majority was fixed in the newer versions hence the request to update.

2 Likes
8

I agree rclone is not a random website however the Debian package maintainers are also not random people. I believe you can find out exactly who maintains which packages. This is why I asked earlier if anyone know who maintains the rclone package.

The other advice about the benefits of testing and maintenance that a package manager system provides is still quite valid. Both methods have pluses and minuses. Websites can be hacked, a malicious version of a binary can be posted for download. A lot of software authors post GPG signatures which can be checked against the binary so users can be assured it came be assured it came from the developer which mitigates this risk. Do you know if something like this is posted for rclone releases?

"The packages in the Debian repositories are known to work well and install properly. "
I have had issues with installing apps that have not been tested with my OS that have caused some major problems. OSes are complex things with a lot of interrelated parts.

I understand generally upgrading to the latest version fixes issues which is why I also did check rclone the change log to see if this specific issue was address and I didn't see anything there.

I don't even know if my issue is a problem or not. I was hoping someone else might have experienced it.

Thanks for sharing your perspective on software upgrades.

9

Yes as I’ve said a few times, upgrade as that has fixes in it for this.

It is not recommend to use the packages from Debian as they are old, dated and have known issues.

1 Like
10

Oh I didn't see where you mentioned earlier that you knew about a specific fix for this issue. I read your messages as though you were giving general advice. Which version fixed the issue?

This is certainly possible. I have also experienced this.

11

It’s best to grab the latest and go from there.

12

It is usually to do with retries - rclone ends up transferring some of the contents twice.

13

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.