What is the problem you are having with rclone?
When having output redirection like piping to tee, progress output gets duplicated on every stats update (once per second on my case). On older versions, the terminal was updated to have only the most recent stats update.
What is your rclone version (output from rclone version
)
Issue happens on every version starting from v1.50, up to current beta, it does not happen up to v1.49.5.
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Debian 10 (buster) 64-bit
Which cloud storage system are you using? (eg Google Drive)
Found the issue on Google Drive, however tried reproducing it on SFTP and it also happened. I think it's storage independent.
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
rclone copy server:copyfile . -P --stats 5s --bwlimit 7M | tee output
A log from the command with the -vv
flag (eg output from rclone -vv copy /tmp remote:tmp
)
From affected version (v1.50):
username@hostname:~/rclonebug/rclone-v1.50.0-linux-amd64$ ./rclone -vv copy server:copyfile . --stats 1s | tee output
2020/04/21 01:37:44 DEBUG : rclone: Version "v1.50.0" starting with parameters ["./rclone" "-vv" "copy" "server:copyfile" "." "--stats" "1s"]
2020/04/21 01:37:44 DEBUG : Using config file from "/home1/username/.config/rclone/rclone.conf"
2020/04/21 01:37:46 DEBUG : sftp://root@server2:60022/copyfile: New connection xxx.xxx.xxx.xxx:33164->xxx.xxx.xxx.xxx:60022 to "SSH-2.0-OpenSSH_7.4p1 Debian-10+deb9u7"
2020/04/21 01:37:47 DEBUG : copyfile: Need to transfer - File not found at Destination
2020/04/21 01:37:48 INFO :
Transferred: 0 / 10 MBytes, 0%, 0 Bytes/s, ETA -
Errors: 0
Checks: 0 / 0, -
Transferred: 0 / 1, 0%
Elapsed time: 700ms
Transferring:
-
copyfile: 0% /10M, 0/s, -
2020/04/21 01:37:49 INFO :
Transferred: 252k / 10 MBytes, 2%, 140.340 kBytes/s, ETA 1m11s
Errors: 0
Checks: 0 / 0, -
Transferred: 0 / 1, 0%
Elapsed time: 1.7s
Transferring:
-
copyfile: 2% /10M, 59.986k/s, 2m46s
2020/04/21 01:37:50 INFO :
Transferred: 6.996M / 10 MBytes, 70%, 2.502 MBytes/s, ETA 1s
Errors: 0
Checks: 0 / 0, -
Transferred: 0 / 1, 0%
Elapsed time: 2.7s
Transferring:
-
copyfile: 69% /10M, 510.012k/s, 6s
2020/04/21 01:37:51 DEBUG : sftp cmd = copyfile
2020/04/21 01:37:51 DEBUG : sftp output = "f1c9645dbc14efddc7d8a322685f26eb copyfile\n"
2020/04/21 01:37:51 DEBUG : sftp hash = "f1c9645dbc14efddc7d8a322685f26eb"
2020/04/21 01:37:51 DEBUG : copyfile: MD5 = f1c9645dbc14efddc7d8a322685f26eb OK
2020/04/21 01:37:51 INFO : copyfile: Copied (new)
2020/04/21 01:37:51 INFO :
Transferred: 10M / 10 MBytes, 100%, 2.716 MBytes/s, ETA 0s
Errors: 0
Checks: 0 / 0, -
Transferred: 1 / 1, 100%
Elapsed time: 3.6s
2020/04/21 01:37:51 DEBUG : 13 go routines active
2020/04/21 01:37:51 DEBUG : rclone: Version "v1.50.0" finishing with parameters ["./rclone" "-vv" "copy" "server:copyfile" "." "--stats" "1s"]
From a non-affected version:
username@hostname:~/rclonebug/rclone-v1.49.5-linux-amd64$ ./rclone -vv copy server:copyfile . -P --stats 1s | tee output
2020/04/21 01:43:43 DEBUG : rclone: Version "v1.49.5" starting with parameters ["./rclone" "-vv" "copy" "server:copyfile" "." "-P" "--stats" "1s"]
2020/04/21 01:43:43 DEBUG : Using config file from "/home1/username/.config/rclone/rclone.conf"
2020/04/21 01:43:45 DEBUG : sftp://root@server2:60022/copyfile: New connection xxx.xxx.xxx.xxx:46966->xxx.xxx.xxx.xxx:60022 to "SSH-2.0-OpenSSH_7.4p1 Debian-10+deb9u7"
2020-04-21 01:43:46 DEBUG : copyfile: Couldn't find file - need to transfer
2020-04-21 01:43:49 DEBUG : copyfile: MD5 = f1c9645dbc14efddc7d8a322685f26eb OK
2020-04-21 01:43:49 INFO : copyfile: Copied (new)
Transferred: 10M / 10 MBytes, 100%, 2.811 MBytes/s, ETA 0s
Errors: 0
Checks: 0 / 0, -
Transferred: 1 / 1, 100%
Elapsed time: 3.5s
2020/04/21 01:43:49 INFO :
Transferred: 10M / 10 MBytes, 100%, 2.811 MBytes/s, ETA 0s
Errors: 0
Checks: 0 / 0, -
Transferred: 1 / 1, 100%
Elapsed time: 3.5s
2020/04/21 01:43:49 DEBUG : 12 go routines active
2020/04/21 01:43:49 DEBUG : rclone: Version "v1.49.5" finishing with parameters ["./rclone" "-vv" "copy" "server:copyfile" "." "-P" "--stats" "1s"]