What is the problem you are having with rclone?
Statistics from the HTTP API core/stats show wrong value in the field totalBytes while a sync is in progress.
The periodic stats logged from rc CLI display correctly a total of 1.762 GiByte, while the core/stats API returns totalBytes: 1510783 bytes (1.5 MB), that is instead the size of the file currently being transferred.
Note: I noticed that after each single file transfer is done, the field totalBytes increases, and its value is the sum of the already transferred files + the file currently being transferred.
What is your rclone version (output from rclone version)
rclone v1.56.2
- os/version: linuxmint 20.1 (64 bit)
- os/kernel: 5.11.0-25-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.16.8
- go/linking: static
- go/tags: none
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Linux Mint, 64bit
Which cloud storage system are you using? (eg Google Drive)
B2, but errors appear also with local backend.
The command you were trying to run (eg rclone copy /tmp remote:tmp)
- Start sync in the first terminal (use very low bwlimit to artificially prolong the execution time)
rclone --verbose --bwlimit=5k --transfers=1 --stats 10s --rc sync /var/log/ /tmp/test-logs/
- In another terminal check statistics:
rclone -vv rc core/stats
The rclone config contents with secrets removed.
[backblaze]
type = b2
account = ************
key = ************
hard_delete = true
A log from the command with the -vv flag
Output from first rclone CLI:
2021/10/22 21:59:42 INFO : Starting bandwidth limiter at 5KiByte/s
2021/10/22 21:59:42 NOTICE: Serving remote control on http://localhost:5572/
2021/10/22 21:59:42 ERROR : boot.log: Failed to copy: failed to open source object: open /var/log/boot.log: permission denied
2021/10/22 21:59:42 ERROR : private: failed to open directory "private": open /var/log/private: permission denied
2021/10/22 21:59:42 ERROR : boot.log.1: Failed to copy: failed to open source object: open /var/log/boot.log.1: permission denied
2021/10/22 21:59:42 ERROR : speech-dispatcher: failed to open directory "speech-dispatcher": open /var/log/speech-dispatcher: permission denied
2021/10/22 21:59:42 NOTICE: teamviewer15/mion00: Can't follow symlink without -L/--copy-links
2021/10/22 21:59:42 ERROR : boot.log.2: Failed to copy: failed to open source object: open /var/log/boot.log.2: permission denied
2021/10/22 21:59:42 ERROR : boot.log.3: Failed to copy: failed to open source object: open /var/log/boot.log.3: permission denied
2021/10/22 21:59:42 ERROR : boot.log.4: Failed to copy: failed to open source object: open /var/log/boot.log.4: permission denied
2021/10/22 21:59:42 ERROR : boot.log.5: Failed to copy: failed to open source object: open /var/log/boot.log.5: permission denied
2021/10/22 21:59:42 ERROR : boot.log.6: Failed to copy: failed to open source object: open /var/log/boot.log.6: permission denied
2021/10/22 21:59:42 ERROR : boot.log.7: Failed to copy: failed to open source object: open /var/log/boot.log.7: permission denied
2021/10/22 21:59:42 ERROR : btmp: Failed to copy: failed to open source object: open /var/log/btmp: permission denied
2021/10/22 21:59:42 ERROR : btmp.1: Failed to copy: failed to open source object: open /var/log/btmp.1: permission denied
2021/10/22 21:59:52 INFO :
Transferred: 60Ki / 1.762 GiByte, 0%, 6.664 KiByte/s, ETA 3d4h59m50s
Errors: 12 (retrying may help)
Checks: 38 / 38, 100%
Transferred: 0 / 203, 0%
Elapsed time: 10.0s
Transferring:
* kern.log: 4% /1.441Mi, 6.663Ki/s, 3m32s
Output from rc core/stats:
2021/10/22 22:00:14 DEBUG : rclone: Version "v1.56.2" starting with parameters ["rclone" "-vv" "rc" "core/stats"]
{
"bytes": 192512,
"checks": 38,
"deletedDirs": 0,
"deletes": 0,
"elapsedTime": 31.680140078,
"errors": 12,
"eta": 288,
"fatalError": false,
"lastError": "failed to open source object: open /var/log/btmp.1: permission denied",
"renames": 0,
"retryError": true,
"speed": 4569.720875775673,
"totalBytes": 1510783,
"totalChecks": 38,
"totalTransfers": 1,
"transferTime": 0,
"transferring": [
{
"bytes": 192512,
"eta": 288,
"group": "global_stats",
"name": "kern.log",
"percentage": 12,
"size": 1510783,
"speed": 6095.685458991245,
"speedAvg": 4569.419699374595
}
],
"transfers": 0
}
2021/10/22 22:00:14 DEBUG : 4 go routines active