What is the problem you are having with rclone?
We are using rcd to run rclone in the background and spawn jobs using the rc command. Due to limitations of the local filesystem we are running sync/copy without checksums and then operations/check to verify the integrity.
It looks like that operations/check only reports stats in the global "group" and not it's own job id or the group specified using the _group argument.
Run the command 'rclone version' and share the full output of the command.
rclone v1.67.0-beta.7796.d9601c78b
- os/version: ubuntu 20.04 (64 bit)
- os/kernel: 5.15.146.1-microsoft-standard-WSL2 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.22.1
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
S3 (Wasabi)
The command you were trying to run (eg rclone copy /tmp remote:tmp)
Start the daemon:
$ rclone rcd -v --rc-web-gui --rc-no-auth --rc-job-expire-interval=7d
As expected stats are empty
$ rclone rc core/stats
{
"bytes": 0,
"checks": 0,
"deletedDirs": 0,
"deletes": 0,
"elapsedTime": 40.629029436,
"errors": 0,
"eta": null,
"fatalError": false,
"renames": 0,
"retryError": false,
"serverSideCopies": 0,
"serverSideCopyBytes": 0,
"serverSideMoveBytes": 0,
"serverSideMoves": 0,
"speed": 0,
"totalBytes": 0,
"totalChecks": 0,
"totalTransfers": 0,
"transferTime": 0,
"transfers": 0
}
copy to ensure everything is up to date (checks are counted in the stats):
rclone rc -v sync/copy srcFs=$(pwd)/f dstFs=wasabi:testing/f _async=true
{
"jobid": 26
}
rclone rc core/stats --json '{"group": "job/26"}'
{
"bytes": 0,
"checks": 8,
"deletedDirs": 0,
"deletes": 0,
"elapsedTime": 15.220809316,
"errors": 0,
"eta": null,
"fatalError": false,
"renames": 0,
"retryError": false,
"serverSideCopies": 0,
"serverSideCopyBytes": 0,
"serverSideMoveBytes": 0,
"serverSideMoves": 0,
"speed": 0,
"totalBytes": 0,
"totalChecks": 8,
"totalTransfers": 0,
"transferTime": 0.255414075,
"transfers": 0
}
check both with and without custom group name doesn't count the stats:
rclone rc -v operations/check srcFs=$(pwd)/f dstFs=wasabi:testing/f _async=true
{
"jobid": 40
}
rclone rc core/stats --json '{"group": "job/40"}'
{
"bytes": 0,
"checks": 0,
"deletedDirs": 0,
"deletes": 0,
"elapsedTime": 0.000017819,
"errors": 0,
"eta": null,
"fatalError": false,
"renames": 0,
"retryError": false,
"serverSideCopies": 0,
"serverSideCopyBytes": 0,
"serverSideMoveBytes": 0,
"serverSideMoves": 0,
"speed": 0,
"totalBytes": 0,
"totalChecks": 0,
"totalTransfers": 0,
"transferTime": 0,
"transfers": 0
}
rclone rc -v operations/check _group=check srcFs=$(pwd)/f dstFs=wasabi:testing/f _async=true
{
"jobid": 62
}
rclone rc core/stats --json '{"group": "check"}'
{
"bytes": 0,
"checks": 0,
"deletedDirs": 0,
"deletes": 0,
"elapsedTime": 0.000005756,
"errors": 0,
"eta": null,
"fatalError": false,
"renames": 0,
"retryError": false,
"serverSideCopies": 0,
"serverSideCopyBytes": 0,
"serverSideMoveBytes": 0,
"serverSideMoves": 0,
"speed": 0,
"totalBytes": 0,
"totalChecks": 0,
"totalTransfers": 0,
"transferTime": 0,
"transfers": 0
}
Since we have ran copy once and check twice global stats show 24 checks:
rclone rc core/stats
{
"bytes": 0,
"checks": 24,
"deletedDirs": 0,
"deletes": 0,
"elapsedTime": 239.70510293,
"errors": 0,
"eta": null,
"fatalError": false,
"renames": 0,
"retryError": false,
"serverSideCopies": 0,
"serverSideCopyBytes": 0,
"serverSideMoveBytes": 0,
"serverSideMoves": 0,
"speed": 0,
"totalBytes": 0,
"totalChecks": 24,
"totalTransfers": 0,
"transferTime": 0.619562053,
"transfers": 0
}
The rclone config contents with secrets removed.
$ rclone rc config/dump
{
"wasabi": {
"access_key_id": "xxxx",
"endpoint": "https://s3.eu-central-2.wasabisys.com",
"provider": "Wasabi",
"region": "eu-central-2",
"secret_access_key": "xxxxx",
"type": "s3"
}
}
