Unclear how to determine number of bytes copied?
How to determine number of bytes copied? Is it changed between versions?
It's not clear if I should sum stats.serverSideCopyBytes, stats.serverSideCopyBytes and stats.bytes or just use one value.
Run the command 'rclone version' and share the full output of the command.
rclone v1.66.0-DEV
- os/version: alpine 3.20.2 (64 bit)
- os/kernel: 5.10.223-212.873.amzn2.x86_64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.22.5
- go/linking: dynamic
- go/tags: none
--
rclone v1.65.0-DEV
- os/version: alpine 3.19.3 (64 bit)
- os/kernel: 5.10.223-212.873.amzn2.x86_64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.21.10
- go/linking: static
- go/tags: none
Which cloud storage system are you using? S3
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone copy :s3,provider=AWS,env_auth:<SRC_BUCKET>/<SRC_PATH>/ :s3,provider=AWS,<AUTH>:<DEST_BUCKET>/<DEST_PATH>/ --use-json-log --ignore-checksum --retries-sleep=5s --stats=30s --stats-one-line --stats-log-level=NOTICE --s3-upload-concurrency=12 --multi-thread-streams=12 --transfers=12 --s3-no-head --s3-disable-checksum --checkers=32 --s3-chunk-size=16Mi --use-mmap --files-from-raw <INCLUDE_FILE> --s3-upload-cutoff=0
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
<no config>
A log from the command that you were trying to run with the -vv flag
Version v1.66.0-DEV
Example of output #1 (in region, access/secret auth for destination)
{
"level": "warning",
"msg": " 526 B / 526 B, 100%, 0 B/s, ETA -\n",
"source": "accounting/stats.go:526",
"stats": {
"bytes": 526,
"checks": 0,
"deletedDirs": 0,
"deletes": 0,
"elapsedTime": 0.521704557,
"errors": 0,
"eta": null,
"fatalError": false,
"renames": 0,
"retryError": false,
"serverSideCopies": 0,
"serverSideCopyBytes": 0,
"serverSideMoveBytes": 0,
"serverSideMoves": 0,
"speed": 0,
"totalBytes": 526,
"totalChecks": 0,
"totalTransfers": 8,
"transferTime": 0.387114865,
"transfers": 8
},
"time": "2024-09-06T11:44:45.308998+00:00"
}
Example of output #2 (cross region, env auth for destination)
{
"level": "warning",
"msg": " 473.048 KiB / 473.048 KiB, 100%, 0 B/s, ETA -\n",
"source": "accounting/stats.go:526",
"stats": {
"bytes": 484401,
"checks": 0,
"deletedDirs": 0,
"deletes": 0,
"elapsedTime": 2.699785845,
"errors": 0,
"eta": null,
"fatalError": false,
"renames": 0,
"retryError": false,
"serverSideCopies": 34,
"serverSideCopyBytes": 484401,
"serverSideMoveBytes": 0,
"serverSideMoves": 0,
"speed": 0,
"totalBytes": 484401,
"totalChecks": 0,
"totalTransfers": 34,
"transferTime": 2.257085154,
"transfers": 34
},
"time": "2024-09-05T01:18:57.042554+00:00"
}
Version v1.65.0-DEV
Example output (inner region, env auth)
{
"level": "warning",
"msg": " 0 B / 0 B, -, 0 B/s, ETA -\n",
"source": "accounting/stats.go:512",
"stats": {
"bytes": 0,
"checks": 0,
"deletedDirs": 0,
"deletes": 0,
"elapsedTime": 610.513853335,
"errors": 0,
"eta": null,
"fatalError": false,
"renames": 0,
"retryError": false,
"serverSideCopies": 24054,
"serverSideCopyBytes": 88746450937,
"serverSideMoveBytes": 0,
"serverSideMoves": 0,
"speed": 0,
"totalBytes": 0,
"totalChecks": 0,
"totalTransfers": 24054,
"transferTime": 610.256457238,
"transfers": 24054
},
"time": "2024-08-30T09:31:12.382167+00:00"
}