JSON Stats: Property Definitions

[bdkjones (Bryan)]

When rclone spits out stats in JSON, they look like this:

["deletes": 0,
 "totalTransfers": 10013, 
 "transferTime": 0,
 "elapsedTime": 2.555557916,
 "transfers": 1,
 "totalChecks": 0,
 "totalBytes": 460559951,
 "bytes": 2483173,
 "renames": 0,
 "retryError": 0,
 "eta": 207,
 "errors": 0,
 "fatalError": 0,
 "deletedDirs": 0,
 "checks": 0,
 "speed": 2211152.470209849]

The property names don't seem to be documented anywhere and I have a few questions:

1. Errors vs retryError vs fatalError -- is errors always the sum of retryError + fatalError? If a transfer produces an error but is retried and completes successfully, is retryError then decremented?

2. checks -- to what does this field refer? Scanning already-uploaded files on the remote to verify that the hashes match and the data is therefore not corrupt?

3. totalTransfers -- does this field mean, "These are all the local files that rclone has discovered thus far and has queued for uploading to the remote"?

4. renames -- to what does this field refer, exactly? If I rename a local file that had previously been uploaded to a remote, then run rclone again, will rclone simply rename the remote file rather than upload the renamed local file to the new path on the remote? If this value is non-zero, is the count included in totalTransfers or in addition to totalTransfers?

5. deletes -- does this refer only to deleted files on the remote (when using sync)? Or is this the TOTAL number of deletes, including both files and directories? For remotes that don't support directories (S3, Backblaze, etc.), I assume that deletedDirs can never be non-zero?

[bdkjones (Bryan)]

I guess a better way to ask about checks is: I've noticed this stays at 0 consistently when using copy or sync. Is this field only populated when using rclone's check command?

Similarly, renames is only populated if I use --track-renames or dedupe with the rename strategy?

[ncw (Nick Craig-Wood)]

@bdkjones

The property names don't seem to be documented anywhere

These are exactly the same stats as output by --stats or -P or --progress just formatted differently.

They are documented here: Remote Control / API

@bdkjones

Errors vs retryError vs fatalError -- is errors always the sum of retryError + fatalError? If a transfer produces an error but is retried and completes successfully, is retryError then decremented?

retryError and fatalError are boolean flags indicating if that sort of error has ocurred...

When I look at the JSON stats (using --use-json-log for example) I get "retryError":false indicating that whatever ingested your stats has missed the fact that retryError is a boolean.

@bdkjones

checks -- to what does this field refer? Scanning already-uploaded files on the remote to verify that the hashes match and the data is therefore not corrupt?

Matching a local file to a remote file to see if it needs to be uploaded.

@bdkjones

totalTransfers -- does this field mean, "These are all the local files that rclone has discovered thus far and has queued for uploading to the remote"?

These are number of files actually transferred I think, the number pending is in transferQueue

@bdkjones

• renames -- to what does this field refer, exactly? If I rename a local file that had previously been uploaded to a remote, then run rclone again, will rclone simply rename the remote file rather than upload the renamed local file to the new path on the remote? If this value is non-zero, is the count included in totalTransfers or in addition to totalTransfers?

renames is whether rclone renamed a file or not.

Why it might do that is a different question! One reason is using --track-renames which is the feature you are referring to.

A file renamed is not a file transferred.

@bdkjones

deletes -- does this refer only to deleted files on the remote (when using sync)? Or is this the TOTAL number of deletes, including both files and directories? For remotes that don't support directories (S3, Backblaze, etc.), I assume that deletedDirs can never be non-zero?

This is files deleted. There is a separate deletedDirs flag.

[bdkjones (Bryan)]

Thank you! That's very helpful!