Rclone size output different

Help and Support
1

What is the problem you are having with rclone?

The output of the rclone size command is non-standard. Did it change between versions? Or is it an OS difference?

One machine outputs results in the following format:

Total objects: 86213
Total size: 259.607 GBytes (278751338420 Bytes)

The other outputs in this format:

Total objects: 86213
Total size: 259.607 GiByte (278751338420 bytes)

What is your rclone version (output from rclone version)

rclone v1.51.0
- os/arch: windows/amd64
- go version: go1.13.7

rclone v1.57.0-beta.5642.6a9ef27b0
- os/version: slackware 14.2+ (64 bit)
- os/kernel: 4.19.107-Unraid (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.17
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone size remote: --fast-list

The rclone config contents with secrets removed.

[remote]
type = drive
client_id = 
client_secret = 
scope = drive
token = 
root_folder_id =

A log from the command with the -vv flag

Paste  log here
2

as i understand it, rclone always used GiByte internally.
in the past, the text was GBytes.
now, the text is GiByte

https://rclone.org/docs/#human-readable

note: v1.57.0 is has been released, no need for the beta.

perhaps @albertony has a comment?

1 Like
3

The only reason I mention it is because I was (badly) parsing the output and it took me ages to work out that things were breaking because of the changing capitalisation of "Bytes" and the dropping of the "s" and addition of binary prefix in the human-readable version.

I couldn't find any documentation about whether this was an intentional change or not.

4

It's an intended change in 1.57.

Pull request details are here:

1 Like
5

Thanks. I couldn't find that pull request searching for that as both versions seemed to have the human readable option.

What's the reasoning behind dropping the plural "s"?

6

I wouldn't want to assume anything so @albertony could comment.

One a different note, you might want to check out json output and parse that as it's a bit cleaner than text based parsing.

rclone size DB: --fast-list --json
{"count":5356,"bytes":22417249486264}

and you can parse using anything you want, but here's a jq example

felix@gemini:~$ rclone size DB: --fast-list --json | jq
{
  "count": 5358,
  "bytes": 22422646464479
}

rclone size DB: --fast-list --json | jq '.bytes'
22468674386219
1 Like
7

It was a bash script and using old and dirty code so text seemed easier than parsing json :slight_smile:

8

In general, rclone has always shown values in binary units (powers of 2, or rather 1024) when referring to sizes, and decimal units (powers of 10) for counts.

Version 1.55 and older used the SI standard notation, which traditionally has been used for binary units as well, even if strictly speaking indicating decimal units. Rclone used some different forms, such as: M, MB, MBytes.

Version 1.56 changed to use of IEC standard notation for sizes, to avoid any confusion. And, if I remember correctly, there was also some adjustments to make the form more consistent accross rclone, Mi vs MiB vs MiByte: It was then chosen to use the long form MiByte as primary, with short form Mi as secondary, with support for just M in config input for backwards-compatibility. This is Use binary/IEC prefix for base-2 and decimal/SI prefix for base-10 numbers by albertony · Pull Request #5087 · rclone/rclone · GitHub.

Version 1.57 included an adjustment of the primary form from MiByte to the more used MiB form. This was done as part of the introduction of --human-readable flag to switch output between raw values and values with suffix. This is Introduce global flag --human-readable by albertony · Pull Request #5237 · rclone/rclone · GitHub.

1 Like
9

Thanks for the explanation, but what was the reason for dropping the plural? Perhaps for consistency when only 1 GiB for example? Or is that the standard IEC notation (I couldn't find anything about the plural)?

10

I think in many cases it can be considered more like a mathematical unit symbol, and then should be plain singular form. Like with centimeters: "75 cm", not "75 cms". Its not always clear cut though, e.g. when part of more natural sentences it may make sense to use plural. The mentioned PRs tried to make it more correct and consistent across rclone, but as can be seen from the descriptions I did not find it to be all clear cut decisions...

1 Like
11

Now it seems to have changed again!

Instead of:

Total objects: 86213
Total size: 259.607 GiByte (278751338420 bytes)

the B is now capitalised again and "yte" has been dropped.

Total objects: 86213
Total size: 259.607 GiB (278751338420 Bytes)

Is this going to change any more?

12

Without knowing what versions you are comparing against, it's answer.

13

Quite possibly... I recommend parsing the JSON output which won't change, rather than the human readable output which gets tweaked to improve the user experience continuously.

$ rclone size --json ~/go/src/github.com/rclone/rclone
{"count":4414,"bytes":961001099}
1 Like
14

This was mentioned in the accepted solution:

1 Like
15

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.