Need only # of files and total size copied in stats

Help and Support
1

What is the problem you are having with rclone?

I'm using a PowerShell script to do versioned backups of my local drives to OneDrive and want to see the total no. of files and the total MB copied each time I run the sync. Something like:

Files copied: 35, deleted: 4
Total Size: 453MB

I have read numerous previous posts on this and have tried --stats-one-line which does not print anything as well as --stats-one-line -v which prints too much info. I also tried just --stats but that fails.

Run the command 'rclone version' and share the full output of the command.

rclone v1.61.1
- os/version: Microsoft Windows 10 Pro 22H2 (64 bit)
- os/kernel: 10.0.19045.2546 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.19.4
- go/linking: static
- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

OneDrive Personal

The command you were trying to run (eg rclone copy /tmp remote:tmp)

.\rclone.exe sync C:\rclone od:testing --stats-one-line

The above is a test command that copies just the single rclone folder on C:\

The rclone config contents with secrets removed.

[od]
type = onedrive
token = 
drive_id = 
drive_type = personal

A log from the command with the -vv flag

.\rclone.exe : 2023/01/22 22:11:50 DEBUG : rclone: Version "v1.61.1" starting with parameters ["C:\\rclone\\rclone.exe" "sync" "C:\\rclone" "od:testing" 
"--stats-one-line" "-vv"]
At line:1 char:1
+ .\rclone.exe sync C:\rclone od:testing --stats-one-line -vv
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (2023/01/22 22:1...ne-line" "-vv"]:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
 
2023/01/22 22:11:50 DEBUG : Creating backend with remote "C:\\rclone"
2023/01/22 22:11:50 DEBUG : Using config file from "C:\\Users\\Ashim Jain\\AppData\\Roaming\\rclone\\rclone.conf"
2023/01/22 22:11:50 DEBUG : fs cache: renaming cache item "C:\\rclone" to be canonical "//?/C:/rclone"
2023/01/22 22:11:50 DEBUG : Creating backend with remote "od:testing"
2023/01/22 22:11:50 DEBUG : logfile.txt: Size and modification time the same (differ by -330.2417ms, within tolerance 1s)
2023/01/22 22:11:50 DEBUG : logfile.txt: Unchanged skipping
2023/01/22 22:11:50 DEBUG : global-exclude.txt: Size and modification time the same (differ by -469.1374ms, within tolerance 1s)
2023/01/22 22:11:50 DEBUG : OneDrive root 'testing': Waiting for checks to finish
2023/01/22 22:11:50 DEBUG : global-exclude.txt: Unchanged skipping
2023/01/22 22:11:50 DEBUG : BackDate, 2023-01-21.txt: Size and modification time the same (differ by -815.5394ms, within tolerance 1s)
2023/01/22 22:11:50 DEBUG : BackDate, 2023-01-21.txt: Unchanged skipping
2023/01/22 22:11:50 DEBUG : rclone.exe: Size and modification time the same (differ by 0s, within tolerance 1s)
2023/01/22 22:11:50 DEBUG : rclone.exe: Unchanged skipping
2023/01/22 22:11:50 DEBUG : OneDrive root 'testing': Waiting for transfers to finish
2023/01/22 22:11:50 DEBUG : Backup_to_OD.ps1: Starting multipart upload
2023/01/22 22:11:50 DEBUG : exclude-file.txt: Starting multipart upload
2023/01/22 22:11:51 DEBUG : exclude-file.txt: Uploading segment 0/75 size 75
2023/01/22 22:11:51 DEBUG : Backup_to_OD.ps1: Uploading segment 0/9208 size 9208
2023/01/22 22:11:51 DEBUG : exclude-file.txt: sha1 = cc7b4ec0af360c7dc55537b014ca66ba4c6ed551 OK
2023/01/22 22:11:51 INFO  : exclude-file.txt: Copied (new)
2023/01/22 22:11:51 DEBUG : Backup_to_OD.ps1: sha1 = 33e7365a9736674538690d17c547208e417ecf18 OK
2023/01/22 22:11:51 INFO  : Backup_to_OD.ps1: Copied (new)
2023/01/22 22:11:51 DEBUG : Waiting for deletions to finish
2023/01/22 22:11:51 INFO  :     9.065 KiB / 9.065 KiB, 100%, 0 B/s, ETA -
2023/01/22 22:11:51 DEBUG : 8 go routines active
2

Hi Ashim,

I guess this will get you pretty close to the desired output:

.\rclone.exe sync C:\rclone od:testing --progress --stats=24h

Here is an example:

PS> .\rclone.exe copy --ignore-times .\makefile1GB\ .\makefile1GBtarget\  --progress --stats=24h
Transferred:            1 GiB / 1 GiB, 100%, 562.244 MiB/s, ETA 0s
Checks:                 1 / 1, 100%
Transferred:            1 / 1, 100%
Elapsed time:         1.7s

You may increase the 24h if needed, just make it longer than your sync time.

3

What you want is --stats-one-line --stats-log-level NOTICE to just print the one line stats without the -v output.

Use the --stats parameter to control how often it gets printed. If you only want it to be printed at the end, set it to a large duration --stats 100y

4

Thank you @Ole . The printout is very close!:

Transferred:   	    9.065 KiB / 9.065 KiB, 100%, 9.058 KiB/s, ETA 0s
Checks:                 4 / 4, 100%
Transferred:            2 / 2, 100%
Elapsed time:         3.0s

Greatly appreciate your prompt support.... you all are light-years ahead of the paid corporations support!

1 Like
5

Hi @ncw , Thank you for your support. When I run:

PS C:\rclone> .\rclone.exe sync C:\rclone od:testing --stats-one-line --stats-log-level NOTICE

I get:

.\rclone.exe : 2023/01/23 09:23:15 NOTICE:     9.063 KiB / 9.063 KiB, 100%, 9.032 KiB/s, ETA 0s
At line:1 char:1
+ .\rclone.exe sync C:\rclone od:testing --stats-one-line --stats-log-l ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (2023/01/23 09:2...2 KiB/s, ETA 0s:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

Due to the above "NativeCommandError", I probably did something wrong but can't immediately figure out what.

Gracias otra vez.

6

This looks like the format of the logs you should be expecting with those flags.

That probably means the sync ended with an error. You'd need to look at the full log to see.

7

It doesn't look like a sync error because exactly the same files to be deleted and copied succeed with one set of flags but fail with the other set ---

Succeeds (the one by @Ole):
--progress --stats=24h

Fails:
--stats-one-line --stats-log-level NOTICE --stats=24h

I didn't check the log because of the above but can if you tell me to.

Actually, the solution by @Ole works perfectly so it is on longer important to figure out why the above line fails. In fact, that also gives the time taken, which I was implementing in PowerShell separately!

Thank you a ton for your kind, prompt and continuous support.

My thanks to everyone here too.

8

Your are welcome, happy we found a good solution!

Strange, I don't think the error message you see is (directly) related to the flags. I cannot reproduce.

9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.