How can I audit the files I am copying with rclone?

What is the problem you are having with rclone?

I would like to count the number of files that have been copied remotely.

Run the command 'rclone version' and share the full output of the command.

rclone 1.60.1

  • os/version: fedora 37 (64 bit)
  • os/kernel: 6.0.11-300.fc37.x86_64 (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.19.3
  • go/linking: dynamic
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone ls gdrive:path/to/files | wc -l

Hello, everyone. Sorry if I didn't follow this template 100%. I'm not sure if I am asking this in the correct place.
Anyway, what I am trying to do is audit the files I am copying using rclone. For example, let's say I have a local path like:


Let's assume that path has two folders each with two files (four files total).

What is a good way to count what's been copied remotely so that I can compare to what I have locally?

I was thinking of doing a simple LS but pipe it to a line counter but I'm not sure how accurate that is or how it deals with directories.

Any ideas on a better way to audit what is uploaded vs. what I have locally? Keeping in mind I have over 1 millions files I probably would need to break down the audits into smaller folders.

Thanks for any suggestions.

Hi cbmeeks,

You could use rclone lsl something like this:

rclone lsl /files/mydata/path/to/files | wc -l
rclone lsl gdrive:path/to/files | wc -l

Edit: You may have to add --stats=0 to disable stats from interfering, not sure.

Here are some other possibilities:

rclone sync --dry-run /files/mydata/path/to/files gdrive:path/to/files

rclone check /files/mydata/path/to/files gdrive:path/to/files
rclone check --download /files/mydata/path/to/files gdrive:path/to/files

rclone md5sum gdrive:path/to/files
rclone md5sum --download /files/mydata/path/to/files

You can see an overview of all the available commands here:

Thanks! I looked at the docs but obviously not close enough.

I will do more research next time.

No problem, the docs can be overwhelming, especially when you don't know exactly what you are looking for.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.