Rclone check only checking one file at a time

Help and Support
1

What is the problem you are having with rclone?

  • rclone check is only checking one file at a time. does not seem to respect --checkers
  • is there a way to get rclone to display the progress of checking each file.
    when uploading a file, rclone will display progress as percent complete and the ETA.

thanks in advance

What is your rclone version (output from rclone version)

rclone v1.52.0
- os/arch: windows/amd64
- go version: go1.14.3

Which OS you are using and how many bits (eg Windows 7, 64 bit)

win10.2004.64

Which cloud storage system are you using? (eg Google Drive)

wasabi s3

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone.exe check "b:\en07.veaam_checkfiles_wasabi_20200607.152939\EN07\en07.veaamfull" wasabieast2:vserver03\en07.veaam\rclone\backup\ --stats=0 --progress --checkers=8 --log-level=DEBUG --log-file=C:\data\rclone\logs\en07.veaam_checkfiles_wasabi\20200607.152939\en07.veaam_20200607.152939_rclone.log

The rclone config contents with secrets removed.

not a config issue

A log from the command with the -vv flag

2020/06/07 15:29:43 DEBUG : rclone: Version "v1.52.0" starting with parameters ["C:\\data\\rclone\\scripts\\rclone.exe" "check" "b:\\en07.veaam_checkfiles_wasabi_20200607.152939\\EN07\\en07.veaamfull" "wasabieast2:vserver03\\en07.veaam\\rclone\\backup\\" "--stats=0" "--progress" "--checkers=8" "--log-level=DEBUG" "--log-file=C:\\data\\rclone\\logs\\en07.veaam_checkfiles_wasabi\\20200607.152939\\en07.veaam_20200607.152939_rclone.log"]
2020/06/07 15:29:43 DEBUG : Using RCLONE_CONFIG_PASS password.
2020/06/07 15:29:43 DEBUG : Using config file from "C:\\data\\rclone\\scripts\\rclone.conf"
2020/06/07 15:29:43 DEBUG : fs cache: renaming cache item "b:\\en07.veaam_checkfiles_wasabi_20200607.152939\\EN07\\en07.veaamfull" to be canonical "//?/b:/en07.veaam_checkfiles_wasabi_20200607.152939/EN07/en07.veaamfull"
2020/06/07 15:29:43 DEBUG : S3 bucket vserver03 path en07.veaam/rclone/backup: Waiting for checks to finish
2020/06/07 15:29:43 ERROR : EN07/EN07.vbm: Sizes differ

image

2

Rclone only checks one file from a directory at the moment.

Are all the files in one directory?

We should really fix this!

3

yes, in this case all files are in one directory.

a fix would be nice.

  • respect --checkers
  • give ETA and percentage done.

thanks

1 Like
4

I just had a look at the source to remind myself. rclone check will only do one check at once which isn't brilliant!

Can you please make a new issue on github about that please :slight_smile:

Do you mean for checking here? That might be a bit trickier. Rclone can count the numbre of files easily enough but there isn't a general mechanism yet for progress in reading md5sums.

5

yeah,
rclone is getting to be a large project.

i created a new request.

i understand about the md5sums and showing progress being tricky.
not a major problem.

thanks,

6

Thanks! I'll take a look at in in a short while!

7

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.