Incorrect folder count after sync

Help and Support
1

What is the problem you are having with rclone?

I used copy and sync on two remotes and comparing them via Windows Explorer, it seems that they both have the same exact number and size of files however there is a big difference in the folder count. The source has more folders.
The source is a local crypt folder and the remote is a remote crypt inside of Dropbox.

Capture
Capture826×168 32.2 KB

What is your rclone version (output from rclone version)

rclone v1.57.0

  • os/version: Microsoft Windows 10 Pro 2009 (64 bit)
  • os/kernel: 10.0.19043.1288 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.17.2
  • go/linking: dynamic
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Dropbox and local (both encrypted)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

./rclone.exe sync -P pics-local: pics-dropbox:

The rclone config contents with secrets removed.

[dropbox]
type = dropbox
token = {"access_token":"","token_type":"bearer","refresh_token":"","expiry":""}

[backup-local]
type = alias
remote = D:\Backup

[pics-local]
type = crypt
remote = backup-local:pics
password =
password2 = 

[encrypted-dropbox]
type = crypt
remote = dropbox:/Encrypted
password = 
password2 = 

[pics-dropbox]
type = crypt
remote = dropbox:Backup/pics
password = 
password2 =

A log from the command with the -vv flag

The output was too big that I couldn't even capture it all.

2

Do you have 30 empty folders in source, by any chance? These will be ignored, unless this flag is used:

--create-empty-src-dirs   Create empty source dirs on destination after sync
3

hi,

i could be wrong, as still on first morning coffee.

  • that screenshot shows c:\...\dropbox\backup but both the rclone sync command and config makes no mention of it?
    what is that directory, something used by the dropbox client?
  • need to compare the two dir in the rclone sync command
4

The directories under the C and D drive are the actual locations.
How do I compare two directories? I have already used the sync command to sync them.

5

rclone check should work

6 
./rclone.exe check -P pics-local: pics-dropbox:
2021-11-02 19:35:34 ERROR : No common hash found - not using a hash for checks
2021-11-02 19:36:00 NOTICE: Encrypted drive 'pics-dropbox:': 0 differences found
2021-11-02 19:36:00 NOTICE: Encrypted drive 'pics-dropbox:': 7578 hashes could not be checked
2021-11-02 19:36:00 NOTICE: Encrypted drive 'pics-dropbox:': 7578 matching files
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:              7578 / 7578, 100%
Elapsed time:        27.9s

Looking at the documentation for the check command, it seems like it doesn't care about folders; only files.

7

getting confused, i thought you wanted to check two local directories.

./rclone.exe check -P pics-local: pics-dropbox:
that is local to cloud, and both are crypted remotes, correct?
if correct, perhaps try

8

One of the remotes is a local encrypted remote and the other copy of it is encrypted on Dropbox.
What I did is that I displayed the properties for each (not mounted and the Dropbox was not downloaded) because otherwise I believe if I wanted to mount the Dropbox copy and check its properties then it would have to download all its content.
So basically I am comparing two encrypted remotes; one locally and one on Dropbox.

Trying cryptcheck yielded the same result as check

➜  rclone-v1.57.0-windows-amd64 ./rclone.exe cryptcheck -P pics-local: pics-dropbox:
2021-11-05 22:51:54 NOTICE: too_many_requests/...: Too many requests or write operations. Trying again in 15 seconds.
2021-11-05 23:20:10 NOTICE: Encrypted drive 'pics-dropbox:': 0 differences found
2021-11-05 23:20:10 NOTICE: Encrypted drive 'pics-dropbox:': 7581 matching files
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:              7581 / 7581, 100%
Elapsed time:     35m36.9s

EDIT: The difference in count compared to my last post is because I removed some files.
9

did you look into that?

10

No. How do I check that?

11

if you do a quick internet search, you can find a bunch of ways.
https://www.raymond.cc/blog/easily-removing-empty-directories-from-windows/

for a given dir structure such as

rclone tree d:\root
/
├── 01.notempty
│   └── 01.txt
├── 02.notempty
│   └── 02.txt
├── 03.empty
└── 04.empty

in python this code

for (dirpath, dirnames, filenames) in os.walk(r'd:\root'):
    if len(dirnames) == 0 and len(filenames) == 0: print(dirpath)

would output

d:\root\03.empty
d:\root\04.empty

in powershell, this code

(gci d:\root -r | ? {$_.PSIsContainer -eq $True}) | ? {$_.GetFiles().Count -eq 0} | select FullName

would output

FullName
--------
D:\root\03.empty
D:\root\04.empty
12

Yea that helped a lot. I ended up with 1 folder difference instead of 8+ so I synced the two remotes the other way around so the extra directory got removed.
I still have the same number of files and total size so nothing was lost.

I guess what I learned is that sync doesn't copy empty folders.

Thanks for the help!

13

good that we got it worked out.

for that, you can use
--create-empty-src-dirs Create empty source dirs on destination after sync

14

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.