Source child directory unexpectedly skipped during sync

Suspected Bug
1

What is the problem you are having with rclone?

I have a directory that contains several git repositories that I sync to a network share. I noticed that one repo is not synchronizing the \.git\refs directory, but others are.

For example, the following source repo does have the missing directory:

C:\>dir scm\dotnet\BackupUtility\.git\refs
 Volume in drive C is Windows
 Volume Serial Number is 2AD6-0B09

 Directory of C:\scm\dotnet\BackupUtility\.git\refs

2022-06-02  06:59 PM    <DIR>          .
2022-06-04  08:14 PM    <DIR>          heads
2022-05-28  04:47 PM    <DIR>          remotes
2022-05-23  10:46 AM    <DIR>          tags
               0 File(s)              0 bytes
               4 Dir(s)  919,788,605,440 bytes free

However, the destination is populated with all but that directory:

C:\>dir \\storage\backup\hosts\wkst.activenetwerx.com\scm\dotnet\BackupUtility\.git\refs
 Volume in drive \\storage\backup is backup
 Volume Serial Number is 8629-4E9E

 Directory of \\storage\backup\hosts\wkst.activenetwerx.com\scm\dotnet\BackupUtility\.git

File Not Found

I have included a verbose log, and I don't see any mention of the source directory.

Run the command 'rclone version' and share the full output of the command.

rclone v1.58.1
- os/version: Microsoft Windows 11 Enterprise 21H2 (64 bit)
- os/kernel: 10.0.22000.675 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.17.9
- go/linking: dynamic
- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

I am not using an external provider, I use rclone to sync from a local drive to a Windows network share.

The command you were trying to run (eg rclone copy /tmp remote:tmp)

"C:\Program Files\rclone\rclone.exe"^
  sync^
  -vv^
  --log-file=c:\users\jcasale\desktop\rclone.log^
  --skip-links^
  --checksum^
  --delete-excluded^
  --filter "- /scm/**/.vs/**"^
  --filter "- /scm/**/bin/**"^
  --filter "- /scm/**/obj/**"^
  --filter "- /scm/**/dist/**"^
  --filter "- /scm/**/node_modules/**"^
  --filter "- /scm/**/packages/**"^
  --filter "- /scm/**/wwwroot/dist/**"^
  --filter "- /scm/**/wwwroot/lib/**"^
  --filter "- /scm/**/__pycache__/**"^
  --filter "- /scm/.lib/**"^
  --filter "- /scm/docs/**/*.aux"^
  --filter "- /scm/docs/**/*.dvi"^
  --filter "- /scm/docs/**/*.fls"^
  --filter "- /scm/docs/**/*.lof"^
  --filter "- /scm/docs/**/*.log"^
  --filter "- /scm/docs/**/*.lot"^
  --filter "- /scm/docs/**/*.out"^
  --filter "- /scm/docs/**/*.synctex.gz"^
  --filter "- /scm/docs/**/*.toc"^
  --filter "+ /scm/**"^
  --filter "- /**"^
  C:/^
  //storage/backup/hosts/wkst.activenetwerx.com

The rclone config contents with secrets removed.

I am not using a configuration file.

A log from the command with the -vv flag

I have a gist created at: Rclone source child directory unexpectedly skipped during sync ยท GitHub

2

My first thought is that your filter is somehow skipping it.

Can you change your script to do lsf instead and see if it is in the output? Then try removing the filters and see if it re-apppears?

The other thought is - are there any files in any of the subdirectories? If not, rclone won't create the empty directories - you need --create-empty-src-dirs for that.

3

Hi,
The source had files in the directories, I was watching it throughout the day as I worked and sync'ed it, and it continued to be skipped.

At some point, the parent directory changed considerably and it started getting synchronized.

I am now watching the whole root and once I see this again, I will run the commands you requested and try to preserve it.

4

Hmm, that sounds more like a bug in either the directory enumeration code or the network filing system.

Let me know if it happens again. Either way, the directory should be missing from an rclone lsf of the parent directory when the problem is happening.

5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.