Rclone not copying subfolders on ACD sync

Not sure if this is a bug or if its me not using it right. I was using SyncBackPro to sync my entire drive on my machine to ACD. Its not a bad tool, but I feel it fails alot and wanted to try rclone as its basically like rsync (which i use already to sync this same drive to another remote machine) and familiar with that.

I tried this as a test and noticed it ran fine and synced a folder (which i added a test) but didnt copy the subfolder inside that (in this case a log folder) which also had a single log file in there. It didnt sync that folder or even add it to ACD.

Here is the command Im running just in case Im not doing something right.

rclone sync d:\ ACD: --checksum --log-file=%LOGFILENAME% --exclude="$RECYCLE.BIN" --exclude=“System Volume Information” --exclude=“2744NVR” -v

You can ignore the LOGFILENAME variable, its part of my BAT file that I run that creates a preformatted logfile name which I can then use to email as well to me after its done.

Dixit

[note that you’ll need the latest beta to use multiple --exclude on the command line]

Was the folder empty? rclone doesn’t copy empty folders.

The folders and subfolders were not empty. Originally I just took the rclone folder itself (also created a logs folder and a log file under that subfolder) and copied it to my D drive as shown in my command above, it copied the Rclone directory, but not the log folder or file.

I then tried this again by just creating a sub folder under the D drive, called testfolder. Then created a text file inside that with just 10 characters in the text file, named it file1. Then did this again with another subfolder under that testfolder directory called subdirectory1, and again same type of text file (different 10 chars). Still didn’t see those folders get created with the files.

Thanks for the heads up on the multiple exclude and need to use the latest beta for that to work, that explains why I kept seeing it look at System Volume Information and all those errors around that directory. Let me try it on latest beta and see what happens. I remember seeing that it would fail after 3 attempts on that directory (System volume information) and then just what seemed to be exiting the rclone process.

Dixit

That is likely the problem - if rclone can’t complete the directory listing, then it will stop transferring.

I tried the following beta “rclone v1.34-75-gcbfec0dβ” and still getting the multiple exclude issue, its still looking at System Volume Information even though its on the exclude list, here is the log of it.

2016/12/31 11:49:37 amazon drive root ‘’: Waiting for checks to finish
2016/12/31 11:49:42 amazon drive root ‘’: Waiting for transfers to finish
2016/12/31 11:49:42 amazon drive root ‘’: not deleting files as there were IO errors
2016/12/31 11:49:42 Attempt 1/3 failed with 0 errors and: error listing source: Local file system at \?\d:: failed to open directory “\\?\d:\System Volume Information”: open \?\d:\System Volume Information: Access is denied.
2016/12/31 11:52:15 amazon drive root ‘’: Waiting for checks to finish
2016/12/31 11:53:02 amazon drive root ‘’: Waiting for transfers to finish
2016/12/31 11:53:02 amazon drive root ‘’: not deleting files as there were IO errors
2016/12/31 11:53:02 Attempt 2/3 failed with 0 errors and: error listing source: Local file system at \?\d:: failed to open directory “\\?\d:\System Volume Information”: open \?\d:\System Volume Information: Access is denied.
2016/12/31 11:55:35 amazon drive root ‘’: Waiting for checks to finish
2016/12/31 11:55:50 amazon drive root ‘’: Waiting for transfers to finish
2016/12/31 11:55:50 amazon drive root ‘’: not deleting files as there were IO errors
2016/12/31 11:55:50 Attempt 3/3 failed with 0 errors and: error listing source: Local file system at \?\d:: failed to open directory “\\?\d:\System Volume Information”: open \?\d:\System Volume Information: Access is denied.
2016/12/31 11:55:50 Failed to sync: error listing source: Local file system at \?\d:: failed to open directory “\\?\d:\System Volume Information”: open \?\d:\System Volume Information: Access is denied.

Same exact command as first post.
So cant test yet if its doing the subfolders.

Dixit

Try

--exclude="System Volume Information/**"

and see if that helps

1 Like

Still not working in my opinion. Notice the first exclude is the one for the Recycle Bin, I just ran this and noticed that folder now exists on my Amazon Drive, so therefore it uploaded/synced that folder to Amazon when it was supposed to be excluded. Exact folder name off Amazon Drive “$RECYCLE.BIN” so matches the exclude option but didn’t seem to work. Seems like its just not even recognizing the exclude option.

Dixit

:frowning:

Is there anything in that directory?

If you add the --dump-filters option to your command line then rclone will print an internal representation of the filters - can you paste that here please and we can see what is going on.

Here is the start of that command and the filters dump.

C:\Program Files\rclone>rclone sync d:\ ACD: --stats=0 --checksum --dump-filters --exclude="$RECYCLE.BIN" --exclude="System Volume Information/**" --exclude="2744NVR" -v
--- start filters ---
--- File filter rules ---
- (^|/)\$RECYCLE\.BIN$
- (^|/)System Volume Information/.*$
- (^|/)2744NVR$
--- Directory filter rules ---
- (^|/)System Volume Information/.*$
--- end filters ---
2017/01/04 22:13:13 rclone: Version "v1.34-75-gcbfec0dβ" starting with parameters ["rclone" "sync" "d:\\" "ACD:" "--stats=0" "--checksum" "--dump-filters" "--exclude=$RECYCLE.BIN" "--exclude=System Volume Information/**" "--exclude=2744NVR" "-v"]
2017/01/04 22:13:13 ACD: Saved new token in config file

The Recycle Bin is a system hidden directory under Windows OS, I think this is where it puts the files you put in the Recycle Bin before you empty it, so don’t want that sync’ed, no need for it. Then the System Volume Information is another Windows OS system folder that’s hidden. The 2744NVR definitely has files in there, maybe 2TB worth. Recycle Bin I just looked and says it has 1 file and 2 folders. System Volume Information has nothing in there.

Dixit

Ok I think I answered my own question/issue after looking at the post (should’ve caught this before submitting the post), but clearly you can see its only ignoring the System Volume Information from a directory perspective because of the /** at the end of it. So I added it to the Recycle Bin and 2744NVR one as well cause I don’t want that synced. Here is the new filters dump.

C:\Program Files\rclone>rclone sync d:\ ACD: --stats=0 --checksum --dump-filters --exclude="$RECYCLE.BIN/**" --exclude="System Volume Information/**" --exclude="2744NVR/**" -v
--- start filters ---
--- File filter rules ---
- (^|/)\$RECYCLE\.BIN/.*$
- (^|/)System Volume Information/.*$
- (^|/)2744NVR/.*$
--- Directory filter rules ---
- (^|/)\$RECYCLE\.BIN/.*$
- (^|/)System Volume Information/.*$
- (^|/)2744NVR/.*$
--- end filters ---
2017/01/04 22:22:08 rclone: Version "v1.34-75-gcbfec0dβ" starting with parameters ["rclone" "sync" "d:\\" "ACD:" "--stats=0" "--checksum" "--dump-filters" "--exclude=$RECYCLE.BIN/**" "--exclude=System Volume Information/**" "--exclude=2744NVR/**" "-v"]
2017/01/04 22:22:08 amazon drive root '': Modify window not supported

My only concern slightly is that the top part of the filters dump shows that if any file has a 2744NVR it will be ignored, or the other two that ended up under the file filter. Let me now run this to make sure its syncing the original test folder I created to make sure its syncing the sub folders and what not which was one of the original issues.

Appreciate it.

Dixit

Ok I think we got it to work, it copied over my test folder, and the two sub folders under that as well as the sample text file we had in there as well. I removed that test folder and re-ran it and noticed it deleted the sample text files but left the directory there and each sub directory was empty.

But after reading your rclone sync guide, it seems that its by design, it only copies the content of the directory, but no the directory. So I guess when it deletes a file it leaves this behind. Is there an option for it really to be the same? I know other paid tools like SyncBackPro does this using Amazon’s APIs. But I have spotty success with that since it fails at times on large files and ends up requiring 2-3 attempts to get it to go through. I’ll keep playing with this more as I like the simply concept of this since it basically matches the rsync concept.

Dixit

That is https://github.com/ncw/rclone/issues/100 - I’ve made a work around with the rclone rmdirs command which will recursively delete empty directories.

Thanks appreciate it Nick. Im going to keep playing with this more.

Dixit