Sync, need to copy only certain subfolders

Help and Support
21

Try this --include="**zip/**"

22

This works in the exact folder structure you use:

C:\rclone>rclone ls F:\TEST --include */zip/**
        0 en07.keepass/zip/somefile in zip.txt

C:\rclone>

I was definitely making a mistake to begin with about the / in the start.
Either you have to make due without it, or else put a star before it.
Else you only match from the root.
Sorry, I'm still not well regimented in using the GLOB syntax even though I think I understand it now :wink:

23

Just FYI, the slash with a single * will restrict the matches to only directories which contain the zip folder. If you also want to match a zip directory in the root, you need to use the ** syntax.

24

very confusing,

i just noticed that for some reason the logfile is not logging the matches, just excludes.
the matches are sent to the command prompt console without the words include or match in it
en07.data/rclone/backup/u/dcmd/plugins/wcx/sevenzip/sevenzip.wcx
that is not what i was expecting???

seems that --include="**zip/**" includes:

  1. filenames containing zip like en07.data/rclone/backup/u/dcmd1/plugins-01/wcx/sevenzip/sevenzip.wcx
  2. foldrenames containting zip like en07.data/rclone/backup/l/go/go/src/compress/gzip/testdata/issue6550.gz
25

Yes, subtle but important distinction. I agree your command is better.
I am still learning :slight_smile:

26

Do you have any directories named zip in your root?

If not, it would be easier to use this instead: --include="**/zip/**"

27

You could also add another include to handle zip directories in the root:
--include="**/zip/**" --include="/zip/**"

28

thanks but that seems to match this, which is not what i want.
en07.rclone/rclone/backup/logs/calibre.zip/11111111.111111/calibre.zip_11111111.111111_exec.cmd

i just want to download subfolders named zip

there are no folders named zip in the root.

29

Did you try the latest command? For clarity, can you post the exact command and its output again?

It seems to be working for me:

Directory Structure:

Media/
├── 1.zip
├── calibre.zip
│   └── 11111111.111111
│       └── calibre.zip_11111111.111111_exec.cmd
├── test
│   ├── 2.zip
│   ├── test1
│   │   ├── 3.zip
│   │   ├── calibre.zip
│   │   │   └── 11111111.111111
│   │   │       └── calibre.zip_11111111.111111_exec.cmd
│   │   └── zip
│   │       └── a
│   └── zip
│       └── b
└── zip
    └── c

Command Output:

$ rclone ls Media/ --include="**/zip/**" --include="/zip/**"
        0 zip/c
        0 test/zip/b
        0 test/test1/zip/a
30

thanks,
now i am going to make this more complicated.
i only want to select folders containt exactly zip that are two levels deep.

so in your media example,
the only downloads i want is for only one folder
Meda/test/zip/ and everything below that.

i do not want
media/zip and media/test/test1/zip

thanks

31

This should do it: --include="/*/*/zip/**"

32

I concur.
--max-depth would might also be useful if you needed something on fodlers on levels 0,1,2 , but not deeper than that (that would exclude any subfolders below that in zip too though). Just for reference, I know that's not exactly what you asked for here :slight_smile:

33

this did not work"/*/zip/**"

but this worked just fine and now it makes sense to me
%root%\rclone.exe ls wasabiwest01:en07 --include="/*/zip/**" %logfile% --dry-run

i had said that i wanted two levels deep but that was including the bucket as a level.

what i needed to ask for was just one level deep based on my source of wasabiwest01:en07
wasabiwest01:en07/*/zip/**

seems so simple now.

super thanks to both of you

34

yes, thanks Darth for the assist :slight_smile:

And yes, it really is a lot LESS complicated than you expect once you manage to wrap your head around the structure. (and don't confuse it with REGEX like I did hehe)

The way I like to think about it in my head is to visualize or write the filepath of what I want to match.
C:\test\en07.keepass\zip\somefile.txt

Then we can literally just subtitute stuff.

  • means "everything except / "
    ** means "everything including / "

So doing just that you are 80% done.
The res is just a few special rules - like anything starting with / matches from the root only. Those special exceptions you just kinda have to memorize, but there aren't many.

check out classes too, like [a-f]*
ie "any file that starts with a,b,c,d,e or f
I use those a lot

35

thanks
........

36

This is conceptually exactly how the filtering works, with the one caveat that the filepath is the relative filepath from the root of the transfer.

...

Maybe I should write a filter explorer

This would be a web page with an example file structure (editable) and a place to put the various rclone filter flags

It would then show you what got filtered.

With a bit of gopherjs it could even be written using the existing rclone code.

37

I'd rather you let us set filters like this in the remote personally - but yea - a lot of people (even the fairly technical users) struggle with understanding the GLOB format. That included me in the start - especially since I thought the format was REGEX - which it is almost, but not quite :stuck_out_tongue:

It's actually fairly easy and intuitive once you manage to wrap your head around it, but that takes a little studying and experimenting unless you are already familiar with it from Linux syntax.

38

perhaps, the filter explorer could use the output from the various rlcone ls commands?

39

That is a really good idea. It would be much more intuitive work on your own structure. Presumably it would not be terribly hard to import from a format like that.

40

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.