@thestigma, help me out. i got a time crunch.
i need to download an entire bucket to my local server.
for that bucket i want to download only subfolders named zip but keep the directory structure.
i tried a few different ways but not working.
I recently picked up some new understanding of this, so I think I can help...
PS: for your reference, this is not regex, but BLOB (from Linux, used in grep)
They are subtly different, so don't go look at regex guides, look at BLOB guides, or else you will spend a lot of time confused like I was
So I think what you need to match a fodler named zip is: --include /zip/ (exact match
or --include /*zip*/ (partial match).
so expand on that, this would be "any files inside a folder called "zip" --include /zip/*
And this would be "any files AND subfolders inside a folder named "zip" --include /zip/**
So the last one should be what you specify you need.
Please do a quick test with that...
rclone ls /location --include /zip/** --max-depth 3
(the last parameter is how far down in the structure to recurse, limit this to avoid unnecessary spam in output)
i tried
%root%\rclone.exe ls wasabiwest01:en07 --include /zip/** %logfile% --dry-run --log-level=DEBUG
and
my previous
%root%\rclone.exe ls wasabiwest01:en07 --include /zip/** %logfile% --dry-run --log-level=DEBUG
and each time the only ouput i get is:
2019/10/30 13:14:48 DEBUG : en07.calibre: Excluded
2019/10/30 13:14:48 DEBUG : en07.data: Excluded
2019/10/30 13:14:48 DEBUG : en07.firefox: Excluded
2019/10/30 13:14:48 DEBUG : en07.kdbx: Excluded
2019/10/30 13:14:48 DEBUG : en07.keepass: Excluded
No, you shouldn't try to match folders on their own. that's not really possible. That was a partial example for illustration. You always need to specify what inside the fodlers you are trying to match for (even if it is "everything", defined by ** )
I am not sure how your folder structure looks so it's a little hard to see what's wrong.
the "zip" folder is next to these files you listed? it should be hitting that with /zip/**
Try /*zip*/** maybe to include any folders that have zip somewhere in the name (partial match).
all of these give the same output
%root%\rclone.exe ls wasabiwest01:en07 --include /zip/ %logfile% --dry-run --log-level=DEBUG
%root%\rclone.exe ls wasabiwest01:en07 --include /zip/ %logfile% --dry-run --log-level=DEBUG
%root%\rclone.exe ls wasabiwest01:en07 --include /zip/* %logfile% --dry-run --log-level=DEBUG
%root%\rclone.exe ls wasabiwest01:en07 --include /zip/** %logfile% --dry-run --log-level=DEBUG
damn it, this forum is mangling my commands, there has to be a way for the forum not to convert and pervert posts!
and the path is as per the command
wasabiwest01:en07
BTW, is this on a windows or bash terminal?
Because on Linux there is a risk that some GLOB commands can be interpreted as REGEX.
TO avoid mistakes of that nature, put quotes around it. It may be that simple.
From your debug output though, it looks like for some reason it is not accepting the folders with periods (.) in the name. That might be the problem.
I don't exactly know why it would be causing an issue though - it should't matter in that pattern
but I will do a quick test here to see if I can replicate that
thanks,
it is not because of a dot in the folder name
the folder firefox does not have a dot and there is a subfolder named zip inside it.
the ls command list not listing that zip subfolder.
i tried --include *zip*/** and still the exact same output each and every time.
%root%\rclone.exe ls wasabiwest01:en07 --include "*zip*/**" %logfile% --dry-run --log-level=DEBUG
%root%\rclone.exe ls wasabiwest01:en07 --include="/zip/" %logfile% --dry-run --log-level=DEBUG
%root%\rclone.exe ls wasabiwest01:en07 --include="/*zip*/" %logfile% --dry-run --log-level=DEBUG
%root%\rclone.exe ls wasabiwest01:en07 --include="/zip/*" %logfile% --dry-run --log-level=DEBUG
%root%\rclone.exe ls wasabiwest01:en07 --include="/zip/**" %logfile% --dry-run --log-level=DEBUG
%root%\rclone.exe ls wasabiwest01:en07 --include="/*zip*/**" %logfile% --dry-run --log-level=DEBUG
