I am trying to copy few files from many subdirectories. I need most of them to be ignored so I played with include/exclude options and filters but dry-run still listed files what should not be.
Folder structure is follow on source:
DIRECTORY1//DIRECTORY3//DIRECTORY4/DIRECTORY5/y=2020/m=12/d=03/h=07/m=00/file
As you can see I have wildcarded directories. There is not a lot of them but then I have also months, days, hours as a folder names and this increase number of directories.
I tried to use for example:
--filter '+ y=2021/m=01/d=29/h=10/m=00//file' --filter ' - *'
but debug mode still show also another folders
y=2020/m=12/d=03/h=07/m=00/file: Excluded
Is there any way how to filter only files defined in include or filter option? Otherwise it will take a lot of time for listing. Also not sure how this will affect transactions.
rclone filters files, not directories.
for a given source path, rclone will visit each and every subfolder, looking for files to include and exclude.
so a listing can take a lot of time.
if the source is local, then there are no transactions.
Thank you for feedback.
I just installed older version from stable repo for Debian. I had also installed latest version but as first tests works with both I kept older version.
So filters will not help me to "target" better needed files. Source is in the cloud, so now I am not sure if this will not touch also "Cold" storage.
If there is some hint how to achieve best results please share.
not sure what you mean about touching the storage.
rclone will have to read a list of files, which will use API calls.
you can reduce that using --fast-list
try to enclose text with backticks so we can see the exact folder structure. DIRECTORY1/*/DIRECTORY3/*/DIRECTORY4/DIRECTORY5/y=2020/m=12/d=03/h=07/m=00/file
versus
DIRECTORY1//DIRECTORY3//DIRECTORY4/DIRECTORY5/y=2020/m=12/d=03/h=07/m=00/file