# Copy multiple directories wildcard?

**URL:** <https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816>\
**Category:** Help and Support\
**Created:** [March 13, 2021, 10:26pm UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816 "2021-03-13T22:26:58Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![aula](https://forum.rclone.org/letter_avatar_proxy/v4/letter/a/7ab992/32.png) [@aula](https://forum.rclone.org/u/aula)\
**Post date:** [March 13, 2021, 10:26pm UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816/1 "2021-03-13T22:26:58Z")

</div>

#### What is the problem you are having with rclone?

I need to copy a directory list. I have  
dir/2021-01-01/dir2  
dir/2021-01-02/dir2  
dir/2021-01-03/dir3  
...  
The dates go from 2021-01-01 until 2021-08-31  
Is there any way to do this with only one command, or with filters file.txt?

#### What is your rclone version (output from `rclone version`)

1.54.1

#### Which OS you are using and how many bits (eg Windows 7, 64 bit)

Debian, 64b

#### Which cloud storage system are you using? (eg Google Drive)

Google Drive

#### The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone copy remote:dir/\*/dir2 /local\_dir

#### The rclone config contents with secrets removed.

#### A log from the command with the `-vv` flag

---

<div class="post-metadata">

**Author:** ![asdffdsa](https://forum.rclone.org/letter_avatar_proxy/v4/letter/a/848f3c/32.png) [@asdffdsa](https://forum.rclone.org/u/asdffdsa)\
**Post date:** [March 13, 2021, 10:54pm UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816/2 "2021-03-13T22:54:00Z")

</div>

you can use  
[https://rclone.org/filtering/#include-from-read-include-patterns-from-file](https://rclone.org/filtering/#include-from-read-include-patterns-from-file)

not sure exactly what you are asking for..  
if you want a one-line command, this could work.  
`rclone ls . --include='/dir/2021-0{1,2,3,4,5,6,7,8}-??/**'`

---

<div class="post-metadata">

**Author:** ![aula](https://forum.rclone.org/letter_avatar_proxy/v4/letter/a/7ab992/32.png) [@aula](https://forum.rclone.org/u/aula)\
**Post date:** [March 14, 2021, 10:24am UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816/3 "2021-03-14T10:24:04Z")

</div>

Hello, and thanks. I have seen an error on my message.  
I need to copy a directory list. I have  
dir/2021-01-01/dir2  
dir/2021-01-02/dir2  
dir/2021-01-03/dir2  
...  
dir/2021-08-31/dir2

Note that the dir2 is the same on all cases. And the data are in Google Drive.  
So, is it correct?  
rclone copy remote --include='/dir/2021-0{1,2,3,4,5,6,7,8}-??/dir2/\*\*'

Another question is the order of the execution. I need run first the more recent dates:

1. rclone copy remote:dir/2021-08-31/dir2 destination
2. rclone copy remote:dir/2021-08-30/dir2 destination
3. rclone copy remote:dir/2021-08-29/dir2 destination  
...  
rclone copy remote:dir/2021-01-01/dir2 destination

Is it possible run with an order?  
Thanks!!

---

<div class="post-metadata">

**Author:** ![aula](https://forum.rclone.org/letter_avatar_proxy/v4/letter/a/7ab992/32.png) [@aula](https://forum.rclone.org/u/aula)\
**Post date:** [March 14, 2021, 12:56pm UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816/4 "2021-03-14T12:56:13Z")

</div>

I have try  
rclone copy remote: destination --include='/dir/2021-0{1,2,3,4,5,6,7,8}-??/dir2/**'  
and it runs ok, but I obtain multiple destination folders of "dir2"  
destination/dir/2021-01-01/dir2/**  
..  
destination/dir/2021-08-31/dir2/\*\*

But I wish to combine all on the same "dir2" folder. I'd like to have all files and folders on  
destination/dir2/\*\*

Could you help me?  
Thanks!

---

<div class="post-metadata">

**Author:** ![asdffdsa](https://forum.rclone.org/letter_avatar_proxy/v4/letter/a/848f3c/32.png) [@asdffdsa](https://forum.rclone.org/u/asdffdsa)\
**Post date:** [March 14, 2021, 3:03pm UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816/5 "2021-03-14T15:03:17Z")

</div>

need an example,  
can you post a detailed example of the source  
and what you want the dest to look like

---

<div class="post-metadata">

**Author:** ![aula](https://forum.rclone.org/letter_avatar_proxy/v4/letter/a/7ab992/32.png) [@aula](https://forum.rclone.org/u/aula)\
**Post date:** [March 14, 2021, 3:24pm UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816/6 "2021-03-14T15:24:56Z")

</div>

Yes,  
I have on SOURCE (Google Drive):  
temp/2021-08-31/folderA  
temp/2021-08-30/folderA  
temp/2021-08-29/folderA  
...  
More than 200 "folderA" in different dates

And I want to merge all "folderA" in one folder in local. With all the contents (subdirs and files)  
On DESTINATION I wish: Only "folderA".

Note:  
I want to save the more recent file (if the file is duplicate).  
Example:  
SOURCE  
temp/2021-08-31/folderA/file1.doc  
temp/2021-03-30/folderA/file1.doc  
Then I want on DESTINATION  
folderA/file1.doc (the file1 of 2021-08-31)

Many thanks

---

<div class="post-metadata">

**Author:** ![asdffdsa](https://forum.rclone.org/letter_avatar_proxy/v4/letter/a/848f3c/32.png) [@asdffdsa](https://forum.rclone.org/u/asdffdsa)\
**Post date:** [March 14, 2021, 3:43pm UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816/7 "2021-03-14T15:43:09Z")

</div>

try `rclone ls /path/to/temp --include="/2021-??-??/folderA/**"`  
get that working first

for the dest, just choose a location like `gdrive:folderA`

as for most recent files, `rclone copy, by default compares size and mod-time.  
if you want to use only mod-time, then add [https://rclone.org/docs/#ignore-size](https://rclone.org/docs/#ignore-size)

---

<div class="post-metadata">

**Author:** ![aula](https://forum.rclone.org/letter_avatar_proxy/v4/letter/a/7ab992/32.png) [@aula](https://forum.rclone.org/u/aula)\
**Post date:** [March 14, 2021, 5:10pm UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816/8 "2021-03-14T17:10:12Z")

</div>

Better, but the problem whit your command is that the result are multiple folders and I only want one folder (merge).  
In the example I have got:  
temp/2021-08-31/folderA/file1.doc  
temp/2021-03-30/folderA/file1.doc

But I'd like to obtain:  
folderA/file1.doc

Thanks!!!

---

<div class="post-metadata">

**Author:** ![asdffdsa](https://forum.rclone.org/letter_avatar_proxy/v4/letter/a/848f3c/32.png) [@asdffdsa](https://forum.rclone.org/u/asdffdsa)\
**Post date:** [March 14, 2021, 5:18pm UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816/9 "2021-03-14T17:18:13Z")

</div>

post your command.

---

<div class="post-metadata">

**Author:** ![aula](https://forum.rclone.org/letter_avatar_proxy/v4/letter/a/7ab992/32.png) [@aula](https://forum.rclone.org/u/aula)\
**Post date:** [March 14, 2021, 5:22pm UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816/10 "2021-03-14T17:22:40Z")

</div>

rclone copy remote:proves --include="/2021-??-??/folderA/\*\*" folderA --ignore-size

---

<div class="post-metadata">

**Author:** ![asdffdsa](https://forum.rclone.org/letter_avatar_proxy/v4/letter/a/848f3c/32.png) [@asdffdsa](https://forum.rclone.org/u/asdffdsa)\
**Post date:** [March 14, 2021, 5:24pm UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816/11 "2021-03-14T17:24:09Z")

</div>

you are copying from gdrive to local?

---

<div class="post-metadata">

**Author:** ![aula](https://forum.rclone.org/letter_avatar_proxy/v4/letter/a/7ab992/32.png) [@aula](https://forum.rclone.org/u/aula)\
**Post date:** [March 14, 2021, 5:24pm UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816/12 "2021-03-14T17:24:40Z")

</div>

yes. Remote is Google drive

---

<div class="post-metadata">

**Author:** ![asdffdsa](https://forum.rclone.org/letter_avatar_proxy/v4/letter/a/848f3c/32.png) [@asdffdsa](https://forum.rclone.org/u/asdffdsa)\
**Post date:** [March 14, 2021, 5:28pm UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816/13 "2021-03-14T17:28:55Z")

</div>

still not sure exactly what you want, need a better example of what the dest should looke like?  
does `folderA` have subfolders? and if yes, what do you want to do with the subfolders?

perhaps try  
`--include="/2021-??-??/folderA/\*"

---

<div class="post-metadata">

**Author:** ![aula](https://forum.rclone.org/letter_avatar_proxy/v4/letter/a/7ab992/32.png) [@aula](https://forum.rclone.org/u/aula)\
**Post date:** [March 14, 2021, 5:35pm UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816/14 "2021-03-14T17:35:07Z")

</div>

Ok, I will try to put an easy example.  
SOURCE (Google Drive)  
proves/2021-01-01/folderA/myfile.jpg  
proves/2021-01-01/folderA/subfolderX/document.doc  
proves/2021-01-02/folderA/myfile.jpg  
proves/2021-01-03/folderA/myfile.jpg

The folderA can have subfolders

All I want on the DESTINATION (local) is:  
folderA/myfile.jpg (only the most recent)  
folderA/subfolderX/document.doc

---

<div class="post-metadata">

**Author:** ![aula](https://forum.rclone.org/letter_avatar_proxy/v4/letter/a/7ab992/32.png) [@aula](https://forum.rclone.org/u/aula)\
**Post date:** [March 14, 2021, 5:36pm UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816/15 "2021-03-14T17:36:46Z")

</div>

I only want one folderA  
And I obtain multiple folderA

By now I have obtained the same structure that on SOURCE:  
folderA/2021-01-01/folderA/myfile.jpg  
folderA/2021-01-01/folderA/subfolderX/document.doc  
folderA/2021-01-02/folderA/myfile.jpg  
folderA/2021-01-03/folderA/myfile.jpg

---

<div class="post-metadata">

**Author:** ![asdffdsa](https://forum.rclone.org/letter_avatar_proxy/v4/letter/a/848f3c/32.png) [@asdffdsa](https://forum.rclone.org/u/asdffdsa)\
**Post date:** [March 14, 2021, 6:12pm UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816/17 "2021-03-14T18:12:30Z")

</div>

not a linux exper but you want some like

`rclone lsf --format=p --include="2021-??-??/folderA/**" gdrive:folderA --recursive --files-only > files.txt`

then cut the start of each line to remove the `2021-08-31/folderA/`

`rclone copy --files-from-raw files.txt gdrive:folderA /path/to/local`

---

<div class="post-metadata">

**Author:** ![albertony](https://forum.rclone.org/user_avatar/forum.rclone.org/albertony/32/8586_2.png) [@albertony](https://forum.rclone.org/u/albertony)\
**Post date:** [March 14, 2021, 8:24pm UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816/18 "2021-03-14T20:24:52Z")

</div>

I'm not a linux expert either, unfortunately. Just playing around until one appears...

I think something like this should work. It will not be _one_ rclone command, though, it will execute a rclone copy command for each unique directory at level `proves/2021-01-01/folderA`.

```auto
for f in $(rclone lsf --recursive --include="/2021-??-??/folderA/**" --files-only gdrive:proves | cut -d/ -f-2 | sort -ru); do rclone copy $f $(echo $f | cut -d/ -f2) --ignore-existing; done

```

---

<div class="post-metadata">

**Author:** ![aula](https://forum.rclone.org/letter_avatar_proxy/v4/letter/a/7ab992/32.png) [@aula](https://forum.rclone.org/u/aula)\
**Post date:** [March 14, 2021, 8:30pm UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816/19 "2021-03-14T20:30:15Z")

</div>

Thank you both.

---

<div class="post-metadata">

**Author:** ![system](https://forum.rclone.org/uploads/default/original/2X/3/39ad3235f6ffec76ccd0c81616bd55697b4c300d.png) [@system](https://forum.rclone.org/u/system)\
**Post date:** [May 14, 2021, 4:31pm UTC](https://forum.rclone.org/t/copy-multiple-directories-wildcard/22816/20 "2021-05-14T16:31:07Z")

</div>

Closed on May 14, 2021, 4:31pm UTC

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
