Copy Multiple Directorys with 1 rclone command?

Help and Support
1

I want to copy these directories to my gcrypt:

/share/CACHEDEV1_DATA/Test Data
/share/CACHEDEV1_DATA/Container
/share/CACHEDEV1_DATA/Multimedia
/share/CACHEDEV1_DATA/QmailAgent
/share/CACHEDEV1_DATA/Web

I would like to copy them with the same folder structure they have right now (including symlinks).

How does my command need to look like?
Can I just specify multiple sources ?

2

Anyone ? I couldn’t find out how to do this till now

3

Use an include file.

There are some examples here:

https://rclone.org/filtering/

1 Like
4

So this command should work right ?

rclone copy gcrypt2: /share/CACHEDEV1_DATA/Test Data --include /share/CACHEDEV1_DATA/Container --include /share/CACHEDEV1_DATA/Multimedia --include /share/CACHEDEV1_DATA/QmailAgent --include /share/CACHEDEV1_DATA/Web --checkers 3 --fast-list --log-file /share/CACHEDEV1_DATA/rclone/copy.log -v --tpslimit 3 --transfers 3

5

Almost… the --include need to be rooted at the root of the transfer and include files within them, so

rclone copy gcrypt2: /share/CACHEDEV1_DATA --include "/Test Data/**" --include "/Container/**" --include "/Multimedia/**" --include "/QmailAgent/**" --include "/Web/**" --checkers 3 --fast-list --log-file /share/CACHEDEV1_DATA/rclone/copy.log -v --tpslimit 3 --transfers 3

You can also put those includes in a file and use --include-from which may be easier to manage.

1 Like
6

Thank you for the quick answer. Then I will go with the file and include from.

So my command and include from file content would need to look like this?

rclone copy gcrypt2: /share/CACHEDEV1_DATA --include-from /share/CACHEDEV1_DATA/rclone/include.txt --checkers 3 --fast-list --log-file /share/CACHEDEV1_DATA/rclone/copy.log -v --tpslimit 3 --transfers 3

and the include.txt would look like this:

/Test Data/**
/Container/**
/Multimedia/**
/QmailAgent/**
/Web/**
7

Yes, that looks spot on :smile:

1 Like
8

@ncw
If i want to exclude a specific folder and its subfolders within the same command can i just add

--exclude /Multimedia/Video/** to the command ?

So my command would look like this ?

rclone copy gcrypt2: /share/CACHEDEV1_DATA --include-from /share/CACHEDEV1_DATA/rclone/include.txt --exclude /Multimedia/Video/** --checkers 3 --fast-list --log-file /share/CACHEDEV1_DATA/rclone/copy.log -v --tpslimit 3 --transfers 3

9

If you are using includes and excludes then it is best to use a filter file with --filter-from. You prefix each line with a + for an include and a - for an exclude

+ /Test Data/**
+ /Container/**
- /Multimedia/Video/**
+ /Multimedia/**
+ /QmailAgent/**
+ /Web/**
- *

Note the rules are checked in order, so put the more specific ones first.

1 Like
10

Thank you for your answer. I must have overlooked that function in the docs.

I only have one more question for what exactly is the last line needed -* ?

11

@ncw
Right now my log file is just filling up with empty stats did I do something wrong?

LOG:

2018/11/24 04:19:38 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:                 0 / 0, -
Transferred:            0 / 0, -
Elapsed time:     48m2.2s

2018/11/24 04:20:38 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:                 0 / 0, -
Transferred:            0 / 0, -
Elapsed time:     49m2.2s

2018/11/24 04:21:38 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:                 0 / 0, -
Transferred:            0 / 0, -
Elapsed time:     50m2.2s

My command:

rclone copy gcrypt:shared /share/CACHEDEV1_DATA --filter-from /share/CACHEDEV1_DATA/rclone/uploadfilter.txt --checkers 3 --fast-list --log-file /share/CACHEDEV1_DATA/rclone/backupdata.log -v --tpslimit 3 --transfers 3

12

You want to exclude everything else, so something which doesn’t match any of the above needs to be excluded.

When you use --include it puts an implicit exclude on the end.

1 Like