Flag to only copy files and not directory path

Help and Support
1

Is there a flag so I can copy only files to the destination while traversing a directories on the source?

Example:

/mypath/
/mypath/dir1/file1a
/mypath/dir1/file2a
/mypath/dir2/file1b
/mypath/dir2/file2b

desired output
/mydest/file1a
/mydest/file2a
/mydest/file1b
/mydest/file2b

Actual output
/mydest/dir1/file1a
/mydest/dir1/file2a
/mydest/dir2/file1b
/mydest/dir2/file2b

2

find /mypath -type f -exec rclone copy {} remote:/mydest ;

Will be slow, but it should work.

3

Edit: I forgot my brain. Mine won’t work. :slight_smile:

If you produce an include list and put it in a file instead, then you can feed into rclone as a --files-from parameter. Would be much faster.

6

Thank you! I figured it out (with your help)!

find "/mypath/" -mtime -1 -type l -exec rclone copy {} drive:tv -v -L \;

7

Hello, I know i'm late to the party here, but has anyone figured out a better/easier way to do this? I have 58 million files and can't execute rclone each time, and an include file would be crazy big. Is it possible to just set the object key = to the filename?

8

Thanks, when I try files-from I still get the object showing up with the full path:

Key: /layer_test/layer2/21294_1000-1-1-1-8-1-1_1575900260.613.ts

my filter1.txt has:

a sample filter rule file

/layer_test/layer2/21294_1000-1-1-1-8-1-1_1575900260.613.ts

and I call it with:
rclone copy --files-from C:\R_Clone2\filter1.txt C:\R_Clone2 leyb_large:inear-root/ --no-check-certificate

it copies the 1 file ok, but It still shows up with the object key having the directories there, I need it to not have the /layer_test/layer2/ dragged along with it as the object key.

what am I doing wrong here? Thanks for the help.

9

forgot to add a log, here is what it looks like with lsl -vv: just trying to get rid of the layer_test/layer2/ part, The file lives here on local, filter finds it ok, just can't have all this showing up in S3.
C:\R_Clone2\layer_test\layer2\21294_1000-1-1-1-8-1-1_1575900260.613.ts

C:\R_Clone>rclone --no-check-certificate lsl leyb_large:inear-root/ -vv
2019/12/13 10:38:20 DEBUG : rclone: Version "v1.50.1-047-gec09127b-fix-netapp-creation-time-beta" starting with parameters ["rclone" "--no-check-certificate" "lsl" "leyb_large:inear-root/" "-vv"]
2019/12/13 10:38:20 DEBUG : Using config file from "C:\Users\pema9013\.config\rclone\rclone.conf"
19135956 2019-12-09 09:09:21.000000000 **layer_test/layer2/**21294_1000-1-1-1-8-1-1_1575900260.613.ts
2019/12/13 10:38:21 DEBUG : 5 go routines active
2019/12/13 10:38:21 DEBUG : rclone: Version "v1.50.1-047-gec09127b-fix-netapp-creation-time-beta" finishing with parameters ["rclone" "--no-check-certificate" "lsl" "leyb_large:inear-root/" "-vv"]

Thanks, Nick got me over a few hurdles so far, almost at the finish line here, much appreciated.

10

If you don't want the layer_test/layer2 part then you need to change your --files-from so it isn't present in there. This might mean you need a different files from for each directory in the source.

Rclone doesn't currently have a flatten operation which would copy all the files in the source into the same directory in the destination which is what I think you are asking for.

11

thank you Sir, you have been a great help. We figured out a way to loop through the folders and issue an rclone copy for each, we don't have to cd into every directory to do this which is great. I still owe you a pizza! Happy holidays :o)

1 Like
12

I'll share our solution when it works!

1 Like