Leave root with rclone move

agneev · June 6, 2020, 12:21pm

I'm trying to create an "inprogress" folder when moving files like this:

rclone mkdir putio:files_inprogress \
&& rclone move putio:files putio:files_inprogress \
--delete-empty-src-dirs \

There are sub folders inside that I would like to delete and are getting deleted with the last flag.

But the root folder is also getting deleted.

Is there a way to preserve the root folder and moving all objects inside it with rclone?
In Unix I'd use

mkdir /mnt/putio/files_inprogress && mv /mnt/putio/files/* /mnt/putio/files_inprogress

calisro · June 6, 2020, 12:57pm

Probably could add the same flag (leave-root) as rmdirs to the move command. For now you can probably remove that -delete-empty-src-dirs flag and do a second step with:

https://rclone.org/commands/rclone_rmdirs/

Or tack a && mkdir dir on your existing command.

Might want to create an enhancement issue

agneev · June 6, 2020, 1:15pm

It seems that rclone move is deleting the source.

agneev@pi4:~ $ mkdir test
agneev@pi4:~ $ cd test
agneev@pi4:~/test $ mkdir test1/test11 test2/test22
agneev@pi4:~/test $ rclone move test2 test3
agneev@pi4:~/test $ tree
.
├── test1
│   └── test11
└── test3
    └── test22

4 directories, 0 files

So how do I do a second step when the folder is already getting deleted?

calisro · June 6, 2020, 1:30pm

Maybe I misunderstood what you were asking. but looking at your recent example, you've asked rclone to move test2 to test3. What do you think should have been left behind?.

Are you looking for the result to look like this?

├── test1
│   └── test11
├── test2
└── test3
    └── test22

agneev · June 6, 2020, 1:34pm

Yeah but I guess that would be the solution to my issue.

calisro · June 6, 2020, 1:43pm

rclone -v move test2 test3 --include=/** --delete-empty-src-dirs

Does that work for you?

@dell-rob:~/test$ rclone tree .
/
├── case.tar
├── file1
├── test1
│   ├── file1
│   └── test11
│       └── file2
└── test2
    ├── file4
    └── test22
        └── file3

4 directories, 6 files
@dell-rob:~/test$ rclone -v move test2 test3 --include=/** --delete-empty-src-dirs
2020/06/06 09:44:37 INFO  : file4: Moved (server side)
2020/06/06 09:44:37 INFO  : test22/file3: Moved (server side)
2020/06/06 09:44:37 INFO  : There was nothing to transfer
2020/06/06 09:44:37 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks:                 2 / 2, 100%
Renamed:                2
Elapsed time:         0.0s

@dell-rob:~/test$ rclone tree .
/
├── case.tar
├── file1
├── test1
│   ├── file1
│   └── test11
│       └── file2
├── test2
└── test3
    ├── file4
    └── test22
        └── file3

5 directories, 6 files

agneev · June 6, 2020, 1:51pm

It does. Maybe the proposed issue can utilize that under the hood as --leave-root.

calisro · June 6, 2020, 2:04pm

I responded in the issue.

agneev · June 10, 2020, 3:26pm

What's the difference between --include=/** and --include=/* ?

calisro · June 10, 2020, 3:28pm

the one star only matches files in the current directory up the the first slash. the two stars descends into subdirectories. It matches everything forward.

agneev · June 11, 2020, 9:07am

How do I preserve the three folders inside the root, while deleting everything else inside those folders with rclone move?

putio/
├── test2
├── test3
└── test4

calisro · June 11, 2020, 2:11pm

@dell-rob:~$ rclone tree putio
/
├── test2
│   └── file2
├── test3
│   └── file3
└── test4
    └── file4

3 directories, 3 files

 @dell-rob:~$ rclone move putio putio1 --include=/*/** -vv
 2020/06/11 10:09:44 DEBUG : rclone: Version "v1.52.0-001-g1cceadaf-beta" starting with parameters ["rclone" "move" "putio" "putio1" "--include=/*/**" "-vv"]
 2020/06/11 10:09:44 DEBUG : Using config file from "/home/xx/.rclone.conf"
 2020/06/11 10:09:44 DEBUG : fs cache: renaming cache item "putio" to be canonical "/home/xx/putio"
 2020/06/11 10:09:44 DEBUG : fs cache: renaming cache item "putio1" to be canonical "/home/xx/putio1"
 2020/06/11 10:09:44 INFO  : test3/file3: Moved (server side)
 2020/06/11 10:09:44 INFO  : test2/file2: Moved (server side)
 2020/06/11 10:09:44 DEBUG : Local file system at /home/xx/putio1: Waiting for checks to finish
 2020/06/11 10:09:44 DEBUG : Local file system at /home/xx/putio1: Waiting for transfers to finish
 2020/06/11 10:09:44 INFO  : test4/file4: Moved (server side)
 2020/06/11 10:09:44 INFO  : There was nothing to transfer
 2020/06/11 10:09:44 INFO  : 
 Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
 Checks:                 3 / 3, 100%
 Renamed:                3
 Elapsed time:         0.0s
 
 2020/06/11 10:09:44 DEBUG : 2 go routines active
 
 
 @dell-rob:~$ rclone tree putio
 /
 ├── test2
 ├── test3
 └── test4
 
 0 directories, 0 files
 @dell-rob:~$ rclone tree putio1
 /
 ├── test2
 │   └── file2
 ├── test3
 │   └── file3
 └── test4
     └── file4
 
 3 directories, 3 files

Note though that subdirectories would be left. If you have subdirectories, you'll likely need to run a second command to delete them. or use the --delete-empty-source-dirs and then recreate those directories you need.

@dell-rob:~$ rclone move putio putio1 --include=/** 
@dell-rob:~$ rclone tree putio
/
├── test2
│   └── dir1
├── test3
└── test4
    └── dir1

0 directories, 0 files
@dell-rob:~$ rclone tree putio1
/
├── test2
│   ├── dir1
│   │   └── file0
│   └── file2
├── test3
│   └── file3
└── test4
    ├── dir1
    │   └── file6
    └── file4

5 directories, 5 files

Alternatively, you'd run several move commands on each initial run for each directory to move its contents.

calisro · June 11, 2020, 2:46pm

THere's also some hacky ways you could do to make it simple for you without running second commands... If you leave a placeholder in the directories then you could run the --delete-empty-source-directories without deleting the folders you care about. Then you'd need to, of course, exclude those placeholder files from moving.

xx@dell-rob:~$ rclone tree putio
/
├── test2
│   ├── dir1
│   │   └── file0
│   └── file2
├── test3
│   └── file3
└── test4
    ├── dir1
    │   └── file6
    └── file4

5 directories, 5 files
xx@dell-rob:~$ find putio
putio
putio/test4
putio/test4/dir1
putio/test4/dir1/file6
putio/test4/file4
putio/test4/.placeholder
putio/test2
putio/test2/file2
putio/test2/dir1
putio/test2/dir1/file0
putio/test2/.placeholder
putio/test3
putio/test3/file3
putio/test3/.placeholder
putio/.placeholder

@dell-rob:~$ rclone move putio putio1 --exclude=**/.placeholder -vv  --delete-empty-src-dirs
2020/06/11 10:44:56 DEBUG : rclone: Version "v1.52.0-001-g1cceadaf-beta" starting with parameters ["rclone" "move" "putio" "putio1" "--exclude=**/.placeholder" "-vv" "--delete-empty-src-dirs"]
2020/06/11 10:44:56 DEBUG : Using config file from "/home/xx/.rclone.conf"
2020/06/11 10:44:56 DEBUG : fs cache: renaming cache item "putio" to be canonical "/home/xx/putio"
2020/06/11 10:44:56 DEBUG : fs cache: renaming cache item "putio1" to be canonical "/home/xx/putio1"
2020/06/11 10:44:56 INFO  : .placeholder: Moved (server side)
2020/06/11 10:44:56 DEBUG : test4/.placeholder: Excluded
2020/06/11 10:44:56 DEBUG : test2/.placeholder: Excluded
2020/06/11 10:44:56 INFO  : test2/file2: Moved (server side)
2020/06/11 10:44:56 INFO  : test4/file4: Moved (server side)
2020/06/11 10:44:56 INFO  : test4/dir1/file6: Moved (server side)
2020/06/11 10:44:56 INFO  : test2/dir1/file0: Moved (server side)
2020/06/11 10:44:56 DEBUG : test3/.placeholder: Excluded
2020/06/11 10:44:56 DEBUG : Local file system at /home/xx/putio1: Waiting for checks to finish
2020/06/11 10:44:56 INFO  : test3/file3: Moved (server side)
2020/06/11 10:44:56 DEBUG : Local file system at /home/xx/putio1: Waiting for transfers to finish
2020/06/11 10:44:56 DEBUG : test4/dir1: Removing directory
2020/06/11 10:44:56 DEBUG : test4: Removing directory
2020/06/11 10:44:56 DEBUG : test4: Failed to Rmdir: remove /home/xx/putio/test4: directory not empty
2020/06/11 10:44:56 DEBUG : test3: Removing directory
2020/06/11 10:44:56 DEBUG : test3: Failed to Rmdir: remove /home/xx/putio/test3: directory not empty
2020/06/11 10:44:56 DEBUG : test2/dir1: Removing directory
2020/06/11 10:44:56 DEBUG : test2: Removing directory
2020/06/11 10:44:56 DEBUG : test2: Failed to Rmdir: remove /home/xx/putio/test2: directory not empty
2020/06/11 10:44:56 DEBUG : Local file system at /home/xx/putio: failed to delete 3 directories
2020/06/11 10:44:56 DEBUG : Local file system at /home/xx/putio: deleted 2 directories
2020/06/11 10:44:56 INFO  : There was nothing to transfer
2020/06/11 10:44:56 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks:                 6 / 6, 100%
Renamed:                6
Elapsed time:         0.0s

2020/06/11 10:44:56 DEBUG : 2 go routines active
@dell-rob:~$ find putio
putio
putio/test4
putio/test4/.placeholder
putio/test2
putio/test2/.placeholder
putio/test3
putio/test3/.placeholder
dell-rob:~$ find putio1
putio1
putio1/test4
putio1/test4/dir1
putio1/test4/dir1/file6
putio1/test4/file4
putio1/test2
putio1/test2/file2
putio1/test2/dir1
putio1/test2/dir1/file0
putio1/test3
putio1/test3/file3
putio1/.placeholder
@dell-rob:~$

agneev · June 11, 2020, 3:04pm

Its part of a script that runs from a Cron Job, so I've added a extra line for each of those folders that need to remain in place.

calisro · June 11, 2020, 5:00pm

Yes you're right. Its essentially running an rmdirs on the dest remote. This was the original issue that necessitated that flag.

If that flag isn't doing what you want, you could omit it loop through what you'd want removed.

system · August 11, 2020, 1:09pm

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.