Differential move, leave existing files in source

Help and Support
1

Odd query: I want to move files from Folder1 (F1) to Folder2 (F2), but wish to leave files that already exist in F2 in F1.

Example:

F1: A, B, C
F2: A, B

rclone move gdrive:F1 gdrive:F2 --max-delete=0 does a server size move of C and yields

F1: A, B
F2: A, B, C

Let's say I then add C and D into F1.

F1: A, B, C, D
F2: A, B, C

rclone move gdrive:F1 gdrive:F2 --max-delete=0 does a server size move of D and yields

F1: A, B, C
F2: A, B, C, D

If I run the above rclone move command this works perfectly if F1 and F2 are folders in the same google My Drive account, doing a server side move of the missing files in F2 but leaving existing files in F1.

However, if F1 is a folder in a Team Drive and F2 is a folder in My Drive then the same rclone move command aborts with the following error:

ERROR : Cancelling sync due to fatal error: --max-delete threshold reached

If I omit the --max-delete=0 flag then the server side move works normally. But of it then deletes existing files from the source, as it supposed to do, leaving:

F1:
F2: A, B, C, D

Of course an rclone copy command achieves 'almost' the same result and will work server side, but it leaves the differential file in the source rather than moving it.

Any ideas about how to get this to work from a Team Drive to My Drive? It's almost there, but for some reason --max-delete works differently if the source is My Drive versus Team Drive.

2

Maybe rclone move --ignore-existing might work? Not sure - test carefully!

I think the server side moves are failing for Team Drive to My Drive and rclone is falling back to copy+delete. Check the log with -vv to make sure.

3

Tx @ncw tbh I was surprised that --max-delete=0 was working, even within My Drive.

server-side moves are working within and across both My Drive and Team Drives with move, copy and sync, which is very handy. I don't think that is the issue.

I had tried --ignore-existing with both My Drive and Team Drives but rclone move still deleted files on the source side if I omitted --max-delete=0

Did a bit of further testing and came up with the following:

When I did the original test I was moving files within My Drive, from Folder1 to Folder2. That worked fine.

When I started the secondary tests I included files in sub-folders within Folder1, and that did not work (Folder1/sample/file1). It appears moving folders may be what is screwing up --max-delete=0. When I manually delete the folders and only have files in the root of Folder1 it all works again, both within and between My Drive and Team Drives.

Guessing (you likely know): Does a file 'move' done server-side allow changing the pointer within My Drive and Team Drives without deleting anything? But appears that a folder move forces creating a new folder and deleting the old folder? If so, that would be what is killing causing --max-delete=0 to terminate the rclone move.

If anyone has a clever workaround I would be grateful. Perhaps a find command (?) or piping output of rclone check selectively to rclone copy?

4

Here is another approach. All completely untested :wink:

 rclone lsf -R --files-only F1: > list-of-files-in-f1
 rclone lsf -R --files-only F2: > list-of-files-in-f2
 comm -23 list-of-files-in-f1  list-of-files-in-f2 > list-of-files-only-in-f1
 rclone move --files-from list-of-files-only-in-f1 F1: F2
5

@ncw Thank you. I knew it should be something simple.

I'll give that a try.

1 Like
6

@ncw Would this move the files in list-of-files-only-in-f1 into a single folder all at root of F2? Or it would also create the nested folder structure from F1? I think the former, but...

7

It will create the nested folder structure :slight_smile:

8

Brilliant. Thank you, as always.

Now looking up the comm -23 command. Always good learning here.

1 Like
9

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.