How can I delete files in the destination that matches source?

Help and Support
1

What is the problem you are having with rclone?

I want to delete the files on Google Drive that I have on my local drive or flash drive. Any files that match should be moved to a new folder like --backup dir does. I should also be able to run the --c command to ensure that I am deleting the correct files. Is it possible ? Thank you.

What is your rclone version (output from rclone version)

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10, 64 bit and Windows 7 32 bit

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

I am looking for the command.

The rclone config contents with secrets removed.

-->

2

You can use rclone check from the latest beta to help with this.

If you do

rclone check --one-way /path/to/local drive:dir --match matched-files.txt

Then matched-files.txt will have a list of all the files which match so which are on the drive already.

You can then do something like this to move the files (notes the -i for testing - another beta feature)

rclone move -i --files-from-raw matched-files.txt drive:dir drive:backup-dir
1 Like
3

I tried it out. Thank you. But it says:

C:\Users\redacted>rclone check --one-way "D:\ANBU\All is one\reassembling\Mu
sic\song\mp3" "redacted:\Copy of Reassembling\Music\song\mp3" --match m
atched-files.txt
Error: unknown flag: --match
Usage:
rclone check source:path dest:path [flags]

Flags:
--download Check by downloading rather than with hash.
-h, --help help for check
--one-way Check one way only, source files must exist on remote

Use "rclone [command] --help" for more information about a command.
Use "rclone help flags" for to see the global flags.
Use "rclone help backends" for a list of supported services.

2020/08/14 21:43:56 Fatal error: unknown flag: --match

Here is my rclone version.

C:\Users\redacted>rclone version
rclone v1.51.0-001-g4d1c616e-beta

  • os/arch: windows/386
  • go version: go1.13.7

Thank you. Please tell me where I am going wrong.

4

You've managed to pick a very old beta.. the latest is here

https://beta.rclone.org/v1.52.3-300-g3d5a6360-beta/

Try that

5

My bad. I picked the first file thinking it was the latest version. I should have checked. I am trying it out now. Thank you.

1 Like
6

I tested it with dry run. Both command appears to work as intended.

I will use the Check command on the new directory to ensure that all the files I have on my new Google folder are in my local drives as well.

Is it possible to use the Google Drive as a source and my local drive as the destination when using the check command with --one-way ?

Like this:
rclone check --one-way "redacted:\Copy of Reassembling\Music\song\mp3"
"D:\ANBU\All is one\reassembling\Mu
sic\song\mp3" .

I am trying to ensure all the files I have on my Google Drive are in my local drive as well.

And instead of a list of files that match, is it possible to get a list of files that do not with ```
rclone check --one-way /path/to/local drive:dir --missing-on-src missing-files.txt


rclone move -i --files-from-raw missing-files.txt drive:dir drive:backup-dir
? 

Thank you for your help.
8

You can run the command either way round that is fine.

However you can remove the --one-way flag and investigate the other flags here: https://tip.rclone.org/commands/rclone_check/

I think they will do what you want

9

Thank you. I am testing it out right now. I will let you know the results.

10

I have tested it out. Everything seems to be okay.

Problem is, the directory structure are no longer the same on my source and destination even though most of the files are the same.

Is it possible to make it ignore the directory structure and just find the missing files when using check ?

Something similar to what --track-rename does during sync ?

Thank you for your help.

11

Not at the moment.

Is you are just interested in the content you might be better off using rclone md5sum and a bit of scripting to find the missing files. (Do rclone md5sum on the source and dest, sort the files then use the comm utility to find the missing files)

12

not sure exactly what you want to do,

perhaps you could use rclone mount gdrive as a local drive.
and then use any number of windows programs to dedupe.
i have used

13

Thank you. I will try out your solution.

I was looking around the forum and it seems rclone mount can also solve it. This thread helped me.

asdffdsa also suggested the same. I will try it out and let you know the results. Thank you again.

14

I am trying to delete the files I have on my local drive from the cloud. Then I want to download only the files I do not have on my my local drive.

Thank you for the suggestions. I will try it out and let you know.

15

This might do the trick for you. If not, perhaps it will give you some ideas.

I'll update it to reflect the wonderful new flags in the recent rclone betas when they are committed to master. But this script will work with both old and new rclone.

16

Thank you. DiffList seems to be the solution. Unfortunately, I have never used a shell script before.

I am on Windows 7 32 bit. Could you please give me some instructions on how to run it ?

Thanks again.

17

you would need to run bash on windows.
there are a few ways to do that.

18

I got Win-bash. Completely stuck after that. When I run start_shell.bat it says bash$.
I can use rclone check from there like I use it from rclone.exe when I open it from command line.

What do I do after that ? I am sorry. I have never used a script before.
Thank you

19

given your lack of experience with rclone, bash, scripting, using that diff bash script and the possibility make a mistake that you cannot recover from, i am not comfortable to help you with that.
perhaps someone else can.

but i did offer a possible solution.
mount the gdrive as a drive letter on your computer and use a dedupe program

20

I will use your solution. I think I will take some time to learn this stuff.

Thank you for the help.