Did something stupid. Now need to undo purge

[haynes (Charles Haynes)]

What is the problem you are having with rclone?

Accidentally did "rclone purge" on a gdrive crypted backend, and would like to undo it. Forgot it doesn't take an —include flag. Oops. I checked in my gdrive trash to see if the files were there. Nope. I'm also admin of the GSuite account (I'm an enterprise customer) and I did a "Restore Data" from the last two days for that user. That doesn't seem to have helped either - though I'm not sure how long the restore data job would take, I've got about 50TB of files in that data directory.

Any suggestions on how I might be able to recover the data? I realize it was my own fault, and rclone is working correctly. I thought I'd read that purge moved things to the bin, but I've gone to drive.google.com logged in a haynes, and don't see them in the bin. There were some files there that I restored, but they are just a directory I'd deleted a few days ago.

Am I just out of luck? I have about 10TB of the original files still lying around, and I have an old backup for maybe half of the others but I'd like to avoid the pain of recovering files piecemeal if at all possible (and losing about half of the data.)

Run the command 'rclone version' and share the full output of the command.

rclone v1.66.0

• os/version: darwin 14.5 (64 bit)
• os/kernel: 23.5.0 (arm64)
• os/type: darwin
• os/arch: arm64 (ARMv8 compatible)
• go/version: go1.22.1
• go/linking: dynamic
• go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive (crypted)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone purge --fast-list gcrypt:/home/haynes/data --include="temp"

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[gcrypt]
type = crypt
remote = gdrive:crypt
filename_encryption = standard
password = XXX
password2 = XXX
directory_name_encryption = true

[gdrive]
type = drive
client_id = XXX
client_secret = XXX
token = XXX
root_folder_id = XXX
team_drive =

A log from the command that you were trying to run with the -vv flag

Yeah I don't think so.

[haynes (Charles Haynes)]

[Edited to add]
Used the admin tools to look at the GDrive log, and it does say that things got moved to the bin, so will go looking for other bins that might be possible to undo.

Talked to Google Support, they say that the user "Restore Data" project should eventually restore the files and send an email when it's complete. Will update this when/if that happens.

[haynes (Charles Haynes)]

Did not get an email, but using the drive log investigation tool shows as many items removed from bin as were put in the bin in the appropriate time period, so I'm going to assume the "Restore Data" did the trick. That's a relief.

As a small suggestion, maybe the purge command could check for "include*" "exclude*" "filter*" or "files-from*" flags and either error out or ask for confirmation if there are any present.

[asdffdsa (jojothehumanmonkey)]

@haynes

That's a relief

good that you recovered the files.

@haynes

maybe the purge command could check for "include*" "exclude*" "filter*" or "files-from*" flags

given the damage that purge can do, that sounds like a good idea.

perhaps, purge would default to --interactive mode and have another flag to override that.
something like, --enable-auto-purge-mode

[jwink3101 (Justin Winokur)]

@asdffdsa

purge would default to --interactive mode and have another flag to override that.
something like, --enable-auto-purge-mode

The problem with this is that if a remote supports directory delete, then purge uses that so it may not even have the chance. When I purge on OneDrive, it deletes the whole directory as opposed to each file.

[asdffdsa (jojothehumanmonkey)]

@jwink3101

The problem with this is that if a remote supports directory delete, then purge uses that so it may not even have the chance. When I purge on OneDrive, it deletes the whole directory as opposed to each file.

yes, you are correct about that but imho, the OP issue is is not about that.
purge is a dangerous command, and the OP made a mistake and there are other topics about that.

purge does not support filters but also, purge does not complain if a filter is used.
as a safety, perhaps rclone should error out in that case.

in addition, i am suggesting that before purge does its dirty deed, rclone should display a warning or error out.
even dumb dos has a warning.

del d:\files\tmp\* /s
d:\files\tmp\*, Are you sure (Y/N)?

and to override that,

del d:\files\tmp /s /q