Some oddities when recovering files from trash from gdrive

What is your rclone version (output from rclone version)

latest rclone v1.53.1

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Win 10, 64 bit

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The rclone config contents with secrets removed.

[mdrive]
type = drive
client_id = xxx
client_secret = yyy
service_account_file = 
token = zzz
root_folder_id = vvv

[trash]
type = drive
client_id = xxx
client_secret = yyy
service_account_file = 
token = zzz
root_folder_id = vvv
trashed_only = true

[mcrypt]
type = crypt
remote = mdrive:backup
filename_encryption = standard
directory_name_encryption = true
password = www
password2 = 

[trashcrypt]
type = crypt
remote = trash:backup
filename_encryption = standard
directory_name_encryption = true
password = www
password2 = 

What is the problem you are having with rclone?

There are several.
Issue 1:

A sync went bad and a folder structure within a folder on mcrypt, like mcrypt:/a was accidentally deleted.
To support server side move/copy, I issued all commmands with trash:/ as source and mdrive:/ as dest.
I identified the corresponding folders using lsd --crypt-show-mapping.

let's say mcrypt:/a/b is a folder I want to revocer that was mdrive:/backup/c/d in encrypted form, I first tried with this command:

rclone move trash:/backup/c/d mdrive:/backup/c/d --drive-server-side-across-configs -v

Logs show moving seemingly successful with "Moved (server side)"
However on mdrive or mcrypt, only the folder shows up but is empty. trash or trashcrypt still show the files there.

Next I tried was creating a new empty folder mcrypt:/a/e that is mdrive:/backup/c/f encrypted. I then issued this command:

rclone move trash:/backup/c/d mdrive:/backup/c/f --drive-server-side-across-configs -v

Again seemingly successful, rclone showing "Moved (server side)" the folder mcrypt:/a/e and mdrive:/backup/c/f show empty.
trash:/backup/c/d shows empty. Files gone?
I go ahead and delete the "empty" folder mcrypt:/a/e
Now when I list trash:/backup/c/f, all the files show up.
So it seems the move took place, but the files are still kind of flagged as deleted by the system.

Doing a copy however works as intended:

rclone copy trash:/backup/c/d mdrive:/backup/c/d --drive-server-side-across-configs -v

show Copied (server side copy) and the files actually show up on mdrive:/backup/c/d and trash:/backup/c/d shows as empty.

So Question to issue 1 is: Can recovery be done with the move command?

Issue 2:

I discovered that the sync was bad when I tried to update a folder that no longer existed and then went to check the upload.
Say there was a mcrypt:/a/g folder with several subfolders mcrypt:/a/g/a1, mcrypt:/a/g/a2 etc.
I copied some new content that created mcrypt:/a/g/b1, mcrypt:/a/g/b2 etc.
At the time of the "update" the whole folder mcrypt:/a/g was gone to the bad sync

This is where it gets weird.

rclone lsd trashcrypt:/a

now shows 2 identical folders, both with the name trashcrypt:/a/g.
Looking at the date, one is 2 years old where the original folder was creaded, the other is just a few hours old of the time I did the supposedly update.
When I enter the folder, both show the same mcrypt:/a/g/b1, mcrypt:/a/g/b2 etc.
I did no deletions in or of the mcrypt:/a/g/ folder after the "update". I'm 100% sure of that.
So my question is, how can I recover the original mcrypt:/a/g/a1, mcrypt:/a/g/a2 etc. folders?

Issue 3:

This is probably totally unrelated, but when I recovered a subfolder of mcrypt:/a with the copy command I closely inspected if all files are there.
That was when I realized that 2 out of ~300 files are listed double.
Doing

rclone lsf "mcrypt:/a/h" >out.txt

I find 2 pairs of identical lines.
I suspect this might have already been the case before the deletion but the question is how to get rid of the dupe and how this can happen in the first place.

hello and welcome to the forum,

gdrive support multiple files with the same name.
https://rclone.org/drive/#duplicated-files

you can run this to get rid of dupes
https://rclone.org/commands/rclone_dedupe/

1 Like

First of all, thanks asdffdsa for solving Issue 3

I made progress with issue 2.
After being hesitant for some time I decided to experiment a little - and it was a success.
I decided to rename the folder mcrypt:/a/g to a never before used name mcrypt:/a/x
When I switch back to trashcrypt drive, trashcrypt:/a/x still shows up with complete subfolder structure, but no files.
Also, trashcrypt:/a/g that before the rename sowed the wrong (current) data now again shows the originally deleted files :slight_smile:

So it seems like the original deleted files somehow got masked by the newly copied files.
Not sure if this is rclone's or google's fault.

Steps to (probably) reproduce the issue:

  1. create a folder like volume:/a and put some files and/or folders it in: volume:/a/content1
  2. delete volume:/a - all should now be displayed in trash as expected
  3. create the same folder volume:/a again and fill it with other content: colume:/a/content2
  4. Trash should no longer show content1

You've probably got duplicated names... which rclone dedupe will sort out for you.

I get duplicate names of folders only, not files. and only in the trashbin.
this seems to happen 100% reproducible with the steps I mentioned.
Can you use dedupe on trashed files, too?

It's not only a sole matter of dupe files because all the originally deleted files were not visible, while other files were displayed in trashbin view that were never deleted but actually in the folder.

This is a known issue I think.

You can dedupe files and folders - deduping folders will merge the folders together under one name.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.