Okay, so long story short, through my own stupidity, the use of many variables, and my own lack of foresight to review scripts after variable changes, I managed to do an rclone move dir:/ dir:/. Yeah… stupid stupid stupid.
Worse yet, I let it run the full drive because I didn’t take the time to review what was going on.
So, I now have hundreds of thousands of files in the trash, waiting to be restored.
How can I accomplish restoring them back to their original directory structure (encrypted)?
I was thinking of something like using rclone mount dir:/ /mnt/dirtrash—use-trash-only and rclone mount dir:/ /mnt/dir and doing a mv /mnt/dirtrash /mnt/dir between the two, but would that download then re-upload everything?
I’ve got fiber at 1gb, so it’s not out of the question, but that 750gb limit is going to be hit for sure, like daily for the next 4 months…
Hundreds of thousands of files, unfortunately. Maybe even over a million. I tried to do the restore by web interface, but my browsers keep crashing when I reach a few hundred, and anything less, and I’ll be at that longer than reasonable.
I’ve also thought about doing a full delete, then a full restore… but that requires admin and I only have an edu account from an old school that I’m not enrolled in any longer, so I couldn’t count on any assistance from them.
Correct, ncw. It had to do with the way the variables were in use, not the program. That’s why I went out of my way to say it was my fault alone.
I didn’t want to go into the long story, and I don’t expect any new development on the issue I experienced.
So, here’s the long story:
I had a setup with plexdrive. I’m moving to rclone with caching now that it is working very well.
I have a drive mounted with rclone mount GDriveCacheCrypt:/ /mnt/disks/GDriveCacheCrypt/
I have another set of scripts with variables which call /mnt/disks/GDriveCacheCrypt to move things in and out of a staging directory at /mnt/disks/Staging.
I have these directories mounted with a unionfs to /mnt/disks/UnionFS/
I had a script that ran rclone move /mnt/disks/Staging/ GDriveCrypt:/
Unfortunately, my script called a variable for $Staging, which I neglected to notice…
I changed the staging variable to GDriveCacheCrypt:/ Things did not go well after that. I had the rclone move setup so it would check the file if it existed and was newer, do nothing, but delete the file from $Staging…
This is the actual script that ran:
rclone move $Staging/Media/ GDriveCrypt:/Media/ --transfers=12 --checkers=24 --min-age 1m --buffer-size 150M --no-traverse --bwlimit=“06:00,3M 00:00,3M” --fast-list --tpslimit 1 --tpslimit-burst 2 -v
So, you see, $Staging/Media became /mnt/disks/GDriveCacheCrypt/Media/, and the rest is history. It tried to move the file, only to find that when it deleted to overwrite the file, the original was mysteriously gone.
Again, I can only blame myself for this. There is nothing that could have been expected of rclone to protect me in this instance. Garbage in, garbage out.
Dude… awesome… I’m running it right now. We’ll see how I fare afterwards. I can’t tell if it’s running or not, but the cursor is flashing, and I got a debug or two about retries. I’ll take that as a sign it’s running.
--max-allowed-deletions x
Require user confirmation if more than x files are to be deleted at end of transfer.
x's default value is 100.
If x is set to 0, no deletions are allowed.
Set x to -1 to allow unlimited deletions. This can cause massive data loss if used improperly.
Use --dry-run to test runs first for safety.
personally I think the default should be -1
it’s rare for someone to use the move command and not want things deleted, because well, you have the copy command.
I pretty much always use the move command as if it were a purge command (only much safer because it checks everything first).
I came across someone on reddit that did something similar, and pointed him here. I’m wondering if there could be an “RClone Related Tools” section, where this, browser, and any others could be stickied?
I apologize for replying to such an old post, however since I couldn't find much information regarding restoring deleted files in the Google Drive trash, I thought this would be beneficial to future Rcloners. After playing with it a while I was able to find a way to restore the deleted files in the Drive trash back to its original location with a little bit of massaging. The good news is all of the operations were performed server-side so nothing was downloaded/re-uploaded, and it took nothing but the rclone utility and some massaging.
# Verify files are actually in the trash and do a comparison count on trashed and un-trashed files
rclone ls --drive-trashed-only --crypt-show-mapping gcrypt-usmba:/p0ds0smb/video-shows/dir1
rclone ls --drive-trashed-only gcrypt-usmba:/p0ds0smb/video-shows/dir1|wc -l
rclone ls gcrypt-usmba:/p0ds0smb/video-shows/dir1|wc -l
# Perform server-side copy operation of trashed files into new destination (appended "-Trash"). I tried move which didn't work, but copy did.
rclone copy --progress --drive-trashed-only --drive-server-side-across-configs gcrypt-usmba:/p0ds0smb/video-shows/dir1 gcrypt-usmba:/p0ds0smb/video-shows/dir1-Trash
# Count the restored files
rclone ls gcrypt-usmba:/p0ds0smb/video-shows/dir1-Trash|wc -l
# Mounted the drive and counted restored files as alternative method
find /media/visualblind/gcrypt-usmba/video-shows/dir1-Trash -type f -print | wc -l
# Purge the original directory that originally contained the files that were accidentally deleted
rclone purge gcrypt-usmba:/p0ds0smb/video-shows/dir1
# Rename the restored directory back to the original
rclone move --drive-server-side-across-configs gcrypt-usmba:/p0ds0smb/video-shows/dir1-Trash gcrypt-usmba:/p0ds0smb/video-shows/dir1
# Re-verify the restored file count
rclone ls gcrypt-usmba:/p0ds0smb/video-shows/dir1|wc -l
I'm surprised that there is no flag to enable prompts prior to deletion. even just rclone purge can potentially wipe entire remotes without so much as a Y/n
not that rm -r requires confirmation either, so not prompting is still consistent behavior...
Well rclone is a unix power tool so comes with the cutting foot off attachment as standard That said it did at least put the data in the trash as standard (guess how that got to be the default!).
The docs recommends --dry-run for destructive operations.
rm does have a -i or --interactive flag which gives prompts - maybe rclone should too?