Filtering issue

It seems I misunderstood the filter rules, and have inadvertently got encrypted files and non-encrypted files in the same folder.

I tried to fix this, and have sadly made the problem worse.

I used --include '*.*' --filter '- *.bin' to try and move the files that were encrypted out of the folder but instead, it moved the encrypted files into another folder, so I now have two folders containing both encrypted and non-encrypted files.

How would I move the non-encrypted and encrypted files using filters?

I dont think you can use filters to do it since you can't 100% test what kind of file it is I don't think based solely from a file name. There may be an easier way but what I would do is:

  1. create a CRYPT (with the keys) remote entry in the .rclone.conf that points to that directory (you may already have it).
  2. run a "rclone lsjson --files-only -R --encrypted --no-modtime crypt-remote:path" to list out all the files that can be decrypted in that remote. This would give you a list to 'move' in json format you'd need to massage.
  3. I'd parse that output and create 'files-from' file that you can later feed into rclone to select only those files. The EncryptedPath is really what you need from that output.
  4. You can then use the move command on the unencrypted remote or the delete command or any other command that fits your use-case to selectively action JUST the encrypted files/folders and leave the others against non-crypt remote. Example rclone move uncrypted-remote1:path remote1:somewhereelse --files-from filefromabove.txt
  5. you'll need to run a rclone rmdirs to delete the empty folders left behind.

****** And make sure you TEST before you run any destructive stuff with --dry-run FIRST*****

THe above was assuming you have file/directory name encryption enabled btw. If you don't, it is far easier as long as you can assume that you don't have non-crypt files called ______.bin.

Can you move just the encrypted files instead? Is that viable for you?
Because this would be a lot easier (than moving just the unencrypted files).

While looking at a location via crypt - all files that can not be decrypted wihh your curent key/salt (including non-encrypted files) will just be ignored (with a non-critical error in the log).
So if you just move everything rclone can see via crypt to myremote:\encrypted or something like that, then the remainder will be unencrypted. Seems to me like this would basically produce the same result.

Folder structures should be retained and rclone won't be able to delete any folders that still have files in them, even if doesn't show you these other (unencrypted) files, so I am pretty sure this is safe to do.

like calisro says, trying to filter them on just name may be difficult because there is not really any hard indicator in the name that it is an encrypted file.

The only thing I can really think of is that you might have decent success by including all files that contain the character "." (directly from the remote, not via crypt) as most normal files will have an extension while encrypted files will never (they only use 0-9 and A-V, see base32hex definition):

But this is not foolproof as files don't have to have an extension. It is just fairly uncommon that they don't, for normal users anyway. I think there are much better options that don't have this risk of false-positives.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.