Prespecified File Structre Backend Idea/Command Argument

Feature
1

rclone lsjson -R has a json representation of the file structure and basic file information of that file path recursively. An idea is that could this file be used to specify what files exist on a remote, and then ignore the files that are not specified. For example, with the json

[
{"Path":"abc.txt","Name":"abc.txt","Size":3,"MimeType":"text/plain","IsDir":false}
]

applied to a remote with the files

abc.txt
xyz.json

then the only files displayed would be

abc.txt

and the xyz.json would be ignored. The use case idea for this is if you have rclone copying between two directories instead of syncing to prevent mass deletion on the second remote due to something going wrong on the first remote, and you have an lsjson -R from the first remote, you could have both remotes match, while still having no chance of deletion of files in the first remote causing deletion of files in the second remote. This also could be a solution to Ncdu: support saving and restoring, because it could allow faster directory listing access, as the directory listings would be already listed locally in the lsjson file. I know these use cases might be fringe use cases, but perhaps it might be applicable to other things.

2

Sorry if I am misunderstanding the idea - but could you not accomplish the same thing using the current filtering system? (you can also have complex predefined filters in files).

If you only included a subset of a remote with filtering then anything outside that wouldn't be touched even if something went badly wrong.

Also - specifically as it related to emergency-recovery cases like that, you might find --backup-dir useful if you are not already aware of it. With this you can pretty easily script all changes to files to be saved in a sort of versioning system and recover easy - be it from a single file being accidentally deleted, or some horrendous mistake in syncing that wiped all your files (well as long you keep that backup-dir outside your working directory tree that is hehe).

4

Let me know if you need any help in that regard (after reading the documentation).
Unless you are already familiar with regular expressions or specifically GLOB in this case (usually from Linux shell experience) then the filtering stuff can be a little hard to wrap one's brain around.

I will be happy to assist. Not gonna call myself an expert on the field, but I know enough to use it regularly to make my life easier with rclone :slight_smile:

5

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.