Once a year I run rclone lsl on my full cloud storage. I pipe it from the command line to text files. This takes up about 900megabytes of plaintext to describe 100TB of files. Is there anyway to tell rclone to read this "fake" data instead of checking my actual cloud storage? So I can use rclone lsl or rclone size or rclone ncdu on this fake exported data? This would solve SO MANY of my problems!
I’ve made a bunch of threads about decoupling data from metadata. Only to realize. Maybe I don’t need a magically all in one solution. Maybe I don’t need gzip, compress, intelligent cloud storage, or any of that.
All I really need is a way to feed rclone false metadata. Because asking rclone to export my metadata is easy. Once exported it is in my control entirely. But there appears to be no way to import this false metadata back to rclone.
This could in theory save a TON of api calls to s3 storage buckets if rclone could just rely on a fake set of metadata before it began to sync or copy new data.
Now what I am describing sounds like Cache but cache was depreciated and never worked for me anyways. Of course cache was replaced by rclone mount which by all accounts works great. BUT I do not value mount at all. I do not particularly care for rclone mount. rclone mount sounds like a terrible combination for s3 archive tier cold storage!
I guess what I need is a way to have vfs-cache SAVED and then RELOADED without being recreated.
Now over on rclone mount I have read that…..
”If run with -vv rclone will print the location of the file cache. The files are stored in the user cache file area which is OS dependent but can be controlled with --cache-dir or setting the appropriate environment variable.”
This means that vfs cache is exporting it’s cache…. the problem is… well… when I terminate the mount what happens? What happens if I shut down my computer and start the computer the next day? Will the vfs cache simply resume from the cache saved on my harddrive? Or will it consider itself a fresh new mount freshly created and in need of a new cache?
That would be disastrous for any sort of colder s3 storage. Additionally I don’t see anyway to feed vfs-cache files into rclone ncdu or rclone lsl…. Hmm.
TLDR: Is what I want impossible? Or am I totally misunderstanding how powerful vfs-cache is and this is already a solved problem?