Is there a simple way (or flag to pass) to force rclone to use RAM for the vfs cache?
Or do I need to manually create a ram disk and then pass that disk as the cache location via --cache-dir
AFAIK there is no built-in function to use RAM for the rclone VFS.
That said, there are a myriad ways of making a RAMdisk with third-party software. It's usually not hard and there are free options too.
Do consider the drawbacks to it however. A moved files that only exists in RAM until uploaded is not very safe. A system crash or a power outage will mean that file is irrevocably lost. Also - a VFS-cache that is an order of magnitude faster than your bandwidth won't really help much. Sure - it will ingest files fast - but that's really the only time you are going to save on. Additionally, you will be limited to filesizes that can fit inside the RAMdisk, so that will will be quite a big limitation.
If I were you I'd rather look into a RAMcache than a RAMdrive. Adding some read/write memory caching to the folder you use for your VFS would give you most of the benefits, with much lower risks, and few if any of the limitations. if using cache on the writes there is still SOME risk, but at least the risk exposure is very time-limited as it ends up on the drive eventually as it passes through. A read-only cache will still give some performance benefits while being safe against data-loss.
On windows, I've used Primocache for this before (and probably will again as soon as my extra RAM arrives for my server). There are probably several alternatives for this too though - but a good RAMcache is a bit more advanced than a simple RAMdisk so finding a free one that is also good might be slightly more challenging.
If you find other good alternatives, either for Windows or Linux - feel free to tell me and the rest of the community about them 
I'm actually not looking for speed--I want to get the cache off of the disk. I'm backing up my boot drive, and during that process rclone's cache regularly exceeds 5GB. Because the cache is stored in my home folder, I'm backing up a cache of the utility I'm using to upload my backup.
Therefore, I'd like to put the vfs cache somewhere else, so that a backup of ~/ doesn't include 5GB of rclone cache.
Because my backups run in the middle of the night, I don't mind allocating 5GB of ram to the operation at 4am, and then flushing the cache when it's done.
Oh, well if this is the only thing you need then it would be much more easily solved by using filtering.
for example
rclone sync C:\ remote: --exclude C:\RcloneVFSCache
Or at least something close to that...
Then this folder (and any other folders you may not want to sync like most OS files) can simply be ignored by the sync operation.
Do you feel like this can be a solution? It seems to attack the problem much more directly than the whole idea of a RAMdisk if you aren't even looking for speed.
And let me just say that as far as backuping up the OS goes (rather than user files on the boot disk) rclone will not be able to do a good job of that. At least trying to restore a broken OS from such a "backup" would be futile. An OS is more than a mere collection of files, and rclone won't be able to read most of the in-use files anyway and will just skip them. To do this you need to make use of shadowcopies and the like (which is actually what many backup solutions do). If you are actually looking to backup the OS then you should use a backup program for that job - and simply let rclone upload that backup-image instead. Windows Backup (included in all Windows above home edition) will do the trick, but of course there are loads of good backup program options.
Or... if you want to be real fancy you can use shadowcopies directly and thus let rclone read otherwise unreadable files. Asdfdsa make a wiki article on that - but this is still not a good OS backup in terms of hoping to ever use it to restore, so it's a rather niche cache.
It would be nice if you can clarify if your objective is to backup the OS, or if you just want a backup of your user files in documents and other various files on the boot-disk.. because depending on the answer the ideal strategy will vary.
I'm actually using borg, with an rclone mount
I have another thread going on that haha
Overall good results, with some minor issues that are not rclone's fault.
Sooo... is it borg that is picking up the cache then and not rclone?
I'm sure bog has some way to ignore certain folders too...
But I mean, if all else fails, you could simply set up a separate partition to have the VFScache on. Then it's not logically connected at all. This just seems like a very "dumb" solution for something that should easily and elegantly be solved via exclusion filtering however.
Borg is just making a copy of everything in ~/
And since rclone's default cache location is ~/Library/Caches/rclone/vfs
It's going to pull in the vfs cache as part of the backup.
I think --cache-dir is your best option, though you could set the equivalent environment variable too.
I'd probably exclude ~/Library/Caches from the backup though as you'll get a lot of thrashing in your backup for browser caches etc.
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.