Caching to the HDD

Hello,

I have a question. Is it possible to cache files locally to the hard drive, and upload them to the remote in the background? I know there is a software called unisonfs-fuse, does it do that? Am I understanding it correctly? If I’m wrong, can someone explain what it does?

Thanks!

Unionfs takes two or more file systems and presents them as one. Using your use case, you can have /cache and /data combined and shown under /combined. You can mark the /cache filesystem as RW while the /data as RO. Then when you write to the /combined ( new, modified, or delete files ) it will only write to /cache and yet still show the two combined as if you had modified /data directly and look consistent with the actions you performed.

Now… If /data was a rclone mount it would combine results of that local cache and the remote /data rclone mount.

In the “background” now you could take the /cache contents and move them to /data (which would upload them on the rclone mount). The /combined will still look the same pre/post move…

Hope that helps!

Hey,

So, I’ll throw a scenario at you. Say I had /cache/file1 and /cache was in a unisonfs mount (/mnt/unisonfs/) with /mnt/data that points to my rclone remote.

So, say I ran “rclone move /cache/file1 remote:” … Would my unisonfs mount look and act the same? Other than the fact that file1 would load slower obviously. But other than that the file structure would look the same right?

Thanks!

Yes it would except that the rclone mount currently caches the file structure so when you use that move command the /data would not allow that file until the directory structure cache expires. So you have a few options to avoid that:

  1. only copy/move on the mount rather than outside the mount
  2. set the cache time on the rclone mount extremely low or off.

Btw this is valid but it will hurt your head. :slight_smile::

Rclone sync /unionfs /data.

Thanks a ton for your help!

That sync command is pretty cool. I’ll definitely use that!