RClone Cache Chunk Path/Memory


#1

So, I have my --cache-chunk-path pointed to /dev/shm (formatted to tmpfs) which is basically memory as per my understanding. In this scenario, would it be recommended to turn off the in-memory storage of chunks using the --cache-chunk-no-memory flag or would that storage still be beneficial somehow?


#2

Personally I turned off in-memory storage of chunks with --cache-chunk-no-memory when using /dev/shm for cache.

One headache I have is that rclone mount seem to increase in RAM usage (not cache or /dev/shm/ usage) over time (weeks) after extensive usage. The RAM usage gets quite big after weeks. The /dev/shm has no problem though.


#3

I tried out the --cache-chunk-no-memory option but the speeds were very low. Normally I get ~150 MBps on rsync from the rclone mount point but with that option enabled, I was getting only 1 MBps or so. Quite a drastic difference.


#4

Has no one faced a similar issue?


#5

I found turning off cache chunk memory to be slower as well.


#6

Yeah, I can understand a little slower, was expecting something like 100 MBps instead of the 150 but 1 MBps? Doesn’t that seem too low?


#7

@ncw @B4dM4n

Sorry to tag you, but wanted to confirm that this is the expected behavior and I don’t need to check any issues with the configuration.


#8

It seems very slow, but I don’t have any idea why! Unless maybe using /dev/shm pushed your machine into swap.


#9

Ok so it’s not expected then. Let me collect some debug logs and see if that helps you.

BTW, there was still plenty of memory available so I don’t think that’s the cause.