A short primer on pre-caching (or pre-listing) technique

@Monarch_de_Boulogne requested that I share some of the information I gave him via PM so that it might benefit other users. At some point I will try to aggregate some more of this information and practical guides into some sort of megathread and or wiki article, but for now I will just leave this here for those who are interested.

If anyone has followup questions on this topic I will also take the time to reply to this here. Maybe I can use this as a reference in the future for others who are curious about the topic...

1 Like

One thing I forgot to mention above is that these two flags wil also be required for the mount command:

--attr-timeout 8760h
--dir-cache-time 8760h

Usually these values are quite low, invalidating the cache after a short time. These values are low by default because not all backends have polling, but on a backend with polling we can keep the cache data indefinitely and rely on updates to keep the information fresh instead of re-gathering the full listing every time we need them.
The specifics numbers here are just an example - I've just set them to be a year to effectively "disable" them. Without setting these timers to a high value it won't make much sense to pre-list as we will just evict most of that information from cache very soon thereafter (1 second default for attr and I think 5 min default for dir).

attr refers to caching file-attributes (size, modtime, name and createdtime for most backends).
dir refers to caching the directory information/structure.

@Monarch_de_Boulogne This information is relevant to you as I've forgotten to tell you this before, sorry :slight_smile: