The caching of the directory and file structure is only memory.
There is no persistent cache for that. Once a mount is restarted, you have to list it all out again.
There is already a feature request out there for a persistent metadata cache.
That should not be confused with the vfs-cache-mode features for full/writes as that is for file caching to local disk only.
They are not connected to each other.