Rclone mount cache seems to be rebuilt despite being present?

[jalam]

Thanks Nick,

I'll have to do more digging around and will update the forum if I find anything meaningful...

In the meantime, and out of interest, if I were to just mount the S3 bucket (but don't access the files/application on the S3 bucket/mounted drive), would it try to read/rebuild the cache?

Or would the reading/rebuilding of the cache only happens when I try to access the files/application on the S3 bucket/mounted drive)?

Many thanks,
James

[jalam]

Sorry, more questions from me Nick..

Let's say the cache is there (and don't need to be rebuilt)... When I launch the application first time, would you expect the "performance" to be slightly slower than the subsequent launches - or would you expect the same "performance"?

I guess in a roundabout way, I wanted to understand if the first launch would take longer than subsequent launches (because although the cache dir/files are there, it would still need to load/cache it to the memory/RAM). And subsequent launches would be relatively faster than the first launch because it would just get it from the RAM (and don't need to store it in the memory/RAM)?

Many thanks,
James

[ncw (Nick Craig-Wood)]

@jalam

In the meantime, and out of interest, if I were to just mount the S3 bucket (but don't access the files/application on the S3 bucket/mounted drive), would it try to read/rebuild the cache?

Rclone will read any cached files that are there on startup.

It won't rebuild any missing files though.

@jalam

Or would the reading/rebuilding of the cache only happens when I try to access the files/application on the S3 bucket/mounted drive)?

If the file isn't in the cache, it won't be fetched until it is accessed.

@jalam

Let's say the cache is there (and don't need to be rebuilt)... When I launch the application first time, would you expect the "performance" to be slightly slower than the subsequent launches - or would you expect the same "performance"?

It will take very slightly longer for rclone to startup because it reads the metadata on the cached files. If you have millions of files in the cache this can be very noticeable.

However you are talking about your application launching, so the first time what will be happening is OS caches will be built. Rclone doesn't do any extra caching here.

@jalam

I guess in a roundabout way, I wanted to understand if the first launch would take longer than subsequent launches (because although the cache dir/files are there, it would still need to load/cache it to the memory/RAM). And subsequent launches would be relatively faster than the first launch because it would just get it from the RAM (and don't need to store it in the memory/RAM)?

The first launch will be slower as the OS caches are built up. The subsequent ones should be quicker.

[jalam]

Hi Nick,

Thanks for the responses, and that really helped my understanding..

And a bit of update.. I think I might have cracked the case, and it's related to how an AppStream session behaves...

For reference, Use Session Scripts to Manage Your AppStream 2.0 Users' Streaming Experience - Amazon AppStream 2.0 details the process about running a session script before an AppStream session is started. Specifically, step #3 in the diagram mentions

If application settings persistence is enabled for your users, the application settings Virtual Hard Disk (VHD) file that stores your users' customizations and Windows settings is downloaded and mounted. Windows user login is required in this case.

In my case, I was trying to mount the S3 bucket using a setup script (not to be confused with the session script). With session scripts, the Windows user login always occurs before session scripts start executing. However, with setup script, this might not be the case.

In my case, because I had been focusing on reducing the time it takes the execute the setup script, in hindsight, I think the rclone mount command had been executing before the VHD with the application persistence is mounted. This would explain why I would see the cache folder/files in the AppStream session (because the VHD would've been mounted by then), but the log file says otherwise (because it was run before the VHD would not have been mounted by then). And when I launch the application, it took about 10s to load.

When I changed my setup script to check for the existence of the drive (where VHD file is mounted) before performing the rclone mount, then the logs would show something like

INFO : vfs cache: cleaned: objects 36 (was 36) in use 0, to upload 0, uploading 0, total size 94.971Mi (was 94.971Mi)

which as you've suggested, means that it was able to find the cache. Following that, when I launch the application, it now takes about 4s the first time. And this 4s, like you said, would be attributed to the time it takes for the OS cache to build - and subsequent launches would be close to instant (once the OS cache builds).

So yeah, I think the mystery's been solved, and I feel the discussion can closed now. Thanks again for sticking around and offering your help, time and insight Nick!

Many thanks,
James

[ncw (Nick Craig-Wood)]

@jalam

This would explain why I would see the cache folder/files in the AppStream session (because the VHD would've been mounted by then), but the log file says otherwise (because it was run before the VHD would not have been mounted by then).

Ah ha!

@jalam

So yeah, I think the mystery's been solved, and I feel the discussion can closed now. Thanks again for sticking around and offering your help, time and insight Nick!

Glad it is solved and you are welcome