What is the problem you are having with rclone?
Hello rclone community,
I am new to rclone and hoping you could help me with my particular challenge.
We have an S3 bucket in AWS that we would like to mount (and have read-only access) on an AWS AppStream image on Windows Server 2019.
Just to clarify, the mounting of S3 bucket does work. The only thing is that, the first time it tries to access the S3 bucket (and launch our application which is stored in the S3 bucket), it does take between 8-10s. But subsequent launches are quick to do. I suppose this is expected because the first time round, it will try to build the cache (and we're getting the 8-10s hit), whereas for subsequent launches, it would read from the cache, and the application launches almost immediately.
Coming onto my challenge.. What I would like to do is to reduce the wait time for the cache to be built everytime a new appstream session starts. AWS Appstream have a concept of application persistence where the user profile is stored on a VHD (and mounted on the appstream session). Noting that an appstream instance would always be started from a golden image (in our case, a vanilla windows 2019 image), if we were to store the cache in e.g. c:\temp, that would always need rebuilding everytime a new appstream instance is started (because the golden image wouldn't have this). However, since we enable application persistence, we could store it under C:\Users\PhotonUser\AppData\Local\rclone (i.e. the mounted VHD), so this will persist onto the next appstream instance (if the current instance is terminated).
From what I'm seeing, the cache does get retained when I terminate and start a new appstream session. However, looking at the property of the item (e.g. appclient.uar), I could see that the created date/time doesn't change, but the modified date/time changes everytime a new appstream session is launched. And the same behaviour where it still takes 8-10s to load the application on every appstream session (but almost instant for subsequent launches) is still observed despite the cache being stored in the mounted VHD - I terminated and started a new session within 3 minutes. I suspect that the cache is being rebuilt on each appstream session.
How do I go about speeding up the launch of my application at the first launch of an appstream instance (if there is already a cache present)? E.g. Is there a way to read from the cache without rebuilding it if it already exists? And how would it detect if changes have been made on the S3 bucket (which may not be that often)?
Hope to hear back
Many thanks,
James
Run the command 'rclone version' and share the full output of the command.
rclone v1.59.0
- os/version: Microsoft Windows Server 2019 Datacenter 1809 (64 bit)
- os/kernel: 10.0.17763.3046 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.18.3
- go/linking: static
- go/tags: cmount
Which cloud storage system are you using? (eg Google Drive)
AWS S3 bucket
The command you were trying to run (eg rclone copy /tmp remote:tmp)
c:\rclone\rclone.exe --config c:\rclone\rclone.conf mount sits_s3:mybucketomount S: --vfs-cache-mode full --network-mode --file-perms 0557 --read-only --no-modtime --buffer-size 0 --vfs-fast-fingerprint --cache-dir C:\Users\PhotonUser\AppData\Local\rclone --s3-profile appstream_machine_role
The rclone config contents with secrets removed.
[s3]
type = s3
provider = AWS
env_auth = true
region = ap-southeast-2
location_constraint = ap-southeast-2
server_side_encryption = aws:kms
sse_kms_key_id = <SOME_ARN>
storage_class = STANDARD
A log from the command with the -vv flag
I was about to include it, but it errored saying that the body of the message cannot have more than 32000 characters (and I had 2832000+ characters when I included the logs).. Can someone please let me know how I'd be able to upload the logs without exceeding the character limits? Thank you
