Unable to mount AWS S3 on AWS AppStream user session

Help and Support
1

We are using AWS AppStream to stream applications to user. Our use case with rclone is to mount S3 objects based on the logged in user's IAM permissions validated through IAM Role.

What is the problem you are having with rclone?

As part of AWS AppStream custom image preparation (Image Builder) on Windows OS:

  • copied latest rclone zip

  • Prepared session start up script

  • Using session start up script (.ps1), we have

    • installed winfsp
    • copied rclone.conf to C:\Users\PhotonUser\AppData\Roaming\rclone\ location
    • Generated temporary .aws/credentials file
    • Prepared a .bat file with below content
      C:\rclone\rclone.exe mount s3_mount_v1:bucket_name/user_no/ Z: --vfs-cache-mode full --config C:\Users\ImageBuilderAdmin\AppData\Roaming\rclone\rclone.conf --log-file C:\log_file_path\log.txt

  • Now, we tried to call above mentioned .bat file through .ps1, with below code block

      	$processOptions = @{
  		FilePath = "C:\Appstream\SessionScripts\s3_objects.bat"
  		RedirectStandardOutput = "C:\log_file_path\log_stdout.txt"
  		RedirectStandardError = "C:\log_file_path\log_stderr.txt"
  		WindowStyle = "Hidden"
  	}
  	Start-Process @processOptions

  • When we launched AppStream user session, we are able to see a virtual drive with Z:\ has been created. But, when we tried to double click, it said:
    Z:\ is not accessible.
    The request could not be performed because of an I/O device error

  • Please note that, we are able to mount s3 objects on virtual drive with out any issue if we copy .bat file on to Desktop and double click it. But we do not want users to double click .bat file every time that they want to mount s3 objects. So, we want to automate this by calling .bat inside .ps1. The problem is with this step.

Run the command 'rclone version' and share the full output of the command.

  • C:\rclone>rclone --version
  • rclone v1.62.2
  • os/version: Microsoft Windows Server 2019 Datacenter 1809 (64 bit)
  • os/kernel: 10.0.17763.4131 Build 17763.4131.4131 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.20.2
  • go/linking: static
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

AWS S3

The command you were trying to run (eg rclone copy /tmp remote:tmp)

C:\rclone\rclone.exe mount s3_mount_v1:bucket_name/user_no/ Z: --vfs-cache-mode full --config C:\Users\PhotonUser\AppData\Roaming\rclone\rclone.conf --log-file C:\log_file_path\log.txt

The rclone config contents with secrets removed.

[s3_mount_v1]
type = s3
provider = AWS
profile = default
region = us-east-1
env_auth = true

A log from the command with the -vv flag

2023/06/23 05:46:51 ERROR : /: Dir.Stat error: AccessDenied: Access Denied
	status code: 403, request id: RJ10WE0QCWQA33YX, host id: WZaD3S71wXJtVQFDATXCEQxDbvjU5mnWFei6nlWT8/9BqceIbTLu3HA/hvfHD80Fak++lVPbvEI=
2023/06/23 05:46:51 ERROR : IO error: AccessDenied: Access Denied
	status code: 403, request id: RJ10WE0QCWQA33YX, host id: WZaD3S71wXJtVQFDATXCEQxDbvjU5mnWFei6nlWT8/9BqceIbTLu3HA/hvfHD80Fak++lVPbvEI=
2023/06/23 05:46:51 ERROR : /: Dir.Stat error: AccessDenied: Access Denied

Please let me know if you need any more details

2

If session start up script runs with elevated privileges then it is expected behaviour and there are multiple ways to solve it mentioned in documentation.

There are also few things to take care of when you want to run rclone on Windows as a background process - Install

3

welcome to the forum,

  • Access Denied, rclone does not have permisison to access the bucket.
    make sure rclone can see the credentials, get the rclone debug log.

  • best to test a simple command, before trying rclone mount, something like
    rclone copy ./file.ext s3_mount_v1:bucket_name/user_no -vv --retries=1
    and post the full output

4

There is also the old chestnut - try using --s3-no-check-bucket or no_check_bucket = true in the config file.

5

It does not look like problem with S3 buckets - but with how mount is initiated.

6

@kapitainsky Yes. You are right.

We dont have issue in mounting s3 bucket with out keeping .bat file in aws appsream startup script. We are able to access virtual drive and able to see s3 objects in Appstream 'PhotonUser' session.

But, when we run that .bat file from .ps1 script (start-process cmdlet), we are able to see Z:\ drive but when we double click it, it says z: is not accessible.

You may be aware, AWS Appstream startup scripts run with 'system' privilege.

As you said, this error seems to be 'System' user vs 'photonuser'

7

can hard code the location of the credentials using
AWS_SHARED_CREDENTIALS_FILE

8

without that, i think the log would show

ERROR : : error listing: AccessDenied: User: arn:aws:iam::100000065159:user/user.redacted is not authorized to perform: s3:ListAllMyBuckets on resource: arn:aws:s3:::
        status code: 403
9

Maybe there is something special about mounts made in start-process cmdlet - even if it is run as SYSTEM... I have never used it myself so only speculating.

I would try few things:

  1. Add --no-console to mount command

  2. run mount bat with PsExec

psexec -s rclone.exe mount.....

  1. add --network-mode to mount command
10

@kapitainsky Tried --network-mode and other one. Didn't help

When AWS AppStream Session startup script runs, script will be executed with 'System'/high privilege on 'PhotonUser' session. Looks like it is trying to access 'c:\Windows\System32\config\systemprofile' path on 'PhotonUser' session.

I could not see 'config\systemprofile' directory structure under 'c:\windows\system32' in 'PhtotonUser' session. Also, I am unable to create these directories through even AppStream Session Startup scripts.

Is it looking for rclone conf file in that location? But, I have given '--config' flag with custom path while invoking rclone command. Then, why is it not taking that custom path?

Do you have any idea?

11

Yes it should not look for config when you define its location explicitly.

I am not Windows expert but given that issue seems to be related to how rclone is started maybe it makes sense to change approach and do not start it from your script.

Instead script could create Windows service or Task Scheduler which would start rclone mount later?

12

Maybe it is not rclone but winfsp? I am thinking here that at early stage of session startup maybe some winfsp components are no fully ready? Again I would try to delay rclone mount start for later - your script would finish and Windows Service would trigger mount some time later.

From docs - how to create new service.

New-Service -Name Rclone -BinaryPathName 'c:\rclone\rclone.exe mount remote:/files X: --config c:\rclone\config\rclone.conf --log-file c:\rclone\logs\mount.txt'
13

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.