Rclone on AWS Appstream can't use profile appstream_machine_role

Suspected Bug
1

Hello,

What is the problem you are having with rclone?

I am trying to run simple rclone ls command in AWS Appstream environment, while creating image in the image builder. So, in the image builder session, I do expect that the PhotonRoleCredentialsProvider.exe won't return any creds. But it should at least run. However, the error I get is -
'"C:\Program Files\Amazon\Photon\PhotonRoleCredentialProvider\PhotonRoleCredentialProvider.exe"' is not recognized as an internal or external command,
operable program or batch file.

I did stumble upon the same issue reported last year - Dec 2024 (Rclone fails to run credential_process on AWS AppStream instance). But it seems the resolution for that was to use rclone version v1.67.0.

I am already running a newer version v1.69.1 as shown below. So not sure why this version gives me the same error.

Run the command 'rclone version' and share the full output of the command.

rclone version
rclone v1.69.1

  • os/version: Microsoft Windows Server 2022 Datacenter 21H2 21H2 (64 bit)
  • os/kernel: 10.0.20348.2762 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.24.0
  • go/linking: static
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

AWS S3

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone ls S3:<bucket_name> --s3-profile appstream_machine_role --dump=headers

The rclone config contents with secrets removed.

[S3]
type = s3
provider = AWS
env_auth = true
region = us-east-1
use_accelerate_endpoint = true

A log from the command with the -vv flag

>rclone ls S3:<bucket_name> --s3-profile appstream_machine_role --dump=headers
2025/02/27 08:00:28 NOTICE: Automatically setting -vv as --dump is enabled
2025/02/27 08:00:28 DEBUG : rclone: Version "v1.69.1" starting with parameters ["rclone" "ls" "S3:<bucket_name>" "--s3-profile" "appstream_machine_role" "--dump=headers"]
2025/02/27 08:00:28 DEBUG : Creating backend with remote "S3:<bucket_name>l"
2025/02/27 08:00:28 DEBUG : Using config file from "C:\\Users\\ImageBuilderAdmin\\AppData\\Roaming\\rclone\\rclone.conf"
2025/02/27 08:00:28 DEBUG : S3: detected overridden config - adding "{0uQwe}" suffix to name
2025/02/27 08:00:28 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2025/02/27 08:00:28 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2025/02/27 08:00:28 DEBUG : fs cache: renaming cache item "S3:<bucket_name>" to be canonical "S3{0uQwe}:<bucket_name>"
'\"C:\Program Files\Amazon\Photon\PhotonRoleCredentialProvider\PhotonRoleCredentialProvider.exe\"' is not recognized as an internal or external command,
operable program or batch file.
2025/02/27 08:00:28 DEBUG : 1 go routines active
2025/02/27 08:00:28 NOTICE: Failed to ls: operation error S3: ListObjectsV2, get identity: get credentials: failed to refresh cached credentials, process provider error: error in credential_process: exit status 1

Appreciate all the help.

Thanks,
Urjit

2

welcome to the forum,

the reason might be v1.68.0 - S3 backend updated to use AWS SDKv2

could open an issue at github...

3

Update -

I experimented a bit during the day.
The AWS Appstream comes pre-configured with

credential_process = "C:\Program Files\Amazon\Photon\PhotonRoleCredentialProvider\PhotonRoleCredentialProvider.exe" --role=Machine

and I was seeing the "is not recognized as an internal or external command" error.

So I tried replacing the default command with a different one. The idea was to see if rcolne is at least able to execute some exe file.

I set credential_process = <absolute_path_to_firefox.exe>.
And rclone did run this command as part of its flow to get credentials at runtime. I did not get any "is not recognized as an internal or external command" error this time.

So, now I wonder if the error I am seeing has something to do with the default executable shipped by AWS appstream or the path where it is present. And would that render this issue as "not an rclone issue, but AWS appstream path / permissions issue" ?

In that case, where do I go next? To AWS? To the appstream team?

Thanks,
~ Urjit

4

can you run that on the command line?

5

Yes. The command itself doesn't have a problem.

Some more experiment -

credential_process = "C:\Program Files\Amazon\Photon\PhotonRoleCredentialProvider\PhotonRoleCredentialProvider.exe" --role=Machine

If I drop the trailing --role=Machine from this line in the config/credentials file, then the is not recognized as an internal or external command error goes away.

So it seems to me that rclone is not able to handle the whole command with parameters. It can handle the command itself just fine (w/o the parameters)

I confirmed the behavior with my dummy profile where I had set the command to be filrefox.exe. If I change that line and add a dummy parameter, I get the dreaded is not recognized as an internal or external command error

6

maybe remove the spaces and quote characters in the path, can create a symlink or try something like

credential_process = c:\progra~1\amazon\photon\photonrolecredentialprovider\photonrolecredentialprovider.exe --role=machine

or try to create a .cmd file
credential_process = c:\path\to\file.cmd

7

Thank you for your suggestion.

But the point is - Its appsteram that places the executable file under 'Program Files' (path with a white space). And its appsteram that gives a pre-configured value for 'credential_process' with the --role=Machine parameter.

And so, even if I get things to work by either removing the --role parameter, or modifying quotes, or wrapping up the exe in my own cmd file and use that, all of that still feels like a hack and it would still mean that rclone cannot work with the defaults provided by appstream as is.

I wonder if that was always the case and people were always getting it to work with some hack? Or has this broken recently, either because AWS did something with how appstream sets things up by default, or because something changed in rclone?