Rclone fails to run credential_process on AWS AppStream instance

What is the problem you are having with rclone?

I'm trying to use rclone to list/mount AWS S3 bucket to AWS AppStream builder instance having IAM role assigned to access S3. Credentials are passed through credentials_process command which rclone fails to envoke.

Run the command 'rclone version' and share the full output of the command.

PS C:\rclone> c:\rclone\rclone.exe version
rclone v1.68.2

  • os/version: Microsoft Windows Server 2022 Datacenter 21H2 (64 bit)
  • os/kernel: 10.0.20348.2762 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.23.3
  • go/linking: static
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

AWS S3

The command you were trying to run (eg rclone copy /tmp remote:tmp)

c:\rclone\rclone.exe --config c:\rclone\rclone.conf lsd s3:appstream2-test-shared-bucket --s3-profile appstream_machine_role

The rclone config contents with secrets removed.

[s3]
type = s3
provider = AWS
env_auth = true
region = eu-central-1
location_constraint = eu-central-1
server_side_encryption = AES256

A log from the command with the -vv flag

2024/12/30 13:22:12 DEBUG : rclone: Version "v1.68.2" starting with parameters ["C:\\rclone\\rclone.exe" "--config" "c:\\rclone\\rclone.conf" "lsd" "s3:appstream2-test-shared-bucket" "--s3-profile" "appstream_machine_role" "-vv"]
2024/12/30 13:22:12 DEBUG : Creating backend with remote "s3:appstream2-test-shared-bucket"
2024/12/30 13:22:12 DEBUG : Using config file from "c:\\rclone\\rclone.conf"
2024/12/30 13:22:12 DEBUG : s3: detected overridden config - adding "{0uQwe}" suffix to name
2024/12/30 13:22:12 DEBUG : fs cache: renaming cache item "s3:appstream2-test-shared-bucket" to be canonical "s3{0uQwe}:appstream2-test-shared-bucket"
'\"C:\Program Files\Amazon\Photon\PhotonRoleCredentialProvider\PhotonRoleCredentialProvider.exe\"' is not recognized as an internal or external command,
operable program or batch file.
2024/12/30 13:22:13 ERROR : : error listing: operation error S3: ListObjectsV2, get identity: get credentials: failed to refresh cached credentials, process provider error: error in credential_process: exit status 1
2024/12/30 13:22:13 DEBUG : 2 go routines active
2024/12/30 13:22:13 NOTICE: Failed to lsd with 2 errors: last error was: operation error S3: ListObjectsV2, get identity: get credentials: failed to refresh cached credentials, process provider error: error in credential_process: exit status 1```

aws profile is as follows

[profile appstream_machine_role]
credential_process = "C:\Program Files\Amazon\Photon\PhotonRoleCredentialProvider\PhotonRoleCredentialProvider.exe" --role=Machine

this command works outside rclone. aws cli uses this profile without any issue.

welcome to the forum,

S3 backend updated to use AWS SDKv2 as v1 is now unsupported.

as a test, i would try v1.67.0

Thanks. I tried 1.67 and it works. But I wonder if this should be fixed on newer versions for SDKv2?

ok. good.

can you post the command and debug output.

for a deeper look, run the command with --dump=headers