GCS remote is getting created for with Application Default Credentials (IAM) based auth but config/create response shows 500 error

Help and Support
1

Hello Experts,

I am trying access GCS bucket with gcs remote which uses ADC based auth to access bucket.

I am running rclone (in remote control mode) in k8 pod which runs on google machine and setup with service account IAM.

Tried to create remote with below inputs:
POST url: http://localhost:5572/config/create
body:
{
"name": "gcs-remote-mi25",
"type": "google cloud storage",
"parameters": {
"provider":"google cloud storage",
"bucket_policy_only": "true"
},
"_config": {
"env_auth": "true"
}
}

Remote is getting created in rclone config :
/data # rclone config show gcs-remote-mi

[gcs-remote-mi]
type = google cloud storage
provider = google cloud storage
bucket_policy_only = true

And if I try to use this remote to access bucket , it works fine -
/data # rclone ls gcs-remote-mi:/o9dl-gcpgt10083
7 sampledir/sample.txt/sample.txt

But issue is remote creation (http://localhost:5572/config/create) api responds with 500 error (though its creating remote in config)

Error body :
{
"error": "config failed to refresh token: failed to start auth webserver: listen tcp 127.0.0.1:53682: bind: address already in use",
"input": {
"_config": {
"env_auth": "true"
},
"name": "gcs-remote-mi25",
"parameters": {
"bucket_policy_only": "true",
"provider": "google cloud storage"
},
"type": "google cloud storage"
},
"path": "config/create",
"status": 500
}

Looks like during remote creation its trying to create access token y opening browser etc.

How can I tell Rclone not to go for access token and just use application default credential way?

I see " --gcs-env-auth" option which looks to be the solution, but did not understand how to use it in my rcd mode config/create request

image
image1185×617 38.4 KB

Any suggestion is much appreciated.

Thanks
Mahendra

2

not an expert but google cloud storage does not have provider.
tho, the odds are rclone ignores that.

3

@asdffdsa you are right, since aws s3 needs that we are passing that for all 3 clouds remote.
Rclone ignores that field for GCS/ADLS as per knowledge

4

I think I got the answer for my issue.
I am using rclove version 1.57 and "env_auth" option is introduced only in 1.62 !

5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.