Rclone with Gcloud WIF configuration

Help and Support
1

What is the problem you are having with rclone?

I want to know how I connect rclone with Gcloud WIF( Workload Identity Federation ). Like which credentials I need to pass in and any guides will be really useful.

2

That looks to be an IAM tool by Google. Not quite sure how it ties into rclone, which is a cloud storage tool.

Not sure how they tie together.

3

hi, took a very quick read of google documentation.

https://cloud.google.com/iam/docs/workload-identity-federation
The token exchange flow returns a federated access token. You can use this token to impersonate a service account and obtain a short-lived OAuth 2.0 access token. The short-lived access token lets you call any Google Cloud APIs that the service account has access to.

this looks like the same issue i deal with AWS S3 when using MFA.
rclone does not support Multi Factor Authentication.
so i need to call a function using AWS S3 api, not with rclone.
that function returns a short-lived access token that i feed that to rclone.

so i think that you need to do something similar

  1. get that short-lived access token using the gcloud api.
  2. create an on-the-fly rclone remote using that token.
  3. use that remote with your rclone commands.

tho i am using AWS S3 api,
i can share the python function that gets the token and creates that on-the-fly remote.

4

Hello @asdffdsa thank you very much for the help. Don't we have any env variable or some config from rclone end directly to make use of this short-lived access token without writing custom code in between to handle this ?

5

yes, rclone can use env variables and standard .json service account file.
https://rclone.org/googlecloudstorage/#gcs-service-account-credentials

for example, here rclone is using a service account file, as downloaded direct from google website.

rclone lsd "gdrive,service_account_file='service.account.file.json':"
          -1 2022-03-03 09:33:10        -1 zork
6

Does the above( standard .json service account file) work when doing auth using WIF from gcloud ? or to use WIF do I need to create custom code mandatory to handle with WIF ?

7

as far as i know, rclone has no concept of WIF.

once you have the token, should be able to use
--gcs-token

beyond that and the official gcloud detailed examples, i do not know.

hopefully, someone will stop by to assist further.

8

You can use the oauth flow directly with google cloud storage - did you try that @Magesh ? Just run through rclone config. I know nothing about WIF though, so maybe that is barking up the wrong tree!