What is the problem you are having with rclone?
When trying to list folders in Google Cloud Storage I am getting could not find default credentials
error. Details bellow.
What is your rclone version (output from rclone version
)
rclone v1.55.1
- os/type: linux
- os/arch: amd64
- go/version: go1.16.3
- go/linking: static
- go/tags: none
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Linux, Ubuntu 20.04, 64 bit
Which cloud storage system are you using? (eg Google Drive)
Google Cloud Storage
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
rclone lsd gcs-auth:seldon-models
The rclone config contents with secrets removed.
[gcs-auth]
type = google cloud storage
env_auth = false
client_id = <redacted client id>
client_secret = <redacted client secret>
A log from the command with the -vv
flag
$ rclone lsd gcs-auth:seldon-models -vv
<7>DEBUG : Using config file from "/home/rskolasinski/.config/rclone/rclone.conf"
<7>DEBUG : rclone: Version "v1.55.1" starting with parameters ["/home/rskolasinski/.asdf/installs/rclone/1.55.1/bin/rclone" "lsd" "gcs-auth:seldon-models" "-vv"]
<7>DEBUG : rclone: systemd logging support activated
<7>DEBUG : Creating backend with remote "gcs-auth:seldon-models"
Failed to create file system for "gcs-auth:seldon-models": failed to configure Google Cloud Storage: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
Extra information
- Listing same folder works if I use
anonymous = true
as it is public one
[gs]
type = google cloud storage
anonymous = true
and
$ rclone lsd gs:seldon-models -vvv
<7>DEBUG : Using config file from "/home/rskolasinski/.config/rclone/rclone.conf"
<7>DEBUG : rclone: Version "v1.55.1" starting with parameters ["/home/rskolasinski/.asdf/installs/rclone/1.55.1/bin/rclone" "lsd" "gs:seldon-models" "-vvv"]
<7>DEBUG : rclone: systemd logging support activated
<7>DEBUG : Creating backend with remote "gs:seldon-models"
0 2021-05-13 14:46:59 -1 alibi
0 2021-05-13 14:46:59 -1 alibi-detect
0 2021-05-13 14:46:59 -1 custom
0 2021-05-13 14:46:59 -1 keras
0 2021-05-13 14:46:59 -1 mab
0 2021-05-13 14:46:59 -1 mlflow
0 2021-05-13 14:46:59 -1 mlserver
0 2021-05-13 14:46:59 -1 odcd
0 2021-05-13 14:46:59 -1 openvino
0 2021-05-13 14:46:59 -1 pytorch
0 2021-05-13 14:46:59 -1 sklearn
0 2021-05-13 14:46:59 -1 spacy
0 2021-05-13 14:46:59 -1 tempo
0 2021-05-13 14:46:59 -1 tensorrt
0 2021-05-13 14:46:59 -1 test
0 2021-05-13 14:46:59 -1 tfserving
0 2021-05-13 14:46:59 -1 triton
0 2021-05-13 14:46:59 -1 trtis
0 2021-05-13 14:46:59 -1 xgboost
<7>DEBUG : 3 go routines active
- Using client id and secret works with
mc
tool
$ mc config host add gcs https://storage.googleapis.com <redacted client id> <redacted client secret>
$ mc ls gcs/seldon-models/
[2021-05-13 14:46:07 BST] 0B alibi-detect/
[2021-05-13 14:46:07 BST] 0B alibi/
[2021-05-13 14:46:07 BST] 0B custom/
[2021-05-13 14:46:07 BST] 0B keras/
[2021-05-13 14:46:07 BST] 0B mab/
[2021-05-13 14:46:07 BST] 0B mlflow/
[2021-05-13 14:46:07 BST] 0B mlserver/
[2021-05-13 14:46:07 BST] 0B odcd/
[2021-05-13 14:46:07 BST] 0B openvino/
[2021-05-13 14:46:07 BST] 0B pytorch/
[2021-05-13 14:46:07 BST] 0B sklearn/
[2021-05-13 14:46:07 BST] 0B spacy/
[2021-05-13 14:46:07 BST] 0B tempo/
[2021-05-13 14:46:07 BST] 0B tensorrt/
[2021-05-13 14:46:07 BST] 0B test/
[2021-05-13 14:46:07 BST] 0B tfserving/
[2021-05-13 14:46:07 BST] 0B triton/
[2021-05-13 14:46:07 BST] 0B trtis/
[2021-05-13 14:46:07 BST] 0B xgboost/