What is the problem you are having with rclone?
I would like to use rclone to sync from a local S3 storage to a Google Cloud Storage bucket. The local config seems to work and I need help configuring the target (GCS).
What is your rclone version (output from rclone version
)
Latest Official provided rclone Docker image
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Latest Official provided rclone Docker image will be running on Kubernetes. ATM it's running on my local machine (Ubuntu 18.04).
Which cloud storage system are you using? (eg Google Drive)
Local: MinIO S3
Remote: Google Cloud Storage (GCS)
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
sudo docker run --env-file ./rclone.env rclone/rclone:latest sync SOURCES3:rclone-test-onprem TARGETS3:rclone-test-duryto
The rclone config contents with secrets removed.
RCLONE_CONFIG_SOURCES3_TYPE=s3
RCLONE_CONFIG_SOURCES3_ACCESS_KEY_ID=removed
RCLONE_CONFIG_SOURCES3_SECRET_ACCESS_KEY=removed
RCLONE_CONFIG_SOURCES3_ENDPOINT=removed
RCLONE_CONFIG_TARGETS3_TYPE=gcs
RCLONE_CONFIG_TARGETS3_CLIENT_ID=removed
RCLONE_CONFIG_TARGETS3_CLIENT_SECRET=removed
RCLONE_CONFIG_TARGETS3_PROJECT_NUMBER=removed
A log from the command with the -vv
flag
2020/11/16 09:24:01 DEBUG : rclone: Version "v1.53.2" starting with parameters ["rclone" "sync" "-vv" "SOURCES3:rclone-test-onprem" "TARGETS3:rclone-test-duryto"]
2020/11/16 09:24:01 DEBUG : Creating backend with remote "SOURCES3:rclone-test-onprem"
2020/11/16 09:24:01 NOTICE: Config file "/config/rclone/rclone.conf" not found - using defaults
2020/11/16 09:24:01 DEBUG : Creating backend with remote "TARGETS3:rclone-test-duryto"
2020/11/16 09:24:01 Failed to create file system for "TARGETS3:rclone-test-duryto": failed to configure Google Cloud Storage: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
I guess I'm using the ENV variables for configuring the GCS remote the wrong way. I don't want to use the JSON file for the Service Account, so I got an ACCESS/SECRET key combination to access the Storage bucket.