What is the problem you are having with rclone?
Hi All,
I'm testing out running rclone within my EKS cluster. Within my org's federated AWS sub's we don't have the ability to create user accounts; therefore unable to create Access Key and Secret credentials for authentication. So instead, I'm trying to use the OIDC provider I have associated with my EKS cluster, combined with a service account, IAM Role and Policy which includes the necessary permissions to be able to use the S3 bucket I have created.
Despite setting all of that up I am getting ACCESSDENIED when trying to copy to the S3 bucket
Run the command 'rclone version' and share the full output of the command.
/data # rclone version
rclone v1.64.0
- os/version: alpine 3.18.3 (64 bit)
- os/kernel: 5.15.117 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.21.1
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
AWS S3
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone copy --s3-no-check-bucket /mnt/a s3:/transfer/input/ -vv
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
/data # rclone config redacted
[s3]
type = s3
provider = AWS
env_auth = true
region = eu-west-2
location_constraint = EU
A log from the command that you were trying to run with the -vv flag
2023/10/01 13:32:24 DEBUG : rclone: Version "v1.64.0" starting with parameters ["rclone" "copy" "--s3-no-check-bucket" "/mnt/a" "s3:/transfer/input/" "-vv"]
2023/10/01 13:32:24 DEBUG : Creating backend with remote "/mnt/a"
2023/10/01 13:32:24 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/10/01 13:32:24 DEBUG : Creating backend with remote "s3:/transfer/input/"
2023/10/01 13:32:24 DEBUG : s3: detected overridden config - adding "{Dn7qA}" suffix to name
2023/10/01 13:32:24 DEBUG : fs cache: renaming cache item "s3:/transfer/input/" to be canonical "s3{Dn7qA}:transfer/input"
2023/10/01 13:32:24 ERROR : S3 bucket mfts-transfer path input: error reading destination root directory: AccessDenied: Access Denied
status code: 403, request id: KTJBKAWFP0AF5G7Q, host id: ktEn3BMakIHLGA6uaZ32hPDYZFZ77BqfFM0KPmxzgWgQkNO3pSbSOVpSbxgxdaoZCW2xwCU45Ew=
2023/10/01 13:32:24 DEBUG : S3 bucket transfer path input: Waiting for checks to finish
2023/10/01 13:32:24 DEBUG : S3 bucket transfer path input: Waiting for transfers to finish
2023/10/01 13:32:24 ERROR : Attempt 1/3 failed with 1 errors and: AccessDenied: Access Denied
status code: 403, request id: KTJBKAWFP0AF5G7Q, host id: ktEn3BMakIHLGA6uaZ32hPDYZFZ77BqfFM0KPmxzgWgQkNO3pSbSOVpSbxgxdaoZCW2xwCU45Ew=
2023/10/01 13:32:24 ERROR : S3 bucket transfer path input: error reading destination root directory: AccessDenied: Access Denied
status code: 403, request id: KTJFMA6FQ1MERYM1, host id: vgIj7L/beq7Myl0wg85y8Cs0uCg1HiIANVrsRjfF35tX4NuolNma196cAlHXc8ZhKXf9zn0Kueo=
2023/10/01 13:32:24 DEBUG : S3 bucket transfer path input: Waiting for checks to finish
2023/10/01 13:32:24 DEBUG : S3 bucket transfer path input: Waiting for transfers to finish
2023/10/01 13:32:24 ERROR : Attempt 2/3 failed with 1 errors and: AccessDenied: Access Denied
status code: 403, request id: KTJFMA6FQ1MERYM1, host id: vgIj7L/beq7Myl0wg85y8Cs0uCg1HiIANVrsRjfF35tX4NuolNma196cAlHXc8ZhKXf9zn0Kueo=
2023/10/01 13:32:24 ERROR : S3 bucket transfer path input: error reading destination root directory: AccessDenied: Access Denied
status code: 403, request id: KTJ47NBNJ27WEKSR, host id: y7w7+0TwysW7SNoAN9D3KhfFGw30CXvZ9I6lS4RYQjc87Bp7bICTIf07HeS8t6d7hVVILcqE+3U=
2023/10/01 13:32:24 DEBUG : S3 bucket transfer path input: Waiting for checks to finish
2023/10/01 13:32:24 DEBUG : S3 bucket transfer path input: Waiting for transfers to finish
2023/10/01 13:32:24 ERROR : Attempt 3/3 failed with 1 errors and: AccessDenied: Access Denied
status code: 403, request id: KTJ47NBNJ27WEKSR, host id: y7w7+0TwysW7SNoAN9D3KhfFGw30CXvZ9I6lS4RYQjc87Bp7bICTIf07HeS8t6d7hVVILcqE+3U=
2023/10/01 13:32:24 INFO :
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.1s
2023/10/01 13:32:24 DEBUG : 6 go routines active
2023/10/01 13:32:24 Failed to copy: AccessDenied: Access Denied
status code: 403, request id: KTJ47NBNJ27WEKSR, host id: y7w7+0TwysW7SNoAN9D3KhfFGw30CXvZ9I6lS4RYQjc87Bp7bICTIf07HeS8t6d7hVVILcqE+3U=
Here's the IAM Policy and Trusted Entities I have set on the role I'm using:
Policy
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:DeleteObject",
"s3:GetObject",
"s3:PutObject",
"s3:PutObjectAcl",
"s3:GetObjectTagging"
],
"Resource": [
"arn:aws:s3:::transfer/*",
"arn:aws:s3:::transfer"
]
},
{
"Effect": "Allow",
"Action": "s3:ListAllMyBuckets",
"Resource": "arn:aws:s3:::*"
}
]
}
Trusted Entities
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::<AWS_SUB_ID>:oidc-provider/oidc.eks.eu-west-2.amazonaws.com/id/<OIDC_ID>"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.eu-west-2.amazonaws.com/id/<OIDC_ID>:sub": "system:serviceaccount:rclone:rclone-sa"
}
}
}
]
}
And for those familiar with k8's, here's my Deployment, Service Account and ConfigMap
apiVersion: apps/v1
kind: Deployment
metadata:
name: rclone
spec:
replicas:
selector:
matchLabels:
app: rclone
template:
metadata:
labels:
app: rclone
spec:
containers:
- name: rclone
image: rclone/rclone
command: ["rclone"]
args: ["copy", "--s3-no-check-bucket", "/mnt/a", "s3:/transfer/input/", "-vv"]
volumeMounts:
- name: rclone-config
mountPath: /root/.config/rclone
- name: mount1
mountPath: /mnt/a
volumes:
- name: rclone-config
configMap:
name: rclone-config
- name: mount1
hostPath:
path: /mnt/a
type: DirectoryOrCreate
---
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/name: rclone
name: rclone-sa
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::<AWS_SUB_ID>:role/s3-rclone
---
apiVersion: v1
kind: ConfigMap
metadata:
name: rclone-config
data:
rclone.conf: |
[s3]
type = s3
provider = AWS
env_auth = true
region = eu-west-2
location_constraint = EU
Any help greatly appreciated!
