Rclone to minio behind nginx

Suspected Bug
1

What is the problem you are having with rclone?

I'm trying to do a sync (or put) to minio behind nginx; it works well from other services, but rclone fails to put (read is Ok).

Same server works Ok with other tools (like aws's s3, third party S3 things) + it works well with ```rclone ls backup_s3:ms-files`.

My setup is that I'm running minio behind nginx. I tried to setup rclone as per minio docs - rclone-with-minio-server.html guide, tried both HTTP and HTTPS (was worried about plain HTTP), but no luck.

Any help would be much appreciated!!

What is your rclone version (output from rclone version)

rclone v1.55.1
- os/type: linux
- os/arch: amd64
- go/version: go1.16.3
- go/linking: static
- go/tags: none

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Ubuntu 18.04

Which cloud storage system are you using? (eg Google Drive)

Self hosted MinIO

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Paste command here

rclone sync /tmp/backup/ backup_s3:ms-files -vv --dump responses --retries 1


#### The rclone config contents with secrets removed.  

https://pastebin.com/zPTryHik


#### A log from the command with the `-vv` flag  

https://pastebin.com/h4yCHjSx
2

hello and welcome to the forum,

have you tested rclone direct to minio, without nginx, over a vpn or ssh session?

3

The error in the log is

<Error>
<Code>NotImplemented</Code>
<Message>A header you provided implies functionality that is not implemented</Message>
<Key>emptyFile</Key>
<BucketName>ms-files</BucketName>
<Resource>/ms-files/emptyFile</Resource><RequestId>167CD02BDA0E0869</RequestId>
<HostId>7050e9ff-e26f-4aa7-a51e-c3af7be3f888</HostId></Error>

Note that rclone uses a different PUT method to most s3 tools (for complicated reasons to do with missing interfaces in the SDK). It looks like this

2021/05/07 17:47:32 DEBUG : PUT /ms-files/emptyFile?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=BzNIlPY1ODBQZbBLB9%2F20210507%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210507T144732Z&X-Amz-Expires=900&X-Amz-SignedHeaders=content-md5%3Bcontent-type%3Bhost%3Bx-amz-acl%3Bx-amz-meta-mtime%3Bx-amz-server-side-encryption%3Bx-amz-server-side-encryption-aws-kms-key-id&X-Amz-Signature=7967266f20eaaccb2f41c5373dd6291355538d5f200836643f9091a078938a21 HTTP/1.1
Host: store.server.name
User-Agent: rclone/v1.55.1
Content-Length: 0
content-md5: 1B2M2Y8AsgTpgAmY7PhCfg==
content-type: application/octet-stream
x-amz-acl: private
x-amz-meta-mtime: 1620398807.249955783
x-amz-server-side-encryption: AES256
x-amz-server-side-encryption-aws-kms-key-id: XXX
Accept-Encoding: gzip

All of those are pretty standard, except for the x-amz-server-side-encryption ones.

Can you try a put without those to see if it works (comment out server_side_encryption = AES256) in the config file.

1 Like
4

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.