NotSupported: The header x-amz-copy-source implies functionality that is not supported

Help and Support
1

What is the problem you are having with rclone?

I'm trying to copy files from AWS S3 to local Netapp S3 bucket, and getting this error:

2023/04/30 13:56:17 ERROR : path/file.dat: Failed to set modification time: NotSupported: The header x-amz-copy-source implies functionality that is not supported.
	status code: 501, request id: , host id:

Run the command 'rclone version' and share the full output of the command.

rclone v1.63.0-beta.6998.ae6874170
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 5.19.0-40-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.20.3
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

As said, aws s3

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy s3_aws:company_name netapp:new_bucket/new_path --config ~/.rclone.conf --metadata --metadata-include tier=STANDARD --no-check-certificate  --stats-log-level DEBUG

The rclone config contents with secrets removed.

[s3_aws]
type = s3
access_key_id = 
provider = AWS
region = eu-west-1
secret_access_key = 

[netapp]
type = s3
access_key_id = 
endpoint = netapp.app.local
provider = Other
secret_access_key =

A log from the command with the -vv flag

This log attached is on debug mode

Edit:
I found an article related, but can't access it: Using sync feature in an S3 client returns x-amz-copy-source error - NetApp Knowledge Base

2

ACL/policy issue?

did you post the rclone debug log?

not able to create a free account?

3

This means that rclone failed to set the modification time of an existing object.

This seems to be because the Netapp doesn't support server side copy which is necessary for that to work.

There are a number of ways to work around this. You can use a --checksum or a --size-only sync.

You can also request rclone not update modification times with

  --no-update-modtime    Don't update destination mod-time if files identical
4

It's a great answer, I'm checking with NetApp - seems the current version of ONTAP does not support modification, only Write/Read.
What if I use the flag --no-update-modtime for file that changed? is it going to replace it (delete old and upload the new)?

and if I don't use this flag, I do a normal copy, does copy replace modify time for all the files in bucket each time I do copy?

5

Yes it will delete the old and upload the new file.

If you upload a file then it will have a new modification date set. That will work without server side copy.

6

OK the issue is found here: Using sync feature in an S3 client returns x-amz-copy-source error - NetApp Knowledge Base

The solution for this error is having ONTAP version > 9.12.1, so we upgrade our ONTAP to the latest.
But now new error:

Failed to set modification time: InvalidArgument: Copy source and destination cannot be the same.

I've tried with -vvv, here example:

path/Dependencies.ini: Size and modification time the same (differ by 0s, within tolerance 1ns)
path/Dependencies.ini: Unchanged skipping

Not sure what is the meaning the error is not understood.
Are the files trying to copy are same, so it's skipped for the next file? if so it's great, that we looking for when doing rsync, to keep going if the files are the same.

7

The modification time is stored as metadata on the object.

The only way to update metadata in the s3 protocol is to COPY the object onto itself.

So in this case the source and the destination should be the same.

This looks like a bug in the NetApp where it isn't following the s3 protocol properly.

8

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.