What is the problem you are having with rclone?
I am copying large number of fairly small files [.jpg, .png] from the Digital Ocean Spaces Object Storage to the AWS s3 . running - initially - rclone copy DO AWS-s3 and planning to run rclony sync later.
Operation works fine for a lot of files [I have over 2 mln of them] but fails for some. To get max details about the issue I run rclone copyto for a "failing" file and it is getting same error api error InvalidArgument: aws-chunked encoding is not supported when x-amz-content-sha256 UNSIGNED-PAYLOAD is supplied.
I did run rclone coputo to the local directory and this works OK. As the number of failing files is many thousands I need a "proper" solution - if possible. Workaroud - copy to local storage then upload to s3 is not practical/acceptable.
Run the command 'rclone version' and share the full output of the command.
Release v1.70.3 OS requirementsWhich cloud storage system are you using? (eg Google Drive)
Source - Digital Ocean Spaces Object Storage , Target - AWS s3
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone copyto -vvv --metadata do-weedvader:weedvader/static/1508204081-OG-Medicinals_Triangle_Kush.jpg s3-weedvader:weedvader-objects/static/test01
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
[do-weedvader]
type = s3
provider = DigitalOcean
access_key_id = XXX
secret_access_key = XXX
endpoint = sfo3.digitaloceanspaces.com
acl = bucket-owner-full-control
[s3-weedvader]
type = s3
provider = AWS
access_key_id = XXX
secret_access_key = XXX
region = us-west-1
location_constraint = us-west-1
acl = bucket-owner-full-control
storage_class = STANDARD
A log from the command that you were trying to run with the -vv flag
2025/08/02 14:13:28 DEBUG : rclone: Version "v1.70.3" starting with parameters ["rclone" "copyto" "-vv" "--metadata" "--s3-chunk-size" "200M" "do-weedvader:weedvader/static/1508204081-OG-Medicinals_Triangle_Kush.jpg" "s3-weedvader:weedvader-objects/static/test01.jpg"]
2025/08/02 14:13:28 DEBUG : Creating backend with remote "do-weedvader:weedvader/static/1508204081-OG-Medicinals_Triangle_Kush.jpg"
2025/08/02 14:13:28 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2025/08/02 14:13:28 DEBUG : do-weedvader: detected overridden config - adding "{JYX5F}" suffix to name
2025/08/02 14:13:28 DEBUG : fs cache: renaming child cache item "do-weedvader:weedvader/static/1508204081-OG-Medicinals_Triangle_Kush.jpg" to be canonical for parent "do-weedvader{JYX5F}:weedvader/static"
2025/08/02 14:13:28 DEBUG : Creating backend with remote "s3-weedvader:weedvader-objects/static/"
2025/08/02 14:13:28 DEBUG : s3-weedvader: detected overridden config - adding "{JYX5F}" suffix to name
2025/08/02 14:13:28 DEBUG : fs cache: renaming cache item "s3-weedvader:weedvader-objects/static/" to be canonical "s3-weedvader{JYX5F}:weedvader-objects/static"
2025/08/02 14:13:28 DEBUG : 1508204081-OG-Medicinals_Triangle_Kush.jpg: Need to transfer - File not found at Destination
2025/08/02 14:13:28 ERROR : 1508204081-OG-Medicinals_Triangle_Kush.jpg: Failed to copy: operation error S3: PutObject, https response error StatusCode: 400, RequestID: 4PDW9PQ388Q4XSKT, HostID: 5/gWJyGRTR6giFGPtoWRJE/8pQVx4KnggmNeEpm6FzvKw5eXP2fmhDPbwncdujjr89D2XQv4HtU=, api error InvalidArgument: aws-chunked encoding is not supported when x-amz-content-sha256 UNSIGNED-PAYLOAD is supplied.
2025/08/02 14:13:28 ERROR : Attempt 1/3 failed with 1 errors and: operation error S3: PutObject, https response error StatusCode: 400, RequestID: 4PDW9PQ388Q4XSKT, HostID: 5/gWJyGRTR6giFGPtoWRJE/8pQVx4KnggmNeEpm6FzvKw5eXP2fmhDPbwncdujjr89D2XQv4HtU=, api error InvalidArgument: aws-chunked encoding is not supported when x-amz-content-sha256 UNSIGNED-PAYLOAD is supplied.
2025/08/02 14:13:28 DEBUG : 1508204081-OG-Medicinals_Triangle_Kush.jpg: Need to transfer - File not found at Destination
2025/08/02 14:13:28 ERROR : 1508204081-OG-Medicinals_Triangle_Kush.jpg: Failed to copy: operation error S3: PutObject, https response error StatusCode: 400, RequestID: 4PDXJ0T707FV089Q, HostID: VMn5ekhPEKV33H8omU62CaOrReIhao3GExXOaUFRPUKcC3APpJPybYfrtqf8cs6Y/Z0NL1zAGBM=, api error InvalidArgument: aws-chunked encoding is not supported when x-amz-content-sha256 UNSIGNED-PAYLOAD is supplied.
2025/08/02 14:13:28 ERROR : Attempt 2/3 failed with 1 errors and: operation error S3: PutObject, https response error StatusCode: 400, RequestID: 4PDXJ0T707FV089Q, HostID: VMn5ekhPEKV33H8omU62CaOrReIhao3GExXOaUFRPUKcC3APpJPybYfrtqf8cs6Y/Z0NL1zAGBM=, api error InvalidArgument: aws-chunked encoding is not supported when x-amz-content-sha256 UNSIGNED-PAYLOAD is supplied.
2025/08/02 14:13:29 DEBUG : 1508204081-OG-Medicinals_Triangle_Kush.jpg: Need to transfer - File not found at Destination
2025/08/02 14:13:29 ERROR : 1508204081-OG-Medicinals_Triangle_Kush.jpg: Failed to copy: operation error S3: PutObject, https response error StatusCode: 400, RequestID: B2HXPRWDW1NMXE5H, HostID: YjSRNssboU8PzAeJpJ3jMLSLl6OzP2t3voFPvTuqVVPMqMOM04fn4xb8rRm5FTAlL+w9qbPugIA=, api error InvalidArgument: aws-chunked encoding is not supported when x-amz-content-sha256 UNSIGNED-PAYLOAD is supplied.
2025/08/02 14:13:29 ERROR : Attempt 3/3 failed with 1 errors and: operation error S3: PutObject, https response error StatusCode: 400, RequestID: B2HXPRWDW1NMXE5H, HostID: YjSRNssboU8PzAeJpJ3jMLSLl6OzP2t3voFPvTuqVVPMqMOM04fn4xb8rRm5FTAlL+w9qbPugIA=, api error InvalidArgument: aws-chunked encoding is not supported when x-amz-content-sha256 UNSIGNED-PAYLOAD is supplied.
2025/08/02 14:13:29 INFO :
Transferred: 262.641 KiB / 262.641 KiB, 100%, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.7s
2025/08/02 14:13:29 DEBUG : 4 go routines active
2025/08/02 14:13:29 NOTICE: Failed to copyto: operation error S3: PutObject, https response error StatusCode: 400, RequestID: B2HXPRWDW1NMXE5H, HostID: YjSRNssboU8PzAeJpJ3jMLSLl6OzP2t3voFPvTuqVVPMqMOM04fn4xb8rRm5FTAlL+w9qbPugIA=, api error InvalidArgument: aws-chunked encoding is not supported when x-amz-content-sha256 UNSIGNED-PAYLOAD is supplied.
Additional information . I have just tried flag --s3-use-presigned-request and it allowed rclonu to perform a copy.
So now I have a question - what adding that flag actually means ? As I don't have an option to use it for "failing files" and not use for the rest ...
Here is result of the same command but with "--s3-use-presigned-request" flag
2025/08/02 14:33:41 DEBUG : rclone: Version "v1.70.3" starting with parameters ["rclone" "copyto" "-vv" "--metadata" "--s3-use-presigned-request" "do-weedvader:weedvader/static/1508204081-OG-Medicinals_Triangle_Kush.jpg" "s3-weedvader:weedvader-objects/static/test02.jpg"]
2025/08/02 14:33:41 DEBUG : Creating backend with remote "do-weedvader:weedvader/static/1508204081-OG-Medicinals_Triangle_Kush.jpg"
2025/08/02 14:33:41 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2025/08/02 14:33:41 DEBUG : do-weedvader: detected overridden config - adding "{7pn3y}" suffix to name
2025/08/02 14:33:41 DEBUG : fs cache: renaming child cache item "do-weedvader:weedvader/static/1508204081-OG-Medicinals_Triangle_Kush.jpg" to be canonical for parent "do-weedvader{7pn3y}:weedvader/static"
2025/08/02 14:33:41 DEBUG : Creating backend with remote "s3-weedvader:weedvader-objects/static/"
2025/08/02 14:33:41 DEBUG : s3-weedvader: detected overridden config - adding "{7pn3y}" suffix to name
2025/08/02 14:33:41 DEBUG : fs cache: renaming cache item "s3-weedvader:weedvader-objects/static/" to be canonical "s3-weedvader{7pn3y}:weedvader-objects/static"
2025/08/02 14:33:41 DEBUG : 1508204081-OG-Medicinals_Triangle_Kush.jpg: Need to transfer - File not found at Destination
2025/08/02 14:33:41 DEBUG : 1508204081-OG-Medicinals_Triangle_Kush.jpg: md5 = 1c52b19479d0ad99be8a5fe7734a5fb3 OK
2025/08/02 14:33:41 INFO : 1508204081-OG-Medicinals_Triangle_Kush.jpg: Copied (new) to: test02.jpg
2025/08/02 14:33:41 INFO :
Transferred: 87.547 KiB / 87.547 KiB, 100%, 0 B/s, ETA -
Transferred: 1 / 1, 100%
Elapsed time: 0.3s
2025/08/02 14:33:41 DEBUG : 6 go routines active