What is the problem you are having with rclone?
Trying to copy to an AWS S3 bucket with object lock enabled (don't know if it's relevant), using --no-traverse option sometime I get 403 errors.
Trying to debug with -vv --dump headers it seems that, after the HEAD call for a file to copy, the program tries a "PUT /", that ends in a 403 error.
Without the --no-traverse option all works ok (in the debug output I see the PUT with the correct key).
Run the command 'rclone version' and share the full output of the command.
rclone v1.58.1
- os/version: centos 7.6.1810 (64 bit)
- os/kernel: 3.10.0-957.21.3.el7.x86_64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.17.9
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
AWS S3
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone --no-traverse --max-age 10d --bwlimit 30M --transfers 10 --update -vv --dump headers copy /XXXXXXXX/YYYY/ZZZZ/WWWW/elasticsearch.backup/ bucket-2:bucket-worm-3/NAS_CO/YYYY/ZZZZ/WWWW/elasticsearch.backup/
The rclone config contents with secrets removed.
[bucket-2]
type = s3
provider = AWS
access_key_id =
secret_access_key =
region = eu-south-1
location_constraint = eu-south-1
storage_class = GLACIER_IR
A log from the command with the -vv flag
# /root/rclone-v1.58.1-linux-amd64/rclone --no-traverse --max-age 10d --bwlimit 30M --transfers 10 --update -vv --dump headers copy /XXXXXXXX/YYYY/ZZZZ/WWWW/elasticsearch.backup/ bucket-2:bucket-worm-3/NAS_CO/YYYY/ZZZZ/WWWW/elasticsearch.backup/
2022/07/04 21:36:57 INFO : Starting bandwidth limiter at 30Mi Byte/s
2022/07/04 21:36:57 DEBUG : --max-age 1.4285714285714286w to 2022-06-24 21:36:57.43788841 +0200 CEST m=-863999.931602630
2022/07/04 21:36:57 DEBUG : rclone: Version "v1.58.1" starting with parameters ["/root/rclone-v1.58.1-linux-amd64/rclone" "--no-traverse" "--max-age" "10d" "--bwlimit" "30M" "--transfers" "10" "--update" "-vv" "--dump" "headers" "copy" "/XXXXXXXX/YYYY/ZZZZ/WWWW/elasticsearch.backup/" "bucket-2:bucket-worm-3/NAS_CO/YYYY/ZZZZ/WWWW/elasticsearch.backup/"]
2022/07/04 21:36:57 DEBUG : Creating backend with remote "/XXXXXXXX/YYYY/ZZZZ/WWWW/elasticsearch.backup/"
2022/07/04 21:36:57 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2022/07/04 21:36:57 DEBUG : Creating backend with remote "bucket-2:bucket-worm-3/NAS_CO/YYYY/ZZZZ/WWWW/elasticsearch.backup/"
2022/07/04 21:36:57 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2022/07/04 21:36:57 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2022/07/04 21:36:57 DEBUG : fs cache: renaming cache item "bucket-2:bucket-worm-3/NAS_CO/YYYY/ZZZZ/WWWW/elasticsearch.backup/" to be canonical "bucket-2:bucket-worm-3/NAS_CO/YYYY/ZZZZ/WWWW/elasticsearch.backup"
2022/07/04 21:36:57 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/07/04 21:36:57 DEBUG : HTTP REQUEST (req 0xc000c88200)
2022/07/04 21:36:57 DEBUG : HEAD /NAS_CO/YYYY/ZZZZ/WWWW/elasticsearch.backup/elastic_2022-07-03.tgz HTTP/1.1
Host: bucket-worm-3.s3.eu-south-1.amazonaws.com
User-Agent: rclone/v1.58.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20220704T193657Z
2022/07/04 21:36:57 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/07/04 21:36:57 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/07/04 21:36:57 DEBUG : HTTP REQUEST (req 0xc000c88000)
2022/07/04 21:36:57 DEBUG : HEAD /NAS_CO/YYYY/ZZZZ/WWWW/elasticsearch.backup/elastic_2022-07-04.tgz HTTP/1.1
Host: bucket-worm-3.s3.eu-south-1.amazonaws.com
User-Agent: rclone/v1.58.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20220704T193657Z
2022/07/04 21:36:57 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/07/04 21:36:57 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/07/04 21:36:57 DEBUG : HTTP RESPONSE (req 0xc000c88000)
2022/07/04 21:36:57 DEBUG : HTTP/1.1 404 Not Found
Connection: close
Content-Type: application/xml
Date: Mon, 04 Jul 2022 19:36:57 GMT
Server: AmazonS3
X-Amz-Id-2: 18YdqO6azWinoUSAPthh8KUG7QSobYHeMNiOxR5L0Lj+xzWx9+VfNVD16lrebgWaUZRcoJLqCRw=
X-Amz-Request-Id: J9MEE196SS1B0YXY
2022/07/04 21:36:57 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/07/04 21:36:57 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/07/04 21:36:57 DEBUG : HTTP RESPONSE (req 0xc000c88200)
2022/07/04 21:36:57 DEBUG : HTTP/1.1 404 Not Found
Connection: close
Content-Type: application/xml
Date: Mon, 04 Jul 2022 19:36:56 GMT
Server: AmazonS3
X-Amz-Id-2: k05ULHMFggEGOLlLg3MHU52YZHBJqgakZdJpyXUmzH11EZhHZ0dMrrKQteiXnDleiwWxiGwmLRQ=
X-Amz-Request-Id: J9M72PQNVXK1JKTA
2022/07/04 21:36:57 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/07/04 21:36:57 DEBUG : S3 bucket bucket-worm-3 path NAS_CO/YYYY/ZZZZ/WWWW/elasticsearch.backup: Waiting for checks to finish
2022/07/04 21:36:57 DEBUG : S3 bucket bucket-worm-3 path NAS_CO/YYYY/ZZZZ/WWWW/elasticsearch.backup: Waiting for transfers to finish
2022/07/04 21:36:57 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/07/04 21:36:57 DEBUG : HTTP REQUEST (req 0xc00023cb00)
2022/07/04 21:36:57 DEBUG : PUT / HTTP/1.1
Host: bucket-worm-3.s3.eu-south-1.amazonaws.com
User-Agent: rclone/v1.58.1
Content-Length: 154
Authorization: XXXX
X-Amz-Acl: private
X-Amz-Content-Sha256: e9370cd051ec4b975c093ccf78845337b62b8341e005eedb73c5d9fa198fd14c
X-Amz-Date: 20220704T193657Z
Accept-Encoding: gzip
2022/07/04 21:36:57 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/07/04 21:36:57 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/07/04 21:36:57 DEBUG : HTTP RESPONSE (req 0xc00023cb00)
2022/07/04 21:36:57 DEBUG : HTTP/1.1 403 Forbidden
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Mon, 04 Jul 2022 19:36:56 GMT
Server: AmazonS3
X-Amz-Id-2: EqpEs3Tk4xUibUH0gRZwo8ZdGKLU71bhGVNfxrI+F5pq8bycAK+kQp7apub3p6hTS92kluBGwV8=
X-Amz-Request-Id: J9M1G68CY0PN67D2
2022/07/04 21:36:57 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<