2023/12/28 15:51:55 DEBUG : Setting --config "rclone.conf" from environment variable RCLONE_CONFIG="rclone.conf"
2023/12/28 15:51:55 NOTICE: --dump-bodies is obsolete - please use --dump bodies instead
2023/12/28 15:51:55 DEBUG : rclone: Version "v1.65.0" starting with parameters ["rclone" "copy" "-vvv" "--dump-bodies" "--retries" "1" "index.txt" "cf:ofm-monaco"]
2023/12/28 15:51:55 DEBUG : Creating backend with remote "index.txt"
2023/12/28 15:51:55 DEBUG : Using config file from "scripts/tile_gen/rclone.conf"
2023/12/28 15:51:55 DEBUG : fs cache: adding new entry for parent of "index.txt", "scripts/tile_gen"
2023/12/28 15:51:55 DEBUG : Creating backend with remote "cf:ofm-monaco"
2023/12/28 15:51:55 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2023/12/28 15:51:55 DEBUG : Resolving service "s3" region "us-east-1"
2023/12/28 15:51:55 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2023/12/28 15:51:55 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/12/28 15:51:55 DEBUG : HTTP REQUEST (req 0x14000952300)
2023/12/28 15:51:55 DEBUG : HEAD /ofm-monaco/index.txt HTTP/1.1
Host: xxx.r2.cloudflarestorage.com
User-Agent: rclone/v1.65.0
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20231228T145155Z
2023/12/28 15:51:55 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/12/28 15:51:55 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/12/28 15:51:55 DEBUG : HTTP RESPONSE (req 0x14000952300)
2023/12/28 15:51:55 DEBUG : HTTP/1.1 404 Not Found
Connection: close
Cf-Ray: 83ca9fc67df4c2e1-VIE
Connection: keep-alive
Content-Type: text/plain;charset=UTF-8
Date: Thu, 28 Dec 2023 14:51:55 GMT
Server: cloudflare
2023/12/28 15:51:55 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/12/28 15:51:55 DEBUG : index.txt: Need to transfer - File not found at Destination
2023/12/28 15:51:55 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/12/28 15:51:55 DEBUG : HTTP REQUEST (req 0x14000b20500)
2023/12/28 15:51:55 DEBUG : PUT /ofm-monaco HTTP/1.1
Host: xxx.r2.cloudflarestorage.com
User-Agent: rclone/v1.65.0
Content-Length: 0
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20231228T145155Z
Accept-Encoding: gzip
2023/12/28 15:51:55 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/12/28 15:51:55 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/12/28 15:51:55 DEBUG : HTTP RESPONSE (req 0x14000b20500)
2023/12/28 15:51:55 DEBUG : HTTP/1.1 403 Forbidden
Transfer-Encoding: chunked
Cf-Ray: 83ca9fc73f2dc2e1-VIE
Connection: keep-alive
Content-Type: application/xml
Date: Thu, 28 Dec 2023 14:51:55 GMT
Server: cloudflare
Vary: Accept-Encoding
6e
<?xml version="1.0" encoding="UTF-8"?><Error><Code>AccessDenied</Code><Message>Access Denied</Message></Error>
0
2023/12/28 15:51:55 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/12/28 15:51:55 ERROR : index.txt: Failed to copy: failed to prepare upload: AccessDenied: Access Denied
status code: 403, request id: , host id:
2023/12/28 15:51:55 ERROR : Attempt 1/1 failed with 1 errors and: failed to prepare upload: AccessDenied: Access Denied
status code: 403, request id: , host id:
2023/12/28 15:51:55 INFO :
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s
2023/12/28 15:51:55 DEBUG : 9 go routines active
2023/12/28 15:51:55 Failed to copy: failed to prepare upload: AccessDenied: Access Denied
status code: 403, request id: , host id:
This is not the most clever rclone behaviour. Not sure what is the history of such approach - without --s3-no-check-bucket rclone always tries to create bucket regardless if it exist or not. And if permissions do not allow buckets' creation, rclone command fails even when bucket exist.
aws cli (sort of golden standard of S3 access tools) works in such situations without any issues - more logical I think. Such AccessDenied errors could be ignored by rclone maybe... as @ncw suggested in the thread mentioned by @asdffdsa
The strange thing is that it works perfectly for directory sync or copy. So even after many years of using rclone, this really puzzled me, as why would it work perfectly on directories but not on files?
The strange thing is that at this point: HEAD /ofm-monaco/index.txt it perfectly knows that I want to copy a file inside a bucket. Then later on, it tries to create the bucket...
I am sure there is old legacy story behind it. And legacy things usually last forever:) Should not be very difficult to change it if somebody feels like doing it.
Does not help that this error message is completely misleading in this case. Maybe it would be enough to replace it with something more clear.
Or maybe no_check_bucket should be true by default? It would be very easy to change.
well, not sure that is correct.
at that point, rclone has no idea if the object named /ofm-monaco/index.txt exists or not.
and then cloudlare returns 404 Not Found, so still, rclone does not know about /ofm-monaco/index.txt
in your case, the source is a file, but what about the dest, is the dest a file or a dir?
try rclone copyto, as in that case, rclone knows the dest is a file, not a dir.
