So there are lots of posts on here about problems with using rclone with Cloudflare r2 for very large files. However they all seem to treat it as a "solved" problem. I have absolutely not been able to resolve it despite days of trying.
I have been able to upload trivially small files to r2 with rclone, no problem. Which makes me think that it is NOT some kind of authentication error, but an issue with multipart file uploads.
What happens is that no matter what settings I try, it gives me the dreaded SignatureDoesNotMatch error on chunks of the file.
The file is about 4GB in size. I upped the retries to 1,000 and let it run all night, and it managed to "send" 80GB of data but only 105MB was actually delivered to the file. I have checked every post on here on this. I have set things up the way that Cloudflare recommends. No dice.
Here's my rclone version:
rclone version
rclone v1.64.0
- os/version: Microsoft Windows 10 Pro 20H2 (64 bit)
- os/kernel: 10.0.19042.1055 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.21.1
- go/linking: static
- go/tags: cmount
Here's my rclone.conf:
[r2]
type = s3
provider = Cloudflare
access_key_id = <redacted>
secret_access_key = <redacted>
region = auto
endpoint = https://<redacted>.r2.cloudflarestorage.com
acl = private
Here's an example of a recent attempt I made and the log file output. Following the advice I've seen again and again, I've tried to limit how much data is sent and the rate it is sent.
D:\maptiles>rclone copy nasa.pmtiles r2:osmtiles/ --transfers=1 --tpslimit=1 --s3-upload-concurrency=1 --s3-upload-cutoff=100M --s3-chunk-size=100M -v
2023/09/16 23:13:52 INFO : Starting transaction limiter: max 1 transactions/s with burst 1
2023/09/16 23:14:22 ERROR : nasa.pmtiles: Failed to copy: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 104857600 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023/09/16 23:14:22 ERROR : Attempt 1/3 failed with 1 errors and: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 104857600 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023/09/16 23:14:33 ERROR : nasa.pmtiles: Failed to copy: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 104857600 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023/09/16 23:14:33 ERROR : Attempt 2/3 failed with 1 errors and: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 104857600 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023/09/16 23:14:52 INFO :
Transferred: 95.781 MiB / 3.841 GiB, 2%, 1.834 MiB/s, ETA 34m52s
Transferred: 0 / 1, 0%
Elapsed time: 1m0.0s
Transferring:
* nasa.pmtiles: 0% /3.781Gi, 2.689Mi/s, 23m47s
2023/09/16 23:15:15 ERROR : nasa.pmtiles: Failed to copy: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 104857600 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023/09/16 23:15:15 ERROR : Attempt 3/3 failed with 1 errors and: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 104857600 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023/09/16 23:15:15 INFO :
Transferred: 153.125 MiB / 153.125 MiB, 100%, 2.247 MiB/s, ETA 0s
Errors: 1 (retrying may help)
Elapsed time: 1m23.5s
2023/09/16 23:15:15 Failed to copy: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 104857600 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
I've tried about every combination of settings I can imagine. No changing in the concurrency, tpslimit, transfers, etc. seems to matter. Always the same errors. (Retrying does not help.) I've tried it with huge chunks, tiny chunks, in the middle chunks. The thing is, this file is only the "smaller" of the two files I want to upload -- the other is +100GB.
Here's what I get with huge chunks (a vain attempt to keep it from spamming Cloudflare):
D:\maptiles>rclone copy nasa.pmtiles r2:osmtiles/ --s3-disable-checksum --s3-chunk-size=1G --s3-upload-cutoff=5G --s3-upload-concurrency=1 --progress -vv
2023/09/16 23:25:16 DEBUG : rclone: Version "v1.64.0" starting with parameters ["rclone" "copy" "nasa.pmtiles" "r2:osmtiles/" "--s3-disable-checksum" "--s3-chunk-size=1G" "--s3-upload-cutoff=5G" "--s3-upload-concurrency=1" "--progress" "-vv"]
2023/09/16 23:25:16 DEBUG : Creating backend with remote "nasa.pmtiles"
2023/09/16 23:25:16 DEBUG : Using config file from "C:\\Users\\User\\AppData\\Roaming\\rclone\\rclone.conf"
2023/09/16 23:25:16 DEBUG : fs cache: adding new entry for parent of "nasa.pmtiles", "//?/D:/maptiles"
2023/09/16 23:25:16 DEBUG : Creating backend with remote "r2:osmtiles/"
2023/09/16 23:25:16 DEBUG : r2: detected overridden config - adding "{F7-Ct}" suffix to name
2023/09/16 23:25:16 DEBUG : Resolving service "s3" region "auto"
2023/09/16 23:25:16 DEBUG : fs cache: renaming cache item "r2:osmtiles/" to be canonical "r2{F7-Ct}:osmtiles"
2023-09-16 23:25:16 DEBUG : nasa.pmtiles: Sizes differ (src 4060292083 vs dst 52428800)
2023-09-16 23:25:32 DEBUG : nasa.pmtiles: open chunk writer: started multipart upload: AFq0lLMFtNCxyeJzhNbWYlVBqsoFAQ1RiLXFj-z_maWc4x_UtCfwLVHgbY9vvetMxenOVv4VkymFIs7p3ypGhU58TpqGnp28A7wDfCUw0_mdGRiPPqADV1-7mSNlyG2e_YZy2ICd2QEDoejjVXnaBvHyPW2EKlYTDX85AbZ-64uCTpz19r7hZD8kJG6rnFiQxP6DIMMkyeYL2WlqOy9vJMvgDREvOI2ifvath0m-wpT0equJjJFtkQUIcrsNVMgrshJZ-wU7OgC5l1eMZJxLOXeWr0RyXGutHabmt0yc6CP1Dm6SMRpr9MXrYAxcXxcB0tF3IzHnSelLXw5VaHeNQZI
2023-09-16 23:25:32 DEBUG : nasa.pmtiles: multi-thread copy: using backend concurrency of 1 instead of --multi-thread-streams 4
2023-09-16 23:25:32 DEBUG : nasa.pmtiles: Starting multi-thread copy with 4 chunks of size 1Gi with 1 parallel streams
2023-09-16 23:25:32 DEBUG : nasa.pmtiles: multi-thread copy: chunk 1/4 (0-1073741824) size 1Gi starting
2023-09-16 23:25:37 DEBUG : nasa.pmtiles: multi-thread copy: chunk 1/4 failed: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023-09-16 23:25:37 DEBUG : nasa.pmtiles: multi-thread copy: chunk 2/4 (1073741824-2147483648) size 1Gi starting
2023-09-16 23:25:40 DEBUG : nasa.pmtiles: multi-thread copy: chunk 2/4 failed: multi-thread copy: failed to write chunk: failed to upload chunk 2 with 1073741824 bytes: RequestCanceled: request context canceled
caused by: context canceled
2023-09-16 23:25:40 DEBUG : nasa.pmtiles: multi-thread copy: cancelling transfer on exit
2023-09-16 23:25:40 DEBUG : nasa.pmtiles: multipart upload "AFq0lLMFtNCxyeJzhNbWYlVBqsoFAQ1RiLXFj-z_maWc4x_UtCfwLVHgbY9vvetMxenOVv4VkymFIs7p3ypGhU58TpqGnp28A7wDfCUw0_mdGRiPPqADV1-7mSNlyG2e_YZy2ICd2QEDoejjVXnaBvHyPW2EKlYTDX85AbZ-64uCTpz19r7hZD8kJG6rnFiQxP6DIMMkyeYL2WlqOy9vJMvgDREvOI2ifvath0m-wpT0equJjJFtkQUIcrsNVMgrshJZ-wU7OgC5l1eMZJxLOXeWr0RyXGutHabmt0yc6CP1Dm6SMRpr9MXrYAxcXxcB0tF3IzHnSelLXw5VaHeNQZI" aborted
2023-09-16 23:25:40 ERROR : nasa.pmtiles: Failed to copy: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023-09-16 23:25:40 ERROR : Attempt 1/3 failed with 1 errors and: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023-09-16 23:25:41 DEBUG : nasa.pmtiles: Sizes differ (src 4060292083 vs dst 52428800)
2023-09-16 23:25:46 DEBUG : nasa.pmtiles: open chunk writer: started multipart upload: ALIoUnDVNTuyolutqGGiqSvPm0VwBKVdAY6yaQ3QBCTgBTRgQsAjz15AgOxVFIBJc-k6eHZdy4UBlHo7UZFBd9FWs6I-eUYQDglw4x11d8BL443AfwqqULmiQGbxyuBT8Wr2uEZcueisP2WEYR3p5vsKP7bvwicAWRltP8jefEKqZ7gzSl95u297vcRfwKI3ViKa1BkWov8MEbRQH6ZMJzvsYbQyYeTWVe2JQE7gSNvjzlVTyGrb9DwtNBu3Do3jnQBvHA5UJu7VweygyKls0UzCPY7Ef6yFvVMj-namp7sgjy9uOTBfPl85RBJGEB1ghTkhGL8XlIVtmjyoO3om6sI
2023-09-16 23:25:46 DEBUG : nasa.pmtiles: multi-thread copy: using backend concurrency of 1 instead of --multi-thread-streams 4
2023-09-16 23:25:46 DEBUG : nasa.pmtiles: Starting multi-thread copy with 4 chunks of size 1Gi with 1 parallel streams
2023-09-16 23:25:46 DEBUG : nasa.pmtiles: multi-thread copy: chunk 1/4 (0-1073741824) size 1Gi starting
2023-09-16 23:26:19 DEBUG : nasa.pmtiles: multi-thread copy: chunk 1/4 failed: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023-09-16 23:26:19 DEBUG : nasa.pmtiles: multi-thread copy: chunk 2/4 (1073741824-2147483648) size 1Gi starting
2023-09-16 23:26:23 DEBUG : nasa.pmtiles: multi-thread copy: chunk 2/4 failed: multi-thread copy: failed to write chunk: failed to upload chunk 2 with 1073741824 bytes: RequestCanceled: request context canceled
caused by: context canceled
2023-09-16 23:26:23 DEBUG : nasa.pmtiles: multi-thread copy: cancelling transfer on exit
2023-09-16 23:26:23 DEBUG : nasa.pmtiles: multipart upload "ALIoUnDVNTuyolutqGGiqSvPm0VwBKVdAY6yaQ3QBCTgBTRgQsAjz15AgOxVFIBJc-k6eHZdy4UBlHo7UZFBd9FWs6I-eUYQDglw4x11d8BL443AfwqqULmiQGbxyuBT8Wr2uEZcueisP2WEYR3p5vsKP7bvwicAWRltP8jefEKqZ7gzSl95u297vcRfwKI3ViKa1BkWov8MEbRQH6ZMJzvsYbQyYeTWVe2JQE7gSNvjzlVTyGrb9DwtNBu3Do3jnQBvHA5UJu7VweygyKls0UzCPY7Ef6yFvVMj-namp7sgjy9uOTBfPl85RBJGEB1ghTkhGL8XlIVtmjyoO3om6sI" aborted
2023-09-16 23:26:23 ERROR : nasa.pmtiles: Failed to copy: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023-09-16 23:26:23 ERROR : Attempt 2/3 failed with 1 errors and: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023-09-16 23:26:23 DEBUG : nasa.pmtiles: Sizes differ (src 4060292083 vs dst 52428800)
2023-09-16 23:26:29 DEBUG : nasa.pmtiles: open chunk writer: started multipart upload: AJYdKGi5ZfdVpmtktRqXke34KURxUy8Rn7iacnfPQ4s5Z_MdpNeM8jzD9S4FheM0YLGrBYROqBQju5iyzkxSdSjvuH1xSBBK2FiQUsTtUSIAgN2-C2igcM9kJbhhSKtpq-9VXuj-qRK6S9MlLkcl2LCxBh9kfraeo4UnelEEo9GyrXjNS7UvM38vvkAZg_PHiTIRqWW2CJgNwJctQ9-0WbTX273P_br4iBllCYfSoFonDI5J6xawdztL-TYZoRGMAy_DKhkudeWFk7C43eFWW078HJ2iwUPF0-6x2tDU8_MumHEkkM5B8A5fyECvNwZGuGNcvTIODhTtsrify-j57s8
2023-09-16 23:26:29 DEBUG : nasa.pmtiles: multi-thread copy: using backend concurrency of 1 instead of --multi-thread-streams 4
2023-09-16 23:26:29 DEBUG : nasa.pmtiles: Starting multi-thread copy with 4 chunks of size 1Gi with 1 parallel streams
2023-09-16 23:26:29 DEBUG : nasa.pmtiles: multi-thread copy: chunk 1/4 (0-1073741824) size 1Gi starting
2023-09-16 23:26:56 DEBUG : pacer: low level retry 1/2 (error RequestError: send request failed
caused by: Put "https://084252fb5c3aefb95e32902d03530612.r2.cloudflarestorage.com/osmtiles/nasa.pmtiles?partNumber=1&uploadId=AJYdKGi5ZfdVpmtktRqXke34KURxUy8Rn7iacnfPQ4s5Z_MdpNeM8jzD9S4FheM0YLGrBYROqBQju5iyzkxSdSjvuH1xSBBK2FiQUsTtUSIAgN2-C2igcM9kJbhhSKtpq-9VXuj-qRK6S9MlLkcl2LCxBh9kfraeo4UnelEEo9GyrXjNS7UvM38vvkAZg_PHiTIRqWW2CJgNwJctQ9-0WbTX273P_br4iBllCYfSoFonDI5J6xawdztL-TYZoRGMAy_DKhkudeWFk7C43eFWW078HJ2iwUPF0-6x2tDU8_MumHEkkM5B8A5fyECvNwZGuGNcvTIODhTtsrify-j57s8": write tcp [2601:8c:4881:8100:dc22:26c1:6182:aae0]:58275->[2606:4700::6812:85a]:443: wsasend: An existing connection was forcibly closed by the remote host.)
2023-09-16 23:26:56 DEBUG : pacer: Rate limited, increasing sleep to 10ms
2023-09-16 23:27:13 DEBUG : pacer: Reducing sleep to 0s
2023-09-16 23:27:13 DEBUG : nasa.pmtiles: multi-thread copy: chunk 1/4 failed: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023-09-16 23:27:13 DEBUG : nasa.pmtiles: multi-thread copy: chunk 2/4 (1073741824-2147483648) size 1Gi starting
2023-09-16 23:27:16 DEBUG : nasa.pmtiles: multi-thread copy: chunk 2/4 failed: multi-thread copy: failed to write chunk: failed to upload chunk 2 with 1073741824 bytes: RequestCanceled: request context canceled
caused by: context canceled
2023-09-16 23:27:16 DEBUG : nasa.pmtiles: multi-thread copy: cancelling transfer on exit
2023-09-16 23:27:16 DEBUG : nasa.pmtiles: multipart upload "AJYdKGi5ZfdVpmtktRqXke34KURxUy8Rn7iacnfPQ4s5Z_MdpNeM8jzD9S4FheM0YLGrBYROqBQju5iyzkxSdSjvuH1xSBBK2FiQUsTtUSIAgN2-C2igcM9kJbhhSKtpq-9VXuj-qRK6S9MlLkcl2LCxBh9kfraeo4UnelEEo9GyrXjNS7UvM38vvkAZg_PHiTIRqWW2CJgNwJctQ9-0WbTX273P_br4iBllCYfSoFonDI5J6xawdztL-TYZoRGMAy_DKhkudeWFk7C43eFWW078HJ2iwUPF0-6x2tDU8_MumHEkkM5B8A5fyECvNwZGuGNcvTIODhTtsrify-j57s8" aborted
2023-09-16 23:27:16 ERROR : nasa.pmtiles: Failed to copy: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023-09-16 23:27:16 ERROR : Attempt 3/3 failed with 1 errors and: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
Transferred: 1.181 GiB / 1.181 GiB, 100%, 1.730 MiB/s, ETA 0s
Errors: 1 (retrying may help)
Elapsed time: 2m0.1s
2023/09/16 23:27:16 INFO :
Transferred: 1.181 GiB / 1.181 GiB, 100%, 1.730 MiB/s, ETA 0s
Errors: 1 (retrying may help)
Elapsed time: 2m0.1s
2023/09/16 23:27:16 DEBUG : 6 go routines active
2023/09/16 23:27:16 Failed to copy: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
What is the issue? It is ridiculous that Cloudflare has no alternative to uploading large files than r2 (other than uploading the whole thing to AWS S3 first and then slurping it, at some cost), and it clearly doesn't work that well.
Is there any update on this issue? I am at my wit's end here. The consensus seems to be that this is the (confusing, unhelpful) way that r2 signals that you are pinging them too quickly or something. I see no way to actually reduce things so that it works. These files are not immeasurably large; I would gladly wait days for them to upload if that would make it happen. As it is, I cannot upload them at all.
Here's one where I tried to increase the delay between API calls dramatically. Note that it fails WHILE UPLOADING THE CHUNK -- it aborts it. Something is seriously wrong here.
D:\maptiles>rclone copy nasa.pmtiles r2:osmtiles/ --s3-disable-checksum --s3-chunk-size=1G --s3-upload-cutoff=5G --s3-upload-concurrency=1 --drive-pacer-burst=1 --drive-pacer-min-sleep=300ms --progress -vv
2023/09/16 23:31:11 DEBUG : rclone: Version "v1.64.0" starting with parameters ["rclone" "copy" "nasa.pmtiles" "r2:osmtiles/" "--s3-disable-checksum" "--s3-chunk-size=1G" "--s3-upload-cutoff=5G" "--s3-upload-concurrency=1" "--drive-pacer-burst=1" "--drive-pacer-min-sleep=300ms" "--progress" "-vv"]
2023/09/16 23:31:11 DEBUG : Creating backend with remote "nasa.pmtiles"
2023/09/16 23:31:11 DEBUG : Using config file from "C:\\Users\\User\\AppData\\Roaming\\rclone\\rclone.conf"
2023/09/16 23:31:11 DEBUG : fs cache: adding new entry for parent of "nasa.pmtiles", "//?/D:/maptiles"
2023/09/16 23:31:11 DEBUG : Creating backend with remote "r2:osmtiles/"
2023/09/16 23:31:11 DEBUG : r2: detected overridden config - adding "{F7-Ct}" suffix to name
2023/09/16 23:31:11 DEBUG : Resolving service "s3" region "auto"
2023/09/16 23:31:11 DEBUG : fs cache: renaming cache item "r2:osmtiles/" to be canonical "r2{F7-Ct}:osmtiles"
2023-09-16 23:31:11 DEBUG : nasa.pmtiles: Sizes differ (src 4060292083 vs dst 52428800)
2023-09-16 23:31:17 DEBUG : nasa.pmtiles: open chunk writer: started multipart upload: AMmyxgJRPX4Uou7OvPyWP0Z-rn0fdb6621vffz7HMfoSC7OMjLe6eGeiNMn6lUTSWt_Yq3aw9KgQPO6IkDR8_aMOxc7wok_fouthGhzRdbNaNxR4EKlaAuDDU06hzmAb3devERXPPrnmvkLCtS2pugBtok8SLnQ36fw5-oDho9yF4FtXuB4iROlZS0KCVHcqGtbaKXAEaIBUwhrKJLVRb62zc1ABWVd0m-C65F20-sJtTnt_4H3qPBnXmvVQ3o5jj9RT7RtAGLmzAb3raGGl3jK5KEyu_ZVrivr3Td4akUTGs9kJ0oSGMDg-GQn7yxLLp1ZUTUILVmn0nUim1S0vtMA
2023-09-16 23:31:17 DEBUG : nasa.pmtiles: multi-thread copy: using backend concurrency of 1 instead of --multi-thread-streams 4
2023-09-16 23:31:17 DEBUG : nasa.pmtiles: Starting multi-thread copy with 4 chunks of size 1Gi with 1 parallel streams
2023-09-16 23:31:17 DEBUG : nasa.pmtiles: multi-thread copy: chunk 1/4 (0-1073741824) size 1Gi starting
2023-09-16 23:32:36 DEBUG : nasa.pmtiles: multi-thread copy: chunk 1/4 failed: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023-09-16 23:32:36 DEBUG : nasa.pmtiles: multi-thread copy: chunk 2/4 (1073741824-2147483648) size 1Gi starting
2023-09-16 23:32:39 DEBUG : nasa.pmtiles: multi-thread copy: chunk 2/4 failed: multi-thread copy: failed to write chunk: failed to upload chunk 2 with 1073741824 bytes: RequestCanceled: request context canceled
caused by: context canceled
2023-09-16 23:32:39 DEBUG : nasa.pmtiles: multi-thread copy: cancelling transfer on exit
2023-09-16 23:32:39 DEBUG : nasa.pmtiles: multipart upload "AMmyxgJRPX4Uou7OvPyWP0Z-rn0fdb6621vffz7HMfoSC7OMjLe6eGeiNMn6lUTSWt_Yq3aw9KgQPO6IkDR8_aMOxc7wok_fouthGhzRdbNaNxR4EKlaAuDDU06hzmAb3devERXPPrnmvkLCtS2pugBtok8SLnQ36fw5-oDho9yF4FtXuB4iROlZS0KCVHcqGtbaKXAEaIBUwhrKJLVRb62zc1ABWVd0m-C65F20-sJtTnt_4H3qPBnXmvVQ3o5jj9RT7RtAGLmzAb3raGGl3jK5KEyu_ZVrivr3Td4akUTGs9kJ0oSGMDg-GQn7yxLLp1ZUTUILVmn0nUim1S0vtMA" aborted
2023-09-16 23:32:39 ERROR : nasa.pmtiles: Failed to copy: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023-09-16 23:32:39 ERROR : Attempt 1/3 failed with 1 errors and: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023-09-16 23:32:39 DEBUG : nasa.pmtiles: Sizes differ (src 4060292083 vs dst 52428800)
2023-09-16 23:32:45 DEBUG : nasa.pmtiles: open chunk writer: started multipart upload: ANCTP6erKY5_PW1NoXUp1LjfG6c3M39qUAh4Z_dWW1QC_PDex_819Z3iMET-sZ_RGaMCR01NvoR-Fxmpw_3XFGGTEm6EHAAhKyEqdll1F2dfdLl1rdbEyzpXyAPFH7vhI8FPZvtxfX83lPMBEvgJRIySpPNFQZQj1tPjLI-102EHQT3qU071_TagH4oMk8K2RW-UWsnC62CVcrS0L25doVtdAIJ46nxAv51Yeysmg0Mb7JrPF3SBc5tYCMbDTlAoduFO6dcEVOuUlV1pFe6oVjUG7kqtfQ1f5cvCymPClSWKV4QO3fHNasgdPh3iyA0kKIdesx9pIAuMj_TIODVH2t0
2023-09-16 23:32:45 DEBUG : nasa.pmtiles: multi-thread copy: using backend concurrency of 1 instead of --multi-thread-streams 4
2023-09-16 23:32:45 DEBUG : nasa.pmtiles: Starting multi-thread copy with 4 chunks of size 1Gi with 1 parallel streams
2023-09-16 23:32:45 DEBUG : nasa.pmtiles: multi-thread copy: chunk 1/4 (0-1073741824) size 1Gi starting
2023-09-16 23:33:37 DEBUG : nasa.pmtiles: multi-thread copy: chunk 1/4 failed: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023-09-16 23:33:37 DEBUG : nasa.pmtiles: multi-thread copy: chunk 2/4 (1073741824-2147483648) size 1Gi starting
2023-09-16 23:33:40 DEBUG : nasa.pmtiles: multi-thread copy: chunk 2/4 failed: multi-thread copy: failed to write chunk: failed to upload chunk 2 with 1073741824 bytes: RequestCanceled: request context canceled
caused by: context canceled
2023-09-16 23:33:40 DEBUG : nasa.pmtiles: multi-thread copy: cancelling transfer on exit
2023-09-16 23:33:40 DEBUG : nasa.pmtiles: multipart upload "ANCTP6erKY5_PW1NoXUp1LjfG6c3M39qUAh4Z_dWW1QC_PDex_819Z3iMET-sZ_RGaMCR01NvoR-Fxmpw_3XFGGTEm6EHAAhKyEqdll1F2dfdLl1rdbEyzpXyAPFH7vhI8FPZvtxfX83lPMBEvgJRIySpPNFQZQj1tPjLI-102EHQT3qU071_TagH4oMk8K2RW-UWsnC62CVcrS0L25doVtdAIJ46nxAv51Yeysmg0Mb7JrPF3SBc5tYCMbDTlAoduFO6dcEVOuUlV1pFe6oVjUG7kqtfQ1f5cvCymPClSWKV4QO3fHNasgdPh3iyA0kKIdesx9pIAuMj_TIODVH2t0" aborted
2023-09-16 23:33:40 ERROR : nasa.pmtiles: Failed to copy: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023-09-16 23:33:40 ERROR : Attempt 2/3 failed with 1 errors and: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023-09-16 23:33:40 DEBUG : nasa.pmtiles: Sizes differ (src 4060292083 vs dst 52428800)
2023-09-16 23:33:46 DEBUG : nasa.pmtiles: open chunk writer: started multipart upload: AJdV13lW3LYY7cPAXh40o2roqa0K6ZHZ7FLLi6MwKf-koM1jBdO1vtUOyn9OnKZc7BVTOUyt610_f2VV5xGxRQFU8oS3G6SoshyiyngIfXoTx-boZOIdCrF7ZIs0cQ9qZpoqEdYFp71GqnuLUtELQni0aC50BXVBm5U9P6MXtHKIqErBH2bx4qfTE3FGTO0eXjydRUs6J48JILDBFJTR_Q8nFmSHd19Q88Gm5kRgCq7YYWEIkBfSplJGJlaT440kRFXztRu0LqTHfySBbycq5e3yoVQh6FvkB3mAGA5eSpshTMxTTFj9jgw0CHW46rQ30mEoyvRrjEls6aDBQTG1bk4
2023-09-16 23:33:46 DEBUG : nasa.pmtiles: multi-thread copy: using backend concurrency of 1 instead of --multi-thread-streams 4
2023-09-16 23:33:46 DEBUG : nasa.pmtiles: Starting multi-thread copy with 4 chunks of size 1Gi with 1 parallel streams
2023-09-16 23:33:46 DEBUG : nasa.pmtiles: multi-thread copy: chunk 1/4 (0-1073741824) size 1Gi starting
2023-09-16 23:33:53 DEBUG : nasa.pmtiles: multi-thread copy: chunk 1/4 failed: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023-09-16 23:33:53 DEBUG : nasa.pmtiles: multi-thread copy: chunk 2/4 (1073741824-2147483648) size 1Gi starting
2023-09-16 23:33:56 DEBUG : nasa.pmtiles: multi-thread copy: chunk 2/4 failed: multi-thread copy: failed to write chunk: failed to upload chunk 2 with 1073741824 bytes: RequestCanceled: request context canceled
caused by: context canceled
2023-09-16 23:33:56 DEBUG : nasa.pmtiles: multi-thread copy: cancelling transfer on exit
2023-09-16 23:33:56 DEBUG : nasa.pmtiles: multipart upload "AJdV13lW3LYY7cPAXh40o2roqa0K6ZHZ7FLLi6MwKf-koM1jBdO1vtUOyn9OnKZc7BVTOUyt610_f2VV5xGxRQFU8oS3G6SoshyiyngIfXoTx-boZOIdCrF7ZIs0cQ9qZpoqEdYFp71GqnuLUtELQni0aC50BXVBm5U9P6MXtHKIqErBH2bx4qfTE3FGTO0eXjydRUs6J48JILDBFJTR_Q8nFmSHd19Q88Gm5kRgCq7YYWEIkBfSplJGJlaT440kRFXztRu0LqTHfySBbycq5e3yoVQh6FvkB3mAGA5eSpshTMxTTFj9jgw0CHW46rQ30mEoyvRrjEls6aDBQTG1bk4" aborted
2023-09-16 23:33:56 ERROR : nasa.pmtiles: Failed to copy: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
2023-09-16 23:33:56 ERROR : Attempt 3/3 failed with 1 errors and: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
Transferred: 342.344 MiB / 342.344 MiB, 100%, 2.167 MiB/s, ETA 0s
Errors: 1 (retrying may help)
Elapsed time: 2m45.4s
2023/09/16 23:33:56 INFO :
Transferred: 342.344 MiB / 342.344 MiB, 100%, 2.167 MiB/s, ETA 0s
Errors: 1 (retrying may help)
Elapsed time: 2m45.4s
2023/09/16 23:33:56 DEBUG : 6 go routines active
2023/09/16 23:33:56 Failed to copy: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 1073741824 bytes: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
status code: 403, request id: , host id:
Any advice you can give is appreciated. This is driving me nuts...! Thank you.