In some cases, rclone does not use ETag to verify files

[asdffdsa (jojothehumanmonkey)]

@ncw

Adding this check when using --s3-no-head is very nice though and a tiny bit more certainty that things are correct

i am trying to achieve --bullet-proof
This would then enable a bullet proof end to end upload check for multipart objects

the beta worked!

2023/02/13 09:13:37 DEBUG : file.ext: Multipart upload Etag: 195c549125ec2fe75e9f3e8444b4a98c-2 OK
2023/02/13 09:13:37 DEBUG : file.ext: md5 = 7246aff55b4ba5cdd90912d8819d5a0f OK
2023/02/13 09:13:37 INFO  : file.ext: Copied (new)

and here is the full log

rclone copy ./source azork: --s3-no-head --s3-upload-cutoff=0 --s3-chunk-size=19M -vv --retries=1 --files-from=missing-on-dst.txt --dump=headers,responses
2023/02/13 09:13:36 DEBUG : rclone: Version "v1.62.0-beta.6731.63bf57580.s3-multipart-etag" starting with parameters ["./rclone" "copy" "./source" "azork:" "--s3-no-head" "--s3-upload-cutoff=0" "--s3-chunk-size=19M" "-vv" "--retries=1" "--files-from=missing-on-dst.txt" "--dump=headers,responses"]
2023/02/13 09:13:36 DEBUG : Creating backend with remote "./source"
2023/02/13 09:13:36 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/02/13 09:13:36 DEBUG : fs cache: renaming cache item "./source" to be canonical "/root/rclone/scripts/minimal/source"
2023/02/13 09:13:36 DEBUG : Creating backend with remote "azork:"
2023/02/13 09:13:36 DEBUG : Creating backend with remote "wasabi_azork_remote:azork"
2023/02/13 09:13:36 DEBUG : wasabi_azork_remote: detected overridden config - adding "{D0imL}" suffix to name
2023/02/13 09:13:36 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2023/02/13 09:13:36 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2023/02/13 09:13:36 DEBUG : fs cache: renaming cache item "wasabi_azork_remote:azork" to be canonical "wasabi_azork_remote{D0imL}:azork"
2023/02/13 09:13:36 DEBUG : fs cache: renaming cache item "azork:" to be canonical "wasabi_azork_remote{D0imL}:azork"
2023/02/13 09:13:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/13 09:13:36 DEBUG : HTTP REQUEST (req 0xc0009b8000)
2023/02/13 09:13:36 DEBUG : GET /?delimiter=%2F&encoding-type=url&list-type=2&max-keys=1000&prefix= HTTP/1.1
Host: azork.s3.us-east-2.wasabisys.com
User-Agent: rclone/v1.62.0-beta.6731.63bf57580.s3-multipart-etag
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20230213T141336Z
Accept-Encoding: gzip

2023/02/13 09:13:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/13 09:13:36 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/02/13 09:13:36 DEBUG : HTTP RESPONSE (req 0xc0009b8000)
2023/02/13 09:13:36 DEBUG : HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Mon, 13 Feb 2023 14:13:36 GMT
Server: WasabiS3/7.10.1198-2022-12-14-39a7a2e69e (XB27-U41)
X-Amz-Bucket-Region: us-east-2
X-Amz-Id-2: 1s95hhGXzJKysk/ZnyJid1zDIZInZOgnFTYYhgrkx/4j7AqY21il6GUxqkQW9FbEBm5SDMCuGFvu
X-Amz-Request-Id: 545E413EDA79C104:A

124
<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>azork</Name><Prefix></Prefix><KeyCount>2</KeyCount><MaxKeys>1000</MaxKeys><Delimiter>/</Delimiter><EncodingType>url</EncodingType><IsTruncated>false</IsTruncated></ListBucketResult>
0

2023/02/13 09:13:36 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/02/13 09:13:36 DEBUG : S3 bucket azork: Waiting for checks to finish
2023/02/13 09:13:36 DEBUG : S3 bucket azork: Waiting for transfers to finish
2023/02/13 09:13:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/13 09:13:36 DEBUG : HTTP REQUEST (req 0xc0009b8300)
2023/02/13 09:13:36 DEBUG : POST /file.ext?uploads= HTTP/1.1
Host: azork.s3.us-east-2.wasabisys.com
User-Agent: rclone/v1.62.0-beta.6731.63bf57580.s3-multipart-etag
Content-Length: 0
Authorization: XXXX
Content-Type: application/octet-stream
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20230213T141336Z
X-Amz-Meta-Md5chksum: ckav9VtLpc3ZCRLYgZ1aDw==
X-Amz-Meta-Mtime: 1676297614.651237148
Accept-Encoding: gzip

2023/02/13 09:13:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/13 09:13:36 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/02/13 09:13:36 DEBUG : HTTP RESPONSE (req 0xc0009b8300)
2023/02/13 09:13:36 DEBUG : HTTP/1.1 200 OK
Transfer-Encoding: chunked
Date: Mon, 13 Feb 2023 14:13:36 GMT
Server: WasabiS3/7.10.1198-2022-12-14-39a7a2e69e (XB27-U41)
X-Amz-Id-2: xM7ZDigOK5WERgMqaQrD6yeX8EagoLQMSWMXQ8S5FFH3rCD0e1BjpPEoUXkde5gVk/DZKVFqmh7q
X-Amz-Request-Id: ACF3AAA660D96C98:A

154
<?xml version="1.0" encoding="UTF-8"?>
<InitiateMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Bucket>azork</Bucket><Key>file.ext</Key><UploadId>yUEm_XShUnHbEcPd5vrTRv9qQSSUIiWwE7omVHMoNWUT6by6jTVztGX8n_ByHXfrGO_FkM4Dszm-UHOR5hH7KlELJ_MRsHDfKgbtb3CBpk31aDbIl3tZqjhQS6bQsQYR</UploadId></InitiateMultipartUploadResult>
0

2023/02/13 09:13:36 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/02/13 09:13:36 DEBUG : file.ext: multipart upload starting chunk 1 size 19Mi offset 0/20Mi
2023/02/13 09:13:36 DEBUG : file.ext: multipart upload starting chunk 2 size 1Mi offset 19Mi/20Mi
2023/02/13 09:13:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/13 09:13:36 DEBUG : HTTP REQUEST (req 0xc000256800)
2023/02/13 09:13:36 DEBUG : PUT /file.ext?partNumber=2&uploadId=yUEm_XShUnHbEcPd5vrTRv9qQSSUIiWwE7omVHMoNWUT6by6jTVztGX8n_ByHXfrGO_FkM4Dszm-UHOR5hH7KlELJ_MRsHDfKgbtb3CBpk31aDbIl3tZqjhQS6bQsQYR HTTP/1.1
Host: azork.s3.us-east-2.wasabisys.com
User-Agent: rclone/v1.62.0-beta.6731.63bf57580.s3-multipart-etag
Content-Length: 1048576
Authorization: XXXX
Content-Md5: aUHYpoSioPsWAqy4QOT46Q==
X-Amz-Content-Sha256: 66de6f9b5138899a8c8decc8f1cf1cbb99fa8e2c52b2a0e6b45eeeae2dffbf1e
X-Amz-Date: 20230213T141336Z
Accept-Encoding: gzip

2023/02/13 09:13:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/13 09:13:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/13 09:13:36 DEBUG : HTTP REQUEST (req 0xc0009b8200)
2023/02/13 09:13:36 DEBUG : PUT /file.ext?partNumber=1&uploadId=yUEm_XShUnHbEcPd5vrTRv9qQSSUIiWwE7omVHMoNWUT6by6jTVztGX8n_ByHXfrGO_FkM4Dszm-UHOR5hH7KlELJ_MRsHDfKgbtb3CBpk31aDbIl3tZqjhQS6bQsQYR HTTP/1.1
Host: azork.s3.us-east-2.wasabisys.com
User-Agent: rclone/v1.62.0-beta.6731.63bf57580.s3-multipart-etag
Content-Length: 19922944
Authorization: XXXX
Content-Md5: n/jrKfsFqxQ2DIIWZVnx+g==
Expect: 100-Continue
X-Amz-Content-Sha256: 0f033eb7f8431b96208e604338ed327d2c5f398908b6d21cd3e6fb6e4009c363
X-Amz-Date: 20230213T141336Z
Accept-Encoding: gzip

2023/02/13 09:13:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/13 09:13:36 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/02/13 09:13:36 DEBUG : HTTP RESPONSE (req 0xc000256800)
2023/02/13 09:13:36 DEBUG : HTTP/1.1 200 OK
Content-Length: 0
Date: Mon, 13 Feb 2023 14:13:37 GMT
Etag: "6941d8a684a2a0fb1602acb840e4f8e9"
Server: WasabiS3/7.10.1198-2022-12-14-39a7a2e69e (XB27-U41)
X-Amz-Id-2: jhIknZzO17wUJW/travDJpYn+PP735Fjl41pYk2eST4/dzZZ/ioo9/l9/c/pF//srSQBpdR4JzCW
X-Amz-Request-Id: 8879FD174587847E:A

2023/02/13 09:13:36 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/02/13 09:13:37 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/02/13 09:13:37 DEBUG : HTTP RESPONSE (req 0xc0009b8200)
2023/02/13 09:13:37 DEBUG : HTTP/1.1 200 OK
Content-Length: 0
Date: Mon, 13 Feb 2023 14:13:38 GMT
Etag: "9ff8eb29fb05ab14360c82166559f1fa"
Server: WasabiS3/7.10.1198-2022-12-14-39a7a2e69e (B33-U25)
X-Amz-Id-2: 7dZQue+s3nxLMbSvatJCzWxytAUh3cv/NQc6NRwJIKz7zaD7zhlblz5GQ4ykatIFK+HrlCcoIVih
X-Amz-Request-Id: 7716F4824964AC56:A

2023/02/13 09:13:37 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/02/13 09:13:37 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/13 09:13:37 DEBUG : HTTP REQUEST (req 0xc000257500)
2023/02/13 09:13:37 DEBUG : POST /file.ext?uploadId=yUEm_XShUnHbEcPd5vrTRv9qQSSUIiWwE7omVHMoNWUT6by6jTVztGX8n_ByHXfrGO_FkM4Dszm-UHOR5hH7KlELJ_MRsHDfKgbtb3CBpk31aDbIl3tZqjhQS6bQsQYR HTTP/1.1
Host: azork.s3.us-east-2.wasabisys.com
User-Agent: rclone/v1.62.0-beta.6731.63bf57580.s3-multipart-etag
Content-Length: 287
Authorization: XXXX
X-Amz-Content-Sha256: 73cba0622eb3317477cccd8807a7cbec7330475d93b0a18e9f19b05a9d74ee02
X-Amz-Date: 20230213T141337Z
Accept-Encoding: gzip

2023/02/13 09:13:37 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/13 09:13:37 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/02/13 09:13:37 DEBUG : HTTP RESPONSE (req 0xc000257500)
2023/02/13 09:13:37 DEBUG : HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Mon, 13 Feb 2023 14:13:38 GMT
Server: WasabiS3/7.10.1198-2022-12-14-39a7a2e69e (B33-U25)
X-Amz-Id-2: wJbOm4VFbTxgfmci1Uny65eYGimQihIkFBGegvHpUvpCe9tOtgv2ZVDEbxrms9O9gGLZaFS+CjM2
X-Amz-Request-Id: 2B644CD20340587B:A
X-Amz-Version-Id: 001676297616821718304-SHNDnyTeTy

140
<?xml version="1.0" encoding="UTF-8"?>
<CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Location>https://azork.s3.us-east-2.wasabisys.com/file.ext</Location><Bucket>azork</Bucket><Key>file.ext</Key><ETag>&quot;195c549125ec2fe75e9f3e8444b4a98c-2&quot;</ETag></CompleteMultipartUploadResult>
0

2023/02/13 09:13:37 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/02/13 09:13:37 DEBUG : file.ext: Multipart upload Etag: 195c549125ec2fe75e9f3e8444b4a98c-2 OK
2023/02/13 09:13:37 DEBUG : file.ext: md5 = 7246aff55b4ba5cdd90912d8819d5a0f OK
2023/02/13 09:13:37 INFO  : file.ext: Copied (new)
2023/02/13 09:13:37 INFO  : 
Transferred:   	       20 MiB / 20 MiB, 100%, 19.989 MiB/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:         1.8s

2023/02/13 09:13:37 DEBUG : 9 go routines active