GCS support seem to be saying the 400 error on upload is a GCS bug, so hopefully they will fix that. However they are not yet saying the modtime not being stored is a bug. @ncw do you have any thoughts on GCS not storing x-amz-meta-mtime when you upload an object to GCS via S3 protocol.
e.g.
c:\Users\adm-pharvey>dir d:\deleteme\file.zip
Volume in drive D is Data
Volume Serial Number is D2F2-5EA9
Directory of d:\deleteme
08/31/2021 12:56 PM 9,019,415 file.zip
1 File(s) 9,019,415 bytes
0 Dir(s) 92,924,738,273,280 bytes free
c:\Users\adm-pharvey>"c:\Program Files\Rclone\rclone.exe" delete sa-test-s3compatibility:test-s3compatibility/file.zip
c:\Users\adm-pharvey>"c:\program files\rclone\rclone.exe" copy D:\deleteme\ sa-test-s3compatibility:test-s3compatibility/ --log-level=DEBUG --modify-window=1s --dump=headers --retries=1 --low-level-retries=1
2021/09/06 15:29:10 DEBUG : rclone: Version "v1.57.0-beta.5609.719949cf0.fix-5507-bwlimit" starting with parameters ["c:\\program files\\rclone\\rclone.exe" "copy" "D:\\deleteme\\" "sa-test-s3compatibility:test-s3compatibility/" "--log-level=DEBUG" "--modify-window=1s" "--dump=headers" "--retries=1" "--low-level-retries=1"]
2021/09/06 15:29:10 DEBUG : Creating backend with remote "D:\\deleteme\\"
2021/09/06 15:29:10 DEBUG : Using config file from "C:\\Users\\adm-pharvey\\.config\\rclone\\rclone.conf"
2021/09/06 15:29:10 DEBUG : fs cache: renaming cache item "D:\\deleteme\\" to be canonical "//?/D:/deleteme/"
2021/09/06 15:29:10 DEBUG : Creating backend with remote "sa-test-s3compatibility:test-s3compatibility/"
2021/09/06 15:29:10 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2021/09/06 15:29:10 DEBUG : fs cache: renaming cache item "sa-test-s3compatibility:test-s3compatibility/" to be canonical "sa-test-s3compatibility:test-s3compatibility"
2021/09/06 15:29:10 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/06 15:29:10 DEBUG : HTTP REQUEST (req 0xc000774600)
2021/09/06 15:29:10 DEBUG : GET /test-s3compatibility?delimiter=%2F&max-keys=1000&prefix= HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.57.0-beta.5609.719949cf0.fix-5507-bwlimit
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20210906T222910Z
Accept-Encoding: gzip
2021/09/06 15:29:10 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/06 15:29:10 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/06 15:29:10 DEBUG : HTTP RESPONSE (req 0xc000774600)
2021/09/06 15:29:10 DEBUG : HTTP/2.0 200 OK
Content-Length: 268
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private, max-age=0
Content-Type: application/xml; charset=UTF-8
Date: Mon, 06 Sep 2021 22:29:10 GMT
Expires: Mon, 06 Sep 2021 22:29:10 GMT
Server: UploadServer
X-Goog-Metageneration: 13
X-Guploader-Uploadid: ADPycdtg4Tq15FgCWM-A4BFM4rucFndpEjXGe6DFLDHzozYFnLdErIbP0ccBMcYWJKU18MtiGWlKWmbXii6FjWHHsjXMV3lMmQ
2021/09/06 15:29:10 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/06 15:29:10 DEBUG : S3 bucket test-s3compatibility: Waiting for checks to finish
2021/09/06 15:29:10 DEBUG : S3 bucket test-s3compatibility: Waiting for transfers to finish
2021/09/06 15:29:10 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/06 15:29:10 DEBUG : HTTP REQUEST (req 0xc000774400)
2021/09/06 15:29:10 DEBUG : PUT /test-s3compatibility/file.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EOV5W4XTLER4IQVCLUWBJSYDDCYSZ3TVGLDHDNXFVPWOE6WIXTGXFCBA%2F20210906%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210906T222910Z&X-Amz-Expires=900&X-Amz-SignedHeaders=content-md5%3Bcontent-type%3Bhost%3Bx-amz-acl%3Bx-amz-meta-mtime&X-Amz-Signature=da732b9b00f5ba10b687da41d03aa83c4eee2dd0efc9f9f6ee757e732b24df6f HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.57.0-beta.5609.719949cf0.fix-5507-bwlimit
Content-Length: 9019415
content-md5: FobW6PEcINXdR3hyzxKP6Q==
content-type: application/zip
x-amz-acl: private
x-amz-meta-mtime: 1630439815.0598549
Accept-Encoding: gzip
2021/09/06 15:29:10 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/06 15:29:10 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/06 15:29:10 DEBUG : HTTP RESPONSE (req 0xc000774400)
2021/09/06 15:29:10 DEBUG : HTTP/2.0 200 OK
Content-Length: 0
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Content-Type: text/html; charset=UTF-8
Date: Mon, 06 Sep 2021 22:29:10 GMT
Etag: "1686d6e8f11c20d5dd477872cf128fe9"
Server: UploadServer
Vary: Origin
X-Goog-Generation: 1630967350537207
X-Goog-Hash: crc32c=rJfBlQ==
X-Goog-Hash: md5=FobW6PEcINXdR3hyzxKP6Q==
X-Goog-Metageneration: 1
X-Goog-Stored-Content-Encoding: identity
X-Goog-Stored-Content-Length: 9019415
X-Guploader-Uploadid: ADPycdsMBn9_e3K9_5AORZweZWBOtOF3mIbvgX1F0S76LqSKF-HnpMmqlpxtLRLL1O-6bqblv2cMKQqSJAiIv4ftdTJ1Nb7Dkw
2021/09/06 15:29:10 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/06 15:29:10 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/06 15:29:10 DEBUG : HTTP REQUEST (req 0xc000774800)
2021/09/06 15:29:10 DEBUG : HEAD /test-s3compatibility/file.zip HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.57.0-beta.5609.719949cf0.fix-5507-bwlimit
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20210906T222910Z
2021/09/06 15:29:10 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/06 15:29:10 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/06 15:29:10 DEBUG : HTTP RESPONSE (req 0xc000774800)
2021/09/06 15:29:10 DEBUG : HTTP/2.0 200 OK
Content-Length: 9019415
Accept-Ranges: bytes
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private, max-age=0
Content-Type: application/zip
Date: Mon, 06 Sep 2021 22:29:10 GMT
Etag: "1686d6e8f11c20d5dd477872cf128fe9"
Expires: Mon, 06 Sep 2021 22:29:10 GMT
Last-Modified: Mon, 06 Sep 2021 22:29:10 GMT
Server: UploadServer
X-Goog-Hash: crc32c=rJfBlQ==
X-Goog-Hash: md5=FobW6PEcINXdR3hyzxKP6Q==
X-Goog-Storage-Class: NEARLINE
X-Guploader-Uploadid: ADPycdtSC-tmxFYXYILKVob4GQHllR5I-Uq5GCTa9azUu-G-aXCs8KRLYXwjmG1UDYqFzSWdN4KdlXgP_75rFzmK37l3Sdrn3A
2021/09/06 15:29:10 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/06 15:29:10 DEBUG : file.zip: md5 = 1686d6e8f11c20d5dd477872cf128fe9 OK
2021/09/06 15:29:10 INFO : file.zip: Copied (new)
2021/09/06 15:29:10 INFO :
Transferred: 8.602Mi / 8.602 MiByte, 100%, 0 Byte/s, ETA -
Transferred: 1 / 1, 100%
Elapsed time: 0.6s
2021/09/06 15:29:10 DEBUG : 4 go routines active
c:\Users\adm-pharvey>"c:\Program Files\Rclone\rclone.exe" lsl sa-test-s3compatibility:test-s3compatibility
9019415 2021-09-06 15:29:10.000000000 file.zip
If I'm reading that correctly rclone says it is sending the file modtime as 1 Sept 2021 UTC, which is probably correct (haven't converted the timezones), but GCS doesn't seem to store this metadata.
no custom time is set in the metadata if I read it with gsutil.
gsutil stat gs://test-s3compatibility/file.zip
gs://test-s3compatibility/file.zip:
Creation time: Mon, 06 Sep 2021 22:29:10 GMT
Update time: Mon, 06 Sep 2021 22:29:10 GMT
Storage class: NEARLINE
Content-Length: 9019415
Content-Type: application/zip
Hash (crc32c): rJfBlQ==
Hash (md5): FobW6PEcINXdR3hyzxKP6Q==
ETag: CPfXxuyy6/ICEAE=
Generation: 1630967350537207
Metageneration: 1
To test further I manually tried to add this metadata, which did something unexpected, it added not just the x-amz-meta-mtime but also sent Custom-Time, which was unexpected.
gsutil stat gs://test-s3compatibility/file.zip
gs://test-s3compatibility/file.zip:
Creation time: Mon, 06 Sep 2021 22:41:35 GMT
Update time: Mon, 06 Sep 2021 22:44:50 GMT
Storage class: NEARLINE
Content-Length: 9019415
Content-Type: application/zip
Custom-Time: 1970-01-01 00:00:00+00:00
Metadata:
x-amz-meta-mtime: 1630439815.0598549
Hash (crc32c): rJfBlQ==
Hash (md5): FobW6PEcINXdR3hyzxKP6Q==
ETag: CJji3s+16/ICEAI=
Generation: 1630968095420696
Metageneration: 2
And rclone did something even more unexpected with that, it now sees that metadata tag as:
X-Amz-Meta-X-Amz-Meta-Mtime: 1630439815.0598549
However it still seems to go off the upload time, which was expected if x-amz-meta-mtime is not set.
2021/09/06 15:45:13 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/06 15:45:13 DEBUG : S3 bucket test-s3compatibility: Waiting for checks to finish
2021/09/06 15:45:13 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/06 15:45:13 DEBUG : HTTP REQUEST (req 0xc0007c7100)
2021/09/06 15:45:13 DEBUG : HEAD /test-s3compatibility/file.zip HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.57.0-beta.5609.719949cf0.fix-5507-bwlimit
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20210906T224513Z
2021/09/06 15:45:13 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/06 15:45:13 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/06 15:45:13 DEBUG : HTTP RESPONSE (req 0xc0007c7100)
2021/09/06 15:45:13 DEBUG : HTTP/2.0 200 OK
Content-Length: 9019415
Accept-Ranges: bytes
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private, max-age=0
Content-Type: application/zip
Date: Mon, 06 Sep 2021 22:45:13 GMT
Etag: "1686d6e8f11c20d5dd477872cf128fe9"
Expires: Mon, 06 Sep 2021 22:45:13 GMT
Last-Modified: Mon, 06 Sep 2021 22:41:35 GMT
Server: UploadServer
X-Amz-Meta-X-Amz-Meta-Mtime: 1630439815.0598549
X-Goog-Custom-Time: 1970-01-01T00:00:00Z
X-Goog-Hash: crc32c=rJfBlQ==
X-Goog-Hash: md5=FobW6PEcINXdR3hyzxKP6Q==
X-Goog-Storage-Class: NEARLINE
X-Guploader-Uploadid: ADPycduWmT6fzDn7vnmF5qrocmM2cBAPYQUMUIIjsXHtBDJybx6a1gssuQIJZWbFr1IiPSONFKdVsG_wgnWBt3O4AO0B_AlHtQ
2021/09/06 15:45:13 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/06 15:45:13 DEBUG : file.zip: Modification times differ by 146h44m39.9401451s: 2021-08-31 12:56:55.0598549 -0700 PDT, 2021-09-06 22:41:35 +0000 UTC
2021/09/06 15:45:13 DEBUG : file.zip: md5 = 1686d6e8f11c20d5dd477872cf128fe9 OK
2021/09/06 15:45:13 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
if I just set the metadata "mtime" then GCS seems to prepend the X-Amz-Meta- and then everything works as expected.
gsutil stat gs://test-s3compatibility/file.zip
gs://test-s3compatibility/file.zip:
Creation time: Mon, 06 Sep 2021 22:41:35 GMT
Update time: Mon, 06 Sep 2021 22:51:48 GMT
Storage class: NEARLINE
Content-Length: 9019415
Content-Type: application/zip
Custom-Time: 1970-01-01 00:00:00+00:00
Metadata:
mtime: 1630439815.0598549
Hash (crc32c): rJfBlQ==
Hash (md5): FobW6PEcINXdR3hyzxKP6Q==
ETag: CJji3s+16/ICEAM=
Generation: 1630968095420696
Metageneration: 3
"c:\program files\rclone\rclone.exe" copy D:\deleteme\ sa-test-s3compatibility:test-s3compatibility/ --log-level=DEBUG --modify-window=1s --dump=headers --retries=1 --low-level-retries=1
2021/09/06 15:56:48 DEBUG : rclone: Version "v1.57.0-beta.5609.719949cf0.fix-5507-bwlimit" starting with parameters ["c:\\program files\\rclone\\rclone.exe" "copy" "D:\\deleteme\\" "sa-test-s3compatibility:test-s3compatibility/" "--log-level=DEBUG" "--modify-window=1s" "--dump=headers" "--retries=1" "--low-level-retries=1"]
2021/09/06 15:56:48 DEBUG : Creating backend with remote "D:\\deleteme\\"
2021/09/06 15:56:48 DEBUG : Using config file from "C:\\Users\\adm-pharvey\\.config\\rclone\\rclone.conf"
2021/09/06 15:56:48 DEBUG : fs cache: renaming cache item "D:\\deleteme\\" to be canonical "//?/D:/deleteme/"
2021/09/06 15:56:48 DEBUG : Creating backend with remote "sa-test-s3compatibility:test-s3compatibility/"
2021/09/06 15:56:48 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2021/09/06 15:56:48 DEBUG : fs cache: renaming cache item "sa-test-s3compatibility:test-s3compatibility/" to be canonical "sa-test-s3compatibility:test-s3compatibility"
2021/09/06 15:56:48 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/06 15:56:48 DEBUG : HTTP REQUEST (req 0xc000716200)
2021/09/06 15:56:48 DEBUG : GET /test-s3compatibility?delimiter=%2F&max-keys=1000&prefix= HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.57.0-beta.5609.719949cf0.fix-5507-bwlimit
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20210906T225648Z
Accept-Encoding: gzip
2021/09/06 15:56:48 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/06 15:56:48 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/06 15:56:48 DEBUG : HTTP RESPONSE (req 0xc000716200)
2021/09/06 15:56:48 DEBUG : HTTP/2.0 200 OK
Content-Length: 503
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private, max-age=0
Content-Type: application/xml; charset=UTF-8
Date: Mon, 06 Sep 2021 22:56:48 GMT
Expires: Mon, 06 Sep 2021 22:56:48 GMT
Server: UploadServer
X-Goog-Metageneration: 15
X-Guploader-Uploadid: ADPycdvKBmEzNqW5tCyp0Kln9rXSQHA6O7qpR4UVUF0gz0vbmEzjw0dTtIQ7oG09_VQhue9agM70XW3RuOWPy1vd9ca1xFTHkQ
2021/09/06 15:56:48 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/06 15:56:48 DEBUG : S3 bucket test-s3compatibility: Waiting for checks to finish
2021/09/06 15:56:48 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/06 15:56:48 DEBUG : HTTP REQUEST (req 0xc0003c6400)
2021/09/06 15:56:48 DEBUG : HEAD /test-s3compatibility/file.zip HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.57.0-beta.5609.719949cf0.fix-5507-bwlimit
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20210906T225648Z
2021/09/06 15:56:48 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/06 15:56:48 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/06 15:56:48 DEBUG : HTTP RESPONSE (req 0xc0003c6400)
2021/09/06 15:56:48 DEBUG : HTTP/2.0 200 OK
Content-Length: 9019415
Accept-Ranges: bytes
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private, max-age=0
Content-Type: application/zip
Date: Mon, 06 Sep 2021 22:56:48 GMT
Etag: "1686d6e8f11c20d5dd477872cf128fe9"
Expires: Mon, 06 Sep 2021 22:56:48 GMT
Last-Modified: Mon, 06 Sep 2021 22:41:35 GMT
Server: UploadServer
X-Amz-Meta-Mtime: 1630439815.0598549
X-Goog-Custom-Time: 1970-01-01T00:00:00Z
X-Goog-Hash: crc32c=rJfBlQ==
X-Goog-Hash: md5=FobW6PEcINXdR3hyzxKP6Q==
X-Goog-Storage-Class: NEARLINE
X-Guploader-Uploadid: ADPycdtDcbeY8Y7GCUU6wF_BxXGWhJonAeMYQhsBJuo3X757vOSnlBjpayvZlTQkvGLGm1Gx19LJsZgF7r7WY23bJilxKfsStw
2021/09/06 15:56:48 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/06 15:56:48 DEBUG : file.zip: Size and modification time the same (differ by 0s, within tolerance 1s)
2021/09/06 15:56:48 DEBUG : file.zip: Unchanged skipping
2021/09/06 15:56:48 DEBUG : S3 bucket test-s3compatibility: Waiting for transfers to finish
2021/09/06 15:56:48 INFO : There was nothing to transfer
2021/09/06 15:56:48 INFO :
Transferred: 0 / 0 Byte, -, 0 Byte/s, ETA -
Checks: 1 / 1, 100%
Elapsed time: 0.3s
2021/09/06 15:56:48 DEBUG : 3 go routines active
"c:\Program Files\Rclone\rclone.exe" lsl sa-test-s3compatibility:test-s3compatibility
9019415 2021-08-31 12:56:55.059854900 file.zip
I suspect in the backend GCS is storing the mtime with x-goog-meta- and then translating that when rclone uses S3 protocol, but that doesn't explain why they don't store the metadata when rclone sends it.
I'm also going to update the GCP support case with all this extra info.