Rclone backed restore / Oracle OCI

Help and Support
1

Just started to test out rclone with oracle/oci s3 compatible storage. been using it for a long time with azure with great success- Thank you!

s3 sync is working well, but when i try to restore from the archive tier i get an error saying we are missing a header. has anyone used the archive tier with oracle and been able to restore though rclone for this provider?

Thank you!

sh-4.3# rclone version
rclone v1.59.1
- os/version: unknown
- os/kernel: 3.10.105 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.18.5
- go/linking: static
- go/tags: none

rclone backend restore oci:"<<redacted>>/BROLL"

config:

{
    "oci": {
        "access_key_id": "key",
        "acl": "private",
        "endpoint": "https://<<redacted>>.oraclecloud.com",
        "location_constraint": "us-ashburn-1",
        "provider": "Other",
        "secret_access_key": "secret",
        "type": "s3"
    }
}

verbose doesnt say much - you can see the error returned for each item (one shown here):

sh-4.3# rclone backend restore oci:"<<redacted>>/BROLL" -vv
2022/08/22 16:20:42 DEBUG : rclone: Version "v1.59.1" starting with parameters ["rclone" "backend" "restore" "oci:<<redacted>>/BROLL" "-vv"]
2022/08/22 16:20:42 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
[
	{
		"Status": "InvalidRequest: Missing required header for this request: Content-MD5\n\tstatus code: 400, request id: iad-1:TzWm-2yqD-vXrr-EJzj3a_GWxWAt_rP8Ymn-zkqd-YtYs947t4AOBlEOdYNMP0e-, host id: ",
		"Remote": "<<redacted>>"
	},
2

hello and welcome to the forum,

i do not know what the exact issue is

for a more detailed log, add --dump=headers

3

thx - not sure this helps much - but thanks for the tip :slight_smile:

sh-4.3# rclone backend restore oci:"<<redacted>>/BROLL/" --dump=headers
2022/08/22 16:54:50 NOTICE: Automatically setting -vv as --dump is enabled
2022/08/22 16:54:50 DEBUG : rclone: Version "v1.59.1" starting with parameters ["rclone" "backend" "restore" "oci:<<redacted>>/BROLL/" "--dump=headers"]
2022/08/22 16:54:50 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2022/08/22 16:54:50 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2022/08/22 16:54:50 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2022/08/22 16:54:50 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/08/22 16:54:50 DEBUG : HTTP REQUEST (req 0xc000626900)
2022/08/22 16:54:50 DEBUG : GET /<<redacted>>?delimiter=&max-keys=1000&prefix=%2BTech%20Docs%2F<<redacted>>%2FBROLL%2F HTTP/1.1
Host: <<redacted>>.oraclecloud.com
User-Agent: rclone/v1.59.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20220822T205450Z
Accept-Encoding: gzip

2022/08/22 16:54:50 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/08/22 16:54:51 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/08/22 16:54:51 DEBUG : HTTP RESPONSE (req 0xc000626900)
2022/08/22 16:54:51 DEBUG : HTTP/1.1 200 OK
Content-Length: 38457
Content-Type: application/xml; charset=utf-8
Date: Mon, 22 Aug 2022 20:54:51 GMT
Opc-Request-Id: iad-1:wt4FUQ_ve2evnh63iRSowzeB0xnB_IZA0PGxaniUkinTeQ_hJYesDplClTXwlWuC
X-Amz-Bucket-Region: us-ashburn-1
X-Amz-Request-Id: iad-1:wt4FUQ_ve2evnh63iRSowzeB0xnB_IZA0PGxaniUkinTeQ_hJYesDplClTXwlWuC
X-Api-Id: s3-compatible

2022/08/22 16:54:51 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/08/22 16:54:51 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/08/22 16:54:51 DEBUG : HTTP REQUEST (req 0xc000626500)
2022/08/22 16:54:51 DEBUG : POST /<<redacted>>/%2BTech%20Docs/<<redacted>>/BROLL/<<redacted>>.mxf?restore= HTTP/1.1
Host: <<redacted>>.oraclecloud.com
User-Agent: rclone/v1.59.1
Content-Length: 81
Authorization: XXXX
X-Amz-Content-Sha256: a9fafa94059473e18625608e3d0c96ba81dd02a4ed2c12c74e6e196571d599aa
X-Amz-Date: 20220822T205451Z
Accept-Encoding: gzip

2022/08/22 16:54:51 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/08/22 16:54:51 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/08/22 16:54:51 DEBUG : HTTP RESPONSE (req 0xc000626500)
2022/08/22 16:54:51 DEBUG : HTTP/1.1 400 Bad Request
Content-Length: 152
Content-Type: application/xml; charset=utf-8
Date: Mon, 22 Aug 2022 20:54:51 GMT
Opc-Request-Id: iad-1:6Rj-Dbolo8drxHGUnSDyHjPbCZFU7BegZ4kEsSbACYsaBm_JnlBWADAFb2xi1vCp
X-Amz-Request-Id: iad-1:6Rj-Dbolo8drxHGUnSDyHjPbCZFU7BegZ4kEsSbACYsaBm_JnlBWADAFb2xi1vCp
X-Api-Id: s3-compatible
4

i get the same basic error as you do,

in my case,
using rclone version: v1.58.1
aws s3 deep glacier on a small file, should be cheap to restore

rclone lsf --format=phsT aws01:vserver12.en07.rcloner/en07.rcloner/rclone/backup/kdbx/zip/en07.kdbx_20191011.133522.7z 
en07.kdbx_20191011.133522.7z;bca5ad716987fa9ef6411048eb618657;80985;DEEP_ARCHIVE

rclone backend restore --dump=headers aws01:vserver12.en07.rcloner/en07.rcloner/rclone/backup/kdbx/zip --include="en07.kdbx_20191011.133522.7z" 
2022/08/22 17:16:23 NOTICE: Automatically setting -vv as --dump is enabled
2022/08/22 17:16:23 DEBUG : rclone: Version "v1.58.1" starting with parameters ["C:\\data\\rclone\\rclone.exe" "backend" "restore" "--dump=headers" "aws01:vserver12.en07.rcloner/en07.rcloner/rclone/backup/kdbx/zip" "--include=en07.kdbx_20191011.133522.7z"]
2022/08/22 17:16:23 DEBUG : Using config file from "C:\\data\\rclone\\rclone.conf"

2022/08/22 17:16:23 DEBUG : HTTP REQUEST (req 0xc00031ca00)
2022/08/22 17:16:23 DEBUG : GET /vserver12.en07.rcloner?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=en07.rcloner%2Frclone%2Fbackup%2Fkdbx%2Fzip%2F HTTP/1.1
Host: s3.us-east-1.amazonaws.com
User-Agent: rclone/v1.58.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20220822T211623Z
Accept-Encoding: gzip

2022/08/22 17:16:23 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/08/22 17:16:24 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/08/22 17:16:24 DEBUG : HTTP RESPONSE (req 0xc00031ca00)
2022/08/22 17:16:24 DEBUG : HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Mon, 22 Aug 2022 21:16:27 GMT
Server: AmazonS3
X-Amz-Bucket-Region: us-east-1
X-Amz-Id-2: Km7MzhN31+IkjqU58HDLedLwchyvzLVrBcfFV+NCMjpfCYyjJhRH0anbBRDOfYH/0QyNjS09uK0=
X-Amz-Request-Id: 04PW1QXCKZ1NFYP9

2022/08/22 17:16:24 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/08/22 17:16:24 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/08/22 17:16:24 DEBUG : HTTP REQUEST (req 0xc00071a100)
2022/08/22 17:16:24 DEBUG : POST /vserver12.en07.rcloner/en07.rcloner/rclone/backup/kdbx/zip/en07.kdbx_20191011.133522.7z?restore= HTTP/1.1
Host: s3.us-east-1.amazonaws.com
User-Agent: rclone/v1.58.1
Content-Length: 81
Authorization: XXXX
X-Amz-Content-Sha256: a9fafa94059473e18625608e3d0c96ba81dd02a4ed2c12c74e6e196571d599aa
X-Amz-Date: 20220822T211624Z
Accept-Encoding: gzip

2022/08/22 17:16:24 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/08/22 17:16:24 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/08/22 17:16:24 DEBUG : HTTP RESPONSE (req 0xc00071a100)
2022/08/22 17:16:24 DEBUG : HTTP/1.1 400 Bad Request
Connection: close
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Mon, 22 Aug 2022 21:16:25 GMT
Server: AmazonS3
X-Amz-Id-2: O8xGBfOMWe8bmVQjLwM6sBr3fsG3RMwMHIRdADHmP7LoIkPS7XEC0RrXSTNYVeRm1BYphtmv+Zk=
X-Amz-Request-Id: 04PTQEQS5DZQ19FW

2022/08/22 17:16:24 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
[
	{
		"Status": "MalformedXML: The XML you provided was not well-formed or did not validate against our published schema\n\tstatus code: 400, request id: 04PTQEQS5DZQ19FW, host id: O8xGBfOMWe8bmVQjLwM6sBr3fsG3RMwMHIRdADHmP7LoIkPS7XEC0RrXSTNYVeRm1BYphtmv+Zk=",
		"Remote": "en07.kdbx_20191011.133522.7z"
	}
]
5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.