Ibm cloud and Rclone

Help and Support
1

Bonjour,

I try to get access to the 25GB of free ibm cloud via rclone
What did i do ?

Create a account, create a ressource Stockage

rclone1
rclone11158×554 46.7 KB

Create a bucklet

rclone2
rclone21209×414 42.9 KB

Get identification for the service as json file like this :

rclone3
rclone31226×524 81.1 KB

Then make a config for rclone
giving this :
[ibm]
type = s3
provider = IBMCOS
env_auth = false
access_key_id = xxxxxxx
secret_access_key = yyyyyyy
endpoint = https://control.cloud-object-storage.cloud.ibm.com/v2/endpoints
location_constraint =
acl = private

Authentication seems to be ok:

rclone lsd ibm: -vv

2021/01/25 11:32:27 DEBUG : rclone: Version "v1.53.4" starting with parameters ["rclone" "lsd" "ibm:" "-vv"]
2021/01/25 11:32:27 DEBUG : Using config file from "/home/airvb/.config/rclone/rclone.conf"
2021/01/25 11:32:27 DEBUG : Creating backend with remote "ibm:"
2021/01/25 11:32:27 DEBUG : 3 go routines active

Trying to create a bucket :
rclone mkdir ibm:testbucket -vv
2021/01/25 11:36:30 DEBUG : rclone: Version "v1.53.4" starting with parameters ["rclone" "mkdir" "ibm:testbucket" "-vv"]
2021/01/25 11:36:30 DEBUG : Creating backend with remote "ibm:testbucket"
2021/01/25 11:36:30 DEBUG : Using config file from "/home/airvb/.config/rclone/rclone.conf"
2021/01/25 11:36:30 DEBUG : S3 bucket testbucket: Making directory
2021/01/25 11:36:30 ERROR : Attempt 1/3 failed with 1 errors and: SerializationError: failed to unmarshal error message
status code: 404, request id: , host id:
caused by: UnmarshalError: failed to unmarshal error message
00000000 3c 21 44 4f 43 54 59 50 45 20 68 74 6d 6c 3e 0a |.|
00000010 3c 68 74 6d 6c 20 6c 61 6e 67 3d 22 65 6e 22 3e ||
00000020 0a 3c 68 65 61 64 3e 0a 3c 6d 65 74 61 20 63 68 |..<meta ch|
00000030 61 72 73 65 74 3d 22 75 74 66 2d 38 22 3e 0a 3c |arset="utf-8">.<|
00000040 74 69 74 6c 65 3e 45 72 72 6f 72 3c 2f 74 69 74 |title>Error</tit|
00000050 6c 65 3e 0a 3c 2f 68 65 61 64 3e 0a 3c 62 6f 64 |le>..<bod|
00000060 79 3e 0a 3c 70 72 65 3e 43 61 6e 6e 6f 74 20 50 |y>.

Cannot P|

00000070  55 54 20 2f 76 32 2f 65  6e 64 70 6f 69 6e 74 73  |UT /v2/endpoints|

00000080  2f 74 65 73 74 62 75 63  6b 65 74 3c 2f 70 72 65  |/testbucket</pre|

00000090  3e 0a 3c 2f 62 6f 64 79  3e 0a 3c 2f 68 74 6d 6c  |>..</html|

000000a0  3e 0a                                             |>.|

caused by: expected element type  but have 


rclone ls ibm:rclone-cos-standard-kqy -vv

2021/01/25 12:13:00 DEBUG : rclone: Version "v1.53.4" starting with parameters ["rclone" "ls" "ibm:rclone-cos-standard-kqy" "-vv"]

2021/01/25 12:13:00 DEBUG : Using config file from "/home/airvb/.config/rclone/rclone.conf"

2021/01/25 12:13:00 DEBUG : Creating backend with remote "ibm:rclone-cos-standard-kqy"

2021/01/25 12:13:00 DEBUG : 3 go routines active

2021/01/25 12:13:00 Failed to ls: directory not found


Any idea is welcome


Thanks
2

That isn't the right endpoint.

Pick one from that page, eg

    "us": {
      "public": {
        "us-geo": "s3.us.cloud-object-storage.appdomain.cloud",
        "Dallas": "s3.dal.us.cloud-object-storage.appdomain.cloud",
        "Washington": "s3.wdc.us.cloud-object-storage.appdomain.cloud",
        "San Jose": "s3.sjc.us.cloud-object-storage.appdomain.cloud"
      },
3

Thanks for yr reply
I try with s3.eu-de.cloud-object-storage.appdomain.cloud
`rclone ls ibm:rclone-cos-standard-kqy -vv
2021/01/25 13:29:57 DEBUG : rclone: Version "v1.53.4" starting with parameters ["rclone" "ls" "ibm:rclone-cos-standard-kqy" "-vv"]

2021/01/25 13:29:57 DEBUG : Using config file from "/home/airvb/.config/rclone/rclone.conf"
2021/01/25 13:29:57 DEBUG : Creating backend with remote "ibm:rclone-cos-standard-kqy"
2021/01/25 13:29:58 DEBUG : 4 go routines active
2021/01/25 13:29:58 Failed to ls: InvalidAccessKeyId: The AWS Access Key ID you provided does not exist in our records.
status code: 403, request id: a12a9e65-f9fa-43bb-ac71-5e6e61d9fa43, host id:

then with s3.private.eu-de.cloud-object-storage.appdomain.cloud
rclone ls ibm:rclone-cos-standard-kqy -vv
2021/01/25 13:31:51 DEBUG : rclone: Version "v1.53.4" starting with parameters ["rclone" "ls" "ibm:rclone-cos-standard-kqy" "-vv"]
2021/01/25 13:31:51 DEBUG : Using config file from "/home/airvb/.config/rclone/rclone.conf"
2021/01/25 13:31:51 DEBUG : Creating backend with remote "ibm:rclone-cos-standard-kqy"
2021/01/25 13:32:51 DEBUG : pacer: low level retry 1/10 (error RequestError: send request failed
caused by: Get "https://s3.private.eu-de.cloud-object-storage.appdomain.cloud/rclone-cos-standard-kqy?delimiter=&max-keys=1000&prefix=": dial tcp 10.1.129.58:443: i/o timeout)
2021/01/25 13:32:51 DEBUG : pacer: Rate limited, increasing sleep to 10ms

But nothing else append....

Another try with : s3.direct.eu-de.cloud-object-storage.appdomain.cloud
rclone ls ibm:rclone-cos-standard-kqy -vv
2021/01/25 13:34:06 DEBUG : rclone: Version "v1.53.4" starting with parameters ["rclone" "ls" "ibm:rclone-cos-standard-kqy" "-vv"]
2021/01/25 13:34:06 DEBUG : Using config file from "/home/airvb/.config/rclone/rclone.conf"
2021/01/25 13:34:06 DEBUG : Creating backend with remote "ibm:rclone-cos-standard-kqy"

But nothing comes...

Q: How should i choose the endpoints ?

4

This looks like the right setting for endpoint.

Have you got the access_key_id and secret_access_key correct?

5

I explained in my first message the method I used to retrieve the key and the secret.
But having not found much information on the ibm website, I am not sure of my method!
I tried to recreate another key with the same result ...

screenshot of the ibm site

rclone4
rclone41205×676 117 KB

Thanks again for your time.

6

I think the keys you want are these ones

image

COS is the name for IBM's object storage and "access_key_id" and "secret_access_key" are the right names for S3 credentials.

7

Thanks it works ,

As describe here
:access_key_id and secret_access_key
Not very friendly explanations on rclone ibm cloud tutorial .

8

Great, glad it is working now!

Can you just sign up for a free trial? Do you need a credit card? Does it expire?

9

I create a 'lite' free account for 25 GB without need a credit card !!!

https://cloud.ibm.com/docs/account?topic=account-accounts#liteaccount

10

I made one too. It isn't obvious how to use it, but it does seem to work with rclone :slight_smile:

11

Maybe we can improve the specific documentation
i have opened an issue an gitub
https://github.com/ibm-cloud-docs/cloud-object-storage

Personally, it's the point about secrets that posed the most difficulties for me.

12

I had no idea I had to tick the HMAC button (which isn't the default) otherwise you don't get the secrets at all.

13

Just for information :
https://github.com/ibm-cloud-docs/cloud-object-storage/issues/111

1 Like
14

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.