Additional mirror in conjunction with B2 for redundancy

Help and Support
1

Hi,
I have a few sources I mirror to B2 with rclone which works well. I want to add another mirror for redundancy and I was wondering what peoples opinions were for options in this field. I imagine aws or azure blob, what do people people recommend for max 2TB with ~3 gig per day up of changes?

2

AWS and Azure are a lot more expensive than b2… You could try https://wasabi.com if you want something similarly priced.

3

Hi Nick,
I have tried both Glacier and Wasabi, and even confirmed my IAM configuration within Wasabi with their technical support, yet both services return an access denied error. My config for both looks like the following:

[glacier]
type = s3
provider = AWS
env_auth = false
access_key_id = yyy
secret_access_key = xxx
region =
acl = private
storage_class = GLACIER
bucket_acl = private

[wasabi]
type = s3
provider = Wasabi
env_auth = false
access_key_id = yyy
secret_access_key = xxx
endpoint = s3.wasabisys.com
acl =

Any idea what I may be doing wrong? I have confirmed the api user and the keys several times, even carefully recreating them.

4

Can you post the output of rclone and the command you tried?

Try doing rclone lsd -vv glacier: and rclone lsd -vv wasabi: and post the results.

5

Hi Nick,
The following is the output:

[root@storage ~]# rclone lsd -vv glacier:
2019/03/04 13:52:25 DEBUG : rclone: Version "v1.46" starting with parameters ["rclone" "lsd" "-vv" "glacier:"]
2019/03/04 13:52:25 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2019/03/04 13:52:26 DEBUG : pacer: Reducing sleep to 0s
2019/03/04 13:52:26 DEBUG : 4 go routines active
2019/03/04 13:52:26 DEBUG : rclone: Version "v1.46" finishing with parameters ["rclone" "lsd" "-vv" "glacier:"]
[root@storage ~]# rclone lsd -vv glacier:restic
2019/03/04 13:55:24 DEBUG : rclone: Version "v1.46" starting with parameters ["rclone" "lsd" "-vv" "glacier:restic"]
2019/03/04 13:55:24 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2019/03/04 13:55:25 DEBUG : pacer: Reducing sleep to 0s
2019/03/04 13:55:25 ERROR : S3 bucket restic: Failed to update region for bucket: reading bucket location failed: AccessDenied: Access Denied
        status code: 403, request id: 262D3FF9250ADE4C, host id: NpwYSCgHsaBzR7sx4PEF5vKU8Bi17N8q/t7jfgVX/Q9W/dLjWscOjoGmXAJfm3rdZWWUWr32jMk=
2019/03/04 13:55:25 ERROR : : error listing: BucketRegionError: incorrect region, the bucket is not in 'us-east-1' region
        status code: 301, request id: , host id:
2019/03/04 13:55:25 Failed to lsd: BucketRegionError: incorrect region, the bucket is not in 'us-east-1' region
        status code: 301, request id: , host id:
[root@storage ~]# rclone lsd -vv wasabi:
2019/03/04 13:52:42 DEBUG : rclone: Version "v1.46" starting with parameters ["rclone" "lsd" "-vv" "wasabi:"]
2019/03/04 13:52:42 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2019/03/04 13:52:42 DEBUG : pacer: Reducing sleep to 0s
          -1 2019-03-03 03:21:48        -1 restic-hosts
2019/03/04 13:52:42 DEBUG : 4 go routines active
2019/03/04 13:52:42 DEBUG : rclone: Version "v1.46" finishing with parameters ["rclone" "lsd" "-vv" "wasabi:"]
[root@storage ~]# rclone lsd -vv wasabi:restic-hosts
2019/03/04 13:54:09 DEBUG : rclone: Version "v1.46" starting with parameters ["rclone" "lsd" "-vv" "wasabi:restic-hosts"]
2019/03/04 13:54:09 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2019/03/04 13:54:10 DEBUG : pacer: Reducing sleep to 0s
2019/03/04 13:54:10 ERROR : : error listing: AccessDenied: Access Denied
        status code: 403, request id: FEAD0044A216E4BE, host id: 6KHb/BcvoEOVWVzq118ebe6Qqz31hJqMfVIuXHrjYXROCiI2jYN5rI15jQBsC44XW/AZv4xw0DxR
2019/03/04 13:54:10 Failed to lsd: AccessDenied: Access Denied
        status code: 403, request id: FEAD0044A216E4BE, host id: 6KHb/BcvoEOVWVzq118ebe6Qqz31hJqMfVIuXHrjYXROCiI2jYN5rI15jQBsC44XW/AZv4xw0DxR

Although the single bucket in Glacier is not shown, the attempt to list it is.

Thanks!

6

That error indicates you’ve created the bucket in a different region to that in the config file. Either supply --s3-region or change the region in the config for glacier:.

I’m not sure what is causing that error. Try making another bucket with rclone mkdir wasabi:bucket-name and then listing it rclone ls wasabi:bucket-name. You might need to try some different names other than bucket-name until you find one which isn’t in use.

7

Hi,
Creating a new bucket in Wasabi with rclone and uploading worked, not sure why it failed to start, but thanks for the suggestion.

With Glacier, the bucket is in the right region, regardless I dropped and recreated it and also regenerated the users access keys to no avail:
Capture

The remote conf is as follows:

[glacier]
type = s3
provider = AWS
env_auth = false
access_key_id = xxx
secret_access_key = yyy
region = us-east-1
acl = private
storage_class = GLACIER
bucket_acl = private
8

That is good

You appear to be using a vault. That as far as I understand is a separate service to S3 which isn’t supported by rclone.

I think what you want to do is create a bucket and use the GLACIER storage_class like you have.

9

Everything is working now, thanks for the patience:)

10

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.