Chunker has been unavailable

Help and Support
1 
[root@**** mysql]# cat /root/.config/rclone/rclone.conf
 
[contabo-all-backup]
type = s3
provider = Ceph
access_key_id = 7****5
secret_access_key = f****6
endpoint = https://u****m

[my-chunker-name]
type = chunker
remote = contabo-all-backup:
chunk_size = 100Mi

The error message：

[root@aa mysql]#  rclone copy /datastore/backup/mysql/aa_20220417124916.sql.gz my-chunker-name: --transfers 100 --retries 3 -vv --progress  --order-by size,mixed,30 --chunker-chunk-size 100M
2022/04/17 14:29:21 DEBUG : rclone: Version "v1.58.0" starting with parameters ["rclone" "copy" "/datastore/backup/mysql/aa_20220417124916.sql.gz" "my-chunker-name:" "--transfers" "100" "--retries" "3" "-vv" "--progress" "--order-by" "size,mixed,30" "--chunker-chunk-size" "100M"]
2022/04/17 14:29:21 DEBUG : Creating backend with remote "/datastore/backup/mysql/aa_20220417124916.sql.gz"
2022/04/17 14:29:21 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2022/04/17 14:29:21 DEBUG : fs cache: adding new entry for parent of "/datastore/backup/mysql/aa_20220417124916.sql.gz", "/datastore/backup/mysql"
2022/04/17 14:29:21 DEBUG : Creating backend with remote "my-chunker-name:"
2022/04/17 14:29:21 DEBUG : my-chunker-name: detected overridden config - adding "{HvT2Y}" suffix to name
2022/04/17 14:29:21 DEBUG : Creating backend with remote "contabo-all-backup:"
2022/04/17 14:29:21 DEBUG : Reset feature "ListR"
2022/04/17 14:29:21 DEBUG : fs cache: renaming cache item "my-chunker-name:" to be canonical "my-chunker-name{HvT2Y}:"
2022-04-17 14:29:21 ERROR : Attempt 1/3 failed with 1 errors and: InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.
2022-04-17 14:29:21 ERROR : Attempt 2/3 failed with 1 errors and: InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.
2022-04-17 14:29:21 ERROR : Attempt 3/3 failed with 1 errors and: InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.0s
2022/04/17 14:29:21 INFO  : 
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.0s

aa_20220417124916.sql.gz is 2.9GB
Limited 200MB

How should I make Chunker work？
Thank you

2

hello and welcome to the forum,

maybe need to use a bucket

remote = contabo-all-backup:bucketname

and good to test by using --dry-run

3

I changed the configuration：

[my-chunker-name]
type = chunker
remote = contabo-all-backup:all-backup
chunk_size = 100Mi

I tried new commands：

rclone copy /datastore/backup/mysql/aa_20220417124916.sql.gz my-chunker-name:all-backup --transfers 100 --retries 3 -vv --progress  --order-by size,mixed,30 --chunker-chunk-size 100M

The mistakes are different. But it failed

[root@aa ~]# rclone copy /datastore/backup/mysql/aa_20220417124916.sql.gz my-chunker-name:all-backup --transfers 100 --retries 3 -vv --progress  --order-by size,mixed,30 --chunker-chunk-size 100M
2022/04/17 20:26:01 DEBUG : rclone: Version "v1.58.0" starting with parameters ["rclone" "copy" "/datastore/backup/mysql/aa_20220417124916.sql.gz" "my-chunker-name:all-backup" "--transfers" "100" "--retries" "3" "-vv" "--progress" "--order-by" "size,mixed,30" "--chunker-chunk-size" "100M"]
2022/04/17 20:26:01 DEBUG : Creating backend with remote "/datastore/backup/mysql/aa_20220417124916.sql.gz"
2022/04/17 20:26:01 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2022/04/17 20:26:01 DEBUG : fs cache: adding new entry for parent of "/datastore/backup/mysql/aa_20220417124916.sql.gz", "/datastore/backup/mysql"
2022/04/17 20:26:01 DEBUG : Creating backend with remote "my-chunker-name:all-backup"
2022/04/17 20:26:01 DEBUG : my-chunker-name: detected overridden config - adding "{HvT2Y}" suffix to name
2022/04/17 20:26:01 DEBUG : Creating backend with remote "contabo-all-backup:all-backup/all-backup"
2022/04/17 20:26:02 DEBUG : Reset feature "ListR"
2022/04/17 20:26:02 DEBUG : fs cache: renaming cache item "my-chunker-name:all-backup" to be canonical "my-chunker-name{HvT2Y}:all-backup"
2022-04-17 20:26:02 ERROR : Attempt 1/3 failed with 1 errors and: Forbidden: Forbidden
        status code: 403, request id: tx0000084b9f75087520274-00625cda4a-42d6-default, host id:
2022-04-17 20:26:02 ERROR : Attempt 2/3 failed with 1 errors and: Forbidden: Forbidden
        status code: 403, request id: tx0000041270ff9d278940b-00625cda4a-8fdb-default, host id:
2022-04-17 20:26:02 ERROR : Attempt 3/3 failed with 1 errors and: Forbidden: Forbidden
        status code: 403, request id: tx0000048f8148e8a3c2e22-00625cda4a-8efa-default, host id:
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.5s
2022/04/17 20:26:02 INFO  : 
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.5s

2022/04/17 20:26:02 DEBUG : 3 go routines active
2022/04/17 20:26:02 Failed to copy: Forbidden: Forbidden
        status code: 403, request id: tx0000048f8148e8a3c2e22-00625cda4a-8efa-default, host id:
4

[root@aaa ~]# rclone copy /datastore/backup/mysql/aa_20220417124916.sql.gz my-chunker-name:all-backup --transfers 100 --retries 3 -vv --progress --order-by size,mixed,30 --chunker-chunk-size 100M --dry-run
2022/04/17 20:31:50 DEBUG : rclone: Version "v1.58.0" starting with parameters ["rclone" "copy" "/datastore/backup/mysql/aa_20220417124916.sql.gz" "my-chunker-name:all-backup" "--transfers" "100" "--retries" "3" "-vv" "--progress" "--order-by" "size,mixed,30" "--chunker-chunk-size" "100M" "--dry-run"]
2022/04/17 20:31:50 DEBUG : Creating backend with remote "/datastore/backup/mysql/aa_20220417124916.sql.gz"
2022/04/17 20:31:50 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2022/04/17 20:31:50 DEBUG : fs cache: adding new entry for parent of "/datastore/backup/mysql/aa_20220417124916.sql.gz", "/datastore/backup/mysql"
2022/04/17 20:31:50 DEBUG : Creating backend with remote "my-chunker-name:all-backup"
2022/04/17 20:31:50 DEBUG : my-chunker-name: detected overridden config - adding "{HvT2Y}" suffix to name
2022/04/17 20:31:50 DEBUG : Creating backend with remote "contabo-all-backup:all-backup/all-backup"
2022/04/17 20:31:50 DEBUG : Reset feature "ListR"
2022/04/17 20:31:50 DEBUG : fs cache: renaming cache item "my-chunker-name:all-backup" to be canonical "my-chunker-name{HvT2Y}:all-backup"
2022-04-17 20:31:50 ERROR : Attempt 1/3 failed with 1 errors and: Forbidden: Forbidden
status code: 403, request id: tx00000d4a9c6fe59eb2bb1-00625cdba6-42d6-default, host id:
2022-04-17 20:31:50 ERROR : Attempt 2/3 failed with 1 errors and: Forbidden: Forbidden
status code: 403, request id: tx000009686597d42a37cb8-00625cdba6-8fdb-default, host id:
2022-04-17 20:31:50 ERROR : Attempt 3/3 failed with 1 errors and: Forbidden: Forbidden
status code: 403, request id: tx00000cc24063ef19f6d18-00625cdba6-8efa-default, host id:
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.4s
2022/04/17 20:31:50 NOTICE:
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.4s

2022/04/17 20:31:50 DEBUG : 3 go routines active
2022/04/17 20:31:50 Failed to copy: Forbidden: Forbidden
status code: 403, request id: tx00000cc24063ef19f6d18-00625cdba6-8efa-default, host id:

5

403 is a simple permissions issue.
the access_key_id does not have permission to access that bucket.

make sure, able to copy files to contabo-all-backup:all-backup, before trying chunker.
rclone copy somefilename.ext contabo-all-backup:all-backup -vv

6

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.