Wasabi Problems

Help and Support
1

What is the problem you are having with rclone?

copy error. is there anyone who can see what is going wrong here

What is your rclone version (output from rclone version)

1.55

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Ubuntu 2004

Which cloud storage system are you using? (eg Google Drive)

Wasabi -> Scaleway

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Paste command here
rclone copy -P W:online-backup2 S:online-backup2 --checksum --fast-list --checkers 10 --transfers 8


#### The rclone config contents with secrets removed.  
<!--  You should use 3 backticks to begin and end your paste to make it readable.   -->

type = s3
provider = Wasabi
env_auth = false
Access_key_id = XXXXXX
secret_access_key = XXXXXX
endpoint = s3.eu-central-1.wasabisys.com

type = s3
provider = Scaleway
Access_key_id = XXXXXX
secret_access_key = XXXXXX
region = nl-ams
endpoint = s3.nl-ams.scw.cloud
storage_class = GLACIER




#### A log from the command with the `-vv` flag  
<!-- You should use 3 backticks to begin and end your paste to make it readable.  Or use a service such as https://pastebin.com or https://gist.github.com/   -->

Paste log here
rclone copy -P W:online-backup2 S:online-backup2 --checksum --fast-list --checkers 10 --transfers 8

2021-07-10 13:27:41 ERROR : : error reading source directory: InternalError: We encountered an internal error. Please retry the operation again later.
status code: 500, request id: 067DB86B1966E633, host id: qLjQFeU/56SJWwAET6X6Jhaav5oHTyKYvNbHNiHB20tHswwH8a3Q+rAxjrRLn9PpTMF2rJpBZIFt
2021-07-10 13:27:41 ERROR : Attempt 1/3 failed with 1 errors and: InternalError: We encountered an internal error. Please retry the operation again later.
status code: 500, request id: 067DB86B1966E633, host id: qLjQFeU/56SJWwAET6X6Jhaav5oHTyKYvNbHNiHB20tHswwH8a3Q+rAxjrRLn9PpTMF2rJpBZIFt
2021-07-10 13:31:14 ERROR : : error reading source directory: InternalError: We encountered an internal error. Please retry the operation again later.
status code: 500, request id: 5D3B829E2A10007D, host id: /osXKYI49QIhm69qjZtu6vnZf9fhTERKMnvk4ok6E5kLReYRy2NUUjp0UGl6bGf4OfyGClTOfBIy
2021-07-10 13:31:14 ERROR : Attempt 2/3 failed with 1 errors and: InternalError: We encountered an internal error. Please retry the operation again later.
status code: 500, request id: 5D3B829E2A10007D, host id: /osXKYI49QIhm69qjZtu6vnZf9fhTERKMnvk4ok6E5kLReYRy2NUUjp0UGl6bGf4OfyGClTOfBIy
2021-07-10 13:34:45 ERROR : : error reading source directory: InternalError: We encountered an internal error. Please retry the operation again later.
status code: 500, request id: 6B086C4A827B744D, host id: 2r0ylFdVDD4nJG1Wxqj3/Yqk+qFVEJvVYSvabkKjhkmZO1cPKwBJYHoCOxPy9tfO3GFPbxgKDWSs
2021-07-10 13:34:45 ERROR : Attempt 3/3 failed with 1 errors and: InternalError: We encountered an internal error. Please retry the operation again later.
status code: 500, request id: 6B086C4A827B744D, host id: 2r0ylFdVDD4nJG1Wxqj3/Yqk+qFVEJvVYSvabkKjhkmZO1cPKwBJYHoCOxPy9tfO3GFPbxgKDWSs
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 10m53.0s
2021/07/10 13:34:45 Failed to copy: InternalError: We encountered an internal error. Please retry the operation again later.
status code: 500, request id: 6B086C4A827B744D, host id: 2r0ylFdVDD4nJG1Wxqj3/Yqk+qFVEJvVYSvabkKjhkmZO1cPKwBJYHoCOxPy9tfO3GFPbxgKDWSs

2

You have no version, no rclone.conf and no debug log.

Please post all the details.

3

If I understand what you are trying to do, you cannot do that. Files in Flacier need to be pulled back to S3 before you can read them. There is a cost and timeframe to get stuff out of Glacier.

If my note is wrong, try copying you file with an AWS S3 cp command to see if it works. RClone I believe uses the AWS SDK for compatible S3 providers.

4

hi,

  • while that is correct, i do think that is the issue here as the error is
    ERROR : : error reading source directory
    and tho the config file is missing the name of the remotes, i assume that since the source of W:online-backup2 that is Wasabi, which does not a glacier storage class.

  • if you try to copy a file from glacier storage class, you get this in the log.
    ERROR : file.txt: Failed to copy: failed to open source object: Object in GLACIER, restore first: bucket="xxx", key="xxx"

and i have used wasabi for years and never saw a 500
InternalError: We encountered an internal error. Please retry the operation again later. status code: 500

6

hi,

i have used wasabi for years and never saw a 500, this might be a one-time error with wasabi server
but if that is not the case, can you try this and post the rclone.log
rclone copy W:online-backup2 S:online-backup2 --dump=bodies --retries=1 --low-level-retries=1 --log-level=DEBUG --log-file=rclone.log

7

My bad. i had it backwards. You are going from Wasabi to your Glacier storage.

Are you by chance using a custom encrypted Wasabi storage bucket? I would try using AWS sdk cp and ls commands to see if you can get them to work on your Wasabi storage. That would verify your source provider.

I might also try going to your dest bucket -/ normal before trying dest:Glacier. Rationale: S3 has some strong limitations about inter region copies. Wasabi, or RClone, have made this easier but they may be tripping you up.

Good luck.

8

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.