Move Command Syntax Help - Wasabi

Help and Support
1

What is the problem you are having with rclone?

I am having difficulty using the move command. I think I am confused about syntax. I've read some examples and some other posts to no avail.
Specifically, I am getting an "error reading destination directory:access denied" error.

Interestingly, I am able to move files when I have the remote drive VFS mounted. So I know that I have access. That's why I think I am just using the move command incorrectly.

What is your rclone version (output from rclone version)

V1.55.1

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10 64-bits

Which cloud storage system are you using? (eg Google Drive)

Wasabi

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone move e:\ <remote name>:<dir on remote> --include *.<file ext name> --progress

The directory on the remote is right off of the root. Not sure if that is relevant.

The rclone config contents with secrets removed.

[<remotename>]
type = s3
provider = Wasabi
env_auth = false
access_key_id = <access key id>
secret_access_key = <secret access key>
region = us-east-2
endpoint = s3.us-east-2.wasabisys.com

A log from the command with the -vv flag

DEBUG : <filename>.<otherextentsion>: Excluded
2021-07-03 11:20:08 ERROR : : error reading destination directory: AccessDenied: Access Denied
        status code: 403, request id: 98F3F47727DB1AD2, host id: oLNwjHEKHeIIMcg3TJZpuOLk8zZ9Ti/HKo1QLFyIj7fol8FtyuZBCQNFRp1xxLVu+sWkU3ytoNjK
2021-07-03 11:20:08 DEBUG : S3 bucket plots: Waiting for checks to finish
2021-07-03 11:20:08 DEBUG : S3 bucket plots: Waiting for transfers to finish
2021-07-03 11:20:08 ERROR : Attempt 3/3 failed with 1 errors and: AccessDenied: Access Denied
        status code: 403, request id: 98F3F47727DB1AD2, host id: oLNwjHEKHeIIMcg3TJZpuOLk8zZ9Ti/HKo1QLFyIj7fol8FtyuZBCQNFRp1xxLVu+sWkU3ytoNjK
2021-07-03 11:20:08 DEBUG : S3 bucket plots: Waiting for checks to finish
2021-07-03 11:20:08 DEBUG : S3 bucket plots: Waiting for transfers to finish
2021-07-03 11:20:08 ERROR : Attempt 3/3 failed with 1 errors and: AccessDenied: Access Denied
        status code: 403, request id: 98F3F47727DB1AD2, host id: oLNwjHEKHeIIMcg3TJZpuOLk8zZ9Ti/HKo1QLFyIj7fol8FtyuZBCQNFRp1xxLVu+sWkU3ytoNjK
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         3.8s
2021/07/03 11:20:08 NOTICE:
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         3.8s

2021/07/03 11:20:08 DEBUG : 6 go routines active
2021/07/03 11:20:08 Failed to move: AccessDenied: Access Denied
        status code: 403, request id: 98F3F47727DB1AD2, host id: oLNwjHEKHeIIMcg3TJZpuOLk8zZ9Ti/HKo1QLFyIj7fol8FtyuZBCQNFRp1xxLVu+sWkU3ytoNjK
2

hello and welcome to the forum,

without a debug log and the exact command, no idea what is really going on

can you post the rclone.log for
rclone ls <remote name>:<dir on remote> --max-depth=1 --dump=bodies --retries=1 --low-level-retries=1 --log-level=DEBUG --log-file=rclone.log

the s3 permissions needed are doucmented here.
with wasabi, i use a more locked down version but that works.
https://rclone.org/s3/#s3-permissions

3

Thanks for your response. Here is what shows up in the log:
...
2021/07/03 12:22:54 DEBUG : Using config file from "C:\Users\bbymi\.config\rclone\rclone.conf"
2021/07/03 12:22:54 DEBUG : rclone: Version "v1.55.1" starting with parameters ["C:\rclone\rclone.exe" "ls" "jesseremote:jessedir" "--max-depth=1" "--dump=bodies" "--retries=1" "--low-level-retries=1" "--log-level=DEBUG" "--log-file=rclone.log"]
2021/07/03 12:22:54 DEBUG : Creating backend with remote "jesseremote:jessedir"
2021/07/03 12:22:54 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2021/07/03 12:22:54 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/07/03 12:22:54 DEBUG : HTTP REQUEST (req 0xc000555f00)
2021/07/03 12:22:54 DEBUG : GET /jessedir?delimiter=%2F&encoding-type=url&max-keys=1000&prefix= HTTP/1.1
Host: s3.us-east-2.wasabisys.com
User-Agent: rclone/v1.55.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20210703T162254Z
Accept-Encoding: gzip

2021/07/03 12:22:54 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/07/03 12:22:54 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/07/03 12:22:54 DEBUG : HTTP RESPONSE (req 0xc000555f00)
2021/07/03 12:22:54 DEBUG : HTTP/1.1 301 Moved Permanently
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Sat, 03 Jul 2021 16:22:53 GMT
Location: https://s3.wasabisys.com/jessedir?delimiter=%2F&encoding-type=url&max-keys=1000&prefix=
Server: WasabiS3/6.2.4757-2021-05-28-58bef69 (head3)
X-Amz-Bucket-Region: us-east-1
X-Amz-Id-2: lan22oTAB9rO+W8UbQeLUcvPjDmTa+CvOBjqEaYGA2wH2btxnCE87kKX2uYKWC4YhyOC+Y08BO6G
X-Amz-Request-Id: FBA7500851D74967

1b5

<?xml version="1.0" encoding="UTF-8"?>

PermanentRedirectThe bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.jessedirjessedir.s3.wasabisys.comFBA7500851D74967lan22oTAB9rO+W8UbQeLUcvPjDmTa+CvOBjqEaYGA2wH2btxnCE87kKX2uYKWC4YhyOC+Y08BO6G
0

2021/07/03 12:22:54 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/07/03 12:22:54 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/07/03 12:22:54 DEBUG : HTTP REQUEST (req 0xc000408300)
2021/07/03 12:22:54 DEBUG : GET /jessedir?delimiter=%2F&encoding-type=url&max-keys=1000&prefix= HTTP/1.1
Host: s3.wasabisys.com
User-Agent: rclone/v1.55.1
Referer: https://s3.us-east-2.wasabisys.com/jessedir?delimiter=%2F&encoding-type=url&max-keys=1000&prefix=
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20210703T162254Z
Accept-Encoding: gzip

2021/07/03 12:22:54 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/07/03 12:22:54 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/07/03 12:22:54 DEBUG : HTTP RESPONSE (req 0xc000408300)
2021/07/03 12:22:54 DEBUG : HTTP/1.1 403 Forbidden
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Sat, 03 Jul 2021 16:22:53 GMT
Server: WasabiS3/7.0.159-2021-06-24-5f20830 (head05)
X-Amz-Bucket-Region: us-east-1
X-Amz-Id-2: KeNMPoHu/viheganHmBu8Gs7gvOpmJ/2IS3y4dbf6zWKQ8pDB+9A67F14eoWYgIrK1YgxoZJneuC
X-Amz-Request-Id: BDB6F360ADCC511E

f3

<?xml version="1.0" encoding="UTF-8"?>

AccessDeniedAccess DeniedBDB6F360ADCC511EKeNMPoHu/viheganHmBu8Gs7gvOpmJ/2IS3y4dbf6zWKQ8pDB+9A67F14eoWYgIrK1YgxoZJneuC
0

2021/07/03 12:22:54 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/07/03 12:22:54 ERROR : : error listing: AccessDenied: Access Denied
status code: 403, request id: BDB6F360ADCC511E, host id: KeNMPoHu/viheganHmBu8Gs7gvOpmJ/2IS3y4dbf6zWKQ8pDB+9A67F14eoWYgIrK1YgxoZJneuC
2021/07/03 12:22:54 DEBUG : 6 go routines active
2021/07/03 12:22:54 Failed to ls with 2 errors: last error was: AccessDenied: Access Denied
status code: 403, request id: BDB6F360ADCC511E, host id: KeNMPoHu/viheganHmBu8Gs7gvOpmJ/2IS3y4dbf6zWKQ8pDB+9A67F14eoWYgIrK1YgxoZJneuC
...

I'll take a closer look at the permissions policy info you liked to as well. Thanks again.

4

the endpoint used in the rclone config file must match the endpoint location of the bucket at wasabi.

5

It does. It looks to me like Wasabi's us-east-2 endpoint is referring me to s3.wasabisys.com.
us-east-2 is what I have in my configuration file and it is where my bucket is located.

image
image751×313 11.7 KB

Also, my settings are working for mounting the drive and transferring files that way.

6

i would use the endpoint that wasabi is telling you do use.

not sure the exact reason why that config works with mount, but not rclone move

with the rclone move, are you creating a new bucket or using an existing bucket?

and with my wasabi remotes, never used
region =

7

Yes, I am already using the endpoint that Wasabi told me to use. s3.us-east-2.wasabisys.com, which is what I have in my config file.

Whatever that referral is, it is automatic.

I am using an existing bucket. The directory on the remote is already existing.
Are you saying that I should pull the region setting out of the config file? The wizard put it there. I'll pull it out and see how that goes.

8

and i never used region in my wasabi remotes and i do not use the wizard.

type = s3
provider = Wasabi
access_key_id = 
secret_access_key = 
endpoint = s3.us-east-2.wasabisys.com
9

Oh boy! I'm confused.
The endpoint in my config should have my bucket name in it, or should it just say "endpoint = s3.us-east-2.wasabisys.com"?
It looks like the system is currently putting the directory name in automatically at the end of the domain:
Referrer: https://s3.us-east-2.wasabisys.com/jessedir...
Is there a distinction between my directory and my bucket name. jessedir is a directory that exists within, let's call it, jessebucket. Somehow, I've been thinking that a remote corresponds to a bucket, and that the bucket was basically just a root directory.

My apologies for all of the questions. I am a super-newb when it comes to S3 compliant storage and Rclone.

10

try

type = s3
provider = Wasabi
access_key_id = 
secret_access_key = 
endpoint = s3.wasabisys.com

that is not correct for s3.
there is no way to specify a bucket when creating a s3 remote.
by default, wasabi creates a root user and its corresponding id/key and has total permission over every bucket and file.

12

Gotcha. So when putting together a command such as move, the syntax for the remote path would be this?

[remote name]:[bucket name]/[dir on remote]

Am I getting that right?

13

yes, that is correct

remotename:jessebucket/jessedir

14

Thanks so much! I'll give that a try.

15

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.