S3 force-path-style option is ignored

What is the problem you are having with rclone?

The "force path style" option is ignored or being overridden by some other default. Virtual host style is used regardless of what the option is, whether it was set in a remote config, or whether it was specified on the command line. I saw that there's a forced override if the provider is AWS in this post, but the problem does not go away if the provider is set to "other" instead of "AWS", the default.

Run the command 'rclone version' and share the full output of the command.

This was tested on the following versions:

From Gentoo Portage:

- os/version: gentoo 2.9 (64 bit)
- os/kernel: 5.10.158-s (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.19.2
- go/linking: dynamic
- go/tags: none

After rclone selfupdate:

- os/version: gentoo 2.9 (64 bit)
- os/kernel: 5.10.158-s (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.19.4
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Generic S3; QuObjects on QNAP NAS

The command you were trying to run (eg rclone copy /tmp remote:tmp)

This command was run with force-style-path in rclone config set to: Default (blank), False, and True:

rclone ls nas_blobs:ldvr

Then, the command was run with each of the following additional flags, with the same result.

--s3-force-path-style=false
--s3-force-path-style=true
--dump=bodies

The rclone config contents with secrets removed.

- type: s3
- provider: AWS
- env_auth: false
- access_key_id: blobs:SCRUBBED
- secret_access_key: SCRUBBED
- endpoint: http://SCRUBBED_IPv4_ADDRESS:8010
- max_upload_parts: 2048
- force_path_style: true

A log from the command with the -vv flag

r000t@chronos ~$ rclone ls nas_blobs:ldvr --s3-force-path-style=true -vv
2023/01/01 21:21:43 DEBUG : rclone: Version "v1.61.1" starting with parameters ["rclone" "ls" "nas_blobs:ldvr" "--s3-force-path-style=true" "-vv"]
2023/01/01 21:21:43 DEBUG : Creating backend with remote "nas_blobs:ldvr"
2023/01/01 21:21:43 DEBUG : Using config file from "/home/r000t/.config/rclone/rclone.conf"
2023/01/01 21:21:43 DEBUG : nas_blobs: detected overridden config - adding "{ladgk}" suffix to name
2023/01/01 21:21:43 DEBUG : fs cache: renaming cache item "nas_blobs:ldvr" to be canonical "nas_blobs{ladgk}:ldvr"
2023/01/01 21:22:34 DEBUG : 2 go routines active
2023/01/01 21:22:34 Failed to ls: RequestError: send request failed
caused by: Get "http://ldvr.SCRUBBED_IPv4_ADDRESS:8010/?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp: lookup ldvr.SCRUBBED_IPv4_ADDRESS on 8.8.8.8:53: no such host
r000t@chronos ~ $ rclone ls nas_blobs:ldvr --s3-force-path-style=false -vv
2023/01/01 21:24:08 DEBUG : rclone: Version "v1.61.1" starting with parameters ["rclone" "ls" "nas_blobs:ldvr" "--s3-force-path-style=false" "-vv"]
2023/01/01 21:24:08 DEBUG : Creating backend with remote "nas_blobs:ldvr"
2023/01/01 21:24:08 DEBUG : Using config file from "/home/r000t/.config/rclone/rclone.conf"
2023/01/01 21:24:08 DEBUG : nas_blobs: detected overridden config - adding "{0kRUr}" suffix to name
2023/01/01 21:24:08 DEBUG : fs cache: renaming cache item "nas_blobs:ldvr" to be canonical "nas_blobs{0kRUr}:ldvr"
2023/01/01 21:24:54 DEBUG : 2 go routines active
2023/01/01 21:24:54 Failed to ls: RequestError: send request failed
caused by: Get "http://ldvr.SCRUBBED_IPv4_ADDRESS:8010/?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp: lookup ldvr.SCRUBBED_IPv4_ADDRESS on 8.8.8.8:53: no such host

Running with --dump=bodies reflects the problem a bit faster, meaning you won't need to wait a full minute:

2023/01/01 21:27:22 DEBUG : Error: dial tcp: lookup ldvr.SCRUBBED_IPv4_ADDRESS on 8.8.8.8:53: no such host

Regardless of the state of the setting, rclone appears to force the use of virtual-host style paths, which just isn't going to work in this case; it's not supported by the S3 backend being used, and we are connecting directly to an IPv4 address; Those do not have subdomains.

Oddly enough, the desired behavior is supposedly the "default" in rclone, meaning rclone isn't even acting as the defaults would suggest.

Choose provider = Other - you aren't using AWS here.

I thought the application was continuing to use the domain style after I had changed the remote to use the Other provider, but as it turns out, --s3-force-path-style was set to False at the command line when I was testing it.

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.