Authentification issues with webdav server

Help and Support
1

What is the problem you are having with rclone?

I cannot authenticate with a webdav server for one path, but it works for another.

What is your rclone version (output from rclone version)

1.53.3

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Arch Linux, 64 bits

Which cloud storage system are you using? (eg Google Drive)

Other

The command you were trying to run (eg rclone copy /tmp remote:tmp)

RCLONE_WEBDAV_PASS="$(pass show passname | rclone obscure -)" rclone lsf skeletons:

The rclone config contents with secrets removed.

[asm]
type = webdav
url = https://****/asm
vendor = other
user = ****

[skeletons]
type = webdav
url = https://****/skeletons
vendor = other
user = ****

A log from the command with the -vv flag

➜ RCLONE_WEBDAV_PASS="$(pass show passname | rclone obscure -)" rclone lsf skeletons: -vv --dump bodies,auth --low-level-retries 1 --retries 1
2021/01/26 11:46:46 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "lsf" "skeletons:" "-vv" "--dump" "bodies,auth" "--low-level-retries" "1" "--retries" "1"]
2021/01/26 11:46:46 DEBUG : Using config file from "/home/schmitta/.config/rclone/rclone.conf"
2021/01/26 11:46:46 DEBUG : Creating backend with remote "skeletons:"
2021/01/26 11:46:46 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2021/01/26 11:46:46 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/01/26 11:46:46 DEBUG : HTTP REQUEST (req 0xc000190500)
2021/01/26 11:46:46 DEBUG : PROPFIND /skeletons/ HTTP/1.1
Host: ****
User-Agent: rclone/v1.53.3
Authorization: Basic ****
Depth: 1
Referer: https://****/skeletons/
Accept-Encoding: gzip

2021/01/26 11:46:46 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/01/26 11:46:47 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/01/26 11:46:47 DEBUG : HTTP RESPONSE (req 0xc000190500)
2021/01/26 11:46:47 DEBUG : HTTP/1.1 401 Unauthorized
Content-Length: 381
Content-Type: text/html; charset=iso-8859-1
Date: Tue, 26 Jan 2021 10:46:46 GMT
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips
Www-Authenticate: Basic realm="**** iLDAP"

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>401 Unauthorized</title>
</head><body>
<h1>Unauthorized</h1>
<p>This server could not verify that you
are authorized to access the document
requested.  Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.</p>
</body></html>
2021/01/26 11:46:47 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/01/26 11:46:47 ERROR : : error listing: couldn't list files: 401 Unauthorized
2021/01/26 11:46:47 DEBUG : 4 go routines active
2021/01/26 11:46:47 Failed to lsf with 2 errors: last error was: error in ListJSON: couldn't list files: 401 Unauthorized

The strange thing is that the same command works with another folder (called asm).

The even stranger thing is that I can run rclone ls asm: without providing any password. Is there some authentication caching happening?

2

Try putting the root webdav server in the config

Then putting the path on the command line

rclone lsf root:
rclone lsf root:asm
rclone lsf root:skeleton

Does that help? That is the recomended way of setting up webdav. You can then make aliases asm and skeletons if you want.

3

Thank you for the suggestion. I can access the root folder, as well as the asm folder (without giving the password, which is surprising), but I cannot access the skeletons folder.

I have checked in my browser that I have access… so I'm really confused as to what is happening. In particular, how can I authenticate without providing a password?

4

I have additional information. In the asm folder, there is not .htupdate file. In the skeletons folder, there is an .htupdate file with contents

Require user *****

Is there a special configuration I need to do to deal with such htupdate file?

5

I think that is giving a list of allowed users isn't it? Is your user on the list?

6

Yes, and yes. I can log in with Firefox and davfs2, but not rclone. I don't know what I'm doing wrong.

I guess the server I'm logging to does not have to be secret, it's a https serveur called files from inria (tld fr) at port 8443 and folder asm for the webdav access. (Sorry for the circumlocution, new user here).

7

It looks to me like the user or password is wrong. Maybe your setting of RCLONE_WEBDAV_PASS isn't working. Try putting the password in the config file and see if that works.

8

Thank you for the suggestion: putting the password in the config file works.

So I guess my problem is different. Is the password stored in the configuration supposed to be a rclone obscure of the real password? (Later: I guess not, rclone obscure always return something different.)

My issue is then that

RCLONE_WEBDAV_PASS="$(echo '****' | rclone obscure -)" rclone ls root:asm

does not work even when **** is the password I entered in rclone config.

9

Yes it is

That is expected - you get a different value each time you run rclone obscure.

Ah, try echo -n so it doesn't put a newline on the end - I wonder if that is the problem?

RCLONE_WEBDAV_PASS="$(echo -n '****' | rclone obscure -)"
10

That was it! I now have a working solution with pass. I guess I can do better that this, but at least it works:

RCLONE_WEBDAV_PASS="$(pass show passname | head -n 1 | head -c -1 | rclone obscure -)" rclone ls root:asm

Thanks a lot for your help.

11

Rclone obscure should probably ignore the newline at the end of the password - that would fix the issue too.

Try this

v1.54.0-beta.5103.9656c15e0.fix-obscure on branch fix-obscure (uploaded in 15-30 mins)

That should work with your original command

12

I confirm it works. Thanks a lot!

(Later) Looking at the code, I see I also won't need the head -n 1 as it will use the first line for the password. This is great.

13

I've merged this to master now which means it will be in the latest beta in 15-30 mins and released in v1.54