[WebDAV] 401 Unauthorized When Server Redirects to Another Domain

Help and Support
1

What is the problem you are having with rclone?

Rclone gets 401 unauthorized http code when webdav server redirects to another domain.
Example: https://domain.name/dav/personal-files/device-detector-6.0.4.zip >>> https://my.tld/dav/personal-files/device-detector-6.0.4.zip

Run the command 'rclone version' and share the full output of the command.

rclone v1.62.2

  • os/version: almalinux 8.7 (64 bit)
  • os/kernel: 4.18.0-425.19.2.el8_7.x86_64 (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.20.2
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

WebDAV

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone -vvv --no-check-certificate=true --dump=headers copy "admin:/personal-files/device-detector-6.0.4.zip" /storage/tmp

The rclone config contents with secrets removed.

[admin]
type = webdav
url = https://origin.domain.name/dav/personal-files
vendor = other
user = admin
pass = ******

A log from the command with the -vv flag

2023/06/24 14:15:08 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "-vvv" "--no-check-certificate=true" "--dump=headers" "copy" "admin:/personal-files/device-detector-6.0.4.zip" "/storage/tmp"]
2023/06/24 14:15:08 DEBUG : Creating backend with remote "admin:/personal-files/device-detector-6.0.4.zip"
2023/06/24 14:15:08 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/06/24 14:15:08 DEBUG : found headers: 
2023/06/24 14:15:08 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : HTTP REQUEST (req 0xc00097eb00)
2023/06/24 14:15:08 DEBUG : PROPFIND /dav/personal-files/device-detector-6.0.4.zip HTTP/1.1
Host: origin.domain.name
User-Agent: rclone/v1.62.2
Authorization: XXXX
Depth: 1
Referer: https://origin.domain.name/dav/
Accept-Encoding: gzip

2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 DEBUG : HTTP RESPONSE (req 0xc00097eb00)
2023/06/24 14:15:08 DEBUG : HTTP/2.0 207 Multi-Status
Content-Length: 704
Cache-Control: no-cache
Cdn-Cachedat: 06/24/2023 17:15:08
Cdn-Edgestorageid: 883
Cdn-Proxyver: 1.03
Cdn-Pullzone: 1334033
Cdn-Requestcountrycode: NL
Cdn-Requestid: c9498c7c4e01558dceca229f04ff4de0
Cdn-Requestpullcode: 207
Cdn-Requestpullsuccess: True
Cdn-Status: 207
Cdn-Uid: 5356470d-341a-4aed-9205-a5ba8b33a88d
Content-Type: text/xml; charset=utf-8
Date: Sat, 24 Jun 2023 17:15:08 GMT
Server: BunnyCDN-AMS1-879

2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 DEBUG : fs cache: adding new entry for parent of "admin:/personal-files/device-detector-6.0.4.zip", "admin:personal-files"
2023/06/24 14:15:08 DEBUG : Creating backend with remote "/storage/tmp"
2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : HTTP REQUEST (req 0xc000576400)
2023/06/24 14:15:08 DEBUG : PROPFIND /dav/personal-files/device-detector-6.0.4.zip HTTP/1.1
Host: origin.domain.name
User-Agent: rclone/v1.62.2
Authorization: XXXX
Depth: 1
Referer: https://origin.domain.name/dav/
Accept-Encoding: gzip

2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 DEBUG : HTTP RESPONSE (req 0xc000576400)
2023/06/24 14:15:08 DEBUG : HTTP/2.0 207 Multi-Status
Content-Length: 704
Cache-Control: no-cache
Cdn-Cachedat: 06/24/2023 17:15:08
Cdn-Edgestorageid: 883
Cdn-Proxyver: 1.03
Cdn-Pullzone: 1334033
Cdn-Requestcountrycode: NL
Cdn-Requestid: 76914a0e0f755403a41811596c28527e
Cdn-Requestpullcode: 207
Cdn-Requestpullsuccess: True
Cdn-Status: 207
Cdn-Uid: 5356470d-341a-4aed-9205-a5ba8b33a88d
Content-Type: text/xml; charset=utf-8
Date: Sat, 24 Jun 2023 17:15:08 GMT
Server: BunnyCDN-AMS1-879

2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 DEBUG : device-detector-6.0.4.zip: Need to transfer - File not found at Destination
2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : HTTP REQUEST (req 0xc00097e300)
2023/06/24 14:15:08 DEBUG : GET /dav/personal-files/device-detector-6.0.4.zip HTTP/1.1
Host: origin.domain.name
User-Agent: rclone/v1.62.2
Authorization: XXXX
Depth: 0
Referer: https://origin.domain.name/dav/
Accept-Encoding: gzip

2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 DEBUG : HTTP RESPONSE (req 0xc00097e300)
2023/06/24 14:15:08 DEBUG : HTTP/2.0 302 Found
Content-Length: 145
Cache-Control: no-cache
Cdn-Cache: MISS
Cdn-Cachedat: 06/24/2023 17:15:08
Cdn-Edgestorageid: 883
Cdn-Proxyver: 1.03
Cdn-Pullzone: 1334033
Cdn-Requestcountrycode: NL
Cdn-Requestid: 50336236dae12fa06ff4b489b35fefd8
Cdn-Requestpullcode: 302
Cdn-Requestpullsuccess: True
Cdn-Status: 302
Cdn-Uid: 5356470d-341a-4aed-9205-a5ba8b33a88d
Content-Type: text/html
Date: Sat, 24 Jun 2023 17:15:08 GMT
Location: https://target.domain.name/dav/personal-files/device-detector-6.0.4.zip
Redirect-Ref: /dav/personal-files/device-detector-6.0.4.zip
Server: BunnyCDN-AMS1-879

2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : HTTP REQUEST (req 0xc0008cc200)
2023/06/24 14:15:08 DEBUG : GET /dav/personal-files/device-detector-6.0.4.zip HTTP/1.1
Host: target.domain.name
User-Agent: rclone/v1.62.2
Depth: 0
Referer: https://origin.domain.name/dav/personal-files/device-detector-6.0.4.zip
Accept-Encoding: gzip

2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 DEBUG : HTTP RESPONSE (req 0xc0008cc200)
2023/06/24 14:15:08 DEBUG : HTTP/2.0 401 Unauthorized
Content-Length: 14
Content-Type: text/plain; charset=utf-8
Date: Sat, 24 Jun 2023 17:15:08 GMT
Server: nginx/1.23.3
Www-Authenticate: Basic realm="Cells DAV"

2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 ERROR : device-detector-6.0.4.zip: Failed to copy: failed to open source object: Unauthorized.: 401 Unauthorized
2023/06/24 14:15:08 ERROR : Attempt 1/3 failed with 1 errors and: failed to open source object: Unauthorized.: 401 Unauthorized
2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : HTTP REQUEST (req 0xc0008cd100)
2023/06/24 14:15:08 DEBUG : PROPFIND /dav/personal-files/device-detector-6.0.4.zip HTTP/1.1
Host: origin.domain.name
User-Agent: rclone/v1.62.2
Authorization: XXXX
Depth: 1
Referer: https://origin.domain.name/dav/
Accept-Encoding: gzip

2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 DEBUG : HTTP RESPONSE (req 0xc0008cd100)
2023/06/24 14:15:08 DEBUG : HTTP/2.0 207 Multi-Status
Content-Length: 704
Cache-Control: no-cache
Cdn-Cachedat: 06/24/2023 17:15:08
Cdn-Edgestorageid: 883
Cdn-Proxyver: 1.03
Cdn-Pullzone: 1334033
Cdn-Requestcountrycode: NL
Cdn-Requestid: e1ae8ae61424aa95feb14df5099e2480
Cdn-Requestpullcode: 207
Cdn-Requestpullsuccess: True
Cdn-Status: 207
Cdn-Uid: 5356470d-341a-4aed-9205-a5ba8b33a88d
Content-Type: text/xml; charset=utf-8
Date: Sat, 24 Jun 2023 17:15:08 GMT
Server: BunnyCDN-AMS1-879

2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 DEBUG : device-detector-6.0.4.zip: Need to transfer - File not found at Destination
2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : HTTP REQUEST (req 0xc00097e600)
2023/06/24 14:15:08 DEBUG : GET /dav/personal-files/device-detector-6.0.4.zip HTTP/1.1
Host: origin.domain.name
User-Agent: rclone/v1.62.2
Authorization: XXXX
Depth: 0
Referer: https://origin.domain.name/dav/
Accept-Encoding: gzip

2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 DEBUG : HTTP RESPONSE (req 0xc00097e600)
2023/06/24 14:15:08 DEBUG : HTTP/2.0 302 Found
Content-Length: 145
Cache-Control: no-cache
Cdn-Cache: MISS
Cdn-Cachedat: 06/24/2023 17:15:08
Cdn-Edgestorageid: 883
Cdn-Proxyver: 1.03
Cdn-Pullzone: 1334033
Cdn-Requestcountrycode: NL
Cdn-Requestid: 3fa9d542bdc3ce60be9f94dbf9b49e0a
Cdn-Requestpullcode: 302
Cdn-Requestpullsuccess: True
Cdn-Status: 302
Cdn-Uid: 5356470d-341a-4aed-9205-a5ba8b33a88d
Content-Type: text/html
Date: Sat, 24 Jun 2023 17:15:08 GMT
Location: https://target.domain.name/dav/personal-files/device-detector-6.0.4.zip
Redirect-Ref: /dav/personal-files/device-detector-6.0.4.zip
Server: BunnyCDN-AMS1-879

2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : HTTP REQUEST (req 0xc00097e900)
2023/06/24 14:15:08 DEBUG : GET /dav/personal-files/device-detector-6.0.4.zip HTTP/1.1
Host: target.domain.name
User-Agent: rclone/v1.62.2
Depth: 0
Referer: https://origin.domain.name/dav/personal-files/device-detector-6.0.4.zip
Accept-Encoding: gzip

2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 DEBUG : HTTP RESPONSE (req 0xc00097e900)
2023/06/24 14:15:08 DEBUG : HTTP/2.0 401 Unauthorized
Content-Length: 14
Content-Type: text/plain; charset=utf-8
Date: Sat, 24 Jun 2023 17:15:08 GMT
Server: nginx/1.23.3
Www-Authenticate: Basic realm="Cells DAV"

2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 ERROR : device-detector-6.0.4.zip: Failed to copy: failed to open source object: Unauthorized.: 401 Unauthorized
2023/06/24 14:15:08 ERROR : Attempt 2/3 failed with 1 errors and: failed to open source object: Unauthorized.: 401 Unauthorized
2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : HTTP REQUEST (req 0xc0007ea000)
2023/06/24 14:15:08 DEBUG : PROPFIND /dav/personal-files/device-detector-6.0.4.zip HTTP/1.1
Host: origin.domain.name
User-Agent: rclone/v1.62.2
Authorization: XXXX
Depth: 1
Referer: https://origin.domain.name/dav/
Accept-Encoding: gzip

2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 DEBUG : HTTP RESPONSE (req 0xc0007ea000)
2023/06/24 14:15:08 DEBUG : HTTP/2.0 207 Multi-Status
Content-Length: 704
Cache-Control: no-cache
Cdn-Cachedat: 06/24/2023 17:15:08
Cdn-Edgestorageid: 883
Cdn-Proxyver: 1.03
Cdn-Pullzone: 1334033
Cdn-Requestcountrycode: NL
Cdn-Requestid: 8816d3b0d94756285ada535fed995e24
Cdn-Requestpullcode: 207
Cdn-Requestpullsuccess: True
Cdn-Status: 207
Cdn-Uid: 5356470d-341a-4aed-9205-a5ba8b33a88d
Content-Type: text/xml; charset=utf-8
Date: Sat, 24 Jun 2023 17:15:08 GMT
Server: BunnyCDN-AMS1-879

2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 DEBUG : device-detector-6.0.4.zip: Need to transfer - File not found at Destination
2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : HTTP REQUEST (req 0xc0007ea300)
2023/06/24 14:15:08 DEBUG : GET /dav/personal-files/device-detector-6.0.4.zip HTTP/1.1
Host: origin.domain.name
User-Agent: rclone/v1.62.2
Authorization: XXXX
Depth: 0
Referer: https://origin.domain.name/dav/
Accept-Encoding: gzip

2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 DEBUG : HTTP RESPONSE (req 0xc0007ea300)
2023/06/24 14:15:08 DEBUG : HTTP/2.0 302 Found
Content-Length: 145
Cache-Control: no-cache
Cdn-Cache: MISS
Cdn-Cachedat: 06/24/2023 17:15:08
Cdn-Edgestorageid: 883
Cdn-Proxyver: 1.03
Cdn-Pullzone: 1334033
Cdn-Requestcountrycode: NL
Cdn-Requestid: f61eb8e96d538e7e0f6a143f51438867
Cdn-Requestpullcode: 302
Cdn-Requestpullsuccess: True
Cdn-Status: 302
Cdn-Uid: 5356470d-341a-4aed-9205-a5ba8b33a88d
Content-Type: text/html
Date: Sat, 24 Jun 2023 17:15:08 GMT
Location: https://target.domain.name/dav/personal-files/device-detector-6.0.4.zip
Redirect-Ref: /dav/personal-files/device-detector-6.0.4.zip
Server: BunnyCDN-AMS1-879

2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : HTTP REQUEST (req 0xc0008cd300)
2023/06/24 14:15:08 DEBUG : GET /dav/personal-files/device-detector-6.0.4.zip HTTP/1.1
Host: target.domain.name
User-Agent: rclone/v1.62.2
Depth: 0
Referer: https://origin.domain.name/dav/personal-files/device-detector-6.0.4.zip
Accept-Encoding: gzip

2023/06/24 14:15:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 DEBUG : HTTP RESPONSE (req 0xc0008cd300)
2023/06/24 14:15:08 DEBUG : HTTP/2.0 401 Unauthorized
Content-Length: 14
Content-Type: text/plain; charset=utf-8
Date: Sat, 24 Jun 2023 17:15:08 GMT
Server: nginx/1.23.3
Www-Authenticate: Basic realm="Cells DAV"

2023/06/24 14:15:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/06/24 14:15:08 ERROR : device-detector-6.0.4.zip: Failed to copy: failed to open source object: Unauthorized.: 401 Unauthorized
2023/06/24 14:15:08 ERROR : Attempt 3/3 failed with 1 errors and: failed to open source object: Unauthorized.: 401 Unauthorized
2023/06/24 14:15:08 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.2s

2023/06/24 14:15:08 DEBUG : 6 go routines active
2023/06/24 14:15:08 Failed to copy: failed to open source object: Unauthorized.: 401 Unauthorized
2

The server issued a 302 redirect.

This is causing rclone (and the go stdlib) to drop the Authorization: header on the redirect, hence the 401 Unauthorized error.

This dropping the auth is done for security reasons and is generally a good idea however I think it should be preserved in this case probably.

Or maybe it should be an option.

Try this and see if it works

v1.63.0-beta.7086.5cc67a5ad.fix-webdav-redirect on branch fix-webdav-redirect (uploaded in 15-30 mins)

3

Hi. Thanks for your attention.
Now it sends authentication header but got the following error:

[...]
Content-Length: 145
Cache-Control: no-cache
Cdn-Cache: MISS
Cdn-Cachedat: 06/25/2023 15:10:47
Cdn-Edgestorageid: 883
Cdn-Proxyver: 1.03
Cdn-Pullzone: 1334033
Cdn-Requestcountrycode: NL
Cdn-Requestid: e8e85695d31ef2779a11f50d3ec982a1
Cdn-Requestpullcode: 302
Cdn-Requestpullsuccess: True
Cdn-Status: 302
Cdn-Uid: 5356470d-341a-4aed-9205-a5ba8b33a88d
Content-Type: text/html
Date: Sun, 25 Jun 2023 15:10:47 GMT
Location: https://target.domain.name/dav/personal-files/device-detector-6.0.4.zip
Redirect-Ref: /dav/personal-files/device-detector-6.0.4.zip
Server: BunnyCDN-AMS1-879
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Transferred:            0 / 1, 0%
Elapsed time:         0.4s
Transferring:
 *                     device-detector-6.0.4.zip: transferring
2023-06-25 12:10:47 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Transferred:            0 / 1, 0%
Elapsed time:         0.4s
Transferring:
 *                     device-detector-6.0.4.zip: transferringfatal error: sync: RUnlock of unlocked RWMutex
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0xac53d8]

goroutine 1 [running]:
sync.fatal({0x1ff6533?, 0xc000b24a20?})
	runtime/panic.go:1031 +0x1e
sync.(*RWMutex).rUnlockSlow(0xc000336730, 0x18?)
	sync/rwmutex.go:129 +0x39
sync.(*RWMutex).RUnlock(0x100c000b10448?)
	sync/rwmutex.go:119 +0x29
panic({0x1c1cd40, 0x3433620})
	runtime/panic.go:884 +0x213
github.com/rclone/rclone/lib/rest.ClientWithAuthRedirects.func1(0xc000258200, {0xc000012090?, 0x1fa42da?, 0x7?})
	github.com/rclone/rclone/lib/rest/rest.go:224 +0x38
net/http.(*Client).checkRedirect(...)
	net/http/client.go:498
net/http.(*Client).do(0xc0006c4300, 0xc000258200)
	net/http/client.go:682 +0x77b
net/http.(*Client).Do(...)
	net/http/client.go:582
github.com/rclone/rclone/lib/rest.(*Client).Call(0xc000336730, {0x24dfbf0, 0xc000124010}, 0xc0003b2000)
	github.com/rclone/rclone/lib/rest/rest.go:343 +0xcbd
github.com/rclone/rclone/backend/webdav.(*Object).Open.func1()
	github.com/rclone/rclone/backend/webdav/webdav.go:1374 +0x58
github.com/rclone/rclone/fs.pacerInvoker(0xc0006263c0?, 0xc0006d2f80?, 0x40?)
	github.com/rclone/rclone/fs/pacer.go:88 +0x38
github.com/rclone/rclone/lib/pacer.(*Pacer).call(0xc0006263c0, 0xc000b252b0?, 0xa)
	github.com/rclone/rclone/lib/pacer/pacer.go:197 +0x85
github.com/rclone/rclone/lib/pacer.(*Pacer).Call(0xc0006263c0, 0xc0006c42a0?)
	github.com/rclone/rclone/lib/pacer/pacer.go:216 +0x90
github.com/rclone/rclone/backend/webdav.(*Object).Open(0xc000627260, {0x24dfbf0?, 0xc000124010}, {0xc0001b3360, 0x1, 0x1})
	github.com/rclone/rclone/backend/webdav/webdav.go:1373 +0x2dd
github.com/rclone/rclone/fs/operations.(*ReOpen).open(0xc0006ca000)
	github.com/rclone/rclone/fs/operations/reopen.go:116 +0x543
github.com/rclone/rclone/fs/operations.NewReOpen({0x24dfbf0?, 0xc000124010}, {0x24f33f0?, 0xc000627260}, 0xa, {0xc0001b3320, 0x1, 0x1})
	github.com/rclone/rclone/fs/operations/reopen.go:49 +0x19d
github.com/rclone/rclone/fs/operations.Open({0x24dfbf0, 0xc000124010}, {0x24f33f0, 0xc000627260}, {0xc0001b3320, 0x1, 0x1})
	github.com/rclone/rclone/fs/operations/reopen.go:71 +0x6e
github.com/rclone/rclone/fs/operations.Copy({0x24dfbf0, 0xc000124010}, {0x24f1da0, 0xc0005b06e0}, {0x0?, 0x0?}, {0x7ffd3ac65571, 0x19}, {0x24f33f0, 0xc000627260})
	github.com/rclone/rclone/fs/operations/operations.go:446 +0x1c75
github.com/rclone/rclone/fs/operations.moveOrCopyFile({0x24dfbf0, 0xc000124010}, {0x24f1da0, 0xc0005b06e0}, {0x24f3380?, 0xc000a9f7a0?}, {0x7ffd3ac65571, 0x19}, {0x7ffd3ac65571, 0x19}, ...)
	github.com/rclone/rclone/fs/operations/operations.go:2027 +0xb38
github.com/rclone/rclone/fs/operations.CopyFile(...)
	github.com/rclone/rclone/fs/operations/operations.go:2047
github.com/rclone/rclone/cmd/copy.glob..func1.1()
	github.com/rclone/rclone/cmd/copy/copy.go:94 +0x87
github.com/rclone/rclone/cmd.Run(0x1, 0x1, 0x2?, 0xc000b25cd8)
	github.com/rclone/rclone/cmd/cmd.go:257 +0x13b
github.com/rclone/rclone/cmd/copy.glob..func1(0x34505e0?, {0xc0003819e0, 0x2, 0x6})
	github.com/rclone/rclone/cmd/copy/copy.go:90 +0xc5
github.com/spf13/cobra.(*Command).execute(0x34505e0, {0xc000381980, 0x6, 0x6})
	github.com/spf13/cobra@v1.7.0/command.go:944 +0x847
github.com/spf13/cobra.(*Command).ExecuteC(0x344b280)
	github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3bd
github.com/spf13/cobra.(*Command).Execute(...)
	github.com/spf13/cobra@v1.7.0/command.go:992
github.com/rclone/rclone/cmd.Main()
	github.com/rclone/rclone/cmd/cmd.go:567 +0x76
main.main()
	github.com/rclone/rclone/rclone.go:14 +0x17

goroutine 49 [select]:
go.opencensus.io/stats/view.(*worker).start(0xc000634000)
	go.opencensus.io@v0.24.0/stats/view/worker.go:292 +0xad
created by go.opencensus.io/stats/view.init.0
	go.opencensus.io@v0.24.0/stats/view/worker.go:34 +0x96

goroutine 65 [syscall]:
os/signal.signal_recv()
	runtime/sigqueue.go:152 +0x2f
os/signal.loop()
	os/signal/signal_unix.go:23 +0x19
created by os/signal.Notify.func1.1
	os/signal/signal.go:151 +0x2a

goroutine 66 [chan receive]:
github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler.func1()
	github.com/rclone/rclone/fs/accounting/accounting_unix.go:25 +0x2d
created by github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler
	github.com/rclone/rclone/fs/accounting/accounting_unix.go:22 +0xb9

goroutine 40 [IO wait]:
internal/poll.runtime_pollWait(0x7f97ff9ab8a0, 0x72)
	runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc000137780?, 0xc0006ec000?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x32
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000137780, {0xc0006ec000, 0x1300, 0x1300})
	internal/poll/fd_unix.go:167 +0x299
net.(*netFD).Read(0xc000137780, {0xc0006ec000?, 0x416230?, 0xc0006ec25b?})
	net/fd_posix.go:55 +0x29
net.(*conn).Read(0xc00062c098, {0xc0006ec000?, 0xc0006ec25b?, 0x1a?})
	net/net.go:183 +0x45
github.com/rclone/rclone/fs/fshttp.(*timeoutConn).Read(0xc00012e048, {0xc0006ec000?, 0xc000010960?, 0x18?})
	github.com/rclone/rclone/fs/fshttp/dialer.go:104 +0x32
crypto/tls.(*atLeastReader).Read(0xc000010960, {0xc0006ec000?, 0xc000010960?, 0x0?})
	crypto/tls/conn.go:788 +0x3d
bytes.(*Buffer).ReadFrom(0xc000190290, {0x24c32a0, 0xc000010960})
	bytes/buffer.go:202 +0x98
crypto/tls.(*Conn).readFromUntil(0xc000190000, {0x7f97ff835318?, 0xc00012e048}, 0x10aa?)
	crypto/tls/conn.go:810 +0xe5
crypto/tls.(*Conn).readRecordOrCCS(0xc000190000, 0x0)
	crypto/tls/conn.go:617 +0x116
crypto/tls.(*Conn).readRecord(...)
	crypto/tls/conn.go:583
crypto/tls.(*Conn).Read(0xc000190000, {0xc000b0f000, 0x1000, 0x7237a0?})
	crypto/tls/conn.go:1316 +0x16f
bufio.(*Reader).Read(0xc000b0a180, {0xc00036c3c0, 0x9, 0x741425?})
	bufio/bufio.go:237 +0x1bb
io.ReadAtLeast({0x24c2f80, 0xc000b0a180}, {0xc00036c3c0, 0x9, 0x9}, 0x9)
	io/io.go:332 +0x9a
io.ReadFull(...)
	io/io.go:351
net/http.http2readFrameHeader({0xc00036c3c0?, 0x9?, 0xc0008171d0?}, {0x24c2f80?, 0xc000b0a180?})
	net/http/h2_bundle.go:1567 +0x6e
net/http.(*http2Framer).ReadFrame(0xc00036c380)
	net/http/h2_bundle.go:1831 +0x95
net/http.(*http2clientConnReadLoop).run(0xc000abcf98)
	net/http/h2_bundle.go:9187 +0x12e
net/http.(*http2ClientConn).readLoop(0xc0003c0300)
	net/http/h2_bundle.go:9082 +0x6f
created by net/http.(*http2Transport).newClientConn
	net/http/h2_bundle.go:7779 +0xc3c

goroutine 78 [select]:
github.com/rclone/rclone/cmd.startProgress.func3()
	github.com/rclone/rclone/cmd/progress.go:57 +0x107
created by github.com/rclone/rclone/cmd.startProgress
	github.com/rclone/rclone/cmd/progress.go:49 +0x14d

goroutine 24 [select]:
github.com/rclone/rclone/fs/accounting.(*StatsInfo).averageLoop(0xc0002b3180)
	github.com/rclone/rclone/fs/accounting/stats.go:322 +0x165
created by github.com/rclone/rclone/fs/accounting.(*StatsInfo).startAverageLoop.func1
	github.com/rclone/rclone/fs/accounting/stats.go:351 +0x70

goroutine 25 [chan receive]:
github.com/rclone/rclone/lib/atexit.Register.func1.1()
	github.com/rclone/rclone/lib/atexit/atexit.go:45 +0x2f
created by github.com/rclone/rclone/lib/atexit.Register.func1
	github.com/rclone/rclone/lib/atexit/atexit.go:44 +0x6b

goroutine 26 [sleep]:
time.Sleep(0x989680)
	runtime/time.go:195 +0x135
github.com/rclone/rclone/lib/pacer.(*Pacer).beginCall.func1(0x0?)
	github.com/rclone/rclone/lib/pacer/pacer.go:167 +0x25
created by github.com/rclone/rclone/lib/pacer.(*Pacer).beginCall
	github.com/rclone/rclone/lib/pacer/pacer.go:166 +0xf7
4

I see, there is a bug in that.

Try this

v1.63.0-beta.7086.029a0860b.fix-webdav-redirect on branch fix-webdav-redirect (uploaded in 15-30 mins)

5

It still doesn't work. I got 401 unauthorized again.
For some reason, resp.Response is empty, but prevReq.Header.Get("Authorization") is not.
If I replace the variable "resp" with "req" it works, and I'm able to download the file.
Apparently the current request contains the http code 302.

		//resp := prevReq.Response
		switch req.Response.StatusCode {
		case http.StatusMovedPermanently, http.StatusFound, http.StatusSeeOther, http.StatusTemporaryRedirect, http.StatusPermanentRedirect:
			// Reapply Auth (if any) from previous request on redirect
			auth := prevReq.Header.Get("Authorization")
			if auth != "" {
				req.Header.Add("Authorization", auth)
			}
		}
6

Looks like I've misunderstood how CheckRedirect works! I double checked in the Go source and your change is correct.

Hopefully this will work now

v1.63.0-beta.7086.62284c5e6.fix-webdav-redirect on branch fix-webdav-redirect (uploaded in 15-30 mins)

If it does then I think we should discuss whether this should be the default behaviour.

Some webdav servers might redirect to 3rd party servers which don't need auth (eg s3 with a limited time token) and passing your auth to a 3rd party server would be wrong in this case and might even leak your credentials.

Thoughts on this?

BTW does this service work with other WevDAV clients? I guess that tells us whether it should be the default in rclone or not.

7

It works. It's fixed.
In this case, specifically, the webdav server is Pydio Cells v4. However, the redirect is done by the proxy server (Nginx) and not by the webdav server. It works with Cyberduck webdav client.
I think it should be disabled by default.
One can turn it on with --webdav-auth-redirect=true or auth_redirect = true in rclone.conf

8

I've added the flag as you suggested - please give this a go

  --webdav-auth-redirect   Preserve authentication on redirect

This can also go in the config file as auth_redirect = true

v1.63.0-beta.7104.572516e30.fix-webdav-redirect on branch fix-webdav-redirect (uploaded in 15-30 mins)

Let me know what you think!

Thanks

9

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.

10

I've merged this to master now which means it will be in the latest beta in 15-30 minutes and released in v1.69

1 Like