Strange errors from webdav on cpanel shared hosting service

What is the problem you are having with rclone?

Webdav remote is returning 400

output from rclone version

rclone v1.53.2
 os/arch: (debian)linux/amd64
 go version: go1.15.3

The command you were trying to run (eg rclone copy /tmp remote:tmp)

400 error breaks ncdu and size

The rclone config contents with secrets removed.

--------------------
[example-com]
type = webdav
url = https://vr6.websitewelcome.com:2078
vendor = other
user = web@example.com
pass = *** ENCRYPTED ***
--------------------

A log from the command with the -vv flag

2020/11/29 18:06:16 DEBUG : rclone: Version "v1.53.2" starting with parameters ["rclone" "size" "-vv" "example-com:public_html/magento/skin/frontend/default"]
2020/11/29 18:06:16 DEBUG : Using config file from "/home/user/.config/rclone/rclone.conf"
2020/11/29 18:06:16 DEBUG : Creating backend with remote "example-com:public_html/magento/skin/frontend/default"
2020/11/29 18:06:18 ERROR : german/images: error listing: couldn't list files: z7_: 400 Bad Request
2020/11/29 18:06:18 ERROR : blank/images/catalog/product: error listing: couldn't list files: z7_: 400 Bad Request
2020/11/29 18:06:19 DEBUG : 2 go routines active
2020/11/29 18:06:19 Failed to size with 3 errors: last error was: couldn't list files: z7_: 400 Bad Request

An excerpt of log from --dump bodies

2020/11/29 18:15:45 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2020/11/29 18:15:45 DEBUG : HTTP REQUEST (req 0xc00081c200)
2020/11/29 18:15:45 DEBUG : PROPFIND /public_html/magento/skin/frontend/default/blank/images/catalog/product/placeholder/ HTTP/1.1^M
Host: vr6.websitewelcome.com:2078^M
User-Agent: rclone/v1.53.2^M
Authorization: XXXX
Depth: 1^M
Referer: https://vr6.websitewelcome.com:2078/^M
Accept-Encoding: gzip^M
^M
2020/11/29 18:15:45 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2020/11/29 18:15:45 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2020/11/29 18:15:45 DEBUG : HTTP RESPONSE (req 0xc00081c200)
2020/11/29 18:15:45 DEBUG : HTTP/1.1 400 Bad Request^M
Connection: close^M
Cache-Control: no-cache, no-store, must-revalidate, private^M
Date: Mon, 30 Nov 2020 00:15:45 GMT^M
Expires: Fri, 01 Jan 1990 00:00:00 GMT^M
Host: vr6.websitewelcome.com:2078^M
Persistent-Auth: false^M
Server: cPanel^M
Vary: Accept-Encoding^M
^M
^_<8b>^H^@±9Ä_^@^C^C^@^@^@^@^@^@^@^@^@
2020/11/29 18:15:45 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Notes

The error does not persist if I descend into the directories mentioned. Trying to list all files gives many dozens of errors. Perhaps it is related to the total number of objects.
--tpslimit 16 alleviates the error. What is wrong with my host?

Not sure!

What WebDAV server is it?

It doesn't seem to like PROPFIND on the path given. I can't see anything unusual about that request.

Does rclone lsf "example-com:public_html/magento/skin/frontend/default" work OK?

Do the same directories always go wrong? If so maybe it is some security setting on the server?

I don't know. The account is managed through cpanel on a shared hosting service, hostgator. It's a crappy service and is not developer (or user) friendly.
lsf works there, yes. The error does not present consistently. If I descend into sub directories with fewer items, the error goes away. If I use --tpslimit < 20 the error goes away. It might be more rare as the --tpslimit approaches 20, actually.

I would seem it's all their fault though. Support is atrocious, so I don't think there's anywhere to go on this unless someone wants a test account from me.

Thanks for quick reply, Nick! I love rclone and want to learn more about it.

So that seems like it is actually some form of rate limiting. 400 isn't the normal error response for rate limiting (that would be 423) but it could be.

If it is rate limiting and you can find a --tpslimit that goes away then I think you'll just have to live with using that.

Rclone is quite aggressive and can easily 100s or 1000s or transactions per second if you let it!

:slight_smile: Glad you are enjoying rclone. Hanging around in the forums is a good way of learning more stuff, or head over to GitHub and look at code/issues/pull requests!

1 Like

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.