If you are having issues with WebDAV multi-connection downloads for a remote behind Cloudflare
Related to this post (can't reply to it anymore): Webdav server behind cloudflare fails to download correctly
Some users were having issues downloading files from my webDAV server. Some of the issues were:
- Continuous downloading, files never stopped downloading, going from 100% completion to 200, 300% until the download was stopped. This was a huge waste of bandwidth.
- Inconsistent downloads. Some downloads would download just fine, some would have the above issue.
We followed some of the debugging troubleshooting tips to view the headers and realized that using a direct connection, everything was working fine requests had the proper headers, and chunks got properly. With Cloudflare in front of the webDAV server, it would remove the Content-Range header and replace it with a Content-Length header, which Rclone wasn't looking for. Since the range wasn't there, it just downloaded forever.
It turns out that Cloudflare cached the file from the server and served it directly. Usually this would be fine, but in this case, we needed to get only the correct range from the server.
The solution ended up being disabling caching on the hostname the webDAV server was hosted on. This happened to be: https://webdav.xxx.xxx. As soon as we changed this option on the Cloudflare dashboard (by going to Caching->Cache Rules->Create Rule), Cloudflare passed the correct headers which were being given by the origin.