What is the problem you are having with rclone?
I can't upload large files to our company Owncloud, because they are uploaded as single file not in chunks.
Without chunks the reverse proxy Apache 2.5.54+ restricts the max uploadable filesize to 1073741824 bytes = 1GiB.
This is set via LimitRequestBody which was previously unlimited, unlimited is and would open a vulnerability, so changing it back is not an option (which was for several ppl the applied fix).
The upload stops after a few MiB with the error message "413 Request Entity Too Large" and checksum didn't match the client.
Because of the detailed message "The requested resource does not allow request data with PUT requests, or the amount of data provided in the request exceeds the capacity limit." I suspect I could fix it with a chunked upload.
Related post which changed the reverse proxy config, therefore accepted the vulnerability opened by it: Copy files to owncloud leads in checksum mismatch - #20 by BentiGorlich
Related sources:
Apache HTTP Server 2.4 vulnerabilities - The Apache HTTP Server Project „low: core: Possible buffer overflow with very large or unlimited LimitXMLRequestBody” (not exactly LimitRequestBody, but probably identical issue)
Big File Upload Configuration core - Apache HTTP Server Version 2.4
If you have the same issue, you can test this on linux like so, assuming "owncloud" remote is configured:
truncate --size "1073741824" "upload_works.file"
rclone copy -vv "upload_works.file" "owncloud:TestFiles/"
truncate --size "1073741825" "upload_fails.file"
rclone copy -vv upload_fails.file "owncloud:TestFiles/"
What is my question?
Probably rclone doesn't support chunk wise uploading with App tokens? Or does it and the server is missing a configuration?
- It seems like Owncloud does support chunks core/tests/acceptance/features/bootstrap/WebDav.php at master · owncloud/core · GitHub
- The configuration for nextcloud seems to be identical: WebDAV
- There are options for rclone / webdav / chunks but only for Nextcloud --webdav-nextcloud-chunk-size but I cant try them on owncloud because url regex prevents it, well its not a nextcloud url.
- I contacted our servicedesk regarding it, but I will get only information about owncloud point of view not rclone.
rclone version
rclone v1.67.0
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 5.15.0-112-generic (x86_64)
- os/type: linux (same tested on windows)
- os/arch: amd64
- go/version: go1.22.4
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
WebDav / Owncloud of our company
The command you were trying to run (eg rclone copy /tmp remote:tmp)
truncate --size "1073741825" "upload_fails.file" # creates a file of size 1GiB
rclone copy -vv upload_fails.file "owncloud:TestFiles/"
rclone config redacted
[owncloud]
type = webdav
url = https://owncloud.company.de/remote.php/nonshib-webdav
vendor = owncloud
user = XXX
pass = XXX
### Double check the config for sensitive info before posting publicly
A log from the command that you were trying to run with the -vv flag
The log file is with files of 900MB and 1100MB, but the results are identical, it would just cost additional time to rerun it without a benefit.
Thanks for your time!