413 Request Entity Too Large - Chunker, Crypt, OneDrive

What is the problem you are having with rclone?

Hi, I'm trying to upload many files to OneDrive using Crypt and Chunker.
My config:
Chunker -> Crypt -> OneDrive
I'm getting an error:

Failed to copy: 413 Request Entity Too Large:

which is weird, I'm using chunker to avoid such problems and it seems to appear only for files >= 2G. OneDrive support files up to 250G.

Can you please look into my setup?

Run the command 'rclone version' and share the full output of the command.

rclone v1.57.0
- os/version: qts 5.0.0 (20220324) (64 bit)
- os/kernel: 5.10.60-qnap (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.17.2
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

OneDrive for Business

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync /share/homes/ OneDrive-Crypt-Chunker:qnap/backups/homes/ --transfers=15 --checkers=15 --bwlimit 80M --progress --stats=1s --config=/share/homes...  --min-size 2G

The rclone config contents with secrets removed.

[OneDrive]
type = onedrive
client_id = <secret>
client_secret = <secret>
token = {"access_token":"<secret>"}
drive_id = <secret>
drive_type = business
chunk_size = 144834560

[OneDrive-Crypt]
type = crypt
remote = OneDrive:ENC-C/
password = <secret>
password2 = <secret>
filename_encryption = obfuscate

[OneDrive-Crypt-Chunker]
type = chunker
remote = OneDrive-Crypt:
#chunk_size = 96Mi
chunk_size = 64Gi
hash_type = md5quick

A log from the command with the -vv flag

2022-05-14 11:25:21 ERROR : admin: failed to open directory "admin": open /share/homes/admin: permission denied
2022-05-14 11:25:21 NOTICE: <secret>/@Recycle: Can't follow symlink without -L/--copy-links
2022-05-14 11:25:21 NOTICE: kamil/@Recycle: Can't follow symlink without -L/--copy-links
2022-05-14 11:25:28 ERROR : kamil/<secret>: Failed to copy: 413 Request Entity Too Large:
2022-05-14 11:25:28 ERROR : kamil/<secret>: Failed to copy: 413 Request Entity Too Large:
2022-05-14 11:25:28 ERROR : kamil/<secret>: Failed to copy: 413 Request Entity Too Large:
2022-05-14 11:25:29 ERROR : kamil/<secret>: Failed to copy: 413 Request Entity Too Large:
2022-05-14 11:25:29 ERROR : kamil/<secret>: Failed to copy: 413 Request Entity Too Large:
2022-05-14 11:25:30 ERROR : kamil/<secret>: Failed to copy: 413 Request Entity Too Large:
2022-05-14 11:25:30 ERROR : kamil/<secret>: Failed to copy: 413 Request Entity Too Large:
2022-05-14 11:25:30 ERROR : kamil/<secret>: Failed to copy: 413 Request Entity Too Large:

You have your chunk size at 64G which. I'd assume you want your chunk size less than 2G.

Not really, I want to use 64G or even 128G because my Internet Connection is 1Gpbs. I just want to get over OneDrive 250G file limit.
What's interesting is that, even if I have default 2G chunk_size I still get that error.

2nd bit would be to update to the latest and validate it still persists as you are a few revs behind.

And any info here might be applicable as well:

OneDrive + WebDAV: 413 Request Entity Too Large - Help and Support - rclone forum

I try latest stable version and latest beta release. Problem still persist.

rclone --version
rclone v1.58.1
- os/version: qts 5.0.0 (20220324) (64 bit)
- os/kernel: 5.10.60-qnap (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.17.9
- go/linking: static
- go/tags: none

rclone --version
rclone v1.59.0-beta.6142.542c1616b
- os/version: qts 5.0.0 (20220324) (64 bit)
- os/kernel: 5.10.60-qnap (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.18.2
- go/linking: static
- go/tags: none

OneDrive limits:

Ok, I found the problem in OneDrive config.

chunk_size = 144834560

This option causes problems with any file bigger than 2G.
Do you have any recommendations on this option for best performance? Or just leave it by default?

Sorry I personally do not use OneDrive so I would not want to comment.

Perhaps @Ole might have some recommendations.

I use the default chunk_size and doubt an increase would improve performance (until I have seen a test case showing so).

I generally use defaults for OneDrive except --checkers, which I increase to 16 in some of my jobs. This gives optimum performance in most situations.

I therefore recommend you remove the settings of chunk_size, --transfers and --bwlimit, unless you have tested and verified that each of them significantly improves performance in your situation.