Box.com question: googledrive I found goes faster with --drive-chunk-size 128M is there a similiar flag for box.com?

Help and Support
1

rclone-v1.63.1

Apparently -drive-chunk-size SizeSuffix Upload chunk size (default 8Mi) is a global flag? not just for google?

Anyone tested this flag on box.com? I'm getting surprisingly slow speeds right now and I wonder if this flag has a sweet spot for box.com?

I tested not using the flag and got about 17-20MiB/sec and I tested 128M just like I did for google and got about 22MiB/sec but for all I know the sweet spot is some other number like 12 or 64, anyone played with this? Or heck, does this flag even effect any remote's besides googledrive?

2

Somehow this appears to be an answer to my question Box getting 403 errors when using chunker (working fine without it) - #31 by vashp2029 I'm still leaving this thread up though because chunk-size fine tuning is an issue if of itself and it took a long time for me to decide on my favorite chunk size for googledrive....

3

This is not a global flag. The clue is in its name... -drive-chunk-size applies only to drive - google drive.

There no equivalent flag for box.

4

Then why did it make so many 32MB chunks when it was supposed to make only 8MB chunks? Or have I misread the -vv in some manner?

Also isn't the goal of larger chunks to reduce api calls? and box.com in theory has even more limited api call restrictions than googledrive? right? Or am I again misunderstanding entirely the purpose of chunks?

5

Why it was supposed to make 8MB chunks? I think that 32MB is hardcoded in Box API implementation. You would have to check their API specification - maybe it is only size they accept?. Or maybe it was never implemented by rclone.

You are right about chunks vs API calls.

6

Box.com was for me to slow. Only round about 160Mbit/s

7

Transfers

For files above 50 MiB rclone will use a chunked transfer. Rclone will upload up to --transfers chunks at the same time (shared among all the multipart uploads). Chunks are buffered in memory and are normally 8 MiB so increasing --transfers will increase memory use.

Usually if I claim something it's because I read it on /docs

Of course I guess it doesn't really matter why it said 32, it was just so odd, docs told me it should be 8Mib, I tried to set it to 32Mib with a flag that wouldn't work, then the -vv says 32Mib that's how I drew the false conclusion.

8

Indeed you are right.. I feel corrected.

But then it is even worse as it is 8MB only without any option to change it.

Either way --box-chunk-size flag is not implemented so it can not be changed by user. Why? I do not know. Would require to go through Box API to understand if it is possible or not.
In general larger chunks should improve performance at the cost of RAM usage.

9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.