Info Update: Following @ncw’s advice I originally set --transfers to 9 to allow 96M per transfer as suggested. After running that for a while, I started upping that number to see the effect on performance and transfer speeds. Given my rclone is running on a file server that generally has no IO to or from it, I figured leaning on the disk for additional RAM wasn’t going to cause much of an issue.
I’m now running pretty happily at --transfers 24, even though that is pushing past my RAM limit (considering other processes running on the server) and getting about consistently getting ~1.3MBytes/s. That’s 6 times faster than I was originally getting, and I’m very happy with that.
Once my initial backup of all my data is complete my daily backup won’t have too much to do. rclone is a perfect fit for my hand-built mirror backup system, so again I’m very happy!
B2 recommend setting the transfers as high as you like. The RAM trade-off is a bit annoying…
Though I just remembered you can now set --b2-chunk-size to below 96MiB now (which used to be the lower limit), I think the minimum is now 5 MB so if you want more transfers with less RAM you could try that.
Thanks for the additional info. I’ve played with the --transfer and --bw-chunk-size settings and found a sweet-spot for my current transfers; --transfers 24 and --b2-chunk-size 48M. I’m getting ~1.4MBytes/s speeds from this configuration, and I’m not maxing out my available RAM.
I have seen some very strange reports from the --progress readings. I’ve seen some .MOV files of size 250Mb - 350Mb report percentages of over 200%. Eventually they appear to transfer, however such odd behaviour may be throwing off the actual overall transfer speed.
That is a little strange! I think these are caused by chunks which needed to be retried for whatever reason. I think the overall transfer speed will still be accurate though.
