How to manage CPU and memory usage efficiently in async job submit?

Help and Support
1

Hello Experts,

We are running rclone in RC mode and submitting COPY jobs via http apis.
Machine where we are running has 8 core and 64GB ram.
And my use case will have large number of files copy from one cloud to another cloud.

I need to submit multiple copy jobs async.

So question is what is the best way to handle my CPU and memory usage.

If I run 10 copy jobs parallelly with "transfers"=8, how it will impact on CPU and memory usage ?
Will it exhaust the CPU which can impact other applications running in same machine?
Is there anyway to throttle CPU usage by Rclone?

Version : rclone v1.57.0

Thanks in advance,
Mahendra

2

Depends on the system. Best way is to test and see.

Maybe? Depends on the OS.

No. You'd do that in the OS.

3

It depends on what you are being limited by. Most likely you'll be limited by network bandwidth in which case 10 parallel jobs will just use more memory but not much more CPU.

Probably not, but you'll have to try it!

Not directly, no. However limiting the bandwidth --bwlimit often has the same effect.

4

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.