Rclone with rcd mode autoscaling and async setup

[xgenvn (Brian Ng)]

Hi,

I have already set up an instance of rclone which running at rcd mode with docker.
Running copy commands is working perfectly.

I'm trying to use rclone with flag --async and poll job_id for getting status and it's working fine.

My concern is that if I scaling to 2 containers of docker, the job_id can be duplicated between instances. Is there any option to config this kind of issue, or I need to add a layer for routing between rclone instances?

Thanks.

[Floris]

Hello @xgenvn we recently started using a docker rclone volume plugin for this purpose exactly. This seems promising for our replicated docker (in swarm mode) containers.

[ncw (Nick Craig-Wood)]

@xgenvn

My concern is that if I scaling to 2 containers of docker, the job_id can be duplicated between instances.

Very likely

@xgenvn

Is there any option to config this kind of issue, or I need to add a layer for routing between rclone instances?

You'll need to make sure you query the same rclone that you started the job on. Not sure how you'll achieve this in your setup - rclone doesn't have a shared DB mode.