We have designed a program using Rclone to pull data from different clouds, display, and with a single click push to our own storage solution.
We want to do this with multiple users simultaneously.
We could go the rout of 1 user= 1 instance , but that seems problematic.
What we are trying to do is figure out a way to have multiple users (up to 30 or so) on a single instance.
Obviously we need to build a DB for log in credentials, security tokens, etc, which we are working on, but even with all of that, it doesn't appear that the current code supports multiple users on the same instance.
Any help/ suggestions would be greatly appreciated.
Maybe use SysV semaphores
Since the script is not the only program calling rclone, then I think I need to intercept all calls - instead of putting this code in the program, could replace rclone by wrapper that implements the parallelism constraint as above and then calls the real program.
this is the only thought I have on how to accomplish this
That very well may work, we would just need to figure out a way to implement a 2 step verification(for our purposes) into it and get multiple users on the same instance.
I believe this is what we are currently using, which gives us 1 user per instance. The problem we are running into is that when we add another user, we add a second instance running, which works for small scale, 1-100 users, but not so much when we are talking 1,000s of users, and 1000's of instances running .
My apologies, as I completely ignored your question. Currently just using your web API to list directories, but we want to build out our own. In that respect I guess that is up for debate. Is there a certain method that would make the Auth-Prox a more practical option than others.
May serve up via Websockets, but for time constraints on the project, we are slightly shying away from that option.
Again, at this point we are open to just about anything.
What you'd do is make a new backend for each user with different auth in it. These will run concurrently in one rclone rcd instance and when they haven't been used for a while (5 minutes I think) rclone will get rid of them.
We are currently using --rc. From what I gather from above you can have multiple users logged into the same instance at once, which is great (and about half of our problem solved).
I guess the only other question I would have would be if we had multiple active users(which we now know is possible), and they both make an FTP request at the same time, can we pull both simultaneously, or would it stage them?