Max. 'united' remotes before performance drops, or other considerations

[dinosm]

I have started using 'union' to unite several OneDrive 1TB remotes together. 'Union' seems to do exactly what I want and so far, with 4 remotes, it is behaving perfectly.

I am planning to add a lot more remotes into this union.
Initially I was thinking of uniting them in sixes (so one union per MS account), but then I thought it might be fine to unite all of them in one.

I just wanted to ask - will there be any performance drop, or are there any other reasons not to 'unite' e.g. 30 remotes into one?

[kapitainsky]

@dinosm

I just wanted to ask - will there be any performance drop, or are there any other reasons not to 'unite' e.g. 30 remotes into one?

Of course there will be performance drop - but it is impossible to quantify it without testing in your specific environment.

To give you some perspective what people do have a look at this:

How to optimize very large numbers of union - Google Drive?

What is the problem you are having with rclone? I'm working on a project which require me to union 1200 Google Drive account. Each account has their own service_account_file, root_folder_id defined, and for every 100 account they share same client_id and client_secret. I find that copying/moving local file to this union take 4-5 minutes overhead before the actual transfers is started. I tried with --fast-list and it has little to no effect. Is there a way to optimize speed for very large number…

[random404]

The limits of your api will be your limit... as soon you start hitting your api limits you'll get throttled... before that you can expect some delays in some actions but it will not be significant

I don't use onedrive so can't say about their api, but maybe it's better to share the files using their web interface so you can have a single remote and single api ?

Or use different api accounts for each remote

[dinosm]

Do you guys think there is any performance difference between:

• An rclone union of 30 remotes

and

• those 30 remotes individually mounted on the filesystem (bog standard ext4), each one in its own directory?

Is there any reason why I should opt for one (and which one) and not the other?

[Animosity022]

Depending on your use case, you'd want to test and see which works better. Way too many factors to give a quick one vs the other.

[dinosm]

The use case is a Plex server, each remote to contain media files, so mostly reading from them, seldom writing. The main performance bottleneck I am finding (and currently have a combination of one small rclone union and a few more mounts) is when (re)scanning libraries, this takes ages. It seems OneDrive has lower throttling limits than either GDrive or Dropbox, as this is going much slower than it used to when I had most of the data on GD and Dropbox. I do have several separate client IDs, so I am not using the same client ID for each remote, and it's still very slow.
As it wasn't before and I had several mounts (but not as many as I now have in the union), I think it's either the union (new kid on the block) or the fact that I now use OneDrive and not the other two.