Feature request: Allow empty unions

Feature
1

If you try to create a union that has no upstreams, rclone tells you "This value is required and has no default." I was hoping to create such unions, hoping that trying to do operations on them would be no-op.

The reason I want to do this is to make it easier to make and maintain unions of unions (and so on). For example:

Let's say I have three "real" remotes: Dropbox:, Google:, and Amazon:. I pay Google and Amazon, but my Dropbox account is free and therefore has a low storage limit. I want to backup all my stuff to both Google and Amazon, but only certain small things to Dropbox.

To make this easier, I could create a Big: union of Google: and Amazon, and an All: union of Dropbox:, Google:, and Amazon:. However, since unions can contain unions, it would be even easier if I were to instead make All: a union of Big: and Dropbox:. The benefit there is that if I ever add another big "real" remote, I only have to modify the definition of Big:, not the definitions of both Big: and All:.

But note that unions are allowed to have a single entry, so I can make it even easier to maintain by creating a Small: union with just Dropbox: in it, and making All: a union of Big: and Small: instead of Big: and Dropbox:. That way, if I ever add another small "real" remote, I only have to modify the definition of Small:, not the definitions of both All: and Small:.

So that's great as far as it goes, but now how about if I eventually want to add OneDrive, which I pay for, and so is not "small", but it's also not "big", as it has a hard cap on storage, unlike Amazon and Google. Then (as it currently stands) I would add a "real" OneDrive: and a union Medium: which contains OneDrive:, modify All: to contain Big:, Medium:, and Small:, and maybe even add another union, MediumAndAbove:, being Big: and Medium:.

But if empty unions were allowed (and were essentially no-op), I could've set up Medium: before I ever got a OneDrive account. And I could have set up All: to include Medium: right from the start, and that way if I ever add a medium remote, all I have to do is modify the definition of Medium:.

Or even better, I could set up an empty MediumAndAbove:, and defined All: not as Big:, Medium:, and Small:, but Big: and MediumAndAbove:. This would make it easier to maintain if I ever wanted to add, say, Biggish:.

And to be clear, this "space" sort of thing is just an example. I actually want to do multiple different kinds of things, some unrelated to the file storage limits, that would be easier if empty unions were allowed.

Done editing to add the stuff I hadn't yet typed when I accidentally hit Reply

2

Whoops, sorry, accidentally "replied" before having finished. I'll edit my post with the end of it and also something saying that I'm done doing so.

Edit: Done

3

Or I guess an alternative, perhaps more flexible, would be to allow a no-op remote. Then you could define a union of a single no-op.

4

Update:

Where I wrote...

note that unions are allowed to have a single entry

... that was an assumption based on the fact that rclone config will allow you to create a union of one entry, whereas it will not allow you to create a union of zero entries. But now that I'm actually trying it out, I see that rclone will complain about a single-member union when you try to actually do something with it (such as rclone lsf).

So, this feature request should really be "Allow empty and single-member unions".

5

A single entry union is probably better described with an alias backend.

I guess you could make a noop alias if you pointed it to a non existing directory in a place you don't have permission to create directories.

6

Right, I'm currently using aliases to get around the fact that single-member unions don't work, and that's basically fine, but there's a conceptual difference between "This should be one underlying thing" and "This should be a group of things that at the moment happens to have only one member". There's no practical difference between them in terms of day-to-day usage, but there is a practical difference between them in terms of maintainability.

Similarly zero-member unions.

7

For what it's worth, this maintainability issue has now tripped me up. Actually, it happened about two days ago, and I only noticed just now.

I had an alias "Size-Small":

[Size-Small]
type = alias
remote = OneDrive:

A few days ago, I signed up for a pClone account, and so updated my config:

[Size-Small]
type = alias
remote = OneDrive: PCloud:

And uploaded some stuff. I had intended the result to be that the stuff would be copied to two places: OneDrive:/Stuff.stuff, and PCloud:/Stuff.stuff. Instead, it was copied to neither of those places, and instead to one single place, OneDrive: PCloud:/Stuff.stuff. That is, into a directory called PCloud: (with a leading space and a trailing colon) on my OneDrive account.

Obviously this would be moot if I were a more careful man, and had remembered that because I was adding a second remote, I needed to also change the Alias to a Union. But please consider these requests -- allowing zero- and one-member unions, and my other request of allowing intersections -- as they really would allow for improved maintainability.

8

if you wanted to experiment you can try commenting out these lines

Which check for 1 and 0 union entries and see how the union backend behaves.

9

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.