How do I create a union? I seem to be misunderstanding the syntax slightly?

1.63 rclone
rclone config
create new
union

I input for upstreams

truecryptdropbox:nc truecryptdropboxalternatename:

then rclone lsd -v "union:"

no subdirectories found

Now in this situation lsd truecryptdropbox: has tons of content and lsd truecryptdropboxalternatename: has no content

Clearly I'm doing something wrong with the ro or nc rules?

When I I input for upstreams

truecryptdropbox: truecryptdropboxalternatename:

lsd of the union shows perfectly what I wanted.....

But well...

TLDR: I want to make a union of two remotes, and I want all uploads to go to remote2 but I need remote1 to be accessible for rclone check so that when I do

rclone copy "CONTENT" "Union:"

It will skip all the content already in remote1 and only upload the missing content to remote2....

This is what :nc or :ro should do, but I'm using their syntax wrongly?

edit: I tried setting search to epall instead of ff, but that didn't help at all, because I'm using the nc syntax wrongly :frowning:

edit2: This example from the documentation is just too alien to me for me to comprehend

'\"upstreama:test/space:ro dir\" upstreamb:'

It looks like for some reason there are escaped quotations and even though :ro goes at the end of the path there's a weird circumstance with a space and then dir afterwards?

I just need a basic example for how and where to place my :ro in order to be valid.

edit3: Why do I want all new data to go to remote2? well because it's the one with more free space, now I know there are policies for this like epmfs all I actually want is epmfs but I do not see how epmfs can possibly work, since there isn't an rclone command to show remaining free space on a remote.... and most remotes probably don't support this sort of feature anyways. Hence my plan is to just use my human brain to sort it out. remote1 has little free space remote2 has tons, so I will just set :nc or :ro on remote1, this then gives me the effect of epmfs....

edit4: okay so

rclone about remote

Actually about works. So epmfs should still work, but, I still prefer to trust in myself and use :nc if someone can teach me the syntax :slight_smile: Even with epmfs working perfectly there's all these rules surrounding prices and weeks and months between storage capacity increases the managing empfs by hand via :nc makes sense to me. If only I comprehended the syntax to make :nc real :slight_smile:

I ran a test.

epmfs is a total failure

rclone about cannot be relied upon.

I expected this outcome though. cloud storage websites ability to accurately report on free space is not something I thought could be relied upon.

So I'm back to my original desire. I need to learn the syntax for a union

I need it to be

upstream=

remote1:nc remote2:

Unfortunately I do not know the syntax for this and the documentation on the subject confuses me.

change it to:

remote1::nc remote2:

but given your description I think that:

remote1::ro remote2:

would be better.

Oh, I guess I misread the documentation, the magical phrase is ":nc" or ":ro" I misunderstood and just thought that it was a case of it being "no" or "ro" to be placed after the :

no wonder it didn't work for me

I won't be able to run the test again until I'm back at my main computer on tuesday. I'll try to remember to come back and edit this post to remind myself of the success/failure of that extra :

1 Like

I'm now getting an error.

conflict/path/folder

I can't figure it out. It only happens at the start of running a copy command while checking the source against the destination. I guess it's not a big deal, if it's just 4 files out of 1000s I can manually move them without the union, but it is strange that the union I've set up has this flaw atm. I'll have to remember to re-run this later if the automatic retry pass doesn't fix it.

what is your rclone config?

If you see error it would indicate you corrupted union structure by changing policies for already existing data.

I believe I did corrupt the union by messing with the policy settings. Yes. When I was testing out the settings. Trying to get :nc to work when I needed to use ::ro instead. This also explains why it's only the first few files with errors and all the rest are fine? Those are the only files that got transferred to the union when the union was set up incorrectly, and I hit ctrl-c to cancel the program and rethink the settings.

The only issue being that I don't know how to fix this, because the files attached to the errors aren't in the union so I can't simply delete them, hmm.

Maybe I've jumped to a wild conclusion though based on your guess? It's hard to say. I also am not sure that having the wrong policies could've corrupted anything, as the only policy change I made from default had no effect whatsoever (I misunderstood it.) And I also only ran the union in the old configuration for a briefer time. hmm.

I tried to see the issue again, and the exact same files popped the error when I ctrl-c'd and repeated the copy.

At the moment it's setup as "remote::ro remote2:" default policies. In the past it may have been set to something like "remote: remote2::nc" I also messed with the 2nd of the three action type policies, but like I said, I returned them all to the default.

Sorry, I am so bad at this.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.