Union for same cloud

I use a cloud provider and would keep files as

norm: F:\ --> ​Normal unencrypted files
pc1: F:\crypt\01 --> Ecrypted files without name encryption
pc2: F:\crypt\02 - Encrypted file with file and directory name encryption. This folder has another folder where the files go to ensure encrypted files have a proper home

Now, instead of mounting those two encrypted folders separately, I would create a union as
[pc]
type = union
upstreams = pc1: pc2:

Now to ensure that I get my filename encryption proper, I place them in a subfolder of pc2:

The problem is my diskspace for this union is reported doubled - once for pc1 and once for pc2

A simple wish, can the app consider not to add up the disk space if the part before : is common (same drive / same cloud provider)

This would solves 3 issues

  1. Double size not being reported
  2. Querying total space more than once
  3. Reduce number of times available space is queried, currently it would be done twice

Hi Naveed,

It is easy for you to know that the F: drive seen by the two computers are in fact the same drive, because you made the setup.

It is not that easy (if at all possible) if you are a program trying to do the same using the very limited information available from the OS.

If I handed you two PC’s of mine both having an X: drive (with different content), how would you determine if they looked at the exact same could account? (in a way that is uniform across platforms, drives, mounts, cloud providers, accounts, etc.)

Hi Ole,

If its two computers, its going to be two processes and they can have a completely different F: or X: or cloud: drives. As they are two processes, they don't know about each other and will query the sizes independently.

If its the same computer, it can't have two drives with same drive letter. Similarly, rclone won't allow to create two clouds with the same name.

So, anything before the : is precisely pointing to the same underlying drive or cloud and differs only in path and hence be counted only once.

hi,

sorry if this is too off-topic.
the same computer can have two drives with the same drive letter.
those drives can point to rclone mounts, net shares or local resources via subst.
this example uses rclone mount

OK, let’s first get the terminology right.

Are you talking about a config like this:

[norm]
type = alias
remote = F:\

[pc1]
type = crypt
remote = F:\crypt\01

[pc2]
type = crypt
remote = F:\crypt\02

[pc]
type = union
upstreams = pc1: pc2:

where you want [pc] to show the free space on F: ?

In that case be aware that F:\crypt\01 and F:\crypt\02 may point to different drives/accounts/… and that both can be different from F:

This is very easy to make on linux and also possible in Windows. I cannot remember the Windows commands off my head, but I have a feeling that @asdffdsa can do it with one hand tied on his back.

sure, first hand me those two PC and then i will figure it out :wink:

We are on a single PC :grin: and set of possible commands are:

mklink /H F:crypt/01 C:/Somefolder/on/your/computer/drive
mklink /H F:crypt/02 D:/Anotherfolder/mounted/from/a/subfolder/in/GDrive

not sure what the goal here is?

Does the feature the OP want exist?
as i do not use unions, i do not know.

should the OP open a github issue?

The first goal is to make sure we understand the wish of Naveed correctly.

Next I try to explain that the proposed feature is very hard (and probably impossible) to implement, based on my current understanding of the wish.

[pc1]
type = crypt
remote = F:\crypt\01

[pc2]
type = crypt
remote = F:\crypt\02

[pc]
type = union
upstreams = pc1: pc2:

This is how the mounts will look

mklink /H F:crypt/01 C:/Somefolder/on/your/computer/drive
mklink /H F:crypt/02 D:/Anotherfolder/mounted/from/a/subfolder/in/GDrive

In this case, the mounts are as subfolders in the drive. In current implementation, rclone reports space of F: instead of C: and D: repsectively, because it doesn't bother checking if the folder is a separate mount or not.

Even if at a later stage (after lets say this wish is implented), rclone starts checking if folders are separate mounts to report their disk space, it will be naturally extended to the feature requested here.

I am surprised with your screenshots for X:, but I believe this is with way some of the mounts via admin are not visible to others. Anways, is a single rclone process able to see both X drives.

If a single rclone process can see both X drives, then I agree with you, that this feature can't be implemented. However; if can only see 1 X drive, then why query it multiple times and sum up the space for unions

I get it, you are right it should be possible, though probably not as easy as it sounds.

I suggest you make an issue on GitHub https://github.com/rclone/rclone/issues and recommend you include the above config of the three remotes and make the title a bit more descriptive e.g. “Calculation of size for unions using the same drive/account twice”.

Perhaps you want to try yourself?

The code for the union backend is here:
https://github.com/rclone/rclone/blob/master/backend/union/union.go
and the contributing guide can help you make a development environment:
https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md

Thanks Ole. I'll open an issue and give it a Go.

It won't be complicated at all. It goes in the following function

func (f *Fs) About(ctx context.Context) (*fs.Usage, error) {

An array is to be initialised for visited mounts and not add up the Total, used, free if the the mount/drive is already visited.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.