Google Drive Union w/ crypt upstream file distribution

Help and Support
1

What is the problem you are having with rclone?

I am trying to find a way to distribute files across a Union of 2-8+ Google Team Drives backed by a crypt. I attempted to use a eplno create, but I see that this is not supported when referencing a crypt remote as it cannot count object statistics.

My ideal situation is that if there is a create path that matches the current path, it will take action on that upstream crypt. As an example: file ABC.txt appears in Drive1 linked to Drive1Crypt. A new file ZXY.txt is added and it bypasses Drive1Crypt and now distributes to to Drive2 or Drive3 linked to Drive2Crypt or Drive3Crypt in the Union. If however, we have a path matching ABC.txt it will update it in the existing Drive1/Drive1Crypt upstream.

If there is not, it will then distribute it evenly across the next crypt upstreams. In the situation, a few of the remotes should be "modify only if exists, otherwise go to the other remotes and distribute evenly".

Is there a way I can lay this out to be more effective? The remotes all contain the same folder structure when they are union'd together.

Run the command 'rclone version' and share the full output of the command.

``
`rclone v1.62.2

  • os/version: alpine 3.17.3 (64 bit)
  • os/kernel: 5.15.0-60-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.20.2
  • go/linking: static
  • go/tags: none

####  Which cloud storage system are you using? (eg Google Drive)

Google Drive with Team Drives

#### The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone mount UnionCryptDrives:



#### The rclone config contents with secrets removed.

[TeamDriveMerged]
type = union
upstreams = TeamDrive1Crypt: TeamDrive2Crypt: TeamDrive3Crypt: TeamDrive4Crypt:
create_policy = eplno
search_policy = epall

[TeamDrive1CryptDrive]
type = drive
client_id = REDACTED
client_secret = REDACTED
scope = drive
team_drive = REDACTED
service_account_file = REDACTED
root_folder_id =

[TeamDrive1Crypt]
type = crypt
remote = TeamDrive1CryptDrive:
password = REDACTED
password2 = REDACTED

[TeamDrive2CryptDrive]
type = drive
client_id = REDACTED
client_secret = REDACTED
scope = drive
team_drive = REDACTED
service_account_file =REDACTED
root_folder_id =

[TeamDrive2Crypt]
type = crypt
remote = TeamDrive2CryptDrive:
password = REDACTED
password2 = REDACTED

[TeamDrive3CryptDrive]
type = drive
client_id = REDACTED
client_secret = REDACTED
scope = drive
team_drive = REDACTED
service_account_file = REDACTED
root_folder_id =

[TeamDrive3Crypt]
type = crypt
remote = TeamDrive3CryptDrive:
password = REDACTED
password2 = REDACTED

[TeamDrive4CryptDrive]
type = drive
client_id = REDACTED
client_secret = REDACTED
scope = drive
team_drive = REDACTED
service_account_file = REDACTED
root_folder_id =

[TeamDrive4Crypt]
type = crypt
remote = TeamDrive4CryptDrive:
password = REDACTED
password2 = REDACTED

2

It isn't? I don't think google team drive supports objects is the basic problem.

Did you try eplfs instead? That should be what you want.

3

I haven't used eplfs yet. I'm curious if Google Drive being wrapped in a crypt remote will obscure details of free space.

Since Team Drives can have "unlimited" space, would eplfs evaluate the current space utilization of each upstream in consideration?

For instance with eplfs:

Free Space is not supported for upstream TeamDriveCrypt, treating as infinite.

I suppose in reading, if I have identical top level folders (e.g. Crypt 1,2,3 all have a folder called "MyFolder" but with different contents inside of "MyFolder") would combine be better than union in this case?

4

I forgot team drives had no upper limit... Since we don't have epmus how about eprand? That is probably about as close as you are going to get.

Combine won't ever change remote for a direcectory. So you'd need top level directories say drive1, drive2, drive3 and this could all point to MyFolder but you can't get rid of those top level directories.