Chunker Config Help

Help and Support
1

What is the problem you are having with rclone?

I can't seem to figure out the best way to implement Chunker and Compression for an encrypted remote. Any help would be appreciated.

so this is what I'm trying to achieve:

Blomp/
├── Folder1
├── Folder2
├── Folder3
└── E (Crypt)
    ├── Folder4
    ├── Folder5
    └── A (Gzip)
        ├── Folder6
        └── Folder7

Folder E is the encrypted folder within the home directory but it isn't compressed, only Folder A within E is compressed.

Run the command 'rclone version' and share the full output of the command.

rclone v1.66.0
- os/version: Microsoft Windows Server 2022 Standard 21H2 (64 bit)
- os/kernel: 10.0.20348.2402 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.22.1
- go/linking: static
- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Blomp

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

Method 1:


[blomp]
type = swift
user = XXX
key = XXX
auth = https://authenticate.ain.net
tenant = XXX
auth_version = 2
endpoint_type = public
leave_parts_on_error = true
chunk_size = 1P
no_chunk = false

[blompc]
type = crypt
remote = blompc-chunker/E
password = XXX
password2 = XXX

[blompc-chunker]
type = chunker
remote = blomp:x@gmail.com
chunk_size = 4Gi

[blompc-gzip]
type = compress
remote = blompc-chunker:Google-archive
level = 9

The command I'm running with this method is:

rclone copy remote: blompc-gzip:

Method 2:

[blomp]
type = swift
user = XXX
key = XXX
auth = https://authenticate.ain.net
tenant = XXX
auth_version = 2
endpoint_type = public
leave_parts_on_error = true
chunk_size = 1P
no_chunk = false

[blompc]
type = crypt
remote = blomp:x@gmail.com/E
password = XXX
password2 = XXX

[blompc-chunker]
type = chunker
remote = blompc-gzip:
chunk_size = 4Gi

[blompc-gzip]
type = compress
remote = blompc:Google-archive
level = 9

The command I'm running with this meth hod is:

rclone copy remote: blompc-chunker:

These are both giving me different outputs so any help/advice would be great.

2

It is all mess:)

What is x@gmail.com ????

Did you mean blompc-chunker:E

I suggest you do one by one and only add more when all is working.

3

@kapitainsky haha I'm sorry but I've been trying to wrap my head around this for a while now and I've just ended up confusing myself even more.

That's my remote. The Blomp works as `blomp:email'. I've just added x to hide my actual email.

No actually blompc-chunker:E will not work because:

Blomp/
├── Folder1
├── Folder2
├── Folder3
└── E (Crypt)
    ├── Folder4
    ├── Folder5
    └── Google-archive (Gzip)
        ├── Folder6
        └── Folder7

This is the hierarchy. I only want to compress Google-archive. And this is inside E. If I do blompc-chunker:E it will compress the entire E folder.

Another user ppointed out that gzip compression is still being tested in rclone and may have bugs. Therfore, I might just go ahead and upload without it. I was just trying to save space.

4

Does:

work?

can you list top level directories?

rclone lsd blomp:

5

Yes that works perfectly. And blompc: works too. (The encrypted folder).

6

Then this should be:

remote = blompc:E/Google-archive
7

remote = blompc:E/Google-archive

But blompc: is already

[blompc]
type = crypt
remote = blomp:x@gmail.com/E
password = XXX
password2 = XXX

So it already contains the path to E. Anyway, I think I'm just confusing myself and wasting your time here. I will just upload it without the compression and see what happens. Hopefully it will work. If not, I will open a new issue with a clearer mind.

Thanks for your help @kapitainsky. You can close this issue or delete it.

1 Like
8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.