--server-side-across-configs should be ignored when copying/moving from crypt to crypt

Suspected Bug
1

What is the problem you are having with rclone?

After uploading a few TBs to a crypt remote, I learned that I should have set password2 and also used a better filename_encoding than default (from default base32 to base32768).

So I created a new crypt remote with the better configs and started moving my data. It went without warnings, supposedly fine, but gladly I tested opening a file from new remote to check if it was OK, because it wasn't. Files are corrupted.

Thinking about what happened, I guess this is expected because by using --server-side-across-configs the server only re-encodes filename, not re-encrypt file data.

But what worried me was that I saw no warning or error. If my understanding is correct, this could cause important issues like even data loss if user doesn't have a proper backup.

Then I tried moving back the files from new remote to old remote and they are still readable in old remote.

So I'm wondering if rclone should warn users that --server-side-across-configs breaks everything when moving/coping across different crypt remotes, of even discard this option in this case unless password is the same for both remote (supposing this would produce identical files, I don't know).

Run the command 'rclone version' and share the full output of the command.

rclone v1.63.1
- os/version: opensuse-tumbleweed (64 bit)
- os/kernel: 6.5.3-1-default (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.21.0
- go/linking: dynamic
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

IDrive e2 (S3)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone move -P --server-side-across-configs --modify-window 1ms --multi-thread-streams 8 --no-traverse --transfers 8 cicrypt:/dir1/dir2 idrivec:dir1/dir2

The rclone config contents with secrets removed.

[icrypt]
type = s3
provider = IDrive
access_key_id = *
secret_access_key = *
endpoint = *
no_check_bucket = true
chunk_size = 10Mi
upload_concurrency = 8
directory_markers = true

[cicrypt]
type = crypt
remote = icrypt:crypt
password = *

[idrive]
type = s3
provider = IDrive
access_key_id = *
secret_access_key = *
endpoint = *
no_check_bucket = true
chunk_size = 10Mi
upload_concurrency = 8
directory_markers = true

[idrivec]
type = crypt
remote = idrive:idrivec
password = *
password2 = *
filename_encoding = base32768
2

You are right that --server-side-across-configs can break stuff when using crypt with two different configs.

That kind of thing is the reason it isn't the default.

However people do use this for copying between different crypts to change the filename encoding so I can't disable it completely.

I guess the crypt remote could do a check in its Copy / Move / MoveDir routines before it passes the operation down the chain to the actual backend.

I guess it would need to check that the data is encrypted the same, so these parameters would need to be the same in order for it to allow the server side copy:

  --crypt-no-data-encryption             Option to either encrypt file data or leave it unencrypted
  --crypt-password string                Password or pass phrase for encryption (obscured)
  --crypt-password2 string               Password or pass phrase for salt (obscured)

Does that sound right to you?

If so please open a new issue on Github about this - thank you! (Can you put a link to the forum post in).

3

Sometimes rclone sets --server-side-across-configs automatically, right? Yesterday I ran a rclone copy without this option, with the same remote in source and dest, and it was done very quickly due to Server Side Copies.

Yes, I think this is right.

Thanks.

4

That is because you aren't doing it across configs, you are doing it in the same config where it is always safe.

Thanks for making the issue :slight_smile:

5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.