Copy fails with permission erorr after upgrade from 1.60.0 to 1.61.0

Suspected Bug
1

What is the problem you are having with rclone?

After upgrade to 1.61.0 I'm facing with permissions issue while trying copy file from local system to azure blob storage. What I'm doing I'm just generating simple json file locally and trying to move it to cloud. After upgrade it started to fail with this log.

Failed to copy: PUT https://***.windows.net/bucket_name
--------------------------------------------------------------------------------
RESPONSE 409: 409 Public access is not permitted on this storage account.
ERROR CODE: PublicAccessNotPermitted
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>PublicAccessNotPermitted</Code><Message>Public access is not permitted on this storage account.
Time:2022-12-22T16:03:40.6388175Z</Message></Error>
---------------------------------------------------------------

Does rclone requires any additional permissions to copy files? because running same with 1.60.0 still passing without error.
Note: I'm performing rclone sync on different container but in same storage account and with same sas-token and it works as expected

Run the command 'rclone version' and share the full output of the command.

rclone v1.61.0
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 5.15.0-56-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.19.4
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Azure Blob Storage

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy local:$filepath azure:$bucket --config $rclone_conf

The rclone config contents with secrets removed.

[azure]
type = azureblob
account_name = ${account_name}
sas_url = https://${account_name}.blob.core.windows.net?${sas_token}

[local]
type = local

A log from the command with the -vv flag

Sorry can't provide now log with -vv as error happens in production env, I'll be able to add it later if needed

-------------------------------------------------------------------------------
2022/12/22 17:03:40 ERROR : Attempt 2/3 failed with 1 errors and: PUT https://****.net/backup
--------------------------------------------------------------------------------
RESPONSE 409: 409 Public access is not permitted on this storage account.
ERROR CODE: PublicAccessNotPermitted
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>PublicAccessNotPermitted</Code><Message>Public access is not permitted on this storage account.
RequestId:
Time:2022-12-22T16:03:40.6478131Z</Message></Error>
--------------------------------------------------------------------------------
2022/12/22 17:03:40 ERROR : backup_2022_12_22T16_03_40Z.json: Failed to copy: PUT https://****.net/backup
--------------------------------------------------------------------------------
RESPONSE 409: 409 Public access is not permitted on this storage account.
ERROR CODE: PublicAccessNotPermitted
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>PublicAccessNotPermitted</Code><Message>Public access is not permitted on this storage account.
RequestId:
Time:2022-12-22T16:03:40.6568084Z</Message></Error>
--------------------------------------------------------------------------------
2022/12/22 17:03:40 ERROR : backup_2022_12_22T16_03_40Z.json: Not deleting source as copy failed: PUT https://***.net/backup
--------------------------------------------------------------------------------
RESPONSE 409: 409 Public access is not permitted on this storage account.
ERROR CODE: PublicAccessNotPermitted
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>PublicAccessNotPermitted</Code><Message>Public access is not permitted on this storage account.
RequestId:
Time:2022-12-22T16:03:40.6568084Z</Message></Error>
--------------------------------------------------------------------------------
2022/12/22 17:03:40 ERROR : Attempt 3/3 failed with 1 errors and: PUT https://***.net/backup
--------------------------------------------------------------------------------
RESPONSE 409: 409 Public access is not permitted on this storage account.
ERROR CODE: PublicAccessNotPermitted
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>PublicAccessNotPermitted</Code><Message>Public access is not permitted on this storage account.
RequestId:
Time:2022-12-22T16:03:40.6568084Z</Message></Error>
--------------------------------------------------------------------------------
2022/12/22 17:03:40 Failed to moveto: PUT https://***.net/backup
--------------------------------------------------------------------------------
RESPONSE 409: 409 Public access is not permitted on this storage account.
ERROR CODE: PublicAccessNotPermitted
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>PublicAccessNotPermitted</Code><Message>Public access is not permitted on this storage account.
RequestId:
Time:2022-12-22T16:03:40.6568084Z</Message></Error>
--------------------------------------------------------------------------------
2

I've just fixed this here: azureblob: is throwing "409 Public access is not permitted on this storage account" with v1.61.0 · Issue #6645 · rclone/rclone · GitHub - there is a binary linked - testing appreciated :slight_smile:

3

Hi, thanks for fast response!
I'm going to test it tomorrow but also can you tell me when next release is planned?
Is it safe to use this patched version?

4

I'll release a 1.61.1 in the next day or two.

The patched version is 1.61.0 plus the azure blob fix and that plus a couple more fixes will become 1.61.1

5

Thanks a lot, we appreciate it!

While I'm here with you I want to ask you one more question or let me know if should I open another thread :wink:
We had a problem with 1.60 version, some files thrown error

ERROR corrupted on transfer: sizes differ NNN vs MMM

This problem gone when I added flag --s3-decompress, but than I found out that new flag was introduced in 1.61 --s3-might-gzip, so to try it I made upgrade an started to play with it.
First run I tried without setting any flags and issue did not come up again while with 1.60 it still appearing. Can you please tell me did you set something by default to deal with it?

6

This problem was fixed in 1.60.1 and you shouldn't need the --s3-decompress flag.

Rclone will deal with this itself and you shouldn't need the --s3-might-gzip flag either.

7

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.