Rclone copyto ADLS Gen2 with ACL permissions for Azure AD SP fails for single files

NiksomV · December 14, 2022, 5:38pm

What is the problem you are having with rclone?

I am using the rclone copyto /path/to/source/file.txt remote-adlsgen2:container/path/to/destination/file.txt command to copy some local file into the destination container of the Azure Data Lake Storage Gen2 storage account.

I use a service principal for this command, and configured ACLs on the target container (allow read, write and execute):

The full command, that I run then looks as follows:
rclone copyto test/hello2.txt adlsgen2:test-inmoment/hello2.txt --azureblob-account vimadlsgen2 --azureblob-service-principal-file aad/azure-principal-ventx.json

As the result I get an error message, saying that the request is not authorized (see full message in the below section), which is not as expected.

A similar rclone copyto command but for copying all files in a specified folder works fine without any error messages, which is an expected behavior.

The only way to currently make rclone copyto command work properly is either to:

Give the service principal the "Storage Blob Data Contributor" role on the container,
- This is too much permissions, because I need to be able to configure permissions on the folder level inside the container.
Copy all files from a given folder

Run the command 'rclone version' and share the full output of the command.

rclone v1.60.1
- os/version: ubuntu 20.04 (64 bit)
- os/kernel: 5.15.0-56-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.19.3
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Azure Data Lake Storage Gen2

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

This command fails:

$ rclone copyto test/hello2.txt adlsgen2:test-inmoment/hello8.txt --azureblob-account vimadlsgen2 --azureblob-service-principal-file aad/azure-principal-ventx.json

This command works:

$ rclone copyto test adlsgen2:test-inmoment --azureblob-account vimadlsgen2 --azureblob-service-principal-file aad/azure-principal-ventx.json

The rclone config contents with secrets removed.

$ rclone config show

[adlsgen2]
type = azureblob

A log from the command with the `-vv` flag

The output for the failing command:

$ rclone copyto test/hello2.txt adlsgen2:test-inmoment/hello8.txt --azureblob-account vimadlsgen2 --azureblob-service-principal-file aad/azure-principal-ventx.json -vv

<7>DEBUG : rclone: Version "v1.60.1" starting with parameters ["rclone" "copyto" "test/hello2.txt" "adlsgen2:test-inmoment/hello8.txt" "--azureblob-account" "vimadlsgen2" "--azureblob-service-principal-file" "aad/azure-principal-ventx.json" "-vv"]
<7>DEBUG : rclone: systemd logging support activated
<7>DEBUG : Creating backend with remote "test/hello2.txt"
<7>DEBUG : Using config file from "/home/slava/.config/rclone/rclone.conf"
<7>DEBUG : fs cache: adding new entry for parent of "test/hello2.txt", "/home/slava/Projects/Allianz/test/test"
<7>DEBUG : Creating backend with remote "adlsgen2:test-inmoment/"
<7>DEBUG : adlsgen2: detected overridden config - adding "{Vtw6O}" suffix to name
<7>DEBUG : fs cache: renaming cache item "adlsgen2:test-inmoment/" to be canonical "adlsgen2{Vtw6O}:test-inmoment"
<7>DEBUG : hello2.txt: Need to transfer - File not found at Destination
<3>ERROR : hello2.txt: Failed to copy: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, github.com/Azure/azure-storage-blob-go@v0.15.0/azblob/zc_storage_error.go:42
===== RESPONSE ERROR (ServiceCode=AuthorizationFailure) =====
Description=This request is not authorized to perform this operation.
RequestId:78543f7c-301e-0009-28e1-0f83aa000000
Time:2022-12-14T17:26:58.0329875Z, Details: 
   Code: AuthorizationFailure
   PUT https://vimadlsgen2.blob.core.windows.net/test-inmoment?restype=container&timeout=31536001
   Authorization: REDACTED
   User-Agent: [rclone/v1.60.1]
   X-Ms-Client-Request-Id: [9e0af892-bc12-453a-4aa3-e4c398e37a31]
   X-Ms-Version: [2020-10-02]
   --------------------------------------------------------------------------------
   RESPONSE Status: 403 This request is not authorized to perform this operation.
   Content-Length: [246]
   Content-Type: [application/xml]
   Date: [Wed, 14 Dec 2022 17:26:57 GMT]
   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
   X-Ms-Client-Request-Id: [9e0af892-bc12-453a-4aa3-e4c398e37a31]
   X-Ms-Error-Code: [AuthorizationFailure]
   X-Ms-Request-Id: [78543f7c-301e-0009-28e1-0f83aa000000]
   X-Ms-Version: [2020-10-02]


<3>ERROR : Attempt 1/3 failed with 1 errors and: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, github.com/Azure/azure-storage-blob-go@v0.15.0/azblob/zc_storage_error.go:42
===== RESPONSE ERROR (ServiceCode=AuthorizationFailure) =====
Description=This request is not authorized to perform this operation.
RequestId:78543f7c-301e-0009-28e1-0f83aa000000
Time:2022-12-14T17:26:58.0329875Z, Details: 
   Code: AuthorizationFailure
   PUT https://vimadlsgen2.blob.core.windows.net/test-inmoment?restype=container&timeout=31536001
   Authorization: REDACTED
   User-Agent: [rclone/v1.60.1]
   X-Ms-Client-Request-Id: [9e0af892-bc12-453a-4aa3-e4c398e37a31]
   X-Ms-Version: [2020-10-02]
   --------------------------------------------------------------------------------
   RESPONSE Status: 403 This request is not authorized to perform this operation.
   Content-Length: [246]
   Content-Type: [application/xml]
   Date: [Wed, 14 Dec 2022 17:26:57 GMT]
   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
   X-Ms-Client-Request-Id: [9e0af892-bc12-453a-4aa3-e4c398e37a31]
   X-Ms-Error-Code: [AuthorizationFailure]
   X-Ms-Request-Id: [78543f7c-301e-0009-28e1-0f83aa000000]
   X-Ms-Version: [2020-10-02]


<7>DEBUG : hello2.txt: Need to transfer - File not found at Destination
<3>ERROR : hello2.txt: Failed to copy: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, github.com/Azure/azure-storage-blob-go@v0.15.0/azblob/zc_storage_error.go:42
===== RESPONSE ERROR (ServiceCode=AuthorizationFailure) =====
Description=This request is not authorized to perform this operation.
RequestId:78543fa7-301e-0009-50e1-0f83aa000000
Time:2022-12-14T17:26:58.1265752Z, Details: 
   Code: AuthorizationFailure
   PUT https://vimadlsgen2.blob.core.windows.net/test-inmoment?restype=container&timeout=31536001
   Authorization: REDACTED
   User-Agent: [rclone/v1.60.1]
   X-Ms-Client-Request-Id: [fbd2e40d-2555-4b1c-6a24-1b0d88af9fe0]
   X-Ms-Version: [2020-10-02]
   --------------------------------------------------------------------------------
   RESPONSE Status: 403 This request is not authorized to perform this operation.
   Content-Length: [246]
   Content-Type: [application/xml]
   Date: [Wed, 14 Dec 2022 17:26:57 GMT]
   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
   X-Ms-Client-Request-Id: [fbd2e40d-2555-4b1c-6a24-1b0d88af9fe0]
   X-Ms-Error-Code: [AuthorizationFailure]
   X-Ms-Request-Id: [78543fa7-301e-0009-50e1-0f83aa000000]
   X-Ms-Version: [2020-10-02]


<3>ERROR : Attempt 2/3 failed with 1 errors and: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, github.com/Azure/azure-storage-blob-go@v0.15.0/azblob/zc_storage_error.go:42
===== RESPONSE ERROR (ServiceCode=AuthorizationFailure) =====
Description=This request is not authorized to perform this operation.
RequestId:78543fa7-301e-0009-50e1-0f83aa000000
Time:2022-12-14T17:26:58.1265752Z, Details: 
   Code: AuthorizationFailure
   PUT https://vimadlsgen2.blob.core.windows.net/test-inmoment?restype=container&timeout=31536001
   Authorization: REDACTED
   User-Agent: [rclone/v1.60.1]
   X-Ms-Client-Request-Id: [fbd2e40d-2555-4b1c-6a24-1b0d88af9fe0]
   X-Ms-Version: [2020-10-02]
   --------------------------------------------------------------------------------
   RESPONSE Status: 403 This request is not authorized to perform this operation.
   Content-Length: [246]
   Content-Type: [application/xml]
   Date: [Wed, 14 Dec 2022 17:26:57 GMT]
   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
   X-Ms-Client-Request-Id: [fbd2e40d-2555-4b1c-6a24-1b0d88af9fe0]
   X-Ms-Error-Code: [AuthorizationFailure]
   X-Ms-Request-Id: [78543fa7-301e-0009-50e1-0f83aa000000]
   X-Ms-Version: [2020-10-02]


<7>DEBUG : hello2.txt: Need to transfer - File not found at Destination
<3>ERROR : hello2.txt: Failed to copy: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, github.com/Azure/azure-storage-blob-go@v0.15.0/azblob/zc_storage_error.go:42
===== RESPONSE ERROR (ServiceCode=AuthorizationFailure) =====
Description=This request is not authorized to perform this operation.
RequestId:78544011-301e-0009-25e1-0f83aa000000
Time:2022-12-14T17:26:58.2241460Z, Details: 
   Code: AuthorizationFailure
   PUT https://vimadlsgen2.blob.core.windows.net/test-inmoment?restype=container&timeout=31536001
   Authorization: REDACTED
   User-Agent: [rclone/v1.60.1]
   X-Ms-Client-Request-Id: [c3bee8c7-d101-41ae-71a4-1501da71c7bd]
   X-Ms-Version: [2020-10-02]
   --------------------------------------------------------------------------------
   RESPONSE Status: 403 This request is not authorized to perform this operation.
   Content-Length: [246]
   Content-Type: [application/xml]
   Date: [Wed, 14 Dec 2022 17:26:57 GMT]
   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
   X-Ms-Client-Request-Id: [c3bee8c7-d101-41ae-71a4-1501da71c7bd]
   X-Ms-Error-Code: [AuthorizationFailure]
   X-Ms-Request-Id: [78544011-301e-0009-25e1-0f83aa000000]
   X-Ms-Version: [2020-10-02]


<3>ERROR : Attempt 3/3 failed with 1 errors and: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, github.com/Azure/azure-storage-blob-go@v0.15.0/azblob/zc_storage_error.go:42
===== RESPONSE ERROR (ServiceCode=AuthorizationFailure) =====
Description=This request is not authorized to perform this operation.
RequestId:78544011-301e-0009-25e1-0f83aa000000
Time:2022-12-14T17:26:58.2241460Z, Details: 
   Code: AuthorizationFailure
   PUT https://vimadlsgen2.blob.core.windows.net/test-inmoment?restype=container&timeout=31536001
   Authorization: REDACTED
   User-Agent: [rclone/v1.60.1]
   X-Ms-Client-Request-Id: [c3bee8c7-d101-41ae-71a4-1501da71c7bd]
   X-Ms-Version: [2020-10-02]
   --------------------------------------------------------------------------------
   RESPONSE Status: 403 This request is not authorized to perform this operation.
   Content-Length: [246]
   Content-Type: [application/xml]
   Date: [Wed, 14 Dec 2022 17:26:57 GMT]
   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
   X-Ms-Client-Request-Id: [c3bee8c7-d101-41ae-71a4-1501da71c7bd]
   X-Ms-Error-Code: [AuthorizationFailure]
   X-Ms-Request-Id: [78544011-301e-0009-25e1-0f83aa000000]
   X-Ms-Version: [2020-10-02]


<6>INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.9s

<7>DEBUG : 7 go routines active
Failed to copyto: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, github.com/Azure/azure-storage-blob-go@v0.15.0/azblob/zc_storage_error.go:42
===== RESPONSE ERROR (ServiceCode=AuthorizationFailure) =====
Description=This request is not authorized to perform this operation.
RequestId:78544011-301e-0009-25e1-0f83aa000000
Time:2022-12-14T17:26:58.2241460Z, Details: 
   Code: AuthorizationFailure
   PUT https://vimadlsgen2.blob.core.windows.net/test-inmoment?restype=container&timeout=31536001
   Authorization: REDACTED
   User-Agent: [rclone/v1.60.1]
   X-Ms-Client-Request-Id: [c3bee8c7-d101-41ae-71a4-1501da71c7bd]
   X-Ms-Version: [2020-10-02]
   --------------------------------------------------------------------------------
   RESPONSE Status: 403 This request is not authorized to perform this operation.
   Content-Length: [246]
   Content-Type: [application/xml]
   Date: [Wed, 14 Dec 2022 17:26:57 GMT]
   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
   X-Ms-Client-Request-Id: [c3bee8c7-d101-41ae-71a4-1501da71c7bd]
   X-Ms-Error-Code: [AuthorizationFailure]
   X-Ms-Request-Id: [78544011-301e-0009-25e1-0f83aa000000]
   X-Ms-Version: [2020-10-02]

The output for the working command:

$ rclone copyto test adlsgen2:test-inmoment --azureblob-account vimadlsgen2 --azureblob-service-principal-file aad/azure-principal-ventx.json -vv 

<7>DEBUG : rclone: Version "v1.60.1" starting with parameters ["rclone" "copyto" "test" "adlsgen2:test-inmoment" "--azureblob-account" "vimadlsgen2" "--azureblob-service-principal-file" "aad/azure-principal-ventx.json" "-vv"]
<7>DEBUG : rclone: systemd logging support activated
<7>DEBUG : Creating backend with remote "test"
<7>DEBUG : Using config file from "/home/slava/.config/rclone/rclone.conf"
<7>DEBUG : fs cache: renaming cache item "test" to be canonical "/home/slava/Projects/Allianz/test/test"
<7>DEBUG : Creating backend with remote "adlsgen2:test-inmoment"
<7>DEBUG : adlsgen2: detected overridden config - adding "{Vtw6O}" suffix to name
<7>DEBUG : fs cache: renaming cache item "adlsgen2:test-inmoment" to be canonical "adlsgen2{Vtw6O}:test-inmoment"
<7>DEBUG : Azure container test-inmoment: Waiting for checks to finish
<7>DEBUG : Azure container test-inmoment: Waiting for transfers to finish
<7>DEBUG : hello.txt: md5 = da6d88d121960feb8d5cf8f2f2392942 OK
<6>INFO  : hello.txt: Copied (new)
<7>DEBUG : hello2.txt: md5 = da6d88d121960feb8d5cf8f2f2392942 OK
<6>INFO  : hello2.txt: Copied (new)
<7>DEBUG : .api_description: md5 = 00b89ecb3afcd7bbaadb50e71c0d2846 OK
<6>INFO  : .api_description: Copied (new)
<7>DEBUG : .options: md5 = b8e00c61d0e692bdfa5fe570af225c75 OK
<6>INFO  : .options: Copied (new)
<7>DEBUG : hello3.txt: md5 = da6d88d121960feb8d5cf8f2f2392942 OK
<6>INFO  : hello3.txt: Copied (new)
<7>DEBUG : hello4.txt: md5 = da6d88d121960feb8d5cf8f2f2392942 OK
<6>INFO  : hello4.txt: Copied (new)
<6>INFO  : 
Transferred:   	   20.933 KiB / 20.933 KiB, 100%, 0 B/s, ETA -
Transferred:            6 / 6, 100%
Elapsed time:         1.1s

<7>DEBUG : 15 go routines active

ncw · December 15, 2022, 1:12pm

We've re-done the auth for the azureblob backend.

Can you try the latest beta please? The docs for the new auth are here: Microsoft Azure Blob Storage - its backwards compatible but there are some better ways of configuring things.

NiksomV · December 16, 2022, 5:15pm

Hello @ncw ,

with the latest released beta version the error message is different now:

$ rclone_beta copyto test/hello2.txt adlsgen2:test-inmoment/hello8.txt --azureblob-account vimadlsgen2 --azureblob-service-principal-file aad/azure-principal-ventx.json -vv

<7>DEBUG : rclone: Version "v1.61.0-beta.6649.7be9855a7" starting with parameters ["rclone_beta" "copyto" "test/hello2.txt" "adlsgen2:test-inmoment/hello8.txt" "--azureblob-account" "vimadlsgen2" "--azureblob-service-principal-file" "aad/azure-principal-ventx.json" "-vv"]
<7>DEBUG : rclone: systemd logging support activated
<7>DEBUG : Creating backend with remote "test/hello2.txt"
<7>DEBUG : Using config file from "/home/slava/.config/rclone/rclone.conf"
<7>DEBUG : fs cache: adding new entry for parent of "test/hello2.txt", "/home/slava/Projects/Allianz/test/test"
<7>DEBUG : Creating backend with remote "adlsgen2:test-inmoment/"
<7>DEBUG : adlsgen2: detected overridden config - adding "{Vtw6O}" suffix to name
<7>DEBUG : fs cache: renaming cache item "adlsgen2:test-inmoment/" to be canonical "adlsgen2{Vtw6O}:test-inmoment"
<7>DEBUG : hello2.txt: Need to transfer - File not found at Destination
<3>ERROR : hello2.txt: Failed to copy: PUT https://vimadlsgen2.blob.core.windows.net/test-inmoment
--------------------------------------------------------------------------------
RESPONSE 409: 409 Public access is not permitted on this storage account.
ERROR CODE: PublicAccessNotPermitted
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>PublicAccessNotPermitted</Code><Message>Public access is not permitted on this storage account.
RequestId:02f14ab6-301e-0019-3970-1146c2000000
Time:2022-12-16T17:05:28.8825201Z</Message></Error>
--------------------------------------------------------------------------------
<3>ERROR : Attempt 1/3 failed with 1 errors and: PUT https://vimadlsgen2.blob.core.windows.net/test-inmoment
--------------------------------------------------------------------------------
RESPONSE 409: 409 Public access is not permitted on this storage account.
ERROR CODE: PublicAccessNotPermitted
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>PublicAccessNotPermitted</Code><Message>Public access is not permitted on this storage account.
RequestId:02f14ab6-301e-0019-3970-1146c2000000
Time:2022-12-16T17:05:28.8825201Z</Message></Error>
--------------------------------------------------------------------------------
<7>DEBUG : hello2.txt: Need to transfer - File not found at Destination
<3>ERROR : hello2.txt: Failed to copy: PUT https://vimadlsgen2.blob.core.windows.net/test-inmoment
--------------------------------------------------------------------------------
RESPONSE 409: 409 Public access is not permitted on this storage account.
ERROR CODE: PublicAccessNotPermitted
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>PublicAccessNotPermitted</Code><Message>Public access is not permitted on this storage account.
RequestId:02f14ac1-301e-0019-4470-1146c2000000
Time:2022-12-16T17:05:28.9681430Z</Message></Error>
--------------------------------------------------------------------------------
<3>ERROR : Attempt 2/3 failed with 1 errors and: PUT https://vimadlsgen2.blob.core.windows.net/test-inmoment
--------------------------------------------------------------------------------
RESPONSE 409: 409 Public access is not permitted on this storage account.
ERROR CODE: PublicAccessNotPermitted
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>PublicAccessNotPermitted</Code><Message>Public access is not permitted on this storage account.
RequestId:02f14ac1-301e-0019-4470-1146c2000000
Time:2022-12-16T17:05:28.9681430Z</Message></Error>
--------------------------------------------------------------------------------
<7>DEBUG : hello2.txt: Need to transfer - File not found at Destination
<3>ERROR : hello2.txt: Failed to copy: PUT https://vimadlsgen2.blob.core.windows.net/test-inmoment
--------------------------------------------------------------------------------
RESPONSE 409: 409 Public access is not permitted on this storage account.
ERROR CODE: PublicAccessNotPermitted
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>PublicAccessNotPermitted</Code><Message>Public access is not permitted on this storage account.
RequestId:02f14ad6-301e-0019-5570-1146c2000000
Time:2022-12-16T17:05:29.0517769Z</Message></Error>
--------------------------------------------------------------------------------
<3>ERROR : Attempt 3/3 failed with 1 errors and: PUT https://vimadlsgen2.blob.core.windows.net/test-inmoment
--------------------------------------------------------------------------------
RESPONSE 409: 409 Public access is not permitted on this storage account.
ERROR CODE: PublicAccessNotPermitted
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>PublicAccessNotPermitted</Code><Message>Public access is not permitted on this storage account.
RequestId:02f14ad6-301e-0019-5570-1146c2000000
Time:2022-12-16T17:05:29.0517769Z</Message></Error>
--------------------------------------------------------------------------------
<6>INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         1.0s

<7>DEBUG : 7 go routines active
Failed to copyto: PUT https://vimadlsgen2.blob.core.windows.net/test-inmoment
--------------------------------------------------------------------------------
RESPONSE 409: 409 Public access is not permitted on this storage account.
ERROR CODE: PublicAccessNotPermitted
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>PublicAccessNotPermitted</Code><Message>Public access is not permitted on this storage account.
RequestId:02f14ad6-301e-0019-5570-1146c2000000
Time:2022-12-16T17:05:29.0517769Z</Message></Error>
--------------------------------------------------------------------------------

But the overall behavior in general is still same. If I mention the whole folder, then it still works fine:

$ rclone_beta copyto test adlsgen2:test-inmoment --azureblob-account vimadlsgen2 --azureblob-service-principal-file aad/azure-principal-ventx.json -vv 
                     
<7>DEBUG : rclone: Version "v1.61.0-beta.6649.7be9855a7" starting with parameters ["rclone_beta" "copyto" "test" "adlsgen2:test-inmoment" "--azureblob-account" "vimadlsgen2" "--azureblob-service-principal-file" "aad/azure-principal-ventx.json" "-vv"]
<7>DEBUG : rclone: systemd logging support activated
<7>DEBUG : Creating backend with remote "test"
<7>DEBUG : Using config file from "/home/slava/.config/rclone/rclone.conf"
<7>DEBUG : fs cache: renaming cache item "test" to be canonical "/home/slava/Projects/Allianz/test/test"
<7>DEBUG : Creating backend with remote "adlsgen2:test-inmoment"
<7>DEBUG : adlsgen2: detected overridden config - adding "{Vtw6O}" suffix to name
<7>DEBUG : fs cache: renaming cache item "adlsgen2:test-inmoment" to be canonical "adlsgen2{Vtw6O}:test-inmoment"
<7>DEBUG : Azure container test-inmoment: Waiting for checks to finish
<7>DEBUG : Azure container test-inmoment: Waiting for transfers to finish
<7>DEBUG : hello2.txt: md5 = da6d88d121960feb8d5cf8f2f2392942 OK
<6>INFO  : hello2.txt: Copied (new)
<7>DEBUG : hello.txt: md5 = da6d88d121960feb8d5cf8f2f2392942 OK
<6>INFO  : hello.txt: Copied (new)
<7>DEBUG : .options: md5 = b8e00c61d0e692bdfa5fe570af225c75 OK
<6>INFO  : .options: Copied (new)
<7>DEBUG : .api_description: md5 = 00b89ecb3afcd7bbaadb50e71c0d2846 OK
<6>INFO  : .api_description: Copied (new)
<7>DEBUG : hello4.txt: md5 = da6d88d121960feb8d5cf8f2f2392942 OK
<6>INFO  : hello4.txt: Copied (new)
<7>DEBUG : hello3.txt: md5 = da6d88d121960feb8d5cf8f2f2392942 OK
<6>INFO  : hello3.txt: Copied (new)
<6>INFO  : 
Transferred:   	   20.933 KiB / 20.933 KiB, 100%, 0 B/s, ETA -
Transferred:            6 / 6, 100%
Elapsed time:         1.0s

<7>DEBUG : 17 go routines active

Best Regards,
Slava

ncw · December 17, 2022, 10:40am

Try adding this

  --azureblob-no-check-container                   If set, don't attempt to check the container exists or create it

Or put no_check_container = true in the config. I think rclone is attempting to create the container for some reason.

NiksomV · December 19, 2022, 8:28am

Hello!

yes, with the flag --azureblob-no-check-container it worked now:

$ rclone_beta copyto test/hello2.txt adlsgen2:test-inmoment/hello8.txt --azureblob-account vimadlsgen2 --azureblob-service-principal-file aad/azure-principal-ventx.json --azureblob-no-check-container -vv

<7>DEBUG : rclone: Version "v1.61.0-beta.6649.7be9855a7" starting with parameters ["rclone_beta" "copyto" "test/hello2.txt" "adlsgen2:test-inmoment/hello8.txt" "--azureblob-account" "vimadlsgen2" "--azureblob-service-principal-file" "aad/azure-principal-ventx.json" "--azureblob-no-check-container" "-vv"]
<7>DEBUG : rclone: systemd logging support activated
<7>DEBUG : Creating backend with remote "test/hello2.txt"
<7>DEBUG : Using config file from "/home/slava/.config/rclone/rclone.conf"
<7>DEBUG : fs cache: adding new entry for parent of "test/hello2.txt", "/home/slava/Projects/Allianz/test/test"
<7>DEBUG : Creating backend with remote "adlsgen2:test-inmoment/"
<7>DEBUG : adlsgen2: detected overridden config - adding "{u32wq}" suffix to name
<7>DEBUG : fs cache: renaming cache item "adlsgen2:test-inmoment/" to be canonical "adlsgen2{u32wq}:test-inmoment"
<7>DEBUG : hello2.txt: Need to transfer - File not found at Destination
<7>DEBUG : hello2.txt: md5 = da6d88d121960feb8d5cf8f2f2392942 OK
<6>INFO  : hello2.txt: Copied (new) to: hello8.txt
<6>INFO  : 
Transferred:   	         20 B / 20 B, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         0.8s

<7>DEBUG : 7 go routines active

ncw · December 19, 2022, 11:14pm

Glad that is working.

system · January 18, 2023, 11:15pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.