What is the problem you are having with rclone?
Rclone does not consider Azure ACL when using service principal (rclone: --azureblob-service-principal-file) authentication.
When you create a Service Principal file with the --role "Storage Blob Data Reader" and provide the READ, Write & Execute ACL to the Service Principal, then the rclone copy command fails with "This request is not authorized to perform this operation.".
It seems rclone does not consider the ACL assigned to the Service Principal. Rclone seems to only check the assigned role "Storage Blob Data Reader" and then stops to further evaluate the assigned ACL.
This is how Azure works:
During security principal-based authorization, permissions are evaluated in the following order.
- Azure role assignments are evaluated first and take priority over any ACL assignments.
2️. If the operation is fully authorized based on Azure role assignment, then ACLs are not evaluated at all.
3️. If the operation is not fully authorized, then ACLs are evaluated.
=> Step 3 seems to get ignored by Rclone.
Reference to Azure documentation:
[Access control model for Azure Data Lake Storage Gen2 | Microsoft Docs]
Note:
When the azure cli is used to copy a file to the storage it works.
Example of the azure cli commands:
az login --service-principal --username XXXXXXXXXXXXX --tenant XXXXXXXXXXXXXXXX --allow-no-subscriptions
az storage blob upload --name test1 --file something --account-name XXXXXXXXXXXXXX --container-name sbazuredemo2 --auth-mode login
Run the command 'rclone version' and share the full output of the command.
rclone v1.59.1
- os/version: ubuntu 20.04 (64 bit)
- os/kernel: 4.4.0-22000-Microsoft (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.18.5
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
Microsoft Azure Blob Storage
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone -vv copy local:/home/nils/azure/report1.txt azurereader:sbazuredemo2
The rclone config contents with secrets removed.
**rclone.conf**
[azurereader]
type = azureblob
account = sbazuredemo2
service_principal_file = /home/nils/azure/azure-principal2.json
no_head_object = true
**azure-principal2.json**
{
"appId": "xxx",
"displayName": "sbspDataReader",
"password": "xxx",
"tenant": "xxx"
}
A log from the command with the -vv flag
2022/08/30 09:28:17 DEBUG : rclone: Version "v1.59.1" starting with parameters ["rclone" "-vv" "copy" "local:/home/nils/azure/report1.txt" "azurereader:sbazuredemo2"]
2022/08/30 09:28:17 DEBUG : Creating backend with remote "local:/home/nils/azure/report1.txt"
2022/08/30 09:28:17 DEBUG : Using config file from "/home/nils/.config/rclone/rclone.conf"
2022/08/30 09:28:17 DEBUG : fs cache: adding new entry for parent of "local:/home/nils/azure/report1.txt", "/home/nils/azure"
2022/08/30 09:28:17 DEBUG : Creating backend with remote "azurereader:sbazuredemo2"
2022/08/30 09:28:18 DEBUG : report1.txt: Sizes differ (src 6 vs dst 0)
2022/08/30 09:28:18 ERROR : report1.txt: Failed to copy: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, github.com/Azure/azure-storage-blob-go@v0.15.0/azblob/zc_storage_error.go:42
===== RESPONSE ERROR (ServiceCode=AuthorizationFailure) =====
Description=This request is not authorized to perform this operation.
RequestId:abf7805b-c01e-0001-6142-bcf72b000000
Time:2022-08-30T07:28:17.1440655Z, Details:
Code: AuthorizationFailure
PUT https://sbazuredemo2.blob.core.windows.net/sbazuredemo2?restype=container&timeout=31536001
Authorization: REDACTED
User-Agent: [rclone/v1.59.1]
X-Ms-Client-Request-Id: [c237597f-1a1c-4882-43f9-734c8939cbfe]
X-Ms-Version: [2020-10-02]
--------------------------------------------------------------------------------
RESPONSE Status: 403 This request is not authorized to perform this operation.
Content-Length: [246]
Content-Type: [application/xml]
Date: [Tue, 30 Aug 2022 07:28:16 GMT]
Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
X-Ms-Client-Request-Id: [c237597f-1a1c-4882-43f9-734c8939cbfe]
X-Ms-Error-Code: [AuthorizationFailure]
X-Ms-Request-Id: [abf7805b-c01e-0001-6142-bcf72b000000]
X-Ms-Version: [2020-10-02]