Fails to authenticate to Microsoft Blob Storage using the `use_az` flag

el-aasi · December 18, 2024, 2:01pm

What is the problem you are having with rclone?

When trying to mount with the use_az flag, the "mounting" process goes though but then it fails with:

--------------------------------------------------------------------------------
RESPONSE 401: 401 Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
ERROR CODE: NoAuthenticationInformation
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>NoAuthenticationInformation</Code><Message>Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
RequestId: <redacted>
Time:2024-12-18T13:47:16.0138085Z</Message></Error>
--------------------------------------------------------------------------------

Run the command 'rclone version' and share the full output of the command.

rclone v1.68.2
- os/version: Microsoft Windows 11 Pro 23H2 (64 bit)
- os/kernel: 10.0.22631.4602 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.23.3
- go/linking: static
- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Microsoft Blob Storage

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone mount <redacted>:<redacted> \\cloud\<redacted> --poll-interval 5s --dir-cache-time 10s -vvv

The rclone config contents with secrets removed.

[redacted]
type = azureblob
account = <redacted>
use_az = true

A log from the command with the `-vv` flag

2024/12/18 14:55:47 DEBUG : Creating backend with remote "<redacted>:<redacted>"
2024/12/18 14:55:47 DEBUG : Using config file from "C:\\Users\\USERE_NAME\\AppData\\Roaming\\rclone\\rclone.conf"
2024/12/18 14:55:47 INFO  : Azure container <redacted>: poll-interval is not supported by this remote
2024/12/18 14:55:47 DEBUG : vfs cache: root is "C:\\Users\\USERE_NAME\\AppData\\Local\\rclone"
2024/12/18 14:55:47 DEBUG : vfs cache: data root is "\\\\?\\C:\\Users\\USERE_NAME\\AppData\\Local\\rclone\\vfs\\<redacted>\\<redacted>"
2024/12/18 14:55:47 DEBUG : vfs cache: metadata root is "\\\\?\\C:\\Users\\USERE_NAME\\AppData\\Local\\rclone\\vfsMeta\\<redacted>\\<redacted>"
2024/12/18 14:55:47 DEBUG : Creating backend with remote ":local,encoding='Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot':C:/Users/USERE_NAME/AppData/Local/rclone/vfs/<redacted>/<redacted>"
2024/12/18 14:55:47 DEBUG : :local: detected overridden config - adding "{fbZPM}" suffix to name
2024/12/18 14:55:47 DEBUG : fs cache: renaming cache item ":local,encoding='Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot':C:/Users/USERE_NAME/AppData/Local/rclone/vfs/<redacted>/<redacted>" to be canonical ":local{fbZPM}://?/C:/Users/USERE_NAME/AppData/Local/rclone/vfs/<redacted>/<redacted>"
2024/12/18 14:55:47 DEBUG : Creating backend with remote ":local,encoding='Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot':C:/Users/USERE_NAME/AppData/Local/rclone/vfsMeta/<redacted>/<redacted>"
2024/12/18 14:55:47 DEBUG : :local: detected overridden config - adding "{fbZPM}" suffix to name
2024/12/18 14:55:47 DEBUG : fs cache: renaming cache item ":local,encoding='Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot':C:/Users/USERE_NAME/AppData/Local/rclone/vfsMeta/<redacted>/<redacted>" to be canonical ":local{fbZPM}://?/C:/Users/USERE_NAME/AppData/Local/rclone/vfsMeta/<redacted>/<redacted>"
2024/12/18 14:55:47 DEBUG : Forcing --network-mode because mountpoint path is network share UNC format
2024/12/18 14:55:47 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2024/12/18 14:55:47 NOTICE: Assigning drive letter "Z:"
2024/12/18 14:55:47 DEBUG : Network mode mounting is enabled
2024/12/18 14:55:47 DEBUG : Mounting on "Z:" ("\\cloud\\<redacted>")
2024/12/18 14:55:47 DEBUG : Azure container <redacted>: Mounting with options: ["-o" "attr_timeout=1" "-o" "uid=-1" "-o" "gid=-1" "--FileSystemName=rclone" "--VolumePrefix=\\cloud\\<redacted>"]
2024/12/18 14:55:47 DEBUG : Azure container <redacted>: Init:
2024/12/18 14:55:47 DEBUG : Azure container <redacted>: >Init:
2024/12/18 14:55:47 DEBUG : /: Statfs:
2024/12/18 14:55:47 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2024/12/18 14:55:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2024/12/18 14:55:47 DEBUG : /: >Getattr: errc=0
2024/12/18 14:55:47 DEBUG : /: Readlink:
2024/12/18 14:55:47 DEBUG : /: >Readlink: linkPath="", errc=-40
2024/12/18 14:55:47 DEBUG : /: Getxattr: name="non-existant-a11ec902d22f4ec49003af15282d3b00"
2024/12/18 14:55:47 DEBUG : /: >Getxattr: errc=-40, value=""
The service rclone has been started.
2024/12/18 14:55:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2024/12/18 14:55:47 DEBUG : /: >Getattr: errc=0
2024/12/18 14:55:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2024/12/18 14:55:47 DEBUG : /: >Getattr: errc=0
2024/12/18 14:55:47 DEBUG : /: Opendir:
2024/12/18 14:55:47 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2024/12/18 14:55:47 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2024/12/18 14:55:47 DEBUG : /: >Opendir: errc=0, fh=0x0
2024/12/18 14:55:47 DEBUG : /: Releasedir: fh=0x0
2024/12/18 14:55:47 DEBUG : /: >Releasedir: errc=0
2024/12/18 14:55:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2024/12/18 14:55:47 DEBUG : /: >Getattr: errc=0
2024/12/18 14:55:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2024/12/18 14:55:47 DEBUG : /: >Getattr: errc=0
2024/12/18 14:55:47 DEBUG : /: Opendir:
2024/12/18 14:55:47 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2024/12/18 14:55:47 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2024/12/18 14:55:47 DEBUG : /: >Opendir: errc=0, fh=0x0
2024/12/18 14:55:47 DEBUG : /: Releasedir: fh=0x0
2024/12/18 14:55:47 DEBUG : /: >Releasedir: errc=0
2024/12/18 14:55:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2024/12/18 14:55:47 DEBUG : /: >Getattr: errc=0
2024/12/18 14:55:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2024/12/18 14:55:47 DEBUG : /: >Getattr: errc=0
2024/12/18 14:55:47 DEBUG : /: Opendir:
2024/12/18 14:55:47 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2024/12/18 14:55:47 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2024/12/18 14:55:47 DEBUG : /: >Opendir: errc=0, fh=0x0
2024/12/18 14:55:47 DEBUG : /: Releasedir: fh=0x0
2024/12/18 14:55:47 DEBUG : /: >Releasedir: errc=0
2024/12/18 14:55:52 DEBUG : /Desktop.ini: Getattr: fh=0xFFFFFFFFFFFFFFFF
2024/12/18 14:55:52 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2024/12/18 14:55:52 DEBUG : /: >Getattr: errc=0
2024/12/18 14:55:52 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2024/12/18 14:55:52 DEBUG : /: >Getattr: errc=0
2024/12/18 14:55:52 DEBUG : /: Opendir:
2024/12/18 14:55:52 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2024/12/18 14:55:52 DEBUG : pacer: low level retry 1/10 (error GET https://<redacted>.blob.core.windows.net/<redacted>
--------------------------------------------------------------------------------
RESPONSE 401: 401 Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
ERROR CODE: NoAuthenticationInformation
--------------------------------------------------------------------------------
?<?xml version="1.0" encoding="utf-8"?><Error><Code>NoAuthenticationInformation</Code><Message>Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
RequestId: <redacted>
Time:2024-12-18T13:55:52.6280762Z</Message></Error>
--------------------------------------------------------------------------------
)
2024/12/18 14:55:52 DEBUG : pacer: Rate limited, increasing sleep to 10ms
2024/12/18 14:55:52 DEBUG : pacer: low level retry 2/10 (error GET https://<redacted>.blob.core.windows.net/<redacted>
--------------------------------------------------------------------------------

I did login just before, I am able to successfully "access" data using az storage, it works just fine with env_auth=true. I have also tried to "replicate" the auth method, but I am not a go developer, but seems to work just fine:

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/Azure/azure-sdk-for-go/sdk/azidentity"
    "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob"
)

func main() {
    var options = azidentity.AzureCLICredentialOptions{}
    cred, err := azidentity.NewAzureCLICredential(&options)
    if err != nil {
        log.Fatalf("failed to get Azure CLI credentials: %v", err)
    }
    blobServiceURL := "https://<redacted>.blob.core.windows.net"

    serviceClient, err := azblob.NewClient(blobServiceURL, cred, nil)
    if err != nil {
        log.Fatalf("failed to create BlobServiceClient: %v", err)
    }


    ctx := context.Background()
    pager := serviceClient.NewListContainersPager(nil)
    for pager.More() {
        pageResp, err := pager.NextPage(ctx)
        if err != nil {
            log.Fatalf("failed to get containers: %v", err)
        }

        for _, container := range pageResp.ContainerItems {
            fmt.Printf("Container name: %s\n", *container.Name)
        }
    }

    fmt.Println("Container listing complete.")
}

I can track the "changes" back to this PR:

from this issue:

github.com/rclone/rclone

AzureBlobStorage Env_Auth setting forces system managed identity

opened 02:39PM - 16 Sep 24 UTC

closed 03:16PM - 20 Sep 24 UTC

niklashha

enhancement Remote: Azure Blob

#### What is the problem you are having with rclone? We currently use RClone …to connect Azure VMs to an Azure Blob Storage. In the context of the Azure Virtual Desktop, we would like to use the identity of the logged-in user. According to the documentation, it is possible to generate credentials via _az login_ and then use them via the Env_Auth flag. According to the documentation, however, the following sequence is used when setting the **Env_Auth** to true switch: 1. Environment Variables 2. Managed Service Identity Credentials 3. Azure CLI credentials (as used by the az tool) Since we are operating in the Azure context and working with Entra ID, every VM in Azure requires a System Managed Identity, including our AVD machine. There is no option to enforce the credentials via az tool, it always automatically uses the System Managed Identity, which has no authorization, as this lies with the user. Setting the **Use_msi** flag to false also has no effect. According to the documentation, there is also no way to bypass the use of the MSI. I am not sure whether this is a bug or a desired behavior. #### What is your rclone version (output from `rclone version`) 1.67 #### Which OS you are using and how many bits (e.g. Windows 7, 64 bit) Windows Server 22 und Windows Server 25 Preview #### Which cloud storage system are you using? (e.g. Google Drive) Azure Blob Storage #### The command you were trying to run (e.g. `rclone copy /tmp remote:tmp`) `rclone mount` Config File ``` [configName] type = azureblob account = <accountName> env_auth = true use_msi = false ``` #### How to use GitHub * Please use the 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to show that you are affected by the same issue. * Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue. * Subscribe to receive notifications on status change and new comments.

asdffdsa · December 18, 2024, 2:59pm

welcome to the forum,

i think you need to use beta version of rclone
v1.69.0-beta.8324.fbb6687f6.fix-8078-azureblob-credentials on branch fix-8078-azureblob-credentials (uploaded in 15-30 mins)

fwiw, test a simple command use as rclone lsd redacted: -vv

azure blob does not support polling
INFO : Azure container <redacted>: poll-interval is not supported by this remote

el-aasi · December 19, 2024, 6:39am

omg saw that this was fixed somewhere in September and the lastest release was in November and assumed it must be there (didn't even look at the version) - that's what I get for assuming

Anyway tried now with beta, works like a charm!

Also thank you for the tip with the poll interval. Best regard!

system · December 22, 2024, 6:39am

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.