Azure- Service Principal Credentials file not parsing

[0xKush]

I cannot authenticate rclone, running on an ec2 instance in AWS, with a storage account in Azure using a certificate. I have managed this using a secret however in order to meet security requirements- a certificate was required. Hence, I created a cert using SSL and attached it to the service principal the rclone instance was using to get read access to the storage account. However I am unable to connect to the Azure Storage Account.

This is the error command I am receiving

command:
rclone lsd <azure-storage-account>:

error:
Failed to create file system for "<azure-storage-account>:": error parsing service principal credentials file: missing fields in credentials file

Where should I be looking in order to resolve this?

I have tried an azure-principal.json like this:

{
  "displayName": "service-principal-name",
  "appId": "app-id",
  "certId": "cert-id",
  "thumbprint": "cert-thumbprint"
}

My azure-principal.json currently looks like this:

{
  "appId": "app-id",
  "displayName": "service-principal-name",
  "certificatePath": "path-to-certificate.pfx-stored-locally-on-ec2-instance",
  "tenant": "tenant-id"
}

I have also tried the above with a pem file (as well as a pfx file) neither seem to work

Any help would be greatly appreciated

Thank you

[asdffdsa (jojothehumanmonkey)]

hello and welcome to the forum,

can you answer the questions in the help and support template.
--- the redacted config file
--- full output of rclone version
--- full output of rclone lsd <azure-storage-account>: --dump headers

[ncw (Nick Craig-Wood)]

@asdffdsa

--- full output of rclone version

This is particularly important since the auth for azure all changed in v1.61 and if you are using that version or newer then please upgrade.

[0xKush]

thanks- here is the requested information

redacted config file

[<azure-storage-account>]
type = azureblob
account = <azure-storage-account>
service_principal_file = azure-principal.json

[<azure-storage-account-2>]
type = azureblob
account = <azure-storage-account-2>
service_principal_file = azure-principal.json

[s3]
type = s3
provider = AWS
env_auth = true
region = eu-west-2
server_side_encryption = aws:kms

rclone version

# rclone version
rclone v1.62.2
- os/version: amazon 2 (64 bit)
- os/kernel: 5.10.173-154.642.amzn2.x86_64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.20.2
- go/linking: static
- go/tags: none

rclone lsd dump headers

rclone lsd edgeprddataarchivestr: --dump headers
2023/05/05 14:21:49 NOTICE: Automatically setting -vv as --dump is enabled
2023/05/05 14:21:49 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "lsd" "edgeprddataarchivestr:" "--dump" "headers"]
2023/05/05 14:21:49 DEBUG : Creating backend with remote "edgeprddataarchivestr:"
2023/05/05 14:21:49 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/05/05 14:21:49 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2023/05/05 14:21:49 Failed to create file system for "edgeprddataarchivestr:": error parsing service principal credentials file: missing fields in credentials file

[asdffdsa (jojothehumanmonkey)]

how did you generate the credentials, using az cli

this looks like the relavant funciton
https://github.com/rclone/rclone/blob/f226f2dfb1eefdfd0fa044dd319ba81443d9eece/backend/azureblob/azureblob.go#L575

if spCredentials.AppID == "" || spCredentials.Password == "" || spCredentials.Tenant == "" {
	return nil, fmt.Errorf("missing fields in credentials file")

[ncw (Nick Craig-Wood)]

I dont think the azure-principal.json parser is expecting anything other than a password.

I think this will work if you put this in the config file as detailed in this section in the docs:

Service principal with certificate

If these variables are set, rclone will authenticate with a service principal with certificate.

• tenant: ID of the service principal's tenant. Also called its "directory" ID.
• client_id: the service principal's client ID
• client_certificate_path: path to a PEM or PKCS12 certificate file including the private key.
• client_certificate_password: (optional) password for the certificate file.
• client_send_certificate_chain: (optional) Specifies whether an authentication request will include an x5c header to support subject name / issuer based authentication. When set to "true" or "1", authentication requests include the x5c header.

NB client_certificate_password must be obscured - see rclone obscure.

[0xKush]

Thanks
The Service Principal credentials were initially created via a cli using a secret
however since then, the certificate was attached to the same Service Principal

[0xKush]

Thanks Nick,

I've tried this but without success:

{
  "tenant": "tenant-id",
  "appid": "app-id",
  "client_certificate_path": "path to cert"
}

Used the app id as I've been told this should work
No password has been set on the certificate as it is optional, also tried setting the variable to an empty string but it didn't work.

[ncw (Nick Craig-Wood)]

Those things need to go in the config file (find the config file with rclone config file) not in the JSON.