Azure Blob Public Resources

Suspected Bug
1

What is the problem you are having with rclone?

When I run lsf command for a public azure resource I get the following error:

no authentication method configured

Credentials should not be required for public azure blob.

Run the command 'rclone version' and share the full output of the command.

rclone v1.67.0
- os/version: Microsoft Windows 11 Enterprise 22H2 (64 bit)
- os/kernel: 10.0.22621.3737 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.22.4
- go/linking: static
- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Azure Blob

The command you were trying to run (eg rclone copy /tmp remote:tmp)

./rclone.exe lsf :azureblob,env_auth=false,account=gododev:files --azureblob-public-access container -vv

The rclone config contents with secrets removed.

[azurepublic]
type = azureblob
account = gododev

A log from the command with the -vv flag

2024/07/16 11:26:27 DEBUG : rclone: Version "v1.67.0" starting with parameters ["C:\\Users\\tkacha\\Downloads\\rclone-v1.67.0-windows-amd64\\rclone-v1.67.0-windows-amd64\\rclone.exe" "lsf" ":azureblob,env_auth=false,account=gododev:files" "--azureblob-public-access" "container" "-vv"]
2024/07/16 11:26:27 DEBUG : Creating backend with remote ":azureblob,env_auth=false,account=gododev:files"
2024/07/16 11:26:27 DEBUG : Using config file from "C:\\Users\\tkacha\\AppData\\Roaming\\rclone\\rclone.conf"
2024/07/16 11:26:27 DEBUG : :azureblob: detected overridden config - adding "{21ZPg}" suffix to name
2024/07/16 11:26:27 Failed to create file system for ":azureblob,env_auth=false,account=gododev:files": no authentication method configured
2

Hmm, we don't seem to have a way of accessing public content.

Try this

v1.68.0-beta.8096.85b971608.fix-azureblob-anonymous on branch fix-azureblob-anonymous (uploaded in 15-30 mins)

It seems to work on your example

rclone ls :azureblob,account=gododev:files
1 Like
3

Tried it out and everything worked with no issues.

./rclone.exe ls :azureblob,account=gododev:files -vv
2024/07/17 17:00:12 DEBUG : rclone: Version "v1.68.0-beta.8096.85b971608.fix-azureblob-anonymous" starting with parameters ["C:\\Users\\tkacha\\Downloads\\rclone-v1.68.0-beta.8096.85b971608.fix-azureblob-anonymous-windows-amd64\\rclone-v1.68.0-beta.8096.85b971608.fix-azureblob-anonymous-windows-amd64\\rclone.exe" "ls" ":azureblob,account=gododev:files" "-vv"]
2024/07/17 17:00:12 DEBUG : Creating backend with remote ":azureblob,account=gododev:files"
2024/07/17 17:00:12 DEBUG : Using config file from "C:\\Users\\tkacha\\AppData\\Roaming\\rclone\\rclone.conf"
2024/07/17 17:00:12 DEBUG : :azureblob: detected overridden config - adding "{DmeEy}" suffix to name
2024/07/17 17:00:12 DEBUG : fs cache: renaming cache item ":azureblob,account=gododev:files" to be canonical ":azureblob{DmeEy}:files"
   329283 contracts/clients/2039
   329284 contracts/clients/2040
   329289 contracts/clients/2041
   329280 contracts/clients/2042
   329278 contracts/clients/2043
   329277 contracts/clients/2044
   329278 contracts/clients/2045
   329279 contracts/clients/2046
   329278 contracts/clients/2047
   329278 contracts/clients/2048
   329279 contracts/clients/2049
   329279 contracts/clients/2050
   329278 contracts/clients/2051
2024/07/17 17:00:13 DEBUG : 4 go routines active

When will this be merged into main?
I assume once the changes are merged librclone.RPC calls will properly work for Azure Blob public as well.

I was using the librclone.RPC("operations/size", string(sizeRequestJSON)) call to get the size of a public container blob in azure and that's how I found the issue.

Also, if I understand correctly, the --azureblob-public-access flag has no any use in this case right?

4

Thanks for testing.

I've merged this to master now which means it will be in the latest beta in 15-30 minutes and released in v1.68

That is the access rights a container gets when rclone creates it.

1 Like
5

Hey Nick,

When I tested this with RPC I still got that no authentication method configured error.

Initializing librclone...
2024/07/18 16:44:01 ERROR : rc: "operations/size": error: no authentication method configured
GetRemoteDatasetSize librclone.RPC error: Got status 500 and output "{\n\t\"error\": \"no authentication method configured\",\n\t\"input\": {\n\t\t\"fs\": \":azureblob,account=gododev:files\"\n\t},\n\t\"path\": \"operations/size\",\n\t\"status\": 500\n}\n"
6

Are you sure you compiled a new librclone?

Can you try the equivalent with the RC, eg

rclone rc --loopback ...your input params...
7

I'm not running a compiled version. I just use imports from Rclone itself like below

"github.com/rclone/rclone/backend/azureblob" 
"github.com/rclone/rclone/librclone/librclone"

and use go lang to execute rclone RPC commands in the following way

librclone.RPC("operations/size", string(sizeRequestJSON))

I wonder why RPC using imports is not working?

Here's the output from running the rclone rc command with your beta build (https://beta.rclone.org/branch/fix-azureblob-anonymous/v1.68.0-beta.8096.85b971608.fix-azureblob-anonymous/)

./rclone.exe rc --loopback operations/size fs=:azureblob,account=gododev:files
{
        "bytes": 4280640,
        "count": 13,
        "sizeless": 0
}
8

Did you update your go.mod to pull in the fix?

go get github.com/rclone/rclone@master
9

Silly mistake. I did run go mod tidy but forgot go get.

I was able to get the size and download files from a public container blob.

Initializing librclone...
Remote dataset size:  4280640
Remote dataset count:  13
Sync request SrcFs:  :azureblob,env_auth=false,account=gododev:files
Sync request DstFs:  C:\images-azure-public       
Sync request Async:  true
Starting rclone sync/copy...
AsyncCopy librclone.RPC : Got status :  200
AsyncCopy JobID :  2
Waiting for the async job to complete...
Stats: {Bytes:0 Speed:0 Transfers:0 ElapsedTime:0 Errors:0}
Waiting for the async job to complete...
Stats: {Bytes:196608 Speed:0 Transfers:0 ElapsedTime:1.013921 Errors:0}        
Waiting for the async job to complete...
Stats: {Bytes:3622083 Speed:1.40200344713411e+06 Transfers:11 ElapsedTime:2.0155489 Errors:0}
Waiting for the async job to complete...
Stats: {Bytes:4280640 Speed:2.139424861205687e+06 Transfers:13 ElapsedTime:3.0214394 Errors:0}

Everything works properly now.
Thanks you!

1 Like
10

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.