Azure Blob: access-tier not respected for aliases

Suspected Bug
1

What is the problem you are having with rclone?

Using Azure Blob, the "access-tier" setting of an alias seems to be ignored.

Please note that perhaps this is more general problem than just "access_tier" for this one particular backend. I thought I had been overriding a different, similar setting on a different backend via an alias, but I'm not so sure about that now. With that said:

The config contents listed below are the initial state of things. I would expect, based on them, for things uploaded to Raw-AzureUSEast (a "real" Azure Blob backend) to be in the "cold" access tier, and things uploaded to AzureUSEastArchive (an alias to the "real" one) to be in the "archive" tier. However, I instead get:

Raw-AzureUSEast: Cold (as expected)
AzureUSEastArchive: Cold (but expected Archive)

My first thought was maybe the "archive" tier isn't supported on upload, so I changed AzureUSEastArchive's "access_tier" setting to "hot". Got the same results:

Raw-AzureUSEast: Cold (as expected)
AzureUSEastArchive: Cold (but expected Hot)

"Cold" is actually the default for the storage account (i.e. on the Azure end, unrelated to rclone), so I then tried setting Raw-yadda to "cool", and the alias's back to "archive". That showed that rclone is paying attention to the access_tier of Raw-yadda, but it's still not being overwritten by the access_tier of the alias:

Raw-AzureUSEast: Cool (as expected, or at least as hoped)
AzureUSEastArchive: Cool (but hoped Archive)

I then thought maybe it will accept the alias's access_tier if I don't explicitly set one for the Raw_yadda, but alas:

Raw-AzureUSEast: Cold (Inferred) (as expected, or at least as hoped)
AzureUSEastArchive: Cold (Inferred) (but hoped Archive)

To be clear, the parenthetical "Inferred" in those last two is part of what Microsoft shows me in the web browser, not something I'm inserting on my own.

From these tests, it seems to me that the "access_tier" setting of an alias is completely ignored.

Run the command 'rclone version' and share the full output of the command.

rclone v1.68.1

  • os/version: Microsoft Windows 11 Pro 24H2 (64 bit)
  • os/kernel: 10.0.26100.2161 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.23.1
  • go/linking: static
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Azure Blob

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy .\blahblah\ AzureUSEastArchive:Gorilla -P

The rclone config contents with secrets removed.

[AzureUSEastArchive]
type = alias
remote = Raw-AzureUSEast:archive
access_tier = archive

[Raw-AzureUSEast]
type = azureblob
account = ...
key = ...
access_tier = cold

A log from the command with the -vv flag

2024/10/30 21:12:09 DEBUG : rclone: Version "v1.68.1" starting with parameters ["C:\\Users\\bob\\AppData\\Local\\Microsoft\\WinGet\\Links\\rclone.exe" "copy" ".\\blahblah\\" "AzureUSEastArchive:" "-vv"]
2024/10/30 21:12:09 DEBUG : Creating backend with remote ".\\blahblah\\"
2024/10/30 21:12:09 DEBUG : Using config file from "C:\\Users\\bob\\AppData\\Roaming\\rclone\\rclone.conf"
2024/10/30 21:12:09 DEBUG : fs cache: renaming cache item ".\\blahblah\\" to be canonical "//?/C:/Users/bob/BobStuff/Trunkesque/Gorilla/blahblah"
2024/10/30 21:12:09 DEBUG : Creating backend with remote "AzureUSEastArchive:"
2024/10/30 21:12:09 DEBUG : Creating backend with remote "Raw-AzureUSEast:archive"
2024/10/30 21:12:09 DEBUG : fs cache: renaming cache item "AzureUSEastArchive:" to be canonical "Raw-AzureUSEast:archive"
2024/10/30 21:12:10 DEBUG : blah.blah: Need to transfer - File not found at Destination
2024/10/30 21:12:10 DEBUG : Azure container archive: Waiting for checks to finish
2024/10/30 21:12:10 DEBUG : Azure container archive: Waiting for transfers to finish
2024/10/30 21:12:10 DEBUG : blah.blah: md5 = 1ff0a4275f737e5e24f2b707292d3646 OK
2024/10/30 21:12:10 INFO  : blah.blah: Copied (new)
2024/10/30 21:12:10 INFO  :
Transferred:        2.133 MiB / 2.133 MiB, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         0.9s

2024/10/30 21:12:10 DEBUG : 5 go routines active
2

That's by design as that flag isn't for that remote:

Alias

3

Then unless there's a real reason for that design (which I wouldn't doubt), I consider it suboptimal.

4

I mean, your use of the setup seems suboptimal to me :slight_smile:

Feel free to submit a PR if you think it's a good change and valuable to you as I would not imagine to determine what is optimal or suboptimal as only can classify that.

5

I don't mean to be confrontational, and I'm sorry if I came across that way. I will be more explicit about why I consider it "suboptimal":

The reason I think it is suboptimal to disallow aliases to override settings is that allowing it would enable better separation of configuration in a "don't repeat yourself" way. As is, the only way I know to accomplish what I was hoping for -- i.e. two different access tier settings for the same actual backend -- is to configure two complete and essentially identical representations of that backend in rclone. So rather than:

[OneParticularValueForASmallSetting]
type = alias
remote = TheActualRemote
setting = DoItThisWay

[AnotherParticularValueForTheSameSmallSetting]
type = alias
remote = TheActualRemote
setting = DoItADifferentWay

[TheActualRemote]
type = BlahCloud
account = MyAccount
account_security = hunter2
(potentially a billion other settings)
setting = DoItTheDefaultWay

I instead would have:

[TheActualRemoteWithOneParticularValueForASmallSetting]
type = BlahCloud
account = MyAccount
account_security = hunter2
(potentially a billion other settings)
setting = DoItThisWay

[TheActualRemoteWithAnotherValueForTheSameSmallSetting]
type = BlahCloud
account = MyAccount
account_security = hunter2
(potentially a billion other settings)
setting = DoItADifferentWay

[TheActualRemoteWithTheDefaultValueForASmallSetting]
type = BlahCloud
account = MyAccount
account_security = hunter2
(potentially a billion other settings)
setting = DoItTheDefaultWay

Repeating the account, account_security, and the potentially billion other settings over and over again, even though they're exact copies of each other. This makes things more difficult and more error prone, in the way that violations of "don't repeat yourself" often do, especially as configuration gets larger and more complex.

Perhaps I am missing something here, like another way to accomplish the same thing, and if so I'd appreciate any such suggestions. And while I agree that what is or is not "suboptimal" is a matter of opinion, and (again) I did not mean it in a confrontational way, another opinion of mine is that "this violation of DRY is suboptimal" is a reasonable opinion to hold.

6

You definitelyd did not come off that way at all. I thought it was a great suggestion as well.

I think alias was just designed to be a shortcut for a long folder path.

And by all means, submit a feature on github or do a PR.

7

This way doesn't work as discussed above.

However it can be made to work using the connection string syntax

[AzureUSEastArchive]
type = alias
remote = Raw-AzureUSEast,access_tier=archive:archive

[Raw-AzureUSEast]
type = azureblob
account = ...
key = ...
access_tier = cold

Thinking about it, there is no reason why the alias backend couldn't do this itself to make the original syntax work.

2 Likes
8

Excellent. Thank you!

1 Like
9

Just had a chance to try it out, and it worked great. Thanks again!

1 Like
10

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.