Failed to HEAD for download: Unknown 401 (401 unknown)

asdffdsa · August 31, 2023, 12:06am

since you mentioned that ancient rclone did work and old rclone did not work,
i wanted to confirm latest rclone works.
i believe the following proves that. if not please let me know what i got wrong?

:: using master_key
@set B2_APPLICATION_KEY_ID=redacted
@set B2_APPLICATION_KEY=redacted
b2 authorize-account
b2 list-keys --long
00483d2039b668d000000000   thekey   zorkzork   listBuckets,listFiles,writeFiles

:: using locked key as shown above
@set B2_APPLICATION_KEY_ID=00483d2039b668d000000000
@set B2_APPLICATION_KEY=redacted
b2 authorize-account 
Using https://api.backblazeb2.com
b2 ls zorkzork --verbose 
INFO:__main__:// ========================================  3.9.0   ======================================== \\
INFO:__main__:starting command [__main__.Ls] with arguments: ['b2', 'ls', 'zorkzork', '--verbose']
DEBUG:b2sdk.account_info.abstract:calling AbstractAccountInfo.is_same_key(self=<b2sdk.account_info.sqlite_account_info.SqliteAccountInfo object at 0x0000025EB6F2F790>, application_key_id='00483d2039b668d000000000', realm='production')
INFO:__main__:\\ ========================================  exit=0  ======================================== //

file.ext

rclone config show b2locked 
[b2locked]
type = b2
account = 00483d2039b668d000000000
key = redacted

rclone ls b2locked:zorkzork -vv --dump=headers 
2023/08/30 20:19:39 DEBUG : Setting --config "C:\\data\\rclone\\rclone.conf" from environment variable RCLONE_CONFIG="C:\\data\\rclone\\rclone.conf"
2023/08/30 20:19:39 DEBUG : rclone: Version "v1.63.1" starting with parameters ["C:\\data\\rclone\\rclone.exe" "ls" "b2locked:zorkzork" "-vv" "--dump=headers"]
2023/08/30 20:19:39 DEBUG : Creating backend with remote "b2locked:zorkzork"
2023/08/30 20:19:39 DEBUG : Using config file from "C:\\data\\rclone\\rclone.conf"
2023/08/30 20:19:39 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2023/08/30 20:19:39 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/08/30 20:19:39 DEBUG : HTTP REQUEST (req 0xc0009e5500)
2023/08/30 20:19:39 DEBUG : GET /b2api/v1/b2_authorize_account HTTP/1.1
Host: api.backblazeb2.com
User-Agent: rclone/v1.63.1
Authorization: XXXX
Accept-Encoding: gzip

2023/08/30 20:19:39 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/08/30 20:19:40 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/08/30 20:19:40 DEBUG : HTTP RESPONSE (req 0xc0009e5500)
2023/08/30 20:19:40 DEBUG : HTTP/1.1 200 
Content-Length: 601
Cache-Control: max-age=0, no-cache, no-store
Content-Type: application/json;charset=UTF-8
Date: Thu, 31 Aug 2023 00:19:41 GMT

2023/08/30 20:19:40 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/08/30 20:19:40 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/08/30 20:19:40 DEBUG : HTTP REQUEST (req 0xc000746100)
2023/08/30 20:19:40 DEBUG : POST /b2api/v1/b2_list_file_names HTTP/1.1
Host: api004.backblazeb2.com
User-Agent: rclone/v1.63.1
Content-Length: 59
Authorization: XXXX
Content-Type: application/json
Accept-Encoding: gzip

2023/08/30 20:19:40 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/08/30 20:19:40 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/08/30 20:19:40 DEBUG : HTTP RESPONSE (req 0xc000746100)
2023/08/30 20:19:40 DEBUG : HTTP/1.1 200 
Content-Length: 594
Cache-Control: max-age=0, no-cache, no-store
Content-Type: application/json;charset=UTF-8
Date: Thu, 31 Aug 2023 00:19:40 GMT

2023/08/30 20:19:40 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
        1 file.ext

maltfield · August 31, 2023, 12:40am

Thanks!

Just to confirm: did you configure the Backblaze B2 API key to be append-only?

asdffdsa · August 31, 2023, 12:54am

thank you for your ideas about PGP and whatnot.

no idea what append-only means. the original link you posted was from wikipedia, not from B2 website?

you stated that your key has the following capabilities.
tho not sure if you locked that key to a single bucket?

and so i created a key named thekey using the same capabilities as your key.
in addition, i locked that key to a single bucket named zorkzork

b2 create-key thekey listBuckets,listFiles,writeFiles --bucket zorkzork
b2 list-keys --long
00483d2039b668d000000000   thekey   zorkzork   listBuckets,listFiles,writeFiles

maltfield · August 31, 2023, 1:10am

I wrote that append-only wikipedia page, but basically append-only in this context is a property of a backup system that can add content (append) but cannot delete content.

Using an append-only backup solution is important ransomware protection. If your server gets compromised and the attacker can use the key on the server to delete all your backups, then they have much more leverage over you to demand a ransom (since you have no backups).

Backblaze doesn't have a concept of append-only, but I've found that this can be obtained by a combination of the key permissions shown above and some lifecycle rules. For the sake of this bug, the lifecycle rules are not relevant.

As far as I can tell, your test shows that this bug was resolved. But I'm surprised because I couldn't find any bug reports for 401 Unknown in the GitHub..

ncw · August 31, 2023, 8:47am

This is the fix I suspect

github.com/rclone/rclone

b2: fix failed to create file system with application key limited to a prefix

committed 03:13PM - 11 Feb 21 UTC

ncw

+3 -6

Before this change, if an application key limited to a prefix was in use, with t…railing `/` marking the folders then rclone would HEAD the path without a trailing `/` to work out if it was a file or a folder. This returned a permission denied error, which rclone returned to the user. Failed to create file system for "b2:bucket/path/": failed to HEAD for download: Unknown 401 (401 unknown) This change assumes any errors on HEAD will make rclone assume the object does not exist and the path is referring to a directory. See: https://forum.rclone.org/t/b2-error-on-application-key-limited-to-a-prefix/22159/

maltfield · August 31, 2023, 12:53pm

Thank you Looks like it! My bad; I only searched the GitHub Issues.

Is there any easy way to search GitHub Commits in the WUI?

Also thank you for your thorough commit messages

Animosity022 · August 31, 2023, 12:59pm

Take this as I don't mean to be snarky / mean / etc as that's why the first step we always ask is to test on the new version and not an old version and there isn't time wasted on something that's already fixed as that's time spent elsewhere in rclone for @ncw to code

maltfield · August 31, 2023, 1:07pm

Agreed. Hopefully the documentation can be added that allows users to securely install the latest version soon. I see it's targeted for the next release <3

ncw · August 31, 2023, 3:46pm

You can use the search bar at the top of https://github.com/rclone/rclone - this searches everything! If you put 401 in there then click on commits you'll find the commit I referenced.

I make them for me in 6 months time mostly The first line goes in the changelog but unfortunately that didn't ring a bell for you.

system · September 3, 2023, 3:47pm

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.