B2 corrupts data when caps are reached

Suspected Bug
1

rclone has a bug with b2 when b2 reached caps (for example download cap).

The server returns 403, but rclone communicates as if everything is OK. Nothing is printed and the exit code is 0.

For example cat returns nothing without any error at all. copy tells there is nothing to transfer. This can corrupt data in a restore process for example, when a file is downloaded.

Here is with dump bodies.

2022/03/23 23:29:06 NOTICE: Automatically setting -vv as --dump is enabled
2022/03/23 23:29:06 DEBUG : rclone: Version "v1.58.0" starting with parameters ["rclone" "--config=c.conf" "--dump" "bodies" "cat" "b2:a/b.tgz"]
2022/03/23 23:29:06 DEBUG : Creating backend with remote "b2:a/b.tgz"
2022/03/23 23:29:06 DEBUG : Using config file from "c.conf"
2022/03/23 23:29:06 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2022/03/23 23:29:06 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/03/23 23:29:06 DEBUG : HTTP REQUEST (req 0xc0004e0400)
2022/03/23 23:29:06 DEBUG : GET /b2api/v1/b2_authorize_account HTTP/1.1
Host: api.backblazeb2.com
User-Agent: rclone/v1.58.0
Authorization: XXXX
Accept-Encoding: gzip

2022/03/23 23:29:06 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/03/23 23:29:07 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/03/23 23:29:07 DEBUG : HTTP RESPONSE (req 0xc0004e0400)
2022/03/23 23:29:07 DEBUG : HTTP/1.1 200
Content-Length: 750
Cache-Control: max-age=0, no-cache, no-store
Content-Type: application/json;charset=UTF-8
Date: Wed, 23 Mar 2022 23:29:06 GMT

{
  "absoluteMinimumPartSize": 5000000,
  "accountId": "xxx",
  "allowed": {
    "bucketId": "xxx",
    "bucketName": "a",
    "capabilities": [
      "listBuckets",
      "readBuckets",
      "listFiles",
      "readFiles",
      "shareFiles",
      "writeFiles",
      "deleteFiles",
      "readBucketEncryption",
      "writeBucketEncryption"
    ],
    "namePrefix": null
  },
  "apiUrl": "https://api000.backblazeb2.com",
  "authorizationToken": "xxx",
  "downloadUrl": "https://f000.backblazeb2.com",
  "minimumPartSize": 100000000,
  "recommendedPartSize": 100000000,
  "s3ApiUrl": "https://s3.us-west-000.backblazeb2.com"
}
2022/03/23 23:29:07 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/03/23 23:29:07 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/03/23 23:29:07 DEBUG : HTTP REQUEST (req 0xc0009f8600)
2022/03/23 23:29:07 DEBUG : HEAD /file/a/b.tgz HTTP/1.1
Host: f000.backblazeb2.com
User-Agent: rclone/v1.58.0
Authorization: XXXX

2022/03/23 23:29:07 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/03/23 23:29:07 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/03/23 23:29:07 DEBUG : HTTP RESPONSE (req 0xc0009f8600)
2022/03/23 23:29:07 DEBUG : HTTP/1.1 403
Content-Length: 199
Cache-Control: max-age=0, no-cache, no-store
Content-Type: application/json;charset=utf-8
Date: Wed, 23 Mar 2022 23:29:07 GMT

2022/03/23 23:29:07 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/03/23 23:29:07 DEBUG : Couldn't decode error response: EOF
2022/03/23 23:29:07 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/03/23 23:29:07 DEBUG : HTTP REQUEST (req 0xc0004e0400)
2022/03/23 23:29:07 DEBUG : POST /b2api/v1/b2_list_file_names HTTP/1.1
Host: api000.backblazeb2.com
User-Agent: rclone/v1.58.0
Content-Length: 140
Authorization: XXXX
Content-Type: application/json
Accept-Encoding: gzip

{"bucketId":"xxx","startFileName":"b.tgz/","maxFileCount":1000,"prefix":"b.tgz/"}
2022/03/23 23:29:07 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/03/23 23:29:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/03/23 23:29:08 DEBUG : HTTP RESPONSE (req 0xc0004e0400)
2022/03/23 23:29:08 DEBUG : HTTP/1.1 200
Content-Length: 42
Cache-Control: max-age=0, no-cache, no-store
Content-Type: application/json;charset=UTF-8
Date: Wed, 23 Mar 2022 23:29:07 GMT

{
  "files": [],
  "nextFileName": null
}
2022/03/23 23:29:08 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/03/23 23:29:08 DEBUG : 8 go routines active
2

Actually it corrupts data with copy, copyto, etc. everything. It returns error code 0 and tells "INFO : There was nothing to transfer".

The b2 web UI actually tells a meaningful error message, like "download cap reached". But even without nice message, rclone should terminate with error != 0.

3

--- can you post a debug log for rclone copy of a single file.
--- how do you know the data is corrupt?
--- the issue is with uploads and downloads?

4

I've just raised my caps so I'll have to wait again to change back but I believe the simplest replication of corruption would be the following:

  1. Download directory with copy
  2. Process terminates without error and error code 0.

It's not corrupt as in gibberish, but the directory is empty while the process terminated error code 0 and no warning.

5

When run on a folder, it works correctly. Shows a lot of errors including:

Failed to copy with 2 errors: last error was: multipart copy: failed to open source: failed to GET for download: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)

Exit code 7, great!

The problem is with single file behaviour.

rclone  copy b2:a/b.tgz c

=> no output, exit code 0

Here is a debug log:

2022/03/24 00:11:05 DEBUG : rclone: Version "v1.58.0" starting with parameters ["rclone" "--config=d.conf" "copy" "b2:a/b.tgz" "a" "-vv"]
2022/03/24 00:11:05 DEBUG : Creating backend with remote "b2:a/b.tgz"
2022/03/24 00:11:05 DEBUG : Using config file from "/root/.config/rclone/d.conf"
2022/03/24 00:11:06 DEBUG : Couldn't decode error response: EOF
2022/03/24 00:11:06 DEBUG : Creating backend with remote "a"
2022/03/24 00:11:06 DEBUG : fs cache: renaming cache item "a" to be canonical "/root/.config/rclone/a"
2022/03/24 00:11:07 DEBUG : Local file system at /root/.config/rclone/a: Waiting for checks to finish
2022/03/24 00:11:07 DEBUG : Local file system at /root/.config/rclone/a: Waiting for transfers to finish
2022/03/24 00:11:07 INFO  : There was nothing to transfer
2022/03/24 00:11:07 INFO  :
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Elapsed time:         1.9s

2022/03/24 00:11:07 DEBUG : 9 go routines active
6

I'll need to go back to using my account properly as it's a production account. If someone would like to replicate this, it's very simple, set a download cap of $0.01 and it reproduces nicely.

7

If you could do that one with -vv --dump bodies that would be really useful - thank you.

8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.