B2 daily download cap preventing server-side copy

Help and Support
1

What is the problem you are having with rclone?

I performed the following steps:

  • Do a server-side copy on B2 (rclone output clearly shows a server-side copy is taking place)
  • Check on B2 Caps & Alerts page and confirm download quota has not been consumed
  • Do a large download from B2 to local machine
  • Check on B2 Caps & Alerts page and confirm download quota has been exhausted
  • Attempt another server-side copy on B2 (same command as above) but receive the following rclone error:

2019/08/28 09:09:10 ERROR : path/to/file.pdf: Failed to copy: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)

My question is: Does rclone do some kind of download operation before/while doing a server-side copy or is this a B2 problem?

What is your rclone version (output from rclone version)

v1.48.0

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Ubuntu 64bit

Which cloud storage system are you using? (eg Google Drive)

B2

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy :b2:bucket-name/folder1 :b2:bucket-name/folder2 -vvv --stats 1s --stats-one-line --fast-list --transfers 14 --b2-account XXX --b2-key YYY

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

2019/08/28 10:24:21 DEBUG : rclone: Version "v1.48.0" starting with parameters ["/path/to/rclone-v1.48.0-linux-amd64/rclone" "copy" ":b2:bucket-name/folder1" ":b2:bucket-name/folder2" "-vvv" "--stats" "1s" "--stats-one-line" "--fast-list" "--transfers" "14" "--b2-account" "XXX" "--b2-key" "YYY"]
2019/08/28 10:24:21 DEBUG : Using config file from "/path/to/rclone.conf"
2019/08/28 10:24:25 INFO : B2 bucket bucket-name path folder2/: Waiting for checks to finish
2019/08/28 10:24:25 INFO : B2 bucket bucket-name path folder2/: Waiting for transfers to finish
2019/08/28 10:24:25 ERROR : test1 (3rd copy).pdf: Failed to copy: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2019/08/28 10:24:25 ERROR : Cancelling sync due to fatal error: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2019/08/28 10:24:25 ERROR : test1 (7th copy).pdf: Failed to copy: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2019/08/28 10:24:25 INFO : 0 / 1.428 GBytes, 0%, 0 Bytes/s, ETA - (xfr#0/35)
2019/08/28 10:24:25 ERROR : test2.txt: Failed to copy: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2019/08/28 10:24:26 ERROR : test1 (5th copy).pdf: Failed to copy: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2019/08/28 10:24:26 ERROR : test1 (4th copy).pdf: Failed to copy: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2019/08/28 10:24:26 ERROR : test1 (9th copy).pdf: Failed to copy: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2019/08/28 10:24:26 ERROR : test1 (6th copy).pdf: Failed to copy: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2019/08/28 10:24:26 ERROR : test.pdf: Failed to copy: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2019/08/28 10:24:26 ERROR : test1 (copy).pdf: Failed to copy: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2019/08/28 10:24:26 ERROR : test1.pdf: Failed to copy: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2019/08/28 10:24:26 ERROR : test1 (8th copy).pdf: Failed to copy: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2019/08/28 10:24:26 ERROR : test1 (another copy).pdf: Failed to copy: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2019/08/28 10:24:26 ERROR : test1.txt: Failed to copy: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2019/08/28 10:24:26 ERROR : test2.txt: Failed to copy: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2019/08/28 10:24:26 ERROR : Fatal error received - not attempting retries
2019/08/28 10:24:26 Failed to copy with 15 errors: last error was: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)

2

According to the b2 docs b2_copy_file should use a class C transaction.

However rclone has to find the source file with b2_get_file_info which is probably the class B transaction you are looking for. Not sure why that is a class B transaction...

If you want to know for certain, run with -vv --dump headers and rclone will show the transactions and you can look them up in the docs.

3

Thank you, I can try that. Though my main concern is that even if the server-side copy uses some class B transactions, it shouldn't fail if the download cap has been exhausted but the class B transaction cap is still not exhausted.
Thus I was wondering whether rclone for some or other reason performs an actual download during a server-side copy. From your answer, that doesn't seem to be the case, thus I believe B2 is incorrectly blocking the server-side copy when the download cap is exhausted but the class B transaction cap has not been exhausted yet.

4

So I did some more testing with the flags you mentioned. It does seem as if B2 blocks the b2_copy_file operation when the download cap is exhausted but the class B transaction cap is still not exhausted. Have asked Backblaze to look into this...
Thanks for the info!

1 Like
5

Can you report back what they say?

Thanks

6

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.