Why does rclone need to download to set mtime on B2?

Help and Support
1

What is the problem you are having with rclone?

Using rclone sync to send files to B2, nightly.

It has just recently started using up our free Class B (download) transactions. I'm not really worried about this, I'll just increase the cap, but I am curious why rclone needs to download the files to set the mtime, and if this might be changed in a newer version. The error messages from rclone (once the cap is hit) say like "Failed to set modification time: Cannot download file, download bandwidth or transaction (Class B) cap exceeded".

Once we hit the cap, rclone starts spamming its output with errors; the first chunk are all rclonelinks but then it goes on to regular files after that.

Run the command 'rclone version' and share the full output of the command.

rclone v1.53.0

  • os/arch: linux/amd64
  • go version: go1.15

Which cloud storage system are you using? (eg Google Drive)

B2

The command you were trying to run (eg rclone copy /tmp remote:tmp)

	rclone sync --use-mmap --local-no-check-updated --fast-list --links --quiet --transfers=24 --delete-excluded --exclude '*cache/**' --exclude 'tmp/**' --exclude '/journal/**' --bwlimit "07:00,5M 22:00,10M 00:00,off" "$d" "b2:snucbackup/$hostname$d"

The rclone config contents with secrets removed.

[b2]
type = b2
hard_delete = false

A log from the command with the -vv flag

Don't have it with -vv, but here's an excerpt:

2023/03/23 04:50:13 ERROR : rmt.rclonelink: Failed to set modification time: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2023/03/23 04:50:13 ERROR : printcap.rclonelink: Failed to set modification time: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2023/03/23 04:50:13 ERROR : shadow: Failed to set modification time: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2023/03/23 04:50:13 ERROR : pnm2ppa.conf: Failed to set modification time: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2023/03/23 04:50:13 ERROR : timezone: Failed to set modification time: Cannot download file, download bandwidth or transaction (Class B) cap exceeded. See the Caps & Alerts page to increase your cap. (403 download_cap_exceeded)
2

hello and welcome to the forum,

that version is several years old.
please update to lastest stable v1.62.2 and test again.

3

Yes, I know it is. And yet there's no reason for me to do the work to upgrade all of our systems if it still has the same behavior. Which is why I'm asking here.

4

Pick 1 system.
Update it.

rclone touch remote:somefile -vv

Share the results.

[felix@gemini ~]$ rclone touch DB:hosts -vv
2023/03/23 14:18:50 DEBUG : Setting --config "/opt/rclone/rclone.conf" from environment variable RCLONE_CONFIG="/opt/rclone/rclone.conf"
2023/03/23 14:18:50 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "touch" "DB:hosts" "-vv"]
2023/03/23 14:18:50 DEBUG : Creating backend with remote "DB:"
2023/03/23 14:18:50 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2023/03/23 14:18:50 DEBUG : Touch time 2023-03-23 14:18:50.635889427 -0400 EDT m=+0.034258130
2023/03/23 14:18:50 DEBUG : Dropbox root '': Touching "hosts"
2023/03/23 14:18:50 ERROR : Attempt 1/3 failed with 1 errors and: failed to touch: can't set modified time without deleting existing object
2023/03/23 14:18:50 DEBUG : Touch time 2023-03-23 14:18:50.872253381 -0400 EDT m=+0.270622082
2023/03/23 14:18:50 DEBUG : Dropbox root '': Touching "hosts"
2023/03/23 14:18:50 ERROR : Attempt 2/3 failed with 1 errors and: failed to touch: can't set modified time without deleting existing object
2023/03/23 14:18:50 DEBUG : Touch time 2023-03-23 14:18:50.971049252 -0400 EDT m=+0.369417949
2023/03/23 14:18:51 DEBUG : Dropbox root '': Touching "hosts"
2023/03/23 14:18:51 ERROR : Attempt 3/3 failed with 1 errors and: failed to touch: can't set modified time without deleting existing object
2023/03/23 14:18:51 DEBUG : 6 go routines active
2023/03/23 14:18:51 INFO  : Dropbox root '': Committing uploads - please wait...
2023/03/23 14:18:51 Failed to touch: failed to touch: can't set modified time without deleting existing object
5

Rclone isn't downloading the files, it is reading the info about the files which is also a class B transaction b2_get_file_info.

6

Thanks for the suggestion. Yeah, it looks like the current version does the same, so I'll just need to raise the cap. :slight_smile:

./rclone touch -vv b2:snucbackup/test
2023/03/24 13:23:12 DEBUG : rclone: Version "v1.62.2" starting with parameters ["./rclone" "touch" "-vv" "b2:snucbackup/test"]
2023/03/24 13:23:12 DEBUG : Creating backend with remote "b2:snucbackup/"
2023/03/24 13:23:12 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/03/24 13:23:12 Failed to create file system for "b2:snucbackup/": failed to authorize account: failed to authenticate: Transaction cap exceeded, see the Caps & Alerts page to increase your cap (403 transaction_cap_exceeded)
7

Yes sorry, I didn't bother to mention that. I'm not sure why it has to do that, either, since the same information appears to be returned by the various list methods. But ah well!

1 Like
8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.