How to reduce number of listed files on remote during fast sync with max age

Help and Support
1

What is the problem you are having with rclone?

I am running daily backups of my Proxmox Backup Server to Backblaze, and hourly backups of my NAS. Most of the files do not change between syncs, but I’m getting a lot of files being listed on the remote for checks regardless, which incurs a lot of Class C transactions which are costly. I’m starting to spend more on transactions that storage. I thought configuring the --max-age flag would help reduce the number of files listed, or removing the --fast-list, but that does not seem to be the case. Is there anything else I can do to reduce the number of files listed on the remote to bring down my transaction costs?

Run the command 'rclone version' and share the full output of the command.

root@nas:~# rclone version
rclone v1.73.2
- os/version: debian 12.12 (64 bit)
- os/kernel: 6.12.57+deb12-amd64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.25.8
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Backblaze

The command you were trying to run (eg rclone copy /tmp remote:tmp)

```
rclone sync --transfers 48 --b2-chunk-size 100M --fast-list --max-age=24h         --buffer-size=256M /backup/ my-bucket-crypt:
```

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

```
[my-bucket]
type = b2
account = XXX
key = XXX
hard_delete = true

[my-bucket-crypt]
type = crypt
remote = my-bucket:my-bucket
password = XXX
```

A log from the command that you were trying to run with the -vv flag

Contains summarized lines that were repeated many times, original log file is 66MB.

```
2026/03/20 10:32:11 DEBUG : rclone: Version "v1.71.1" starting with parameters ["rclone" "sync" "--transfers" "48" "--b2-chunk-size" "100M" "--fast-list" "--max-age=24h" "--buffer-size=256M" "-vv" "/backup/" "my-bucket-crypt:"]
2026/03/20 10:32:11 DEBUG : Creating backend with remote "/backup/"
2026/03/20 10:32:11 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2026/03/20 10:32:11 DEBUG : fs cache: renaming cache item "/backup/" to be canonical "/backup"
2026/03/20 10:32:11 DEBUG : Creating backend with remote "my-bucket-crypt:"
2026/03/20 10:32:11 DEBUG : Creating backend with remote "my-bucket-config:my-bucket"
2026/03/20 10:32:11 DEBUG : my-bucket-config: detected overridden config - adding "{HvT2Y}" suffix to name
2026/03/20 10:32:12 DEBUG : fs cache: renaming cache item "my-bucket-config:my-bucket" to be canonical "my-bucket-config{HvT2Y}:my-bucket"
[SUMMARY: 143212 lines]
  [143210x] .chunks/XXXX/XXXXXXXX: Excluded
    Example: 2026/03/20 10:32:12 DEBUG : .chunks/0000/0000e622e6ed01867cf1c1bc59f77467bb01e26e811574b068e10bbab5694621: Excluded (ModTime Filter)
  [2x] .lock: Excluded
    Example: 2026/03/20 10:32:12 DEBUG : .lock: Excluded (ModTime Filter)
2026/03/20 10:33:12 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Checks:                 0 / 0, -, Listed 143405
Elapsed time:       1m0.0s

[SUMMARY: 37854 lines]
  [37718x] .chunks/XXXX/XXXXXXXX: Excluded
    Example: 2026/03/20 10:33:12 DEBUG : .chunks/ca3c/ca3c43258bd32f41f76cd62b86d43c58ec0e409cca1e934bf27ca4762145ef5b: Excluded (ModTime Filter)
  [99x] vm/XXX/XXXX-XX-XXTXX:XX:XXZ/FILE: Excluded
    Example: 2026/03/20 10:33:53 DEBUG : vm/101/2026-02-22T04:00:01Z/drive-scsi0.img.fidx: Excluded (ModTime Filter)
  [36x] ct/XXX/XXXX-XX-XXTXX:XX:XXZ/FILE: Excluded
    Example: 2026/03/20 10:33:53 DEBUG : ct/105/2026-02-22T05:00:01Z/root.pxar.didx: Excluded (ModTime Filter)
  [1x] .lock: Excluded
    Example: 2026/03/20 10:33:53 DEBUG : .lock: Excluded (ModTime Filter)
2026/03/20 10:33:53 DEBUG : .gc-status: Size and modification time the same (differ by -576.111µs, within tolerance 1ms)
2026/03/20 10:33:53 DEBUG : .gc-status: Unchanged skipping
[SUMMARY: 112 lines]
  [112x] vm/XXX/XXXX-XX-XXTXX:XX:XXZ/FILE: Excluded
    Example: 2026/03/20 10:33:53 DEBUG : vm/203/owner: Excluded (ModTime Filter)
2026/03/20 10:33:53 DEBUG : vm/101/2026-03-20T04:00:04Z/client.log.blob: Size and modification time the same (differ by -408.744µs, within tolerance 1ms)
2026/03/20 10:33:53 DEBUG : vm/101/2026-03-20T04:00:04Z/client.log.blob: Unchanged skipping
2026/03/20 10:33:53 DEBUG : vm/101/2026-03-20T04:00:04Z/drive-scsi0.img.fidx: Size and modification time the same (differ by -277.026µs, within tolerance 1ms)
2026/03/20 10:33:53 DEBUG : vm/101/2026-03-20T04:00:04Z/drive-scsi0.img.fidx: Unchanged skipping
2026/03/20 10:33:53 DEBUG : vm/101/2026-03-20T04:00:04Z/index.json.blob: Size and modification time the same (differ by -814.802µs, within tolerance 1ms)
2026/03/20 10:33:53 DEBUG : vm/101/2026-03-20T04:00:04Z/index.json.blob: Unchanged skipping
2026/03/20 10:33:53 DEBUG : vm/101/2026-03-20T04:00:04Z/qemu-server.conf.blob: Size and modification time the same (differ by -241.634µs, within tolerance 1ms)
2026/03/20 10:33:53 DEBUG : vm/101/2026-03-20T04:00:04Z/qemu-server.conf.blob: Unchanged skipping
[SUMMARY: 122 lines]
  [70x] vm/XXX/XXXX-XX-XXTXX:XX:XXZ/FILE: Excluded
    Example: 2026/03/20 10:33:53 DEBUG : vm/103/2026-01-25T03:06:22Z/drive-scsi0.img.fidx: Excluded (ModTime Filter)
  [52x] ct/XXX/XXXX-XX-XXTXX:XX:XXZ/FILE: Excluded
    Example: 2026/03/20 10:33:53 DEBUG : ct/105/owner: Excluded (ModTime Filter)
2026/03/20 10:33:53 DEBUG : ct/105/2026-03-20T05:00:05Z/catalog.pcat1.didx: Size and modification time the same (differ by -654.15µs, within tolerance 1ms)
2026/03/20 10:33:53 DEBUG : ct/105/2026-03-20T05:00:05Z/catalog.pcat1.didx: Unchanged skipping
2026/03/20 10:33:53 DEBUG : ct/105/2026-03-20T05:00:05Z/client.log.blob: Size and modification time the same (differ by -770.319µs, within tolerance 1ms)
2026/03/20 10:33:53 DEBUG : ct/105/2026-03-20T05:00:05Z/pct.conf.blob: Size and modification time the same (differ by -832.888µs, within tolerance 1ms)
2026/03/20 10:33:53 DEBUG : ct/105/2026-03-20T05:00:05Z/client.log.blob: Unchanged skipping
2026/03/20 10:33:53 DEBUG : ct/105/2026-03-20T05:00:05Z/pct.conf.blob: Unchanged skipping
2026/03/20 10:33:53 DEBUG : ct/105/2026-03-20T05:00:05Z/index.json.blob: Size and modification time the same (differ by -960.467µs, within tolerance 1ms)
2026/03/20 10:33:53 DEBUG : ct/105/2026-03-20T05:00:05Z/index.json.blob: Unchanged skipping
2026/03/20 10:33:53 DEBUG : ct/105/2026-03-20T05:00:05Z/root.pxar.didx: Size and modification time the same (differ by -654.05µs, within tolerance 1ms)
2026/03/20 10:33:53 DEBUG : ct/105/2026-03-20T05:00:05Z/root.pxar.didx: Unchanged skipping
[SUMMARY: 361410 lines]
  [361108x] .chunks/XXXX/XXXXXXXX: Excluded
    Example: 2026/03/20 10:33:54 DEBUG : .chunks/0003/0003d3db987dba00ca031ddc47fcbd3f71404f23f9eab60458f727f10c36cb1c: Excluded (ModTime Filter)
  [151x] .chunks/XXXX/XXXXXXXX: Size and modification time the same
    Example: 2026/03/20 10:33:54 DEBUG : .chunks/00c2/00c2582287f6eb58d860c85a22cbaf95db0ee615a1aa5a123da9d5a491cb9eb8: Size and modification time the same (differ by -247.033µs, within tolerance 1ms)
  [151x] .chunks/XXXX/XXXXXXXX: Unchanged skipping
    Example: 2026/03/20 10:33:54 DEBUG : .chunks/00c2/00c2582287f6eb58d860c85a22cbaf95db0ee615a1aa5a123da9d5a491cb9eb8: Unchanged skipping
2026/03/20 10:33:58 DEBUG : Encrypted drive 'my-bucket-crypt:': Waiting for checks to finish
2026/03/20 10:33:58 DEBUG : Encrypted drive 'my-bucket-crypt:': Waiting for transfers to finish
2026/03/20 10:33:58 DEBUG : Waiting for deletions to finish
2026/03/20 10:33:58 INFO  : There was nothing to transfer
2026/03/20 10:33:58 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Checks:               161 / 161, 100%, Listed 427629
Elapsed time:      1m46.0s

2026/03/20 10:33:58 DEBUG : 7 go routines active
```
2

--max--age and--fast-list are definitely good start. Add --no-traverse to get max benefit of minimising number of transactions. Very important is also to use copy instead of sync as the --no-traverse does not work with sync:

rclone copy /local/data b2:my-bucket --fast-list --max-age 24h --no-traverse

When using --max--age it is good practice to run full sync periodically too - to ensure that nothing was missed.

3

PS.

Proxmox nowadays supports S3 backup directly:

And Backblaze supports S3. Might be much better (because native) and efficient (it uses a lot of local caching to limit S3 interactions) way to achieve what you want (cloud backup of your PVE VMs).

4

FWIW, Backblaze is also going to make API calls free from 1st May: Backblaze Pricing and Product Updates

Free API calls: Effective May 1, we’re making API calls free for all B2 Cloud Storage customers.* This removes transaction costs and makes it easier to build, scale, and run high-volume workloads subject to our standard platform usage rules and Terms of Service.

2 Likes
5

Changing to copy with no traverse seems to have done the trick. I was running an hourly sync on a NAS with many files which seems to have been the issue.

It seems with what @darthShadow shared this will stop being an issue for now, but until the caps are removed I can get by with a daily sync and an hourly copy to keep the transaction costs down. Thank you both!

6

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.