What is the problem you are having with rclone?
When copying to Azure Blob Storage, larger files often exceed 100%. Sometimes nearing 200%.
I've read other posts on this topic (such as this) and I know a common suggestion is to set --multi-thread-streams 0. I have tried this, but it seems to make no difference for me.
The issue of files exceeding 100% only seems to occur for larger files, particularly those which trigger multi-part uploads. Having run the log (attached below) with -vv, I don't see anything logged about retries, which was my initial thought. The log attached below shows a total data transfer of 1.773 GiB, however the folder being transferred only contains 970 MB of data. It appears to essentially be transferring twice the actual data size.
Any suggestions on what else I can try to remedy this are greatly appreciated!
Run the command 'rclone version' and share the full output of the command.
rclone v1.69.1
- os/version: ubuntu 24.04 (64 bit)
- os/kernel: 6.8.12-8-pve (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.24.0
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
Azure Blob Storage
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone copy /media/NAS1/photos/Archive p5azcrypt: --ignore-existing --filter-from ~/Rclone/Photos/photos-az-filter.txt --bwlimit 1.25M --azureblob-access-tier archive --progress --fast-list --multi-thread-streams 0 --log-file ~/Rclone/Photos/azlogs/`date +"%Y%m%d-%H%M"`.log
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
[AzRCPBackup]
type = azureblob
account = XXX
tenant = XXX
client_id = XXX
client_secret = XXX
[m1crypt]
type = crypt
remote = AzRCPBackup:rc1/m1
password = XXX
password2 = XXX
[p5azcrypt]
type = crypt
remote = AzRCPBackup:rc1/p5
password = XXX
password2 = XXX