What is the problem you are having with rclone?
I have a question regarding the amount of WRITE operations generated for azure blob as destination that should be expected.
I was copying the below amount of data:
Total objects: 210304
Total size: 3.954 TBytes (4346937048495 Bytes)
Using Azure blob (hot) as destination and immediately setting the archive-flag with rclone
after copy I did a destination check as well to confirm:
time rclone -vv size ""
2021/03/05 09:17:25 DEBUG : rclone: Version "v1.54.0" starting with parameters ["rclone" "-vv" "size" ""]
2021/03/05 09:17:25 DEBUG : Using config file from ""
2021/03/05 09:17:25 DEBUG : Creating backend with remote ""
Total objects: 210281
Total size: 3.954 TBytes (4346933420158 Bytes)
2021/03/05 09:19:21 DEBUG : 4 go routines active
( a few files failed so number does not match 100%)
Above worked fine, but, and here is my concern / surprise, uploading these just over 200k objects as described above resulted in ~1.4 million "Tiered Block Blob - Hot LRS - Write Operations" on my azure charge, ie on average every object uploaded seems to have triggered almost 7 hot write operations in azure. any ideas why, and if this can be optimized in any way?
as a reference , the "Tiered Block Blob - Archive LRS - Write Operations" I was charged matches the ~200k quite nicely..
The azure figures comes from my actual monthly charges as published by Microsoft in my portal.
Comments or thoughts?
What is your rclone version (output from rclone version)
rclone v1.54.0
- os/arch: linux/amd64
- go version: go1.15.7
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Debian - bullseye/sid
Which cloud storage system are you using? (eg Google Drive)
Azure Blob (Hot->Archive)
The command you were trying to run (eg rclone copy /tmp remote:tmp)
/usr/bin/rclone --progress --max-depth -1 --checkers 28 --dropbox-impersonate <MASKED> --config <MASKED> --transfers=28 --log-file=$LOGPATH/rclone-<MASKED>-$date.log --log-level INFO
The rclone config contents with secrets removed.
[<MASKED>]
type = azureblob
account = <MASKED>
key = <MASKED>
access_tier = Archive
archive_tier_delete = true
A log from the command with the -vv flag
did not use -vv unfortunately