Azure Blob Storage: Large File Never Uploads

What is the problem you are having with rclone?

When using Azure Recovery Vault to restore backups to filesystem at a path where rclone has mounted an Azure Blob Storage container, all files restore and are copied to Blob storage successfully except extremely large files above 200 gb

The functional account used to run rclone as a registered service has full rights on the machine. Looking at the log it appears as though rclone is marking the file as uploading but it never gets a byte transferred.

Run the command 'rclone version' and share the full output of the command.

rclone v1.61.1
- os/version: Microsoft Windows Server 2019 Datacenter 1809 (64 bit)
- os/kernel: 10.0.17763.4010 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.19.4
- go/linking: static
- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Azure Blob Storage

The command you were trying to run (eg rclone copy /tmp remote:tmp)

 mount database:database c:\database --vfs-cache-mode full -v --log-file=C:\backup\rclone.log 

The rclone config contents with secrets removed.

[database]
type = azureblob
sas_url = XXXREDACTEDXXX

[asdf]
type = azureblob
account = exit
service_principal_file = exit
key = exi
sas_url = exit

A log from the command with the -vv flag

2023/03/14 10:06:07 INFO  : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 0, uploading 1, total size 313.923Gi (was 313.923Gi)
2023/03/14 10:07:05 DEBUG : rclone: Version "v1.61.1" starting with parameters ["C:\\ProgramData\\chocolatey\\lib\\rclone.portable\\tools\\rclone-v1.61.1-windows-amd64\\rclone.exe" "mount" "database:database" "c:\\database" "--vfs-cache-mode" "full" "-vv" "--log-file=C:\\backup\\rclone.log"]
2023/03/14 10:07:05 DEBUG : Creating backend with remote "database:database"
2023/03/14 10:07:05 DEBUG : Using config file from "C:\\Users\\funct_db\\AppData\\Roaming\\rclone\\rclone.conf"
2023/03/14 10:07:05 INFO  : Azure container database: poll-interval is not supported by this remote
2023/03/14 10:07:05 DEBUG : vfs cache: root is "C:\\Users\\funct_db\\AppData\\Local\\rclone"
2023/03/14 10:07:05 DEBUG : vfs cache: data root is "\\\\?\\C:\\Users\\funct_db\\AppData\\Local\\rclone\\vfs\\database\\database"
2023/03/14 10:07:05 DEBUG : vfs cache: metadata root is "\\\\?\\C:\\Users\\funct_db\\AppData\\Local\\rclone\\vfsMeta\\database\\database"
2023/03/14 10:07:05 DEBUG : Creating backend with remote "C:/Users/funct_db/AppData/Local/rclone/vfs/database/database"
2023/03/14 10:07:05 DEBUG : fs cache: renaming cache item "C:/Users/funct_db/AppData/Local/rclone/vfs/database/database" to be canonical "//?/C:/Users/funct_db/AppData/Local/rclone/vfs/database/database"
2023/03/14 10:07:05 DEBUG : Creating backend with remote "C:/Users/funct_db/AppData/Local/rclone/vfsMeta/database/database"
2023/03/14 10:07:05 DEBUG : fs cache: renaming cache item "C:/Users/funct_db/AppData/Local/rclone/vfsMeta/database/database" to be canonical "//?/C:/Users/funct_db/AppData/Local/rclone/vfsMeta/database/database"
2023/03/14 10:07:05 DEBUG : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: vfs cache: truncate to size=337072597504 (not needed as size correct)
2023/03/14 10:07:10 DEBUG : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: vfs cache: setting modification time to 2023-03-14 08:55:52.2324913 -0500 CDT
2023/03/14 10:07:10 INFO  : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: vfs cache: queuing for upload in 5s
2023/03/14 10:07:10 DEBUG : XTEvent: Added virtual directory entry vAddFile: "XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak"
2023/03/14 10:07:10 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak not removed, freed 0 bytes
2023/03/14 10:07:10 INFO  : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 1, uploading 0, total size 313.923Gi (was 313.923Gi)
2023/03/14 10:07:10 DEBUG : Network mode mounting is disabled
2023/03/14 10:07:10 DEBUG : Mounting on "c:\\database" ("database database")
2023/03/14 10:07:10 DEBUG : Azure container database: Mounting with options: ["-o" "attr_timeout=1" "-o" "uid=-1" "-o" "gid=-1" "--FileSystemName=rclone" "-o" "volname=database database"]
2023/03/14 10:07:11 DEBUG : Azure container database: Init: 
2023/03/14 10:07:11 DEBUG : Azure container database: >Init: 
2023/03/14 10:07:11 DEBUG : /: Statfs: 
2023/03/14 10:07:11 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2023/03/14 10:07:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:07:11 DEBUG : /: >Getattr: errc=0
2023/03/14 10:07:11 DEBUG : /: Readlink: 
2023/03/14 10:07:11 DEBUG : /: >Readlink: linkPath="", errc=-40
2023/03/14 10:07:11 DEBUG : /: Getxattr: name="non-existant-a11ec902d22f4ec49003af15282d3b00"
2023/03/14 10:07:11 DEBUG : /: >Getxattr: errc=-40, value=""
2023/03/14 10:07:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:07:11 DEBUG : /: >Getattr: errc=0
2023/03/14 10:07:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:07:11 DEBUG : /: >Getattr: errc=0
2023/03/14 10:07:11 DEBUG : /: Opendir: 
2023/03/14 10:07:11 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:07:11 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:07:11 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:07:11 DEBUG : /: Statfs: 
2023/03/14 10:07:11 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2023/03/14 10:07:11 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:07:11 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:07:12 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:07:12 DEBUG : /: >Getattr: errc=0
2023/03/14 10:07:12 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:07:12 DEBUG : /: >Getattr: errc=0
2023/03/14 10:07:12 DEBUG : /: Opendir: 
2023/03/14 10:07:12 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:07:12 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:07:12 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:07:12 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:07:12 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:07:12 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:07:12 DEBUG : /: >Getattr: errc=0
2023/03/14 10:07:12 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:07:12 DEBUG : /: >Getattr: errc=0
2023/03/14 10:07:12 DEBUG : /: Opendir: 
2023/03/14 10:07:12 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:07:12 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:07:12 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:07:12 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:07:12 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:07:12 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:07:12 DEBUG : /: >Getattr: errc=0
2023/03/14 10:07:12 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:07:12 DEBUG : /: >Getattr: errc=0
2023/03/14 10:07:12 DEBUG : /: Opendir: 
2023/03/14 10:07:12 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:07:12 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:07:12 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:07:12 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:07:12 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:07:12 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:07:12 DEBUG : /: >Getattr: errc=0
2023/03/14 10:07:12 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:07:12 DEBUG : /: >Getattr: errc=0
2023/03/14 10:07:12 DEBUG : /: Opendir: 
2023/03/14 10:07:12 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:07:12 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:07:12 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:07:12 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:07:12 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:07:12 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:07:12 DEBUG : /: >Getattr: errc=0
2023/03/14 10:07:12 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:07:12 DEBUG : /: >Getattr: errc=0
2023/03/14 10:07:12 DEBUG : /: Opendir: 
2023/03/14 10:07:12 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:07:12 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:07:12 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:07:12 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:07:12 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:07:12 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:07:12 DEBUG : /: >Getattr: errc=0
2023/03/14 10:07:12 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:07:12 DEBUG : /: >Getattr: errc=0
2023/03/14 10:07:12 DEBUG : /: Opendir: 
2023/03/14 10:07:12 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:07:12 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:07:12 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:07:12 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:07:12 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:07:12 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:07:12 DEBUG : /: >Getattr: errc=0
2023/03/14 10:07:12 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:07:12 DEBUG : /: >Getattr: errc=0
2023/03/14 10:07:12 DEBUG : /: Opendir: 
2023/03/14 10:07:12 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:07:12 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:07:12 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:07:12 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:07:12 DEBUG : /: >Getattr: errc=0
2023/03/14 10:07:12 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:07:12 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:07:15 DEBUG : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: vfs cache: starting upload
2023/03/14 10:08:10 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak not removed, freed 0 bytes
2023/03/14 10:08:10 INFO  : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 0, uploading 1, total size 313.923Gi (was 313.923Gi)
2023/03/14 10:09:10 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak not removed, freed 0 bytes
2023/03/14 10:09:10 INFO  : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 0, uploading 1, total size 313.923Gi (was 313.923Gi)
2023/03/14 10:09:36 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:09:36 DEBUG : /: >Getattr: errc=0
2023/03/14 10:09:36 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:09:36 DEBUG : /: >Getattr: errc=0
2023/03/14 10:09:36 DEBUG : /: Opendir: 
2023/03/14 10:09:36 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:09:36 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:09:36 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:09:36 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:09:36 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:09:36 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:09:36 DEBUG : /: >Getattr: errc=0
2023/03/14 10:09:36 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:09:36 DEBUG : /: >Getattr: errc=0
2023/03/14 10:09:36 DEBUG : /: Opendir: 
2023/03/14 10:09:36 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:09:36 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:09:36 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:09:36 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:09:36 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:09:36 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:09:36 DEBUG : /: >Getattr: errc=0
2023/03/14 10:09:36 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:09:36 DEBUG : /: >Getattr: errc=0
2023/03/14 10:09:36 DEBUG : /: Opendir: 
2023/03/14 10:09:36 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:09:36 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:09:36 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:09:36 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:09:36 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:09:36 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:09:36 DEBUG : /: >Getattr: errc=0
2023/03/14 10:09:36 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:09:36 DEBUG : /: >Getattr: errc=0
2023/03/14 10:09:36 DEBUG : /: Opendir: 
2023/03/14 10:09:36 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:09:36 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:09:36 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:09:36 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:09:36 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:10:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:10:08 DEBUG : /: >Getattr: errc=0
2023/03/14 10:10:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:10:08 DEBUG : /: >Getattr: errc=0
2023/03/14 10:10:08 DEBUG : /: Opendir: 
2023/03/14 10:10:08 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:10:08 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:10:08 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:10:08 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:10:08 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:10:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:10:08 DEBUG : /: >Getattr: errc=0
2023/03/14 10:10:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:10:08 DEBUG : /: >Getattr: errc=0
2023/03/14 10:10:08 DEBUG : /: Opendir: 
2023/03/14 10:10:08 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:10:08 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:10:08 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:10:08 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:10:08 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:10:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:10:08 DEBUG : /: >Getattr: errc=0
2023/03/14 10:10:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:10:08 DEBUG : /: >Getattr: errc=0
2023/03/14 10:10:08 DEBUG : /: Opendir: 
2023/03/14 10:10:08 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:10:08 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:10:08 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:10:08 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:10:08 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:10:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:10:08 DEBUG : /: >Getattr: errc=0
2023/03/14 10:10:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:10:08 DEBUG : /: >Getattr: errc=0
2023/03/14 10:10:08 DEBUG : /: Opendir: 
2023/03/14 10:10:08 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:10:08 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:10:08 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:10:08 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:10:08 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:10:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:10:08 DEBUG : /: >Getattr: errc=0
2023/03/14 10:10:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:10:08 DEBUG : /: >Getattr: errc=0
2023/03/14 10:10:08 DEBUG : /: Opendir: 
2023/03/14 10:10:08 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:10:08 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:10:08 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:10:08 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:10:08 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:10:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:10:08 DEBUG : /: >Getattr: errc=0
2023/03/14 10:10:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:10:08 DEBUG : /: >Getattr: errc=0
2023/03/14 10:10:08 DEBUG : /: Opendir: 
2023/03/14 10:10:08 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:10:08 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:10:08 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:10:08 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:10:08 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:10:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:10:08 DEBUG : /: >Getattr: errc=0
2023/03/14 10:10:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:10:08 DEBUG : /: >Getattr: errc=0
2023/03/14 10:10:08 DEBUG : /: Opendir: 
2023/03/14 10:10:08 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:10:08 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:10:08 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:10:08 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:10:08 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:10:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:10:08 DEBUG : /: >Getattr: errc=0
2023/03/14 10:10:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:10:08 DEBUG : /: >Getattr: errc=0
2023/03/14 10:10:08 DEBUG : /: Opendir: 
2023/03/14 10:10:08 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:10:08 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:10:08 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:10:08 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:10:08 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:10:10 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak not removed, freed 0 bytes
2023/03/14 10:10:10 INFO  : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 0, uploading 1, total size 313.923Gi (was 313.923Gi)
2023/03/14 10:11:10 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak not removed, freed 0 bytes
2023/03/14 10:11:10 INFO  : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 0, uploading 1, total size 313.923Gi (was 313.923Gi)
2023/03/14 10:11:30 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:11:30 DEBUG : /: >Getattr: errc=0
2023/03/14 10:11:30 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:11:30 DEBUG : /: >Getattr: errc=0
2023/03/14 10:11:30 DEBUG : /: Opendir: 
2023/03/14 10:11:30 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:11:30 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:11:30 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:11:30 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:11:30 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:11:30 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:11:30 DEBUG : /: >Getattr: errc=0
2023/03/14 10:11:30 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:11:30 DEBUG : /: >Getattr: errc=0
2023/03/14 10:11:30 DEBUG : /: Opendir: 
2023/03/14 10:11:30 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:11:30 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:11:30 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:11:30 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:11:30 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:11:30 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:11:30 DEBUG : /: >Getattr: errc=0
2023/03/14 10:11:30 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:11:30 DEBUG : /: >Getattr: errc=0
2023/03/14 10:11:30 DEBUG : /: Opendir: 
2023/03/14 10:11:30 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:11:30 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:11:30 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:11:30 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:11:30 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:11:30 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:11:30 DEBUG : /: >Getattr: errc=0
2023/03/14 10:11:30 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:11:30 DEBUG : /: >Getattr: errc=0
2023/03/14 10:11:30 DEBUG : /: Opendir: 
2023/03/14 10:11:30 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:11:30 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:11:30 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:11:30 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:11:30 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:11:30 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:11:30 DEBUG : /: >Getattr: errc=0
2023/03/14 10:11:30 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:11:30 DEBUG : /: >Getattr: errc=0
2023/03/14 10:11:30 DEBUG : /: Opendir: 
2023/03/14 10:11:30 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:11:30 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:11:30 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:11:30 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:11:30 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:11:30 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:11:30 DEBUG : /: >Getattr: errc=0
2023/03/14 10:11:30 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:11:30 DEBUG : /: >Getattr: errc=0
2023/03/14 10:11:30 DEBUG : /: Opendir: 
2023/03/14 10:11:30 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:11:30 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:11:30 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:11:30 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:11:30 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:11:30 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:11:30 DEBUG : /: >Getattr: errc=0
2023/03/14 10:11:30 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:11:30 DEBUG : /: >Getattr: errc=0
2023/03/14 10:11:30 DEBUG : /: Opendir: 
2023/03/14 10:11:30 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:11:30 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:11:30 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:11:30 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:11:30 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:11:30 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:11:30 DEBUG : /: >Getattr: errc=0
2023/03/14 10:11:30 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/03/14 10:11:30 DEBUG : /: >Getattr: errc=0
2023/03/14 10:11:30 DEBUG : /: Opendir: 
2023/03/14 10:11:30 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/03/14 10:11:30 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/03/14 10:11:30 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/03/14 10:11:30 DEBUG : /: Releasedir: fh=0x0
2023/03/14 10:11:30 DEBUG : /: >Releasedir: errc=0
2023/03/14 10:12:10 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak not removed, freed 0 bytes
2023/03/14 10:12:10 INFO  : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 0, uploading 1, total size 313.923Gi (was 313.923Gi)
2023/03/14 10:13:10 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak not removed, freed 0 bytes
2023/03/14 10:13:10 INFO  : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 0, uploading 1, total size 313.923Gi (was 313.923Gi)
2023/03/14 10:14:10 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak not removed, freed 0 bytes
2023/03/14 10:14:10 INFO  : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 0, uploading 1, total size 313.923Gi (was 313.923Gi)
2023/03/14 10:15:10 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak not removed, freed 0 bytes
2023/03/14 10:15:10 INFO  : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 0, uploading 1, total size 313.923Gi (was 313.923Gi)
2023/03/14 10:16:10 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak not removed, freed 0 bytes
2023/03/14 10:16:10 INFO  : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 0, uploading 1, total size 313.923Gi (was 313.923Gi)


hello and welcome to the forum,

--azureblob-disable-checksum
"rclone will calculate the MD5 checksum of the input before uploading"

fwiw, much easier to test using rclone copy, not rclone mount

1 Like

Thanks @asdffdsa , I applied that change. Now we quickly and easily see there is an issue handling the upload in the logs:

RESPONSE 409: 409 The uncommitted block count cannot exceed the maximum limit of 100,000 blocks.
ERROR CODE: BlockCountExceedsLimit
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>BlockCountExceedsLimit</Code><Message>The uncommitted block count cannot exceed the maximum limit of 100,000 blocks.
RequestId:78ba1aad-201e-0002-672f-573724000000
Time:2023-03-15T11:19:03.1545663Z</Message></Error>
--------------------------------------------------------------------------------
2023/03/15 06:19:03 ERROR : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: vfs cache: failed to upload try #218, will retry in 5m0s: vfs cache: failed to transfer file from cache to remote: multipart upload failed to upload part: PUT https://[redacted]/database/XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak
--------------------------------------------------------------------------------
RESPONSE 409: 409 The uncommitted block count cannot exceed the maximum limit of 100,000 blocks.
ERROR CODE: BlockCountExceedsLimit
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>BlockCountExceedsLimit</Code><Message>The uncommitted block count cannot exceed the maximum limit of 100,000 blocks.
RequestId:78ba1aad-201e-0002-672f-573724000000
Time:2023-03-15T11:19:03.1545663Z</Message></Error>
--------------------------------------------------------------------------------
2023/03/15 06:19:41 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak not removed, freed 0 bytes
2023/03/15 06:19:41 INFO  : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 1, uploading 0, total size 313.923Gi (was 313.923Gi)
2023/03/15 06:20:41 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak not removed, freed 0 bytes
2023/03/15 06:20:41 INFO  : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 1, uploading 0, total size 313.923Gi (was 313.923Gi)
2023/03/15 06:21:41 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak not removed, freed 0 bytes
2023/03/15 06:21:41 INFO  : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 1, uploading 0, total size 313.923Gi (was 313.923Gi)
2023/03/15 06:22:41 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak not removed, freed 0 bytes
2023/03/15 06:22:41 INFO  : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 1, uploading 0, total size 313.923Gi (was 313.923Gi)
2023/03/15 06:23:41 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak not removed, freed 0 bytes
2023/03/15 06:23:41 INFO  : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 1, uploading 0, total size 313.923Gi (was 313.923Gi)
2023/03/15 06:24:03 DEBUG : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: vfs cache: starting upload
2023/03/15 06:24:03 DEBUG : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: size: 313.923Gi, parts: 50000, default: 4Mi, new: 7Mi; default chunk size insufficient, returned new chunk size
2023/03/15 06:24:03 DEBUG : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: Multipart upload session started for 45923 parts of size 7Mi
2023/03/15 06:24:03 DEBUG : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: Uploading part 1/45923 offset 0/313.923Gi part size 7340032
2023/03/15 06:24:03 DEBUG : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: Uploading part 2/45923 offset 7Mi/313.923Gi part size 7340032
2023/03/15 06:24:03 DEBUG : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: Uploading part 3/45923 offset 14Mi/313.923Gi part size 7340032
2023/03/15 06:24:03 DEBUG : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: Uploading part 4/45923 offset 21Mi/313.923Gi part size 7340032
2023/03/15 06:24:03 DEBUG : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: Uploading part 5/45923 offset 28Mi/313.923Gi part size 7340032
2023/03/15 06:24:03 DEBUG : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: Uploading part 6/45923 offset 35Mi/313.923Gi part size 7340032
2023/03/15 06:24:03 DEBUG : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: Uploading part 7/45923 offset 42Mi/313.923Gi part size 7340032
2023/03/15 06:24:03 DEBUG : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: Uploading part 8/45923 offset 49Mi/313.923Gi part size 7340032
2023/03/15 06:24:03 DEBUG : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: Uploading part 9/45923 offset 56Mi/313.923Gi part size 7340032
2023/03/15 06:24:03 DEBUG : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: Uploading part 10/45923 offset 63Mi/313.923Gi part size 7340032
2023/03/15 06:24:03 DEBUG : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: Uploading part 11/45923 offset 70Mi/313.923Gi part size 7340032
2023/03/15 06:24:04 DEBUG : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: Uploading part 12/45923 offset 77Mi/313.923Gi part size 7340032
2023/03/15 06:24:04 ERROR : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: Failed to copy: multipart upload failed to upload part: PUT https://prodgpv2mom.blob.core.windows.net/database/XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak
--------------------------------------------------------------------------------
RESPONSE 409: 409 The uncommitted block count cannot exceed the maximum limit of 100,000 blocks.
ERROR CODE: BlockCountExceedsLimit
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>BlockCountExceedsLimit</Code><Message>The uncommitted block count cannot exceed the maximum limit of 100,000 blocks.
RequestId:78bba98f-201e-0002-1e30-573724000000
Time:2023-03-15T11:24:04.0030681Z</Message></Error>
--------------------------------------------------------------------------------
2023/03/15 06:24:04 ERROR : XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak: vfs cache: failed to upload try #219, will retry in 5m0s: vfs cache: failed to transfer file from cache to remote: multipart upload failed to upload part: PUT https://[redacted]/database/XTEvent/XTEvent_Full_5422571018578846353_2023-03-08_08-24-55.bak
--------------------------------------------------------------------------------
RESPONSE 409: 409 The uncommitted block count cannot exceed the maximum limit of 100,000 blocks.
ERROR CODE: BlockCountExceedsLimit
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>BlockCountExceedsLimit</Code><Message>The uncommitted block count cannot exceed the maximum limit of 100,000 blocks.
RequestId:78bba98f-201e-0002-1e30-573724000000
Time:2023-03-15T11:24:04.0030681Z</Message></Error>

I have a theory it might be related to block / chunk size that I commented on and ncw replied in GitHub:

I see that the limit of 50,000 is applied in this pull request, but my own error complains about 100,000 limit:

So I searched and found the limit error I am hitting is because we are using 50,000 blocks and then once it fails a retry causes the uncommitted blocks from the last attempt to remain and can add new uncommitted blocks on the new attempt:

I'm going to try with a new file and report back.

good, now we know the issue.

in the meantime, can tweak --azureblob-chunk-size
and might test using lastest version....

1 Like

If you want to delete the uncommitted blocks then upload a small (or zero length) file of the same name and path as the failed file (perhaps with rclone touch) then delete it. This will release the uncommitted blocks.

1 Like

I triggered a new backup file download from the vault so we would get a clean test and found it is now passing:


2023/03/15 11:40:02 DEBUG : XTEvent/XTEvent_Full_5422564754304493325_2023-03-10_08-26-43.bak: Uploading part 46187/46190 offset 315.725Gi/315.748Gi part size 7340032
2023/03/15 11:40:02 DEBUG : XTEvent/XTEvent_Full_5422564754304493325_2023-03-10_08-26-43.bak: Uploading part 46188/46190 offset 315.731Gi/315.748Gi part size 7340032
2023/03/15 11:40:02 DEBUG : XTEvent/XTEvent_Full_5422564754304493325_2023-03-10_08-26-43.bak: Uploading part 46189/46190 offset 315.738Gi/315.748Gi part size 7340032
2023/03/15 11:40:02 DEBUG : XTEvent/XTEvent_Full_5422564754304493325_2023-03-10_08-26-43.bak: Uploading part 46190/46190 offset 315.745Gi/315.748Gi part size 2599424
2023/03/15 11:40:03 DEBUG : XTEvent/XTEvent_Full_5422564754304493325_2023-03-10_08-26-43.bak: Dst hash empty - aborting Src hash check
2023/03/15 11:40:03 INFO  : XTEvent/XTEvent_Full_5422564754304493325_2023-03-10_08-26-43.bak: Copied (new)
2023/03/15 11:40:03 DEBUG : XTEvent/XTEvent_Full_5422564754304493325_2023-03-10_08-26-43.bak: vfs cache: fingerprint now "339031337472,2023-03-15 15:53:56.6232384 +0000 UTC,"
2023/03/15 11:40:03 DEBUG : XTEvent/XTEvent_Full_5422564754304493325_2023-03-10_08-26-43.bak: vfs cache: writeback object to VFS layer
2023/03/15 11:40:03 DEBUG : XTEvent: Added virtual directory entry vAddFile: "XTEvent_Full_5422564754304493325_2023-03-10_08-26-43.bak"
2023/03/15 11:40:03 INFO  : XTEvent/XTEvent_Full_5422564754304493325_2023-03-10_08-26-43.bak: vfs cache: upload succeeded try #1
2023/03/15 11:40:11 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item XTEvent/XTEvent_Full_5422564754304493325_2023-03-10_08-26-43.bak not removed, freed 0 bytes
2023/03/15 11:40:11 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item XTEvent/5422564754304493325_metadata.json not removed, freed 0 bytes
2023/03/15 11:40:11 INFO  : vfs cache: cleaned: objects 2 (was 2) in use 0, to upload 0, uploading 0, total size 315.748Gi (was 315.748Gi)

Thanks for the suggestions, looks like all is resolved and working with only that disable checksum flag. I'm going to try a few more things on my end- rearrange, reset, resend, and reconfigure some jobs to ensure all works after the new parameter is set- but, it looks pretty good to me.

imho, would never use that flag, all the more so, as you mention backups
when you need to download the backup file, how will verify the transfer without checksums?

i have the same issue when transferring large veeam backups files on slow REFS using slow spinning disk on a slow CPU.
i just let rclone do its thing.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.