Azure blob storage mount, large file transfer fails with vfs-cache mode off

Help and Support
1

What is the problem you are having with rclone?

Mounting Azure blob storage via winfsp, and trying to achieve fastest transfer possible for large files - in this case a 76GB zip file. I am running an Azure VM, connecting to azure blob storage standard - 10GBe Azure NICs.

Trying to disable caching as it takes too long when first copying 76GB to rclone cache, before beginning the upload.

My read of rclone documentation is that I should be able to upload without cache?

If I use --vfs-cache-mode full, the file will upload to blob successfully.

When transferring with cache disabled, it fails.

WriteFileHandle: Truncate: Can't change size without --vfs-cache-mode >= writes
WriteFileHandle.Write: can't seek in file without --vfs-cache-mode >= writes
WriteFileHandle.New Rcat failed: multipart upload failed to finalize: PUT

I have tried changing chunk size, --azureblob-upload-concurrency, cache mode disabled, moving tmp to another disk to ensure C:\ doesn't run out of space, almost all options I can think of.

Run the command 'rclone version' and share the full output of the command.

rclone v1.63.1

  • os/version: Microsoft Windows 10 Enterprise 22H2 (64 bit)
  • os/kernel: 10.0.19045.3324 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.20.6
  • go/linking: static
  • go/tags: cmount

Are you on the latest version of rclone? You can validate by checking the version listed here: Rclone downloads
--> yes

Which cloud storage system are you using? (eg Google Drive)

Azure Blob Storage

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone.exe mount prod-aus-data:1000094614-test-888 F: --volname "E-1000094614-test-888-ProjectEme" -dir-cache-time=15s --vfs-cache-mode off --azureblob-upload-concurrency 64 --azureblob-disable-checksum --azureblob-directory-markers --azureblob-chunk-size 1G --log-level DEBUG --log-file" "C:\rclone\connectscripts\Log\1000094614-test-888.txt"

The rclone config contents with secrets removed.

[prod-aus-Data]
type = azureblob
account = prodauedata
use_msi = true
scope = drive

A log from the command with the -vv flag

--------------------------------------------------------------------------------
2023/08/21 10:36:21 ERROR : 1000094614-CLIENT-118_22.zip: WriteFileHandle.Flush error: multipart upload failed to finalize: PUT https://prodauedata.blob.core.windows.net/1000094614-test-888/1000094614-CLIENT-118_22.zip
--------------------------------------------------------------------------------
RESPONSE 400: 400 The specified block list is invalid.
ERROR CODE: InvalidBlockList
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidBlockList</Code><Message>The specified block list is invalid.
RequestId:aa5f8d8f-901e-0003-02c7-d3be43000000
Time:2023-08-21T00:36:21.3289295Z</Message></Error>
--------------------------------------------------------------------------------
2023/08/21 10:36:21 ERROR : IO error: multipart upload failed to finalize: PUT https://prodauedata.blob.core.windows.net/1000094614-test-888/1000094614-CLIENT-118_22.zip
--------------------------------------------------------------------------------
RESPONSE 400: 400 The specified block list is invalid.
ERROR CODE: InvalidBlockList
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidBlockList</Code><Message>The specified block list is invalid.
RequestId:aa5f8d8f-901e-0003-02c7-d3be43000000
Time:2023-08-21T00:36:21.3289295Z</Message></Error>
--------------------------------------------------------------------------------
The service rclone has been started.

2023/08/21 10:44:39 ERROR : 1000094614-CLIENT-118_22.zip: WriteFileHandle: Truncate: Can't change size without --vfs-cache-mode >= writes
2023/08/21 10:44:39 NOTICE: 1000094614-CLIENT-118_22.zip: Streaming uploads using chunk size 1Gi will have maximum file size of 48.828Ti
2023/08/21 10:44:40 ERROR : 1000094614-CLIENT-118_22.zip: WriteFileHandle.Write: can't seek in file without --vfs-cache-mode >= writes
2023/08/21 10:44:41 ERROR : 1000094614-CLIENT-118_22.zip: WriteFileHandle.Write: can't seek in file without --vfs-cache-mode >= writes
2023/08/21 10:44:42 ERROR : 1000094614-CLIENT-118_22.zip: WriteFileHandle.Write: can't seek in file without --vfs-cache-mode >= writes
2023/08/21 10:44:45 ERROR : 1000094614-CLIENT-118_22.zip: WriteFileHandle: Truncate: Can't change size without --vfs-cache-mode >= writes
2023/08/21 10:44:46 ERROR : 1000094614-CLIENT-118_22.zip: WriteFileHandle.Write: can't seek in file without --vfs-cache-mode >= writes
2023/08/21 10:44:47 ERROR : 1000094614-CLIENT-118_22.zip: WriteFileHandle.Write: can't seek in file without --vfs-cache-mode >= writes
2023/08/21 10:44:48 ERROR : 1000094614-CLIENT-118_22.zip: WriteFileHandle.Write: can't seek in file without --vfs-cache-mode >= writes


The service rclone has been started.
2023/08/21 11:05:11 ERROR : Empty Folder/1000094614-CLIENT-118_22.zip: WriteFileHandle: Truncate: Can't change size without --vfs-cache-mode >= writes
2023/08/21 11:05:11 NOTICE: Empty Folder/1000094614-CLIENT-118_22.zip: Streaming uploads using chunk size 1Gi will have maximum file size of 48.828Ti
2023/08/21 11:05:12 ERROR : Empty Folder/1000094614-CLIENT-118_22.zip: WriteFileHandle.Write: can't seek in file without --vfs-cache-mode >= writes
2023/08/21 11:05:13 ERROR : Empty Folder/1000094614-CLIENT-118_22.zip: WriteFileHandle.Write: can't seek in file without --vfs-cache-mode >= writes
2023/08/21 11:05:14 ERROR : Empty Folder/1000094614-CLIENT-118_22.zip: WriteFileHandle.Write: can't seek in file without --vfs-cache-mode >= writes
2023/08/21 11:05:16 ERROR : Empty Folder/1000094614-CLIENT-118_22.zip: WriteFileHandle: Truncate: Can't change size without --vfs-cache-mode >= writes
2023/08/21 11:05:18 ERROR : Empty Folder/1000094614-CLIENT-118_22.zip: WriteFileHandle.Write: can't seek in file without --vfs-cache-mode >= writes
2023/08/21 11:05:19 ERROR : Empty Folder/1000094614-CLIENT-118_22.zip: WriteFileHandle.Write: can't seek in file without --vfs-cache-mode >= writes
2023/08/21 11:05:20 ERROR : Empty Folder/1000094614-CLIENT-118_22.zip: WriteFileHandle.Write: can't seek in file without --vfs-cache-mode >= writes
2023/08/21 11:06:33 DEBUG : rclone: Version "v1.63.1" starting with parameters ["c:\\rclone\\rclone.exe" "mount" "prod-aus-data:1000094614-test-888" "F:" "--volname "E-1000094614-test-888-ProjectEme" "--dir-cache-time=15s" "--vfs-cache-mode" "off" "--cache-dir" "e:\\rclone-cache\\" "--transfers" "64" "--azureblob-upload-concurrency" "64" "--azureblob-disable-checksum" "--azureblob-directory-markers" "--azureblob-chunk-size" "1G" "--log-level" "DEBUG" "--log-file" "C:\\rclone\\connectscripts\\Log\\1000094614-test-888.txt"]
2023/08/21 11:06:33 DEBUG : Creating backend with remote "prod-aus-data:1000094614-test-888"
2023/08/21 11:06:33 DEBUG : Using config file from "c:\\rclone\\rclone.conf"
2023/08/21 11:06:33 DEBUG : prod-aus-data: detected overridden config - adding "{xCwtg}" suffix to name
2023/08/21 11:06:33 DEBUG : Azure container 1000094614-test-888: Using directory markers
2023/08/21 11:06:33 DEBUG : fs cache: renaming cache item "prod-aus-data:1000094614-test-888" to be canonical "prod-aus-data{xCwtg}:1000094614-test-888"
2023/08/21 11:06:33 INFO  : Azure container 1000094614-test-888: poll-interval is not supported by this remote
2023/08/21 11:06:33 DEBUG : Network mode mounting is disabled
2023/08/21 11:06:33 DEBUG : Mounting on "F:" ("E-1000094614-test-888-ProjectEme")
2023/08/21 11:06:33 DEBUG : Azure container 1000094614-test-888: Mounting with options: ["-o" "attr_timeout=1" "-o" "uid=-1" "-o" "gid=-1" "--FileSystemName=rclone" "-o" "volname=E-1000094614-test-888-ProjectEme"]


2023/08/21 11:06:57 DEBUG : /Empty Folder: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/08/21 11:06:57 DEBUG : /Empty Folder: >Getattr: errc=0
2023/08/21 11:06:57 DEBUG : /Empty Folder: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/08/21 11:06:57 DEBUG : /Empty Folder: >Getattr: errc=0
2023/08/21 11:06:57 DEBUG : /Empty Folder: Opendir: 
2023/08/21 11:06:57 DEBUG : /Empty Folder: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/08/21 11:06:57 DEBUG : /Empty Folder: >OpenFile: fd=Empty Folder/ (r), err=<nil>
2023/08/21 11:06:57 DEBUG : /Empty Folder: >Opendir: errc=0, fh=0x0
2023/08/21 11:06:57 DEBUG : /Empty Folder: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/08/21 11:06:57 DEBUG : /Empty Folder: >Getattr: errc=0
2023/08/21 11:06:57 DEBUG : /Empty Folder: Releasedir: fh=0x0
2023/08/21 11:06:57 DEBUG : /Empty Folder: >Releasedir: errc=0
2023/08/21 11:06:57 DEBUG : /Empty Folder: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/08/21 11:06:57 DEBUG : /Empty Folder: >Getattr: errc=0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: CreateEx: flags=0x502, mode=0770
2023/08/21 11:06:57 DEBUG : Empty Folder: Re-reading directory (19.1116398s old)
2023/08/21 11:06:57 DEBUG : Empty Folder: Removed virtual directory entry vDel: "1000094614-CLIENT-118_22.zip"
2023/08/21 11:06:57 DEBUG : Empty Folder/1000094614-CLIENT-118_22.zip: Open: flags=O_RDWR|O_CREATE|O_EXCL
2023/08/21 11:06:57 DEBUG : Empty Folder: Added virtual directory entry vAddFile: "1000094614-CLIENT-118_22.zip"
2023/08/21 11:06:57 DEBUG : Empty Folder/1000094614-CLIENT-118_22.zip: >Open: fd=Empty Folder/1000094614-CLIENT-118_22.zip (w), err=<nil>
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >CreateEx: errc=0, fh=0x0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Getattr: errc=0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Getattr: errc=0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Getattr: errc=0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Truncate: size=82350748019, fh=0x0
2023/08/21 11:06:57 ERROR : Empty Folder/1000094614-CLIENT-118_22.zip: WriteFileHandle: Truncate: Can't change size without --vfs-cache-mode >= writes
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Truncate: errc=-1
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Getattr: errc=0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Write: ofst=0, fh=0x0
2023/08/21 11:06:57 DEBUG : Empty Folder: Added virtual directory entry vAddFile: "1000094614-CLIENT-118_22.zip"
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Write: n=1048576
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Getattr: errc=0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Write: ofst=1048576, fh=0x0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Write: n=1048576
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Getattr: errc=0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Write: ofst=2097152, fh=0x0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Write: n=1048576
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Getattr: errc=0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Write: ofst=3145728, fh=0x0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Write: n=1048576
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Getattr: errc=0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Write: ofst=4194304, fh=0x0
2023/08/21 11:06:57 DEBUG : o.meta = map[]
2023/08/21 11:06:57 DEBUG : o.meta = map[mtime:2023-08-21T11:06:57.751600500+10:00]
2023/08/21 11:06:57 NOTICE: Empty Folder/1000094614-CLIENT-118_22.zip: Streaming uploads using chunk size 1Gi will have maximum file size of 48.828Ti
2023/08/21 11:06:57 DEBUG : Empty Folder/1000094614-CLIENT-118_22.zip: Multipart upload session started for -1 parts of size 1Gi
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Write: n=1048576
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Getattr: errc=0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Write: ofst=5242880, fh=0x0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Write: n=1048576
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Getattr: errc=0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Write: ofst=6291456, fh=0x0
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Write: n=1048576
2023/08/21 11:06:57 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x0

023/08/21 11:06:58 DEBUG : /: >Releasedir: errc=0
2023/08/21 11:06:59 DEBUG : Empty Folder/1000094614-CLIENT-118_22.zip: aborting in-sequence write wait, off=272629760
2023/08/21 11:06:59 DEBUG : Empty Folder/1000094614-CLIENT-118_22.zip: failed to wait for in-sequence write to 272629760
2023/08/21 11:06:59 ERROR : Empty Folder/1000094614-CLIENT-118_22.zip: WriteFileHandle.Write: can't seek in file without --vfs-cache-mode >= writes
2023/08/21 11:06:59 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Write: n=-29
2023/08/21 11:06:59 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x0
2023/08/21 11:06:59 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Getattr: errc=0
2023/08/21 11:06:59 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Write: ofst=271581184, fh=0x0
2023/08/21 11:06:59 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Write: n=1048576
2023/08/21 11:06:59 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x0
2023/08/21 11:06:59 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Getattr: errc=0
2023/08/21 11:06:59 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Write: ofst=273678336, fh=0x0
2023/08/21 11:06:59 DEBUG : Empty Folder/1000094614-CLIENT-118_22.zip: waiting for in-sequence write to 273678336 for 1s
2023/08/21 11:07:00 DEBUG : Empty Folder/1000094614-CLIENT-118_22.zip: aborting in-sequence write wait, off=273678336
2023/08/21 11:07:00 DEBUG : Empty Folder/1000094614-CLIENT-118_22.zip: failed to wait for in-sequence write to 273678336
2023/08/21 11:07:00 ERROR : Empty Folder/1000094614-CLIENT-118_22.zip: WriteFileHandle.Write: can't seek in file without --vfs-cache-mode >= writes
2023/08/21 11:07:00 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Write: n=-29
2023/08/21 11:07:00 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x0
2023/08/21 11:07:00 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Getattr: errc=0
2023/08/21 11:07:00 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Write: ofst=274726912, fh=0x0
2023/08/21 11:07:00 DEBUG : Empty Folder/1000094614-CLIENT-118_22.zip: waiting for in-sequence write to 274726912 for 1s
2023/08/21 11:07:01 DEBUG : Empty Folder/1000094614-CLIENT-118_22.zip: aborting in-sequence write wait, off=274726912
2023/08/21 11:07:01 DEBUG : Empty Folder/1000094614-CLIENT-118_22.zip: failed to wait for in-sequence write to 274726912
2023/08/21 11:07:01 ERROR : Empty Folder/1000094614-CLIENT-118_22.zip: WriteFileHandle.Write: can't seek in file without --vfs-cache-mode >= writes
2023/08/21 11:07:01 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Write: n=-29
2023/08/21 11:07:01 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x0
2023/08/21 11:07:01 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Getattr: errc=0
2023/08/21 11:07:01 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x0
2023/08/21 11:07:01 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Getattr: errc=0
2023/08/21 11:07:03 DEBUG : Empty Folder/1000094614-CLIENT-118_22.zip: >Open: fd=<nil *ReadFileHandle>, err=file does not exist
2023/08/21 11:07:03 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >OpenFile: fd=<nil *ReadFileHandle>, err=file does not exist
2023/08/21 11:07:03 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >OpenEx: errc=-2, fh=0xFFFFFFFFFFFFFFFF
2023/08/21 11:07:03 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Unlink: 
2023/08/21 11:07:03 DEBUG : Empty Folder/1000094614-CLIENT-118_22.zip: Remove: 
2023/08/21 11:07:03 DEBUG : Empty Folder: Added virtual directory entry vDel: "1000094614-CLIENT-118_22.zip"
2023/08/21 11:07:03 DEBUG : Empty Folder/1000094614-CLIENT-118_22.zip: >Remove: err=<nil>
2023/08/21 11:07:03 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Unlink: errc=0
2023/08/21 11:07:03 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Flush: fh=0x0
2023/08/21 11:07:03 DEBUG : Empty Folder/1000094614-CLIENT-118_22.zip: Uploading part 1/-1 offset 0/off part size 272629760


023/08/21 11:07:03 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: CreateEx: flags=0x502, mode=0770
2023/08/21 11:07:03 DEBUG : Empty Folder/1000094614-CLIENT-118_22.zip: Open: flags=O_RDWR|O_CREATE|O_EXCL
2023/08/21 11:07:03 DEBUG : Empty Folder: Added virtual directory entry vAddFile: "1000094614-CLIENT-118_22.zip"
2023/08/21 11:07:03 DEBUG : Empty Folder/1000094614-CLIENT-118_22.zip: >Open: fd=Empty Folder/1000094614-CLIENT-118_22.zip (w), err=<nil>
2023/08/21 11:07:03 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >CreateEx: errc=0, fh=0x3
2023/08/21 11:07:03 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x3
2023/08/21 11:07:03 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Getattr: errc=0
2023/08/21 11:07:03 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x3
2023/08/21 11:07:03 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Getattr: errc=0
2023/08/21 11:07:03 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x3
2023/08/21 11:07:03 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Getattr: errc=0
2023/08/21 11:07:03 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Truncate: size=82350748019, fh=0x3
2023/08/21 11:07:03 ERROR : Empty Folder/1000094614-CLIENT-118_22.zip: WriteFileHandle: Truncate: Can't change size without --vfs-cache-mode >= writes
2023/08/21 11:07:03 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Truncate: errc=-1
2023/08/21 11:07:03 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Getattr: fh=0x3
2023/08/21 11:07:03 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: >Getattr: errc=0
2023/08/21 11:07:03 DEBUG : /Empty Folder/1000094614-CLIENT-118_22.zip: Write: ofst=0, fh=0x3
2

You'd be better off just doing a rclone copy directly to the remote rather than using a mount.

It spawns this message

Based on how the application/program/process is writing the data and you can't 'tune' around that with a flag. If it requires a seek to write, you need cache mode writes which makes a local copy first.

3

Thanks for the reply.

I should have been more clear - for this use case it has to be a mounted drive, rather than an rclone copy or azcopy.

Is there anything you are aware of to speed up the process of writing to blob with an rclone mount?

Disabling checksum helps, upload concurrency/transfers helps, but the process of writing 75GB to cache before transfer is the issue.

4

Unless you'd want to re-write that portion of rclone, no as that is how it works.

5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.