Can't disable cache in Windows

Help and Support
1

What is the problem you are having with rclone?

I'm using the following command to connect and mount my target:
...
rclone mount "Veeam Sharepoint": O: --vfs-cache-mode off
...
My targets is mounted just fine in Windows. As soon as I'm pasting a file to the drive it seems that cache is automatically activated in write-mode, even though it should've been disabled. Why is it automatically activated, and how do I prevent it ?

2020/07/16 15:06:38 ERROR : Storage/Kopier data til Onedrive/1GB.bin: WriteFileHandle: Truncate: Can't change size without --vfs-cache-mode >= writes

What is your rclone version (output from rclone version)

rclone v1.52.2

  • os/arch: windows/amd64
  • go version: go1.14.4

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10 Pro, 64bit

Which cloud storage system are you using? (eg Google Drive)

Sharepoint

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount "Veeam Sharepoint": O: --vfs-cache-mode off

The rclone config contents with secrets removed.

[Veeam Sharepoint]
type = onedrive
token = hidden
drive_id = hidden
drive_type = documentLibrary

Thanks in advance
Regards

2

helo and welcome to the forum,

3

That isn't turning on cache mode, but it's telling you to write that file in the way that is required, it needs to be turned on.

4

Tried using this:
...
rclone mount "Veeam Sharepoint": O: --vfs-cache-mode off --ignore-checksum --ignore-size
...

Same happens. I can see that it's quick at copying the file to the drive, but at the same time filling up the cache.

5

Makes sense. So to help me, I guess that it automatically switches the cache from off to writes, and then begins moving the file into the cache before uploading. It works perfectly fine. It's just that I don't want to use the cache.

So basically I have to use cache in write mode in order to upload any files. I've tested with both a bin file and a zip file, in both situations it works, but only because the case is in wiring mode. As far as I understand, it should be doable without using the cache.

6

No, it does not automatically switch or change anything. It is tell you if you want to write that file that you "have" to turn it on.

If you wan to upload directly to the cloud, you have to use rclone move or rclone copy to the remote.

If you copy a file regardless of the cache mode, it writes it locally first, uploads it and removes it locally from the cache area.

7

Okay. It can be that it is only telling me it. But somehow I am certain that it's changing to write cache by itself.. Something is happening, if it ain't that, I don't know what it is. As soon as a paste the file, it tells me that. The file is then quickly moved to the drive, while this happens I see that it consumes more and more disk space at C where my default cache is, also it doesn't start to upload until the file is successfully copied, then I see it is initiating the actual upload... This whole process is just weird to me, as it shouldn't be using any local cache ?

I thought that I would be able to upload directly to the cloud by mounting my Sharepoint as a drive in Windows, but as I understand from your writing, this can't be done.. My goal is to have a location represented either as drive or network drive which I can upload data to without any involvement of a local cache.

Thanks for your help!

8

there should not be a vfs cache, unless you explicitly enable it.
on windows, i copy all the time, using mount, without vfs cache.

where is it and what is in it?

9

I can say with 100% certainty it does not change itself.

No, it depends on 'how' you are writing the files if you need writes or not. The error message in the log you shared says you need it because of how the files are being written.

If you can write with write modes off, you upload directly to the mount and it waits for it upload to come back:

2020/07/16 13:33:03 DEBUG : : Added virtual directory entry vAdd: "jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv"
2020/07/16 13:33:03 DEBUG : &{jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv (w)}: >Flush: err=<nil>
2020/07/16 13:33:03 DEBUG : &{jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv (w)}: Release:
2020/07/16 13:33:03 DEBUG : jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv: WriteFileHandle.Release nothing to do
2020/07/16 13:33:03 DEBUG : &{jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv (w)}: >Release: err=<nil>
2020/07/16 13:33:05 DEBUG : Google drive root '': Checking for changes on remote
2020/07/16 13:33:05 DEBUG : : changeNotify: relativePath="jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv", type=3
2020/07/16 13:33:05 DEBUG : : invalidating directory cache
2020/07/16 13:33:05 DEBUG : : >changeNotify:

If you include a debug as the help template asks for, we can see this in the debug log and walk through it.

10

This is the path:
C:\Users\veeam\AppData\Local\Temp\rclone-spool714211375\Storage\Kopier data til Onedrive
my test file 10GB.zip is in it, which is also the file i pasted to the drive.

11

as a test, just now,

  • created a 20GB file of random data
  • copied the file to my t:, which is the mount.
  • during the copy:
    my hard drive did not decrease in size at al
    and
    the default cache folder, had zero files in it C:\Users\user01\AppData\Local\rclone
12

Okay. Glad to hear that it's just me that is missing something then.
I am writing the file just by copying my 10GB.zip test file from my download folder to the drive.

Here's my logfile: privatebin dot net/?850d9680041a73bd#9FqJX3AQTxkpnozdfs8jbE5iPK6hLWbH3Lqz7McAHCEB

I guess it has something to do with this?
2020/07/16 20:04:24 DEBUG : One drive root '': Target remote doesn't support streaming uploads, creating temporary local FS to spool file

13

always best to use a debug log, as the answer if often there.

14

Okay. Hope my debug log shreds some light on this then.

15

Can't add links to my replies. Hope it makes sense with the link to the debug log.
add https and replace dot

16

Yeah, it's telling you there is has to write local for that remote before it uploads which is what I said a number of posts above with One Drive.

With Google Drive and a number of other remotes, it can stream the upload so no need to write locally first which is my GD example debug log I shared.

17

I guess there's no way to have it upload directly then. Wonder if it would be possible using the webdav option to connect to the Sharepoint site, instead of Onedrive ?

18

You can use rclone copy instead of copying into the mount - that doesn't require a temporary copy.

Only owncloud supports streaming uploads with webdav so I don't think this will help.

1 Like
19

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.