With this mode, when is it removed from the local file system and uploaded to the cloud?
I suppose it is removed when the upload transfer is finished and uploaded immediately.
I confirmed this by regularly doing “mv AMovie.mkv /mnt/GDrive/” in a screen session.
I didn’t see that behavior. I tested with just /etc/hosts which if a few bytes and it didn’t go away.
did the transfer finished? any error? most of the time in my case the files are there locally when rclone didn’t finish transfer, or have error closing the file, or can’t delete the local file (in case of mv command)
I just copied it by doing cp /etc/hosts /gmedia
Jun 3 19:53:17 gemini rclone[25614]: hosts: Copied (new)
Confused… that was just a copy so you would expect it remain, and as its the hosts file i would expect the permissions would stop it from being removed.
I have a mount of my GD at /gmedia
I’m copying /etc/hosts as an example which is a small file.
It gets coped to my vfs-cache-write area as expected by my cache-dir and I can see:
/data/rclone-tmp/vfs/gcrypt/hosts
I see a note saying it’s been uploaded via the log I shared.
I’d expect the temp location to be removed once uploaded but it remains.
i believe rclone only delete file from its temporary upload dir once it is verified that the file is finished uploading and the file is closed for any other operation.
At 53 bytes, I waited a few minutes and saw the log file noting it was done. Nothing was writing to it.
I fixed the subject as I had the wrong question. My question is once uploaded, when does it remove from the cached area.
ah, so the issue is that it is not being removed from the temp location.
Maybe try with a non system file, as even if nothing is writing to the hosts file it is probably still in use by the system
I did have some issues with rclone not removing folders, but files did seem to be removed from the temp dir after upload, but that was using rclone cache.
Probably the worst worded subject/question.
I just did a test mount:
Jun 5 18:15:29 gemini rclone[31870]: hosts: Copied (new)
copied the file.
I do not see it on my GD yet, but it’s on the local mount. I checked the webUI to see if the file was there.
I’m not sure what causes the upload to happen and remove the file from the local cache area as I can see it just copied it:
-rw------- 1 felix felix 351 Jun 5 18:15 hosts
[felix@gemini gcrypt]$ pwd
/data/rclone_upload/vfs/gcryptSo after more testing:
–vfs-cache-max-age 1m
Seems to make it upload and remove after one minute.
With this option, do you find that the API requests go up or Plex speed goes down? Does it remove after a minute of it being uploaded or a minute after it’s finished copying to the vfs tmp folder? I wish the v2 api would work for this as a flag since V3 seems to be much slower for this kind of setup vs Cache since with cache, it’ll be linked to your Gdrive mount before it’s even started uploading unlike here where you need to wait.
@ncw I noticed in the v3 and v2 APIs that there exists resumable uploads for large files, is this implemented with rclone? Would be nice to have it retry chunks instead of the entire file if it isn’t already, since without the vfs-cache-mode writes enabled, I noticed files not copying over fully and needing to be retried from the beginning, causing the daily limit to reach.
I just left it 1m for testing. I moved it back up to 12hours which is fine for me in actual use.
rclone should be retrying chunks if they fail using the resumable upload. Do you have a log with -vv of it failing?
Is that only with the vfs-cache-mode writes flag or will it also work without the cache mode defined? I only started using vfs-cache-mode writes recently and haven’t noticed any failed uploads but before it was enabled, items would fail to upload.
writes just has a different use-case as it’s documented on the page:
I used ‘none’ for a few weeks already and I was testing ‘writes’ as I wanted to ffmpeg convert a few files and that does not work without using writes.
I was going off of the https://tip.rclone.org/commands/rclone_mount/#vfs-cache-mode-writes page. There isn’t any info on it resuming on this page yet.
