Idea of a new Mode called "vfs-cache-mode light"

[Dual-O (Oli)]

Have tested it successfully.

• Libary Scan in the making... No issues so far.
• Works very good for download and upload through mount.
• upload feels much faster.

Found a new directroy called vfsMeta in Cache Folder.
Thanks for the release!

UPDATE: Test half day and got 403 Error on two mounts. Sorry I don't have a debug log.

[jad3675]

I'm getting a few errors, though playback does seem to work just fine.

2020/06/04 09:22:23 DEBUG : rc: "core/stats": reply map[bytes:2259498034 checks:0 deletes:0 elapsedTime:32.61097402 errors:7 fatalError:false lastError:vfs reader: failed to write to cache file: stream abandoned renames:0 retryError:true speed:6.928644614705905e+07 transferring:[map[bytes:128970752 eta:8 group:global_stats name:Shows/DC's Legends of Tomorrow/Season 5/S05E01 - Crisis on Infinite Earths (5).mkv percentage:13 size:985190693 speed:9.897332940256158e+07 speedAvg:9.750415099318166e+07]] transfers:7]:
2020/06/04 09:22:24 DEBUG : rc: "core/stats": reply map[bytes:2425173042 checks:0 deletes:0 elapsedTime:34.190821197 errors:7 fatalError:false lastError:vfs reader: failed to write to cache file: stream abandoned renames:0 retryError:true speed:7.093054338591148e+07 transferring:[map[bytes:294645760 eta:6 group:global_stats name:Shows/DC's Legends of Tomorrow/Season 5/S05E01 - Crisis on Infinite Earths (5).mkv percentage:29 size:985190693 speed:1.0220347132743181e+08 speedAvg:1.0117937329722695e+08]] transfers:7]:
2020/06/04 09:22:28 DEBUG : rc: "core/stats": reply map[bytes:2721920050 checks:0 deletes:0 elapsedTime:37.62940624 errors:7 fatalError:false lastError:vfs reader: failed to write to cache file: stream abandoned renames:0 retryError:true speed:7.233493296161216e+07 transferring:[map[bytes:591392768 eta:4 group:global_stats name:Shows/DC's Legends of Tomorrow/Season 5/S05E01 - Crisis on Infinite Earths (5).mkv percentage:60 size:985190693 speed:9.355232705915609e+07 speedAvg:9.43682559430865e+07]] transfers:7]:
2020/06/04 09:22:29 DEBUG : rc: "core/stats": reply map[bytes:2856137778 checks:0 deletes:0 elapsedTime:38.954890472 errors:7 fatalError:false lastError:vfs reader: failed to write to cache file: stream abandoned renames:0 retryError:true speed:7.331912021843676e+07 transferring:[map[bytes:725610496 eta:2 group:global_stats name:Shows/DC's Legends of Tomorrow/Season 5/S05E01 - Crisis on Infinite Earths (5).mkv percentage:73 size:985190693 speed:9.488822404741956e+07 speedAvg:9.391907120313661e+07]] transfers:7]:
2020/06/04 09:22:33 DEBUG : rc: "core/stats": reply map[bytes:3112244567 checks:0 deletes:0 elapsedTime:41.970510484 errors:7 fatalError:false lastError:vfs reader: failed to write to cache file: stream abandoned renames:0 retryError:true speed:7.415312635252436e+07 transfers:8]:
2020/06/04 09:22:34 DEBUG : rc: "core/stats": reply map[bytes:3112244567 checks:0 deletes:0 elapsedTime:41.970510484 errors:7 fatalError:false lastError:vfs reader: failed to write to cache file: stream abandoned renames:0 retryError:true speed:7.415312635252436e+07 transfers:8]:
2020/06/04 09:22:38 DEBUG : rc: "core/stats": reply map[bytes:3112244567 checks:0 deletes:0 elapsedTime:41.970510484 errors:7 fatalError:false lastError:vfs reader: failed to write to cache file: stream abandoned renames:0 retryError:true speed:7.415312635252436e+07 transfers:8]:
2020/06/04 09:22:39 DEBUG : rc: "core/stats": reply map[bytes:3112244567 checks:0 deletes:0 elapsedTime:41.970510484 errors:7 fatalError:false lastError:vfs reader: failed to write to cache file: stream abandoned renames:0 retryError:true speed:7.415312635252436e+07 transfers:8]:
2020/06/04 09:22:43 DEBUG : rc: "core/stats": reply map[bytes:3112244567 checks:0 deletes:0 elapsedTime:41.970510484 errors:7 fatalError:false lastError:vfs reader: failed to write to cache file: stream abandoned renames:0 retryError:true speed:7.415312635252436e+07 transfers:8]:
2020/06/04 09:22:44 DEBUG : rc: "core/stats": reply map[bytes:3112244567 checks:0 deletes:0 elapsedTime:41.970510484 errors:7 fatalError:false lastError:vfs reader: failed to write to cache file: stream abandoned renames:0 retryError:true speed:7.415312635252436e+07 transfers:8]:
2020/06/04 09:22:48 DEBUG : rc: "core/stats": reply map[bytes:3112244567 checks:0 deletes:0 elapsedTime:41.970510484 errors:7 fatalError:false lastError:vfs reader: failed to write to cache file: stream abandoned renames:0 retryError:true speed:7.415312635252436e+07 transfers:8]:
2020/06/04 09:22:49 DEBUG : rc: "core/stats": reply map[bytes:3112244567 checks:0 deletes:0 elapsedTime:41.970510484 errors:7 fatalError:false lastError:vfs reader: failed to write to cache file: stream abandoned renames:0 retryError:true speed:7.415312635252436e+07 transfers:8]:
2020/06/04 09:22:53 DEBUG : rc: "core/stats": reply map[bytes:3112244567 checks:0 deletes:0 elapsedTime:41.970510484 errors:7 fatalError:false lastError:vfs reader: failed to write to cache file: stream abandoned renames:0 retryError:true speed:7.415312635252436e+07 transfers:8]:
2020/06/04 09:22:54 DEBUG : rc: "core/stats": reply map[bytes:3112244567 checks:0 deletes:0 elapsedTime:41.970510484 errors:7 fatalError:false lastError:vfs reader: failed to write to cache file: stream abandoned renames:0 retryError:true speed:7.415312635252436e+07 transfers:8]:
2020/06/04 09:22:58 DEBUG : rc: "core/stats": reply map[bytes:3112244567 checks:0 deletes:0 elapsedTime:41.970510484 errors:7 fatalError:false lastError:vfs reader: failed to write to cache file: stream abandoned renames:0 retryError:true speed:7.415312635252436e+07 transfers:8]:
2020/06/04 09:22:59 DEBUG : rc: "core/stats": reply map[bytes:3112244567 checks:0 deletes:0 elapsedTime:41.970510484 errors:7 fatalError:false lastError:vfs reader: failed to write to cache file: stream abandoned renames:0 retryError:true speed:7.415312635252436e+07 transfers:8]:
2020/06/04 09:23:03 DEBUG : rc: "core/stats": reply map[bytes:3112244567 checks:0 deletes:0 elapsedTime:41.970510484 errors:7 fatalError:false lastError:vfs reader: failed to write to cache file: stream abandoned renames:0 retryError:true speed:7.415312635252436e+07 transfers:8]:
2020/06/04 09:23:04 DEBUG : rc: "core/stats": reply map[bytes:3112244567 checks:0 deletes:0 elapsedTime:41.970510484 errors:7 fatalError:false lastError:vfs reader: failed to write to cache file: stream abandoned renames:0 retryError:true speed:7.415312635252436e+07 transfers:8]:
2020/06/04 09:23:08 DEBUG : rc: "core/stats": reply map[bytes:3112244567 checks:0 deletes:0 elapsedTime:41.970510484 errors:7 fatalError:false lastError:vfs reader: failed to write to cache file: stream abandoned renames:0 retryError:true speed:7.415312635252436e+07 transfers:8]:

This is my mount command:
/usr/bin/rclone mount gcrypt: /mnt/media --config /root/.config/rclone/rclone.conf --vfs-cache-mode full --user-agent="rclone" --use-mmap --daemon-timeout=10m --vfs-read-chunk-size 10M --vfs-read-chunk-size-limit 512M --cache-dir /home/rclone/vfs_cache --allow-other --dir-cache-time 480h --drive-chunk-size 128M --log-level DEBUG --log-file /var/log/rclone.log --timeout 1h --umask 002 --poll-interval 15s -

[asdffdsa (jojothehumanmonkey)]

what version of rclone are you using?
what is your command?

[root0r]

Opening files now working here, but i have a lot Problems with Playback media.

On Kodi Movies stops after some seconds and i have to pause and play again,
Musicplay stutters often and Long pause between tracks (sometime with stutters over some seconds).

[Animosity022]

It would be great to wait for a feedback thread to get posted rather than haphazardly testing the beta and dropping information here and there so it can be consolidated and looked at.

[ncw (Nick Craig-Wood)]

@Dual-O

Have tested it successfully.

• Libary Scan in the making... No issues so far.
• Works very good for download and upload through mount.
• upload feels much faster.

Found a new directroy called vfsMeta in Cache Folder.

Good testing! Faster uploads are to do with the writeback cache - it returns instantly to explorer and uploads in the background. The vfsMeta dir holds info about the files in the vfs directory.

UPDATE: Test half day and got 403 Error on two mounts. Sorry I don't have a debug log.

Would need more info but probably not related to new code.

@jad3675

lastError:vfs reader: failed to write to cache file: stream abandoned

These errors are normal - I'll stop them being logged. They happen when rclone is downloading stuff in the background and the application closes the file so rclone stops the download.

@root0r

On Kodi Movies stops after some seconds and i have to pause and play again,
Musicplay stutters often and Long pause between tracks (sometime with stutters over some seconds).

Hmm, what kind of disk do you have? Is it very busy? --vfs-cache-mode full uses your disk much more.

@Animosity022

It would be great to wait for a feedback thread to get posted rather than haphazardly testing the beta and dropping information here and there so it can be consolidated and looked at.

Great idea!

I created a new thread here (with a new beta) for the discussions - please reply there to the above - thanks!

Testing for new vfs cache mode features

I'm in the process of completing the changes to the vfs cache This will have these 3 major new features (along with a host of bug fixes!) --vfs-cache-mode full which will only download the parts of the file you need I envision this will become the preferred way of using the vfs cache --vfs-cache-mode writes will do delayed writeback of files this will help with explorer lockups on windows this will help with downloads and renames in quick sucession (eg downloading from a web browser) r…