Version:
rclone v1.52.3-315-g85f9bd1a-beta
Parameters are in my post above:
--allow-other --dir-cache-time 8760h --poll-interval 0 --vfs-cache-mode full --vfs-cache-max-size 250G --vfs-cache-max-age 8760h --drive-use-trash=false --log-level DEBUG --log-file /opt/rclone/rclone.log upload: /media/cry
The thing is, the whole log is GB big...
No, can you check actual rclone as that's just your system CPU.
In /etc/netdata/apps_group.conf, you can add a line like:
rclone: rclone
And you'll see rclone in netdata for the apps section.
Can you share what beta you are running and what config.
Didn't know that, I just added that line and will keep an eye on it.
Rclone version:
rclone v1.52.3-315-g85f9bd1a-beta
Config:
Gdrive -> Crypt (or is it the other way around?)
Paramenters:
--allow-other --dir-cache-time 8760h --poll-interval 0 --vfs-cache-mode full --vfs-cache-max-size 250G --vfs-cache-max-age 8760h --drive-use-trash=false --log-level DEBUG --log-file /opt/rclone/rclone.log upload: /media/cry
@Dual-O, yes I have been using crypt ever since I started using rclone.
@Animosity022, I added the line as suggested and looks like rclone is causing quite a bit of load on my server when reading from vfs cache.
I'm quite a few betas before that.
@ncw - are you aware of any changes since rclone v1.52.3-290-g82975109-beta that might cause something?
I'm not aware so any changes. Running a cou profile via the RC would be the way to find out what is going on
I don't see the issue. I updated to the latest beta and am testing that now.
felix@gemini:~$ rclone version
rclone v1.52.3-326-g45afe97e-beta
What mount command are you using?
I will update to the latest beta and keep an eye on it.
Mine is pretty normal.
/usr/bin/rclone mount gcrypt: /GD --allow-other --dir-cache-time 1000h --log-level INFO --log-file /opt/rclone/logs/rclone.log --poll-interval 15s --umask 002 --rc --rc-addr :5572 --rc-no-auth --cache-dir=/cache --vfs-cache-mode full --vfs-cache-max-size 500G --vfs-cache-max-age 336h
Flag-wise it looks pretty much the same as mine above.
I will update the beta and keep an eye on it.
Can I check the expected behaviour with Emby please. I implemented the new cache mode with systemd service as such
ExecStart=/usr/bin/rclone mount gd:/Media /opt/rclone-media \
--dir-cache-time 1000h \
--config /root/.config/rclone/rclone.conf \
--log-file=/var/log/rclone.log \
--log-level=INFO \
--allow-other \
--allow-non-empty \
--transfers=4 \
--drive-chunk-size=32M \
--cache-dir=/mnt/Cache \
--vfs-cache-mode full \
--vfs-cache-max-size 500G \
--vfs-cache-max-age 336h
So far in testing files which I've streamed download and play without issue. I did notice though that this morning the cache is starting to fill up with movies and TV shows that haven't been watched. I assume this is maybe a side effect of maybe a recently added file or a recently scanned file. I that was the case, would it be executed that the scanned file was completely downloaded to the cache folder or should it be a segment of the file to generate meta data from?
hi,
one of the many things i have learned from @Animosity022; that flag is not considered a good idea, many forums posts about that.
i could be wrong but --transfer does nothing on a mount.
Hi I've been using the new VFS cache for a while and it's fixed all the problems I had with the old cache backend completely so thank you!
I've gone so far as using the latest beta on my Android phone to mount a crypted Google Drive for music streaming, caching data for when I enter a no signal environment (currently on holiday and very common in SW England). Its working far better than I ever expected. The only problem I have is that when I restart my phone, I need an internet connection just to start the mount.
Are there any plans on caching the directory structure and being able to start a VFS cache mount without an internet connection? Using the cached data until a data connection is re-established, essentially acting like a true/full cache?
So it's a bit confusing as it does sparse files. That means if you check the file, you'd see a 10GB file but in actuality, it only has a MB of the file downloaded. So in my example, I have a 25GB file, but only 19MB is in use. If you do a du on the file, you can see the actual use.
I don't think there are plans to make it an offline sync.
Something scanned the files. Sonarr/Radarr/Emby, they are all setup to scan by default every 12 hours. Plex scans periodically too. Most often it's mediainfo or ffmpegprobe calls.
It will just be the data it downloaded and used, so just a partial. Probably around the size of the vfs-chunk-size.
You can adjust the scanning in those apps. Emby look for "tasks" under settings. For plex, look under library in the settings. For sonarr/radarr, you need to use the new UI (preview/beta versions) and then go into the media management settings, find the setting to only scan the disk when doing a manual refresh. Also turn off media analysis, it's pretty useless.
Ahhhh, thank you. Just tested with one file
ls xxx.mkv -l
-rw-rw-r-- 1 emby emby 7911511742 Jul 13 2019 xxx.mkv
and
du xxx.mkv
7726086 xxx.mkvThank you. Removed --allow-non-empty and commented --transfer out until I can verify its not needed with the new vfs caching system. It was a hang over from my original mount system migration.