Rclone mount becoming unresponsive

Drapersniper · October 24, 2021, 11:48am

What is the problem you are having with rclone?

I've temporarily reduced my vfs cache size from 500G to 200G while I'm rebuilding my library.
However, I noticed every so often rclone becomes unresponsive.

When looking at the log file I always see the same thing when this happens:


2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 08/Detective Conan (1996) - S08E015 - 0208 - [UNCUT DVD][8bit][h264][AAC 2.0]-Ruffy.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 06/Detective Conan (1996) - S06E006 - 0140 - [SDTV][10bit][x265][Opus 2.0][JA]-Judas.mkv: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Bleach (2004) {tvdb-74796}/Season 14/Bleach (2004) - S14E001 - 0266 - [HDTV-720p][8bit][x264][AAC 2.0][JA]-Japan.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 08/Detective Conan (1996) - S08E015 - 0208 - [UNCUT DVD][8bit][h264][AAC 2.0]-Ruffy.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 06/Detective Conan (1996) - S06E006 - 0140 - [SDTV][10bit][x265][Opus 2.0][JA]-Judas.mkv: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Bleach (2004) {tvdb-74796}/Season 14/Bleach (2004) - S14E001 - 0266 - [HDTV-720p][8bit][x264][AAC 2.0][JA]-Japan.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 08/Detective Conan (1996) - S08E015 - 0208 - [UNCUT DVD][8bit][h264][AAC 2.0]-Ruffy.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 06/Detective Conan (1996) - S06E006 - 0140 - [SDTV][10bit][x265][Opus 2.0][JA]-Judas.mkv: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Bleach (2004) {tvdb-74796}/Season 14/Bleach (2004) - S14E001 - 0266 - [HDTV-720p][8bit][x264][AAC 2.0][JA]-Japan.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 08/Detective Conan (1996) - S08E015 - 0208 - [UNCUT DVD][8bit][h264][AAC 2.0]-Ruffy.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 06/Detective Conan (1996) - S06E006 - 0140 - [SDTV][10bit][x265][Opus 2.0][JA]-Judas.mkv: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Bleach (2004) {tvdb-74796}/Season 14/Bleach (2004) - S14E001 - 0266 - [HDTV-720p][8bit][x264][AAC 2.0][JA]-Japan.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 08/Detective Conan (1996) - S08E015 - 0208 - [UNCUT DVD][8bit][h264][AAC 2.0]-Ruffy.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 06/Detective Conan (1996) - S06E006 - 0140 - [SDTV][10bit][x265][Opus 2.0][JA]-Judas.mkv: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Bleach (2004) {tvdb-74796}/Season 14/Bleach (2004) - S14E001 - 0266 - [HDTV-720p][8bit][x264][AAC 2.0][JA]-Japan.mp4: In-access item skipped, freed 0 bytes

PurgeClean is always trying to remove an item while it still in use.

In some instances I seen the log file grow to 1TB+ because the same file kept spamming the logs

What is your rclone version (output from `rclone version`)

Which OS you are using and how many bits (eg Windows 7, 64 bit)

rclone v1.57.0-beta.5741.7b66ca132

os/version: Microsoft Windows 10 Pro 2009 (64 bit)
os/kernel: 10.0.22000.258 (x86_64)
os/type: windows
os/arch: amd64
go/version: go1.17.2
go/linking: dynamic
go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

Paste command here

mount Union-crypt:Anime A: 
--volname Anime 
--file-perms 0777 
--dir-perms 0777 
--vfs-cache-mode full 
--dir-cache-time 96h 
--vfs-cache-max-age 48h 
--vfs-read-chunk-size 128M 
--poll-interval 2m 
--vfs-read-chunk-size-limit off 
--vfs-cache-max-size off 
--buffer-size 256M
 --cache-dir "C:\Rclone Cache"
 --vfs-cache-max-size 100G 
--drive-chunk-size 1M  
--drive-server-side-across-configs=true 
--attr-timeout 1000h 
--vfs-cache-poll-interval=1m

I've added --vfs-cache-poll-interval=1m after the issue started happening because from the logs it looked like it was being pooled more often than once per minute

A logs

2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 08/Detective Conan (1996) - S08E015 - 0208 - [UNCUT DVD][8bit][h264][AAC 2.0]-Ruffy.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Bleach (2004) {tvdb-74796}/Season 14/Bleach (2004) - S14E001 - 0266 - [HDTV-720p][8bit][x264][AAC 2.0][JA]-Japan.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 06/Detective Conan (1996) - S06E006 - 0140 - [SDTV][10bit][x265][Opus 2.0][JA]-Judas.mkv: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 08/Detective Conan (1996) - S08E015 - 0208 - [UNCUT DVD][8bit][h264][AAC 2.0]-Ruffy.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Bleach (2004) {tvdb-74796}/Season 14/Bleach (2004) - S14E001 - 0266 - [HDTV-720p][8bit][x264][AAC 2.0][JA]-Japan.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 06/Detective Conan (1996) - S06E006 - 0140 - [SDTV][10bit][x265][Opus 2.0][JA]-Judas.mkv: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 08/Detective Conan (1996) - S08E015 - 0208 - [UNCUT DVD][8bit][h264][AAC 2.0]-Ruffy.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Bleach (2004) {tvdb-74796}/Season 14/Bleach (2004) - S14E001 - 0266 - [HDTV-720p][8bit][x264][AAC 2.0][JA]-Japan.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 06/Detective Conan (1996) - S06E006 - 0140 - [SDTV][10bit][x265][Opus 2.0][JA]-Judas.mkv: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 08/Detective Conan (1996) - S08E015 - 0208 - [UNCUT DVD][8bit][h264][AAC 2.0]-Ruffy.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Bleach (2004) {tvdb-74796}/Season 14/Bleach (2004) - S14E001 - 0266 - [HDTV-720p][8bit][x264][AAC 2.0][JA]-Japan.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 06/Detective Conan (1996) - S06E006 - 0140 - [SDTV][10bit][x265][Opus 2.0][JA]-Judas.mkv: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 08/Detective Conan (1996) - S08E015 - 0208 - [UNCUT DVD][8bit][h264][AAC 2.0]-Ruffy.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Bleach (2004) {tvdb-74796}/Season 14/Bleach (2004) - S14E001 - 0266 - [HDTV-720p][8bit][x264][AAC 2.0][JA]-Japan.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 06/Detective Conan (1996) - S06E006 - 0140 - [SDTV][10bit][x265][Opus 2.0][JA]-Judas.mkv: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 08/Detective Conan (1996) - S08E015 - 0208 - [UNCUT DVD][8bit][h264][AAC 2.0]-Ruffy.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Bleach (2004) {tvdb-74796}/Season 14/Bleach (2004) - S14E001 - 0266 - [HDTV-720p][8bit][x264][AAC 2.0][JA]-Japan.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 06/Detective Conan (1996) - S06E006 - 0140 - [SDTV][10bit][x265][Opus 2.0][JA]-Judas.mkv: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 08/Detective Conan (1996) - S08E015 - 0208 - [UNCUT DVD][8bit][h264][AAC 2.0]-Ruffy.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Bleach (2004) {tvdb-74796}/Season 14/Bleach (2004) - S14E001 - 0266 - [HDTV-720p][8bit][x264][AAC 2.0][JA]-Japan.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 06/Detective Conan (1996) - S06E006 - 0140 - [SDTV][10bit][x265][Opus 2.0][JA]-Judas.mkv: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 08/Detective Conan (1996) - S08E015 - 0208 - [UNCUT DVD][8bit][h264][AAC 2.0]-Ruffy.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Bleach (2004) {tvdb-74796}/Season 14/Bleach (2004) - S14E001 - 0266 - [HDTV-720p][8bit][x264][AAC 2.0][JA]-Japan.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 06/Detective Conan (1996) - S06E006 - 0140 - [SDTV][10bit][x265][Opus 2.0][JA]-Judas.mkv: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 08/Detective Conan (1996) - S08E015 - 0208 - [UNCUT DVD][8bit][h264][AAC 2.0]-Ruffy.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Bleach (2004) {tvdb-74796}/Season 14/Bleach (2004) - S14E001 - 0266 - [HDTV-720p][8bit][x264][AAC 2.0][JA]-Japan.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 06/Detective Conan (1996) - S06E006 - 0140 - [SDTV][10bit][x265][Opus 2.0][JA]-Judas.mkv: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 08/Detective Conan (1996) - S08E015 - 0208 - [UNCUT DVD][8bit][h264][AAC 2.0]-Ruffy.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Bleach (2004) {tvdb-74796}/Season 14/Bleach (2004) - S14E001 - 0266 - [HDTV-720p][8bit][x264][AAC 2.0][JA]-Japan.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache RemoveNotInUse (maxAge=0, emptyOnly=false): item Boruto - Naruto Next Generations (2017) - {tvdb-321285}/Season 01/Boruto - Naruto Next Generations (2017) - S01E189 - 0189 - [HDTV-1080p][8bit][x264][AAC 2.0][JA]-Erai-raws.mkv was removed, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 06/Detective Conan (1996) - S06E006 - 0140 - [SDTV][10bit][x265][Opus 2.0][JA]-Judas.mkv: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 08/Detective Conan (1996) - S08E015 - 0208 - [UNCUT DVD][8bit][h264][AAC 2.0]-Ruffy.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Bleach (2004) {tvdb-74796}/Season 14/Bleach (2004) - S14E001 - 0266 - [HDTV-720p][8bit][x264][AAC 2.0][JA]-Japan.mp4: In-access item skipped, freed 0 bytes
2021/10/24 12:17:48 INFO  : vfs cache purgeClean item.Reset Detective Conan (1996) - {tvdb-72454}/Season 06/Detective Conan (1996) - S06E006 - 0140 - [SDTV][10bit][x265][Opus 2.0][JA]-Judas.mkv: In-access item skipped, freed 0 bytes

Animosity022 · October 24, 2021, 12:32pm

You only gave 1 second of a log so it's a bit tough to see much other than normal polling.

Polls happen based on what the poll interval is set to so you'd see that every minute based on your setup.

You didn't include a rclone.conf so don't really know what your setup is either.

Drapersniper · October 24, 2021, 1:38pm

Sorry for the lack of info, when I'm back in my PC I'll share the config.

You see the issue with the logs is that's all that shows on it when it happens I've had to setup rotating logs so what happened with the screenshot doesn't happen again.

It doesn't reproduce all the time so far I've seen it reproduce three times.

Once before I manually set the pooling to 1m.

Then twice after I set it to 1m.

This screenshot was after the first time it happened.

The setup is an encrypted Union with 10 google drive team drives.

Animosity022 · October 24, 2021, 1:43pm

Your command also doesn't match up as you have INFO level logging but the default command just logs NOTICE.

If you are running something different than what you listed, it also makes it tough as that's why we ask for a debug log as it has the running command / version in the first line of the debug log.

On the surface, it seems like you have logging too high for your config if it's constantly spamming the log.

Drapersniper · October 24, 2021, 2:02pm

Sorry I copied the command from a file and forgot I had further edited it, this is the full command run by nssm to start up the service

mount 
Union-crypt:Anime 
A: 
--volname Anime 
--file-perms 0777 
--dir-perms 0777 
--vfs-cache-mode full 
--dir-cache-time 96h 
--vfs-cache-max-age 48h 
--vfs-read-chunk-size 128M 
--poll-interval 2m 
--vfs-read-chunk-size-limit off 
--vfs-cache-max-size off 
--buffer-size 256M 
--log-level INFO 
--timeout 1h 
--cache-dir "C:\Rclone Cache" 
--vfs-cache-max-size 100G
--transfers 10 
--rc 
--rc-addr localhost:5572 
--rc-pass ... 
--rc-web-gui 
--rc-user ... 
--drive-chunk-size 1M  
--drive-server-side-across-configs=true 
--attr-timeout 1000h 
--vfs-cache-poll-interval=1m

I cant attach the rclone config or links, I've uploaded it here
gist github com/Drapersniper/8a1e0ef862e07784a1e067af5931a8df

On the surface, it seems like you have logging too high for your config if it's constantly spamming the log.

Shouldn't Rclone try to delete see that's being used and then schedule removal for later instead of getting stuck on a loop trying to delete it?

While I was getting these files it looks like it happened again.
gist github com/Drapersniper/b9bf329ff41153234b8ffa248fe6cc51

Animosity022 · October 24, 2021, 2:07pm

It's super tough as you have a single second of a log file and nothing else.

You need to:

Change log level to debug
Try to recreate the issue
Share the full debug log

Drapersniper · October 24, 2021, 2:17pm

gist.github.com

https://gist.github.com/Drapersniper/b9bf329ff41153234b8ffa248fe6cc51

rclone-anime-err.txt

2021/10/24 15:00:10 INFO  : vfs cache purgeClean item.Reset Fairy Tail (2009) {tvdb-114801}/Season 03/Fairy Tail (2009) - S03E004 - 0100 - [Bluray-1080p][8bit][AVC][TrueHD 5.1][EN+JA].mkv: In-access item skipped, freed 0 bytes
2021/10/24 15:00:10 INFO  : vfs cache purgeClean item.Reset Fairy Tail (2009) {tvdb-114801}/Season 03/Fairy Tail (2009) - S03E002 - 0098 - [Bluray-1080p][8bit][AVC][TrueHD 5.1][EN+JA].mkv: In-access item skipped, freed 0 bytes
2021/10/24 15:00:10 INFO  : vfs cache purgeClean item.Reset Fairy Tail (2009) {tvdb-114801}/Season 03/Fairy Tail (2009) - S03E004 - 0100 - [Bluray-1080p][8bit][AVC][TrueHD 5.1][EN+JA].mkv: In-access item skipped, freed 0 bytes
2021/10/24 15:00:10 INFO  : vfs cache purgeClean item.Reset Fairy Tail (2009) {tvdb-114801}/Season 03/Fairy Tail (2009) - S03E002 - 0098 - [Bluray-1080p][8bit][AVC][TrueHD 5.1][EN+JA].mkv: In-access item skipped, freed 0 bytes
2021/10/24 15:00:10 INFO  : vfs cache purgeClean item.Reset Fairy Tail (2009) {tvdb-114801}/Season 03/Fairy Tail (2009) - S03E004 - 0100 - [Bluray-1080p][8bit][AVC][TrueHD 5.1][EN+JA].mkv: In-access item skipped, freed 0 bytes
2021/10/24 15:00:10 INFO  : vfs cache purgeClean item.Reset Fairy Tail (2009) {tvdb-114801}/Season 03/Fairy Tail (2009) - S03E002 - 0098 - [Bluray-1080p][8bit][AVC][TrueHD 5.1][EN+JA].mkv: In-access item skipped, freed 0 bytes
2021/10/24 15:00:10 INFO  : vfs cache purgeClean item.Reset Fairy Tail (2009) {tvdb-114801}/Season 03/Fairy Tail (2009) - S03E004 - 0100 - [Bluray-1080p][8bit][AVC][TrueHD 5.1][EN+JA].mkv: In-access item skipped, freed 0 bytes
2021/10/24 15:00:10 INFO  : vfs cache purgeClean item.Reset Fairy Tail (2009) {tvdb-114801}/Season 03/Fairy Tail (2009) - S03E002 - 0098 - [Bluray-1080p][8bit][AVC][TrueHD 5.1][EN+JA].mkv: In-access item skipped, freed 0 bytes
2021/10/24 15:00:10 INFO  : vfs cache purgeClean item.Reset Fairy Tail (2009) {tvdb-114801}/Season 03/Fairy Tail (2009) - S03E004 - 0100 - [Bluray-1080p][8bit][AVC][TrueHD 5.1][EN+JA].mkv: In-access item skipped, freed 0 bytes
2021/10/24 15:00:10 INFO  : vfs cache purgeClean item.Reset Fairy Tail (2009) {tvdb-114801}/Season 03/Fairy Tail (2009) - S03E002 - 0098 - [Bluray-1080p][8bit][AVC][TrueHD 5.1][EN+JA].mkv: In-access item skipped, freed 0 bytes

This file has been truncated. show original

system · November 23, 2021, 2:18pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.