Unstable Windows mount

What is the problem you are having with rclone?

Unstable mount in windows 10 64bit. Keeps crashing explorer.. Just looking to see if there is any way to tweak my mount code to make it more stable

What is your rclone version (output from rclone version)

rclone v1.48.0

  • os/arch: windows/amd64
  • go version: go1.12.3

Which OS you are using and how many bits (eg Windows 7, 64 bit)

windows 10 64bit

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

mount --log-file "C:\Users\Shenniko.config\rclone\Rclone_Log_Files\Mount.txt" --log-level INFO --allow-non-empty --allow-other --fuse-flag sync_read --tpslimit 10 --tpslimit-burst 10 --dir-cache-time=160h --buffer-size=64M --attr-timeout=1s --vfs-read-chunk-size=2M --vfs-read-chunk-size-limit=2G --vfs-cache-max-age=5m --vfs-cache-mode writes --dir-cache-time 72h gdrive_media_vfs: N: --config "C:\Users\Shenniko.config\rclone\rclone.conf"rclone

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

2019/09/05 01:30:33 DEBUG : rclone: Version "v1.48.0" starting with parameters ["rclone" "mount" "--log-file" "C:\Users\Shenniko\.config\rclone\Rclone_Log_Files\Mount_M.txt" "--allow-non-empty" "--allow-other" "--fuse-flag" "sync_read" "--tpslimit" "10" "--tpslimit-burst" "10" "--dir-cache-time=160h" "--buffer-size=64M" "--attr-timeout=1s" "--vfs-read-chunk-size=2M" "--vfs-read-chunk-size-limit=2G" "--vfs-cache-max-age=5m" "--vfs-cache-mode" "writes" "--dir-cache-time" "72h" "gdrive_media_vfs:" "M:" "--config" "C:\Users\Shenniko\.config\rclone\rclone.conf" "-vv"]
2019/09/05 01:30:33 DEBUG : Using config file from "C:\Users\Shenniko\.config\rclone\rclone.conf"
2019/09/05 01:30:33 INFO : Starting HTTP transaction limiter: max 10 transactions/s with burst 10
2019/09/05 01:30:34 DEBUG : Encrypted drive 'gdrive_media_vfs:': Mounting on "M:"
2019/09/05 01:30:35 DEBUG : vfs cache root is "C:\Users\Shenniko\AppData\Local\rclone\vfs\gdrive_media_vfs"
2019/09/05 01:30:35 DEBUG : Adding path "vfs/forget" to remote control registry
2019/09/05 01:30:35 DEBUG : Adding path "vfs/refresh" to remote control registry
2019/09/05 01:30:35 DEBUG : Adding path "vfs/poll-interval" to remote control registry
2019/09/05 01:30:35 DEBUG : Encrypted drive 'gdrive_media_vfs:': Mounting with options: ["-o" "fsname=gdrive_media_vfs:" "-o" "subtype=rclone" "-o" "max_readahead=131072" "-o" "attr_timeout=1" "-o" "atomic_o_trunc" "-o" "uid=-1" "-o" "gid=-1" "--FileSystemName=rclone" "-o" "volname=gdrive_media_vfs" "-o" "nonempty" "-o" "allow_other" "sync_read"]
Invalid argument "sync_read"
2019/09/05 01:30:35 DEBUG : Encrypted drive 'gdrive_media_vfs:': Init:
2019/09/05 01:30:35 DEBUG : Encrypted drive 'gdrive_media_vfs:': >Init:
2019/09/05 01:30:35 DEBUG : /: Statfs:
2019/09/05 01:30:35 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274509047278 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2019/09/05 01:30:35 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2019/09/05 01:30:35 DEBUG : /: >Getattr: errc=0
2019/09/05 01:30:35 DEBUG : /: Readlink:
2019/09/05 01:30:35 DEBUG : /: >Readlink: linkPath="", errc=-40
The service rclone has been started.
2019/09/05 01:30:35 DEBUG : /: Statfs:
2019/09/05 01:30:35 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274509047278 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2019/09/05 01:30:35 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2019/09/05 01:30:35 DEBUG : /: >Getattr: errc=0
2019/09/05 01:30:35 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2019/09/05 01:30:35 DEBUG : /: >Getattr: errc=0
2019/09/05 01:30:35 DEBUG : /: Opendir:
2019/09/05 01:30:35 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/09/05 01:30:35 DEBUG : /: >OpenFile: fd=/ (r), err=
2019/09/05 01:30:35 DEBUG : /: >Opendir: errc=0, fh=0x0
2019/09/05 01:30:35 DEBUG : /: Releasedir: fh=0x0
2019/09/05 01:30:35 DEBUG : /: >Releasedir: errc=0
2019/09/05 01:30:35 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2019/09/05 01:30:35 DEBUG : /: >Getattr: errc=0
2019/09/05 01:30:35 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2019/09/05 01:30:35 DEBUG : /: >Getattr: errc=0
2019/09/05 01:30:35 DEBUG : /: Opendir:
2019/09/05 01:30:35 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/09/05 01:30:35 DEBUG : /: >OpenFile: fd=/ (r), err=
2019/09/05 01:30:35 DEBUG : /: >Opendir: errc=0, fh=0x0
2019/09/05 01:30:35 DEBUG : /: Releasedir: fh=0x0
2019/09/05 01:30:35 DEBUG : /: >Releasedir: errc=0
2019/09/05 01:30:35 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2019/09/05 01:30:35 DEBUG : /: >Getattr: errc=0
2019/09/05 01:30:35 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2019/09/05 01:30:35 DEBUG : /: >Getattr: errc=0
2019/09/05 01:30:35 DEBUG : /: Opendir:
2019/09/05 01:30:35 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/09/05 01:30:35 DEBUG : /: >OpenFile: fd=/ (r), err=
2019/09/05 01:30:35 DEBUG : /: >Opendir: errc=0, fh=0x0
2019/09/05 01:30:35 DEBUG : /: Releasedir: fh=0x0
2019/09/05 01:30:35 DEBUG : /: >Releasedir: errc=0

Your rclone log doesnt indicate errors and you say that explorer itself crashes.
Does explorer actually crash or just freeze up? ... this is important to be spesific about.
It would help if you specify exactly how and then the explorer crash happens. When does it happen, and does anything spesific trigger it? Is it repeatable? Does it actually crash or just become unresponsive?

This is not something I am very familiar with, but a quick google indicates it performs the operations synchronously, so this will probably make explorer freeze until operations are done. What is your reason for using this, and are you sure this isn't the cause of problems?

Other comments:

This is pretty low, but should only affect performance. Unrelated to the issue, but unless you have a very low bandwidth I'd suggest increasing it. 128M is the default, which is fine for most uses and has good performance.

This shouldn't be necessary and will only limit your performance. The Gdrive pacer should handle this for you automatically and allow you to burst the API within your limits without spamming and getting limited as a result.

Generally not recommended. Mounting into non-empty folder will have potential for a lot of problems to happen. Only do this if you are an advanced user and know exactly what this means and why you need it.

This is already the default value.

Also you have --dir-cache-time set twice. Only the last parameter will actually be used in this case.
A long --dir-cache-time is perfectly ok on a Gdrive as it uses polling to keep up to date.

It would be worth trying the latest release which is v1.49.1

I've updated to the latest version (1.48.0)
And made changes to the mount code as specified:
Im using NSSM to create a service to mount the drive..
mount --log-file "C:\Users\Shenniko\.config\rclone\Rclone_Log_Files\Mount.txt" --log-level INFO --allow-other --dir-cache-time=160h --buffer-size=64M --vfs-read-chunk-size=1G --vfs-cache-max-age=5m --vfs-cache-mode writes gdrive_media_vfs: N: --config "C:\Users\Shenniko\.config\rclone\rclone.conf"

and am still having issues with the mount.

Usually when there is a stable connection, it shows the drive name as "gdrive_media_vfs", but when its having issues, it just shows "Local Disk", i click on it, and i get the spinning wheel, and the explorer becomes unresponsive..

The latest is v1.49.1

Apologies, typo on my part..

rclone v1.49.1

  • os/arch: windows/amd64
  • go version: go1.12.3

Do you think this might be to do with your Internet connection?

Shouldn't be:

Plus I'm connected via Ethernet, so no wifi issues..

Are you doing anything in particular when the mount becomes unresponsive?

The only thing I can think of is using Sonarr, pointing it to directly download to my (mounted) N: Drive

Edit: Yeah, i just tried Sonarr again, and it seems once a download has finished and it tries to move the file to the mounted N Drive.. it becomes unstable..

I erroneously quoted your chunk-limit instead of size earlier, sorry. It's the chunk-size that was a bit low (I see you later increased it). The default is 128M and that's usually a good number for most. In any case I doubt a very high or low number on this is related directly to your problem.

I think it would be worth monitoring your memory usage while sonarr is doing that operation. See that it's not going nuts on concurrent connections. A buffer-size of 64M is not that unreasonable, but that's still pr. connection so it ultimately will depend a bit on how aggressive the software is on concurrent file-access. If that seems to be the case you can either see if the software has settings to adjust this directly, or you can attempt a lower buffer size. The default is 16M and that's rarely a significant limiting factor on performance.

Changed code to the default 16MB also added a few extras to see if that would help.. but still having issues.. Seems to work fine navigating.. but as soon as Sonnar has downloaded the TV show and then looks to move to the mapped drive.. it just dies.. Sonnar can no longer see the drive and explorer crashes when i try and access the drive.. Closing Sonnar, drive is still FUBAR, restarting the service fixes it...

Any ideas how to fix this? New code below:

mount gdrive_media_vfs: N: --allow-other --dir-cache-time=160h --cache-chunk-size=10M --cache-info-age=168h --cache-workers=5 --attr-timeout=1s --vfs-read-chunk-size=128M --vfs-cache-max-age=1h --vfs-cache-mode writes --buffer-size=16M --cache-tmp-upload-path "E:\Rclone_Cache\Upload" --cache-tmp-wait-time 30m --cache-db-path "E:\Rclone_Cache\Cache" --log-file "C:\Users\Shenniko.config\rclone\Rclone_Log_Files\Mount.txt" --log-level INFO --config "C:\Users\Shenniko.config\rclone\rclone.conf"

Put the log file in debug and share the full log when the issue occurs.

ok, so think i figured it out... it seems better with whatever code ive tweaked...

But it seems when the download has finished from Sonarr, and then it moves the file from local to the mounted google drive, and im trying to access during that process.. Explorer bugs out.. it doesnt completely crash like it used to.. just locks and takes a LONG time for it to unlock..

Debug posted below.

http://s000.tinyupload.com/?file_id=88880287175167649497

Edit: Just to add, its definitely when uploading to the Gdrive, i cannot access the folder through windows explorer.

This is likely just a result of Sonarr being very aggressive about it's file-requests and drowning out your own list-request which explorer will have to wait to complete before continuing. This would be expected behavior even if it is not very ideal. There is very little rclone can do on it's end to limit a spesific program from aggressively hogging all the available resources. It will just fill requests from the OS to the best of it's ability.

The best solution here might be to set up a secondary service account with a mount dedicated to Sonarr and related programs, so that you can avoid getting your general-use access (like simply browsing the mount) getting choked out. They can both point to the same drive of course. --poll-interval (default 1min) will determine how quickly you would see the changes appear on your main mount. It can be lowered if needed - but this check does use an API call so keep it reasonable. I've tested down to 10s with no problems and that would be using 1% of default API quota.

Or - if you can figure out a way to configure Sonarr to not run so many concurrent file-operations then that would be even cleaner solution that directly adresses the problem - but of course this entirely depends on Sonarr having such an option (without literally changing the code). I am not familiar enough with Sonarr to say if it does or not...

I'll try mounting another drive.. i dont think Sonarr has that capability :smiley:
Do i have to create another entry in my rclone config? Or just create a new service to mount the drive?

That seems to be part of the log and not the whole log. Can you share the whole log?

Basically just go into your config and make a copy of your existing setup with a new name.

  • Set up a different authorization (I think a service account would be ideal here if you know how, but Oauth would likely work too)
  • Make a copy of your existing mount command to use on the secondary drive, but if you use write caching then you probably want to change the directory for that. You also obviously need to change the name of the remote you are mounting to match the new secondary remote. Not much else should need to change

Then mount both - Set Sonarr and other such "bulk programs" that don't need to be very responsiveto use the secondary mount, while you can use your primary for general/manual and other light use and keep it from being overwhelmed.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.