Gcrpyt Mount hangs while opening folder

All right I will look into it over the next few days and tweak my settings accordingly.

[gdrive]
type = drive
scope = drive
token = {"access_token":"xxxxxxxxxxxxxxxxxxxxxxxx","token_type":"Bearer","refresh_token":"xxxxxxxxxxxxxxxxxxxxxxxx","expiry":"2019-10-02T22:45:04.0657592+05:30"}
client_id = xxxxxxxxxxxxxxxxxxxxxxxx
client_secret = xxxxxxxxxxxxxxxxxxxxxxxx
chunk_size = 8M

REMOVED CACHE

[gcrypt]
type = crypt
remote = gdrive:
filename_encryption = standard
directory_name_encryption = true
password = xxxxxxxxxxxxxxxxxxxxxxxx

That should be all you need.

  • Remove cache
  • Re-route Crypt to point to your Gdrive instead of the Gcache as it did before.
1 Like

All right thanks!

kk, appreciate your uploads btw :wink:

2 Likes

Opsie - i forgot to add link to Animosity's "recommended settings" post.

I edited the posted reference earlier to include it also.

1 Like

BTW I did see you note in another thread that you precached your drive too, but used dir /s for the job.

That does work - but I would advice you look into doing it the way I explained here instead - because dir /s goes via the mount and can not take advantage of --fast-list (which is often 10-15 times faster as well as way less API intensive). Doing this through an RC request makes --fast-list possible though.

I can help with any scripting-related stuff as I assume you are on windows - so I have done all this before for my own use :slight_smile:

1 Like

Saw that, yeah. Will give RC a shot later today. Thanks for the tip!

I saw Ani uses this now as well. Wonder if this really has any noticeable impact on a read-only mount. Same for --poll-interval. I've had both at their defaults.

It's useful specifically for read-operations - so you'd all of the benefits on a read-only. Particularly:

  • When you have to manually navigate through your files
  • When you do a search
  • When anything needs to scan through the hierarchy for whatever reason.

It doesn't really affect download performance in any way. It only makes the drive respond much faster to requests and listings as it already has this data and in most cases never even has to ask the Cloud for the info. Since latency is one of the bigest limitations on a Cliud-drive generally this has a lot of benefits to it.

Please note that polling-support is essential for any such setup. Not all Cloud services can support that. Gdrive can though (and some others) see docs for spesific info. I say this because I don't specifically know what you use.

1 Like

I'll give those two a try as well. In Windows you need all the speed you can get :stuck_out_tongue:

I was able to reproduce the problem. This with my old setup only I have not made any changes yet.
this is the -vv output https://controlc.com/19fc6adb

I had to close the cmd window manually even doing ctrl+c numerous times did not work

I don't see any obvious errors here, but it seems to be performing some quite sizable listing operations. If it's large and slow enough it would make sense to appear unresponsive for a while - even though your experience seems a bit extreme.

But if you haven't adjusted Plex scanning yet then it's very possible that is causing spikes of extreme load. That is very possible. Then it could definitely slow to a crawl for your manual browsing as it happens.

Maybe we can get @Animosity022 in here to take a look. He's much better at reading the logs than I am.

2019/10/03 01:55:50 DEBUG : qlgrhbtc8prp71nm57a3qr8ln0: list: warm 1114 from cache for: qlgrhbtc8prp71nm57a3qr8ln0, expiring on: 2019-10-04 01:55:31.5608724 +0530 IST
2019/10/03 01:55:50 DEBUG : qlgrhbtc8prp71nm57a3qr8ln0: list: cached entries:

This is the part of the log where the window hanged/became not responding. And everything after that in the log happened while the window was hanged.

Yea that is the thing I was suspecting. I am not sure exactly what process this message is referring to though. That's why I called for help :smiley:

I think something (likely a program, possibly Plex) just requested 1114 files (probably over many folders) listed in the cache - but those were already expired so it probably had to list them from the drive again. Since the mount can't fast-list that either it would have to iterate through it and list every folder - which can take some time depending on the complexity of the folder structure. I think it is normal for the mount to do this operation synchronously - or in other words it will pause everything else until it finishes that job. It's just not supposed to stall for that long under most normal use-cases :slight_smile:

Take that hypothesis with a grain of salt. That's just my best approximation of what I get from that debug-line. I'm certainly not familiar with all debug output yet.

Should I use your suggested mount settings or animosity's on his github?
And you told me to refer to anymosity's pre caching script , is it this ? https://github.com/animosity22/homescripts/blob/master/systemd/rclone.service

And I looked at his recommended plex settings. He basically only suggests doing 2 things one is disable generation of thumbnails for each library and to disable enhanced media analysis task. I have done those 2 things and gotten rid of cache in my conf file. Now I just need to properly implement the precaching and mount script and would like some help with that.

@thestigma
These are the scripts I am trying:
Mount script:

start /b rclone mount gcrypt: E: --allow-other --attr-timeout 1000h --buffer-size 128M --dir-cache-time 1000h --log-level INFO --log-file D:\rclone\logs\rclone.log --poll-interval 15s --timeout 1h -rc

::Start cache warmup
call VFSCacheWarmup.bat E 5572

VFSCacheWarmup script:

@echo off
:: If window is not minimized, restart it as minimized
::if not DEFINED IS_MINIMIZED set IS_MINIMIZED=1 && start "" /min "%~dpnx0" %* && exit

::Variables
set driveletter=%1
set RCport=%2

:: Check that the folder is valid, otherwise wait until it is
echo Waiting for %driveletter%: to be ready ...

:LOOP1
vol %driveletter%: >nul 2>nul
if errorlevel 1 (
echo " "| set /p dummyName=.
timeout /t 1 > nul
goto LOOP1
) else (
echo.
echo Drive %driveletter%: OK!,
)

echo.
echo Warming up cache...
echo This may take a few minutes to complete
echo You may use the cloud-drive normally while this runs
echo.

echo Awaiting completion-message from RC...
rclone rc vfs/refresh -v --fast-list recursive=true --rc-addr localhost:5572 --rc-user test --rc-pass test --timeout 1h

::alternative slower method that does not use the remote control
::echo.
::echo verifying old-school ...
::%driveletter%:
::dir /s >nul

echo.
echo Cache warmup for %driveletter%: OK!
echo.

echo Closing windows in 5 seconds...
timeout /t 5 >nul

But I am getting this error :
2019/10/07 19:24:32 Fatal error: unknown shorthand flag: 'r' in -rc

found the issue...I was stupidly using -rc instead of --rc

I'm sorry, I must have confused myself somewhere. I see you are on Windows and Animosity is where I refer people who use Linux. I apologize. Sometimes I can get responses to different users mixed up.

This would be his precaching, but that's not going to work on windows:


What you linked was his his mounting system service by the way. Useful, but not needed to begin with. I can tech you how to do that in Windows later if you want.


You appear to be using my script as intended (calling it with parameters). Good.
This should be correct, assuming that you have rclone, mountscript and cache-warmup all in the same folder.

The error you are getting is due to a very simple mistake:
replace
-rc
with
--rc

Only "alternative shorthand" flags like -v (--verbose) and -P (--progress) use single dashes.

The good news is that when you are on Windows I can coach you directly and help you make what you need.

You got it before I did :wink:

Let me know if you need any more help getting things working or you want something customized to your need.

I see one more potential problem here:

You should only add the user and password here if you actually set a user/pass in the mount to begin with. You don't seem to do that (yet) and it is not needed for this either. Rclone might complain about an invalid login if you try to use send the command with a user/pass that was never set up (or it might ignore it - I'm not sure).

Ok thanks I will get rid of it. Rest of the settings seem fine to you?
Anymosity does not use --fast-list from what I could understand from his script here https://github.com/animosity22/homescripts/blob/master/systemd/GD-find.service . What difference does it make?