Plex playing, buffering, stoppage - rclone mount issues

EkoPyaar · May 4, 2020, 12:37am

Hello @VBB,

I am in the process of creating the bat files. I have a few questions like the last bat file titled: title Rclone Mount READ/WRITE If I don't want to move things within my gdrivecrypt because I already uploaded to the final locations via rclone browser, its no point using this rclone mount READ/WRITE correct?

Currently I use this command to mount the drive, rclone -v -v -v mount gdrivecrypt: Y:

Please correct me, however what your saying is to run bat file titled: Rclone Mount READ ONLY

Than rclone -v -v -v mount gdrivecrypt: Y to mount drive on windows file explorer.

Then final command would be bat file titled: Rclone Prime

Than run plex scan?

I apologies in advance, still learning all this stuff.. As to where I am located, In US, the golden state.

Thanks

VBB · May 4, 2020, 1:05am

If you use RcloneBrowser to move your files to their final destination, then you do not need my READ/WRITE mount, correct. You want to use the READ ONLY mount for Plex. BTW, the reason I only use one -v in my commands is that I don't need to see everything the mount is doing. I only want to see errors. So, again:

Rclone READ ONLY mount
Rclone PRIME (it will be a blank window that will only output something when it's done. If all goes smoothly, it will say "OK")
Plex scan

After discussing some things with @Animosity022 today, I modified the READ ONLY mount slightly:

@echo off
title Rclone Mount READ ONLY
D:\Programs\Rclone\rclone mount --attr-timeout 1000h --buffer-size 256M --dir-cache-time 1000h --poll-interval 0 --rc --read-only --timeout 1h -v Google_Drive_Crypt: G:
pause

EkoPyaar · May 4, 2020, 2:24am

@VBB ran the 2 commands.

Rclone Mount READ ONLY
Rclone Prime

Ran plex scan and looks good. I don't notice any difference in speeds however ill trust your words.

Update: From the file explorer, When navigating within the drive, I am noticing it to be super quick and fast!

In terms of plex playback.. I will seek out your help as was facing issues yesterday still..

EkoPyaar · May 4, 2020, 11:11am

I'm sure I already know however I would like to confirm my thinking is correct

As noted above I ran the 2 commands. READ ONLY and Prime. Now being in this status, I went to rclone browser and uploaded a file. After completing the file, it displays in rclone browser however it does not on mount drive gdrivecypt Y: and plex scan doesn't pick it up.

Is this because I am in the READ ONLY command and that's why from a file explorer perspective it will not show up until I stop the READ ONLY mount via CTRL+C?

2nd question: Being in this command status Rclone Mount READ ONLY & Rclone Prime I stopped it by CTRL+C and the drive gdrivecrypt Y: is no longer under displaying under my PC.

On the command it is asking to Terminate batch job (Y/N)? What does this mean? As I am currently not uploading anything via rclone browser. Do I enter Y or N?

Capture12

Do I have to run READ/WHITE then READ ONLY and than Prime to visually see the file I uploaded and resume the service where plex scan picks it up?

Thanks for reading through my concern..

VBB · May 4, 2020, 6:05pm

This is normal behavior. You do not have to quit the READ ONLY mount in order to refresh it. Simply run PRIME again, and then Plex scan.

To stop the mount, you press CTRL+C, like you did, and you answer "Y". That's the cleanest way to do it

EkoPyaar · May 5, 2020, 11:48am

Thanks @VBB that worked perfect! Also playblack is spot on smooth

Thanks everyone for sharing knowledge. I'll consider this topic successfully closed.

asdffdsa · May 5, 2020, 3:06pm

hi,

the way i prime is dir y: /s

works for all backends
vfs/refresh has no speed advantage as tested on google drive.

rclone rc vfs/refresh recursive=true --fast-list
gdrive: works
onedrvie: works
s3 wasabi: does not work

tested using rclone-v1.51.0-264-g2696e369-fix-4104.exe
one folder with 5000 files

gdrive
	vfs	dir
start	14:24:49	14:25:41
end	14:25:04	14:25:57
duration	0:00:15	0:00:16

Animosity022 · May 5, 2020, 6:42pm

It's got a huge speed advantage. It depends on how many directories and object you have as the refresh has the ability to use fast-list, while the mount does not.

asdffdsa · May 5, 2020, 6:48pm

thanks,
i will edit that post to make this clear.

that my comment was based on the use case in this post; a mount on windows.
in that case, what i wrote seems to be correct.

Animosity022 · May 5, 2020, 6:51pm

That's why @VBB uses the rc command as he's got a big library on Windows.

fast-list makes a big difference vs a dir/find but you need enough items and folders for it to matter as there is a time when they numbers are the same.

asdffdsa · May 5, 2020, 7:00pm

what would be considered a big library?

do you know why vfs/refresh seems to do nothing with s3 wasabi on windows mount?
i have to do a dir /s

Animosity022 · May 5, 2020, 7:09pm

What does the mount say when in debug and you hit the command?

You should see something like:

2020/05/05 15:08:00 DEBUG : rc: "vfs/refresh": with parameters map[recursive:true]
2020/05/05 15:08:00 DEBUG : : Reading directory tree

and the command should be "Ok" when done.

felix@gemini:~$ rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:4321
{
	"result": {
		"": "OK"
	}
}

asdffdsa · May 5, 2020, 7:15pm

maybe i need to add --rc-addr 127.0.0.1:4321 but i did not need that when the mount as gdrive

but yes,
i get the OK but when i try to enter the folder it gets stuck for over one minute.
only after a dir /s, i can enter the folder instantly.
and the folder has only 5000 files of size 1 byte.

C:\data\rclone\scripts\rr\other\mount>C:\data\rclone\scripts\rr\other\mount\rclone-v1.51.0-264-g2696e369-fix-4104.exe rc vfs/refresh recursive=true --fast-list -vv
2020/05/05 15:10:48 DEBUG : rclone: Version "v1.51.0-264-g2696e369-fix-4104-cwd-disappears-beta" starting with parameters ["C:\\data\\rclone\\scripts\\rr\\other\\mount\\rclone-v1.51.0-264-g2696e369-fix-4104.exe" "rc" "vfs/refresh" "recursive=true" "--fast-list" "-vv"]
{
        "result": {
                "": "OK"
        }
}
2020/05/05 15:10:49 DEBUG : 4 go routines active

Animosity022 · May 5, 2020, 7:18pm

What do you see in the mount debug log?

It also runs in one second, which seems odd.

Animosity022 · May 5, 2020, 7:20pm

Not sure I have a 'big' library per se, but my refresh takes 1m 18s with the rc command about 15-20 minutes via a find, which is basically the same as a dir /s.

felix@gemini:/GD$ find . -type f | wc -l
40531
felix@gemini:/GD$ find . -type d | wc -l
4062

I have 40k files and 4k directories.

asdffdsa · May 5, 2020, 7:22pm

yes, one second, i noticed that as well.

here is the log, after i did the vfs/refresh

2020/05/05 15:20:54 DEBUG : rclone: Version "v1.51.0-264-g2696e369-fix-4104-cwd-disappears-beta" starting with parameters ["C:\\data\\rclone\\scripts\\rr\\other\\mount\\rclone-v1.51.0-264-g2696e369-fix-4104.exe" "mount" "remote:" "y:" "--attr-timeout=1000h" "--buffer-size=256M" "--dir-cache-time=1000h" "--poll-interval=0" "--rc" "--timeout=1h" "--log-file=C:\\data\\rclone\\scripts\\rr\\other\\mount\\log.rclone-v1.51.0-264-g2696e369-fix-4104.txt" "--log-level=DEBUG"]
2020/05/05 15:20:54 NOTICE: Serving remote control on http://127.0.0.1:5572/
2020/05/05 15:20:54 DEBUG : Using RCLONE_CONFIG_PASS password.
2020/05/05 15:20:54 DEBUG : Using config file from "c:\\data\\rclone\\scripts\\rclone.conf"
2020/05/05 15:20:54 DEBUG : S3 bucket aliasremote: Mounting on "y:"
2020/05/05 15:20:54 INFO  : S3 bucket aliasremote: poll-interval is not supported by this remote
2020/05/05 15:20:54 DEBUG : Adding path "vfs/forget" to remote control registry
2020/05/05 15:20:54 DEBUG : Adding path "vfs/refresh" to remote control registry
2020/05/05 15:20:54 DEBUG : Adding path "vfs/poll-interval" to remote control registry
2020/05/05 15:20:54 DEBUG : S3 bucket aliasremote: Mounting with options: ["-o" "fsname=wasabieast2:aliasremote" "-o" "subtype=rclone" "-o" "max_readahead=131072" "-o" "attr_timeout=3.6e+06" "-o" "atomic_o_trunc" "-o" "uid=-1" "-o" "gid=-1" "--FileSystemName=rclone" "-o" "volname=wasabieast2 aliasremote"]
2020/05/05 15:20:54 DEBUG : S3 bucket aliasremote: Init: 
2020/05/05 15:20:54 DEBUG : S3 bucket aliasremote: >Init: 
2020/05/05 15:20:54 DEBUG : /: Statfs: 
2020/05/05 15:20:54 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2020/05/05 15:20:54 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/05/05 15:20:54 DEBUG : /: >Getattr: errc=0
2020/05/05 15:20:54 DEBUG : /: Readlink: 
2020/05/05 15:20:54 DEBUG : /: >Readlink: linkPath="", errc=-40
The service rclone-v1.51.0-264- has been started.
2020/05/05 15:20:54 DEBUG : /: Statfs: 
2020/05/05 15:20:54 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2020/05/05 15:20:54 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/05/05 15:20:54 DEBUG : /: >Getattr: errc=0
2020/05/05 15:20:54 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/05/05 15:20:54 DEBUG : /: >Getattr: errc=0
2020/05/05 15:20:54 DEBUG : /: Opendir: 
2020/05/05 15:20:54 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2020/05/05 15:20:54 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2020/05/05 15:20:54 DEBUG : /: >Opendir: errc=0, fh=0x0
2020/05/05 15:20:54 DEBUG : /: Releasedir: fh=0x0
2020/05/05 15:20:54 DEBUG : /: >Releasedir: errc=0
2020/05/05 15:20:54 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/05/05 15:20:54 DEBUG : /: >Getattr: errc=0
2020/05/05 15:20:54 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/05/05 15:20:54 DEBUG : /: >Getattr: errc=0
2020/05/05 15:20:54 DEBUG : /: Opendir: 
2020/05/05 15:20:54 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2020/05/05 15:20:54 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2020/05/05 15:20:54 DEBUG : /: >Opendir: errc=0, fh=0x0
2020/05/05 15:20:54 DEBUG : /: Releasedir: fh=0x0
2020/05/05 15:20:54 DEBUG : /: >Releasedir: errc=0
2020/05/05 15:20:54 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/05/05 15:20:54 DEBUG : /: >Getattr: errc=0
2020/05/05 15:20:54 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/05/05 15:20:54 DEBUG : /: >Getattr: errc=0
2020/05/05 15:20:54 DEBUG : /: Opendir: 
2020/05/05 15:20:54 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2020/05/05 15:20:54 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2020/05/05 15:20:54 DEBUG : /: >Opendir: errc=0, fh=0x0
2020/05/05 15:20:54 DEBUG : /: Releasedir: fh=0x0
2020/05/05 15:20:54 DEBUG : /: >Releasedir: errc=0
2020/05/05 15:20:54 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/05/05 15:20:54 DEBUG : /: >Getattr: errc=0
2020/05/05 15:20:54 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/05/05 15:20:54 DEBUG : /: >Getattr: errc=0
2020/05/05 15:20:54 DEBUG : /: Opendir: 
2020/05/05 15:20:54 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2020/05/05 15:20:54 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2020/05/05 15:20:54 DEBUG : /: >Opendir: errc=0, fh=0x0
2020/05/05 15:20:54 DEBUG : /: Releasedir: fh=0x0
2020/05/05 15:20:54 DEBUG : /: >Releasedir: errc=0
2020/05/05 15:20:54 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/05/05 15:20:54 DEBUG : /: >Getattr: errc=0
2020/05/05 15:20:54 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/05/05 15:20:54 DEBUG : /: >Getattr: errc=0
2020/05/05 15:20:54 DEBUG : /: Opendir: 
2020/05/05 15:20:54 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2020/05/05 15:20:54 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2020/05/05 15:20:54 DEBUG : /: >Opendir: errc=0, fh=0x0
2020/05/05 15:20:54 DEBUG : /: Releasedir: fh=0x0
2020/05/05 15:20:54 DEBUG : /: >Releasedir: errc=0
2020/05/05 15:20:54 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/05/05 15:20:54 DEBUG : /: >Getattr: errc=0
2020/05/05 15:20:54 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/05/05 15:20:54 DEBUG : /: >Getattr: errc=0
2020/05/05 15:20:54 DEBUG : /: Opendir: 
2020/05/05 15:20:54 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2020/05/05 15:20:54 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2020/05/05 15:20:54 DEBUG : /: >Opendir: errc=0, fh=0x0
2020/05/05 15:20:54 DEBUG : /: Releasedir: fh=0x0
2020/05/05 15:20:54 DEBUG : /: >Releasedir: errc=0
2020/05/05 15:20:54 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/05/05 15:20:54 DEBUG : /: >Getattr: errc=0
2020/05/05 15:20:54 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/05/05 15:20:54 DEBUG : /: >Getattr: errc=0
2020/05/05 15:20:54 DEBUG : /: Opendir: 
2020/05/05 15:20:54 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2020/05/05 15:20:54 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2020/05/05 15:20:54 DEBUG : /: >Opendir: errc=0, fh=0x0
2020/05/05 15:20:54 DEBUG : /: Releasedir: fh=0x0
2020/05/05 15:20:54 DEBUG : /: >Releasedir: errc=0
2020/05/05 15:21:07 DEBUG : rc: "vfs/refresh": with parameters map[recursive:true]
2020/05/05 15:21:07 DEBUG : : Reading directory tree
2020/05/05 15:21:08 DEBUG : : Reading directory tree done in 973.8021ms
2020/05/05 15:21:08 DEBUG : rc: "vfs/refresh": reply map[result:map[:OK]]: <nil>
The service rclone-v1.51.0-264- has been stopped.
2020/05/05 15:21:11 DEBUG : S3 bucket aliasremote: Destroy: 
2020/05/05 15:21:11 DEBUG : S3 bucket aliasremote: >Destroy: 
2020/05/05 15:21:11 DEBUG : rclone: Version "v1.51.0-264-g2696e369-fix-4104-cwd-disappears-beta" finishing with parameters ["C:\\data\\rclone\\scripts\\rr\\other\\mount\\rclone-v1.51.0-264-g2696e369-fix-4104.exe" "mount" "remote:" "y:" "--attr-timeout=1000h" "--buffer-size=256M" "--dir-cache-time=1000h" "--poll-interval=0" "--rc" "--timeout=1h" "--log-file=C:\\data\\rclone\\scripts\\rr\\other\\mount\\log.rclone-v1.51.0-264-g2696e369-fix-4104.txt" "--log-level=DEBUG"]
2020/05/05 15:21:11 DEBUG : Calling host.Unmount
2020/05/05 15:21:11 DEBUG : host.Unmount failed

VBB · May 5, 2020, 7:32pm

My library is rather large at 13k movies and 2400 shows. Add to that an nfo file for each one...

So, the RC refresh is super fast compared to a dir /s, which I used to do, but the Plex scan itself is not much faster. At least in the Movies library, but that's because I haven't had the time and patience to split everything. The slowdown in Windows Explorer starts at around 4-5k items within a single folder.

asdffdsa · May 5, 2020, 7:34pm

i agree that the vfs/refresh seems to work with gdrive,
but not with s3 wasabi.
and i do not use windows explorer, i use double commander.

asdffdsa · May 5, 2020, 9:13pm

on s3 wasabi, i created 10 folders with 5000 files per folder.
total of 50,000 files.

i mount that set of folders.

vfs/refresh takes just 6 seconds.
but if i try to enter any folder, there is a 60+ second delay.
in effect, that refresh seems to be do nothing of use.

if i do a

dir /s

on just one folder, it takes 11 seconds.
but when i enter that folder, zero delay

EkoPyaar · May 6, 2020, 2:14am

@VBB Agreed from your statements however, having one folder, i.e. 'Movies' and all the movie folders within directory is so cool I love to view all my movies in one folder! Yes the downside is what your saying..