Slow scan plex tv show

[VBB]

@asdffdsa

--tpslimit 10 --tpslimit-burst 10

Anyway, try the flags that @asdffdsa and I recommended and remove --tpslimit 10 --tpslimit-burst 10 (not necessary). Run the refresh command so that the directories and file names are pre-cached, and then try scanning again. Should be much faster.

[RicarSeg (crazy apprentice)]

@asdffdsa

do you mean command or flags?
--log-file is a flag, not a command.

the order of the flags does not matter.

Excuse me, but yes, I meant flags

@asdffdsa

and not sure you need this
--tpslimit 10 --tpslimit-burst 10

I was guided a bit by the link you shared, I thought it was necessary to limit the number of API requests, I guess all prevention is good

@VBB

Anyway, try the flags that @asdffdsa and I recommended and remove --tpslimit 10 --tpslimit-burst 10 (not necessary). Run the refresh command so that the directories and file names are pre-cached, and then try scanning again. Should be much faster.

Thank you very much for the help you have given me, tomorrow I will try to execute the command and I will comment on the results!

In closing, I have one more question.

I thought that working with cache was noticeably better than working without it, for this reason I started with these flags to be able to share the same cache with two different computers, it is possible?

[VBB]

@RicarSeg

I was guided a bit by the link you shared, I thought it was necessary to limit the number of API requests

These are for Dropbox users, as @Animosity022 does not use GDrive any longer and has since updated his guide to reflect this.

@RicarSeg

I thought that working with cache was noticeably better than working without it, for this reason I started with these flags to be able to share the same cache with two different computers, it is possible?

I can't help you with Rclone cache questions, since I've never used it. I find it's not necessary, if your Internet speed is sufficiently fast to stream your media.

[asdffdsa (jojothehumanmonkey)]

it can also depends on the hardware.

--- i have a pi.zero, that i run rclone mount
its only storage system is sdcard, not really suitable for a vfs file cache for media files

--- there is a fellow rcloner running rclone serve dlna on a 128MB router.

[RicarSeg (crazy apprentice)]

@VBB

Estos son para usuarios de Dropbox, ya que@animosidad022ya no usa GDrive y desde entonces ha actualizado su guía para reflejar esto.

Thank you very much for the clarification!

@asdffdsa

it can also depends on the hardware.

--- i have a pi.zero, that i run rclone mount
its only storage system is sdcard, not really suitable for a vfs file cache for media files

--- there is a fellow rcloner running rclone serve dlna on a 128MB router.

Of course I guessed it!

This type of mount is used for small streams, in my case I usually have a medium number of streams, I think that's why I wanted to test the cache, but I'm still not sure.

I will try everything you have told me to see how it works.

Thank you very much for all your attention!

[RicarSeg (crazy apprentice)]

Download Quota Exceeded & VFS

Context is important here as it matters. For my context, I was speaking in regards to vfs-cache-mode none, which is has no impact on uploads as it's not used. If you turn vfs-cache-mode writes, you see it is used. I don't say this. @ncw does here -> https://github.com/rclone/rclone/issues/2542 vfs-cache-mode full should not be used with Plex streaming as it's counter productive. It has to download the full file for each read. So when you play something in Plex, it would fully download it 3-…

Goodnight!

Reading a bit I found this information that has me quite confused right now.

Surely I am wrong, what I intend with the vfs cache is to have only the content that is being transmitted in the cache.

I have re-scanned my libraries and the size on disk continues to increase considerably, should this be the case?

[asdffdsa (jojothehumanmonkey)]

well, your command uses
"--vfs-cache-max-size" "1500G" "--vfs-cache-max-age" "504h"

rclone uses sparse files for the vfs file cache.
so be careful, about the real size used.
in this case, only 20.0MB used.

[RicarSeg (crazy apprentice)]

True, you're right.

What I really need is that only the file to be transmitted be partially or totally downloaded, in my case I had not realized that the entire library was cached

[asdffdsa (jojothehumanmonkey)]

@RicarSeg

I had not realized that the entire library was cached

that is not possible. as the cache is just 1.5TiB and the library is 60TiB

you told rclone to use up to 1.5TiB for the vfs file cache.

[RicarSeg (crazy apprentice)]

@asdffdsa

that is not possible. as the cache is just 1.5TiB and the library is 60TiB

you told rclone to use up to 1.5TiB for the vfs file cache.

Good morning sir! it is not a joke

2022/06/27 07:09:41 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: >Getattr: errc=0
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: >Getattr: errc=0
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: OpenEx: flags=0x0
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: Open: flags=O_RDONLY
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: newRWFileHandle: 
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: >newRWFileHandle: err=<nil>
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: >Open: fd=PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv (rw), err=<nil>
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: >OpenFile: fd=PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv (rw), err=<nil>
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: >OpenEx: errc=0, fh=0x1
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: Getattr: fh=0x1
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: >Getattr: errc=0
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: Flush: fh=0x1
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv(0xc0006b5380): RWFileHandle.Flush
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: >Flush: errc=0
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: Release: fh=0x1
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv(0xc0006b5380): RWFileHandle.Release
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv(0xc0006b5380): close: 
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv(0xc0006b5380): >close: err=<nil>
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: >Release: errc=0
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: >Getattr: errc=0
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: >Getattr: errc=0
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: OpenEx: flags=0x0
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: Open: flags=O_RDONLY
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: newRWFileHandle: 
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: >newRWFileHandle: err=<nil>
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: >Open: fd=PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv (rw), err=<nil>
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: >OpenFile: fd=PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv (rw), err=<nil>
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: >OpenEx: errc=0, fh=0x1
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: Getattr: fh=0x1
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: >Getattr: errc=0
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: Read: ofst=0, fh=0x1
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv(0xc0006b53c0): _readAt: size=32768, off=0
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv(0xc0006b53c0): openPending: 
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: vfs cache: checking remote fingerprint "5769328178,2020-01-12 12:38:40.261 +0000 UTC,62f231724e7efdaab63c7927c2d93767" against cached fingerprint ""
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: vfs cache: truncate to size=5769328178
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC: Added virtual directory entry vAddFile: "Ant-Man (2015) [MicroHD 1080p][Dual].mkv"
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv(0xc0006b53c0): >openPending: err=<nil>
2022/06/27 07:09:42 DEBUG : vfs cache: looking for range={Pos:0 Size:32768} in [] - present false
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.RangeSeek from -1 to 0 length -1
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at -1 length 4096 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.openRange at 0 length 134217728
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 4096 length 8192 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 12288 length 16384 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 28672 length 32768 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 61440 length 65536 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv(0xc0006b53c0): >_readAt: n=32768, err=<nil>
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: >Read: n=32768
2022/06/27 07:09:42 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: Read: ofst=5769297920, fh=0x1
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv(0xc0006b53c0): _readAt: size=32768, off=5769297920
2022/06/27 07:09:42 DEBUG : vfs cache: looking for range={Pos:5769297920 Size:30258} in [{Pos:0 Size:61440}] - present false
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.RangeSeek from -1 to 5769297920 length -1
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at -1 length 4096 chunkOffset 5769297920 chunkSize 134217728
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.openRange at 5769297920 length 134217728
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 126976 length 131072 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 258048 length 262144 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 520192 length 524288 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 1044480 length 1048576 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 2093056 length 1048576 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 3141632 length 1048576 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 4190208 length 1048576 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 5238784 length 1048576 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 6287360 length 1048576 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 7335936 length 1048576 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:42 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 8384512 length 1048576 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:43 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 9433088 length 1048576 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:43 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 10481664 length 1048576 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:43 DEBUG : /PELICULAS/Marvel y DC: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/27 07:09:43 DEBUG : /PELICULAS/Marvel y DC: >Getattr: errc=0
2022/06/27 07:09:43 DEBUG : /PELICULAS/Marvel y DC: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/27 07:09:43 DEBUG : /PELICULAS/Marvel y DC: >Getattr: errc=0
2022/06/27 07:09:43 DEBUG : /PELICULAS/Marvel y DC: Opendir: 
2022/06/27 07:09:43 DEBUG : /PELICULAS/Marvel y DC: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2022/06/27 07:09:43 DEBUG : /PELICULAS/Marvel y DC: >OpenFile: fd=PELICULAS/Marvel y DC/ (r), err=<nil>
2022/06/27 07:09:43 DEBUG : /PELICULAS/Marvel y DC: >Opendir: errc=0, fh=0x2
2022/06/27 07:09:43 DEBUG : /PELICULAS/Marvel y DC: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/27 07:09:43 DEBUG : /PELICULAS/Marvel y DC: >Getattr: errc=0
2022/06/27 07:09:43 DEBUG : /PELICULAS/Marvel y DC: Releasedir: fh=0x2
2022/06/27 07:09:43 DEBUG : /PELICULAS/Marvel y DC: >Releasedir: errc=0
2022/06/27 07:09:43 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 11530240 length 1048576 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:43 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 5769302016 length 8192 chunkOffset 5769297920 chunkSize 134217728
2022/06/27 07:09:43 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 5769310208 length 16384 chunkOffset 5769297920 chunkSize 134217728
2022/06/27 07:09:43 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 12578816 length 1048576 chunkOffset 0 chunkSize 134217728
2022/06/27 07:09:43 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: ChunkedReader.Read at 5769326592 length 32768 chunkOffset 5769297920 chunkSize 134217728
2022/06/27 07:09:43 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv(0xc0006b53c0): >_readAt: n=30258, err=EOF
2022/06/27 07:09:43 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: >Read: n=30258
2022/06/27 07:09:43 DEBUG : /PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv: Read: ofst=2739044352, fh=0x1
2022/06/27 07:09:43 DEBUG : PELICULAS/Marvel y DC/Ant-Man (2015) [MicroHD 1080p][Dual].mkv(0xc0006b53c0): _readAt: size=32768, off=2739044352
2022/06/27 07:09:43 DEBUG : vfs cache: looking for range={Pos:2739044352 Size:32768} in [{Pos:0 Size:61440} {Pos:5769297920 Size:30258}] - present false

@Animosity022

While you have the mount going, recreate the issue in Plex and share the log of when you have the issue.

https://pastebin.com/TcgK58sC

I made a record of a small library with full cache and when scanning the library it caches all the movies that are on the disk, should this be the case?

EDIT

@asdffdsa

--drive-pacer-min-sleep 10ms --drive-pacer-burst 200

and it is very common to pre-load the vfs dir cache, can speed up plex scanning.
to the rclone mount command add the flags,
--rc --rc-addr :5572 --rc-no-auth

and after the mountpoint is live, run
rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5572

then run the plex scan

I've also tried this with a small library of 49 movies, and it took about 3 minutes.

[asdffdsa (jojothehumanmonkey)]

@RicarSeg

Good morning sir! it is not a joke

everything looks ok to me.

[RicarSeg (crazy apprentice)]

Millions of thanks!

I'll start everything from scratch so I can thoroughly test the various caching modes.

Thank you all for your time!

[VBB]

Let us know how it goes!