Slow scan plex tv show

Help and Support
1

Good morning to all! I apologize if my English is bad as I am using the translator

What is the problem you are having with rclone?Preformatted text

I have problems when I scan media in tv shows, despite having the content already in the cache it is very slow looking for new files

System Windows 10
2tb ssd cache disk
1Gbs conection
Intel core i5 10500
32GB ram ddr4

Run the command 'rclone version' and share the full output of the command.

rclone: Version "v1.58.1"

Which cloud storage system are you using? (eg Google Drive)

GDrive personal account
60TB tv show

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Paste command here
```starting with parameters ["rclone" "mount" "SERIES:" "G:" "--log-file" "C:\\rclone\\log.txt" "--log-level" "DEBUG" "--vfs-read-chunk-size" "1M" "--vfs-read-chunk-size-limit" "1G" "--vfs-cache-mode" "full" "--vfs-cache-max-size" "1500G" "--vfs-cache-max-age" "504h" "--tpslimit" "10" "--tpslimit-burst" "10" "--vfs-cache-poll-interval" "5m" "--cache-dir" "D:"]



#### The rclone config contents with secrets removed.  

[SERIES]
type = drive
client_id = 
client_secret = 
scope = drive
token = 
team_drive = 
root_folder_id =

Paste config here




#### A log from the command with the `-vv` flag

Paste log here

2022/06/26 12:26:55 DEBUG : rclone: Version "v1.58.1" starting with parameters ["rclone" "mount" "SERIES:" "G:" "--log-file" "C:\\rclone\\log.txt" "--log-level" "DEBUG" "--vfs-read-chunk-size" "1M" "--vfs-read-chunk-size-limit" "1G" "--vfs-cache-mode" "full" "--vfs-cache-max-size" "1500G" "--vfs-cache-max-age" "504h" "--tpslimit" "10" "--tpslimit-burst" "10" "--vfs-cache-poll-interval" "5m" "--cache-dir" "D:"]
2022/06/26 12:26:55 DEBUG : Creating backend with remote "SERIES:"
2022/06/26 12:26:55 DEBUG : Using config file from "C:\\rclone\\rclone.conf"
2022/06/26 12:26:55 DEBUG : vfs cache: root is "D:\\"
2022/06/26 12:26:55 DEBUG : vfs cache: data root is "\\\\?\\D:\\vfs\\SERIES"
2022/06/26 12:26:55 DEBUG : vfs cache: metadata root is "\\\\?\\D:\\vfsMeta\\SERIES"
2022/06/26 12:26:55 DEBUG : Creating backend with remote "D://vfs/SERIES/"
2022/06/26 12:26:55 DEBUG : fs cache: renaming cache item "D://vfs/SERIES/" to be canonical "//?/D://vfs/SERIES/"
2022/06/26 12:26:55 DEBUG : Creating backend with remote "D://vfsMeta/SERIES/"
2022/06/26 12:26:55 DEBUG : fs cache: renaming cache item "D://vfsMeta/SERIES/" to be canonical "//?/D://vfsMeta/SERIES/"
2022/06/26 12:27:52 DEBUG : Network mode mounting is disabled
2022/06/26 12:27:52 DEBUG : Mounting on "G:" ("SERIES")
2022/06/26 12:27:52 DEBUG : Google drive root '': Mounting with options: ["-o" "attr_timeout=1" "-o" "uid=-1" "-o" "gid=-1" "--FileSystemName=rclone" "-o" "volname=SERIES"]
2022/06/26 12:27:52 DEBUG : Google drive root '': Init: 
2022/06/26 12:27:52 DEBUG : Google drive root '': >Init: 
2022/06/26 12:27:52 DEBUG : /: Statfs: 
2022/06/26 12:27:52 DEBUG : SERIES: Loaded invalid token from config file - ignoring
2022/06/26 12:27:52 DEBUG : vfs cache RemoveNotInUse (maxAge=1814400000000000, emptyOnly=false): item Series/Series 7/House (2004)/7x15 - House.mkv not removed, freed 0 bytes
2022/06/26 12:27:52 DEBUG : vfs cache RemoveNotInUse (maxAge=1814400000000000, emptyOnly=false): item Series/Telenovelas/Love Is In The Air (2020)/Temporada 01/Love Is In The Air - s01e33.mp4 not removed, freed 0 bytes
2022/06/26 12:27:52 DEBUG : vfs cache RemoveNotInUse (maxAge=1814400000000000, emptyOnly=false): item Series/Series 1/A dos metros bajo tierra (2001)/Temporada 05/A dos metros bajo tierra - S05E03.mkv not removed, freed 0 bytes
2022/06/26 12:27:52 DEBUG : vfs cache RemoveNotInUse (maxAge=1814400000000000, emptyOnly=false): item Series/Series 1/CSI - Las Vegas (2000)/Temporada 12/CSI - Las Vegas - S12E07 - Cerebro Desconocido [720p h264].mkv not removed, freed 0 bytes
2022/06/26 12:27:52 DEBUG : vfs cache RemoveNotInUse (maxAge=1814400000000000, emptyOnly=false): item Series/Series 1/L.A.'s Finest - Policías de Los Ángeles (2019)/Temporada 01/L.A.'s Finest - Policías de Los Ángeles - S01E08.mkv not removed, freed 0 bytes
2022/06/26 12:27:52 DEBUG : vfs cache RemoveNotInUse (maxAge=1814400000000000, emptyOnly=false): item Series/Series 7/9-1-1 Lone Star/Temporada 03/9-1-1 Lone Star - S03E16.mkv not removed, freed 0 bytes
2022/06/26 12:27:52 DEBUG : vfs cache RemoveNotInUse (maxAge=1814400000000000, emptyOnly=false): item Series/Series 7/House (2004)/5x17 - House.mkv not removed, freed 0 bytes
2022/06/26 12:27:52 DEBUG : vfs cache RemoveNotInUse (maxAge=1814400000000000, emptyOnly=false): item Series/Series 1/Aida (2005)/Temporada 01/Aida - S01E11.mkv not removed, freed 0 bytes
2022/06/26 12:27:52 DEBUG : vfs cache RemoveNotInUse (maxAge=1814400000000000, emptyOnly=false): item Series/Series 1/Killing Eve (2018)/Temporada 01/Killing Eve - S01E02.mkv not removed, freed 0 bytes
2022/06/26 12:27:52 DEBUG : vfs cache RemoveNotInUse (maxAge=1814400000000000, emptyOnly=false): item Series/Series 7/El príncipe de Bel-Air (1990)/Temporada 04/El príncipe de Bel-Air (1990) - S04E12.mkv not removed, freed 0 bytes
2022/06/26 12:27:52 DEBUG : vfs cache RemoveNotInUse (maxAge=1814400000000000, emptyOnly=false): item Series/Series 7/La Fortuna (2021)/Season 01/La Fortuna - S01E06.mp4 not removed, freed 0 bytes
2022/06/26 12:27:52 DEBUG : vfs cache RemoveNotInUse (maxAge=1814400000000000, emptyOnly=false): item Series/Series 7/Traces (2019)/Temporada 01/Traces - S01E05.mkv not removed, freed 0 bytes
2022/06/26 12:27:52 DEBUG : vfs cache RemoveNotInUse (maxAge=1814400000000000, emptyOnly=false): item Series/Telenovelas/La Esclava Blanca (2016)/Temporada 01/La esclava blanca - S01E58.mp4 not removed, freed 0 bytes
2022/06/26 12:27:52 DEBUG : vfs cache RemoveNotInUse (maxAge=1814400000000000, emptyOnly=false): item Series/Series 1/Chicas buenas (2018)/Temporada 02/Chicas buenas - S02E10.mkv not removed, freed 0 bytes
2022/06/26 12:27:52 DEBUG : vfs cache RemoveNotInUse (maxAge=1814400000000000, emptyOnly=false): item Series/Series 7/Chicago Fire (2012)/Temporada 03/Chicago Fire - S03E06.mkv not removed, freed 0 bytes

2022/06/26 12:45:26 INFO  : vfs cache: cleaned: objects 45773 (was 45773) in use 0, to upload 0, uploading 0, total size 956.056Gi (was 956.056Gi)
2022/06/26 12:46:14 DEBUG : Google drive root '': Checking for changes on remote
2022/06/26 12:47:14 DEBUG : Google drive root '': Checking for changes on remote
2022/06/26 12:47:46 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:46 DEBUG : /: >Getattr: errc=0
2022/06/26 12:47:46 DEBUG : /: Opendir: 
2022/06/26 12:47:46 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2022/06/26 12:47:46 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2022/06/26 12:47:46 DEBUG : /: >Opendir: errc=0, fh=0x0
2022/06/26 12:47:46 DEBUG : /Desktop.ini: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:46 DEBUG : : Re-reading directory (12m17.4462555s old)
2022/06/26 12:47:46 DEBUG : /Desktop.ini: >Getattr: errc=-2
2022/06/26 12:47:46 DEBUG : /AutoRun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:46 DEBUG : /AutoRun.inf: >Getattr: errc=-2
2022/06/26 12:47:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:47 DEBUG : /: >Getattr: errc=0
2022/06/26 12:47:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:47 DEBUG : /: >Getattr: errc=0
2022/06/26 12:47:47 DEBUG : /: Opendir: 
2022/06/26 12:47:47 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2022/06/26 12:47:47 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2022/06/26 12:47:47 DEBUG : /: >Opendir: errc=0, fh=0x1
2022/06/26 12:47:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:47 DEBUG : /: >Getattr: errc=0
2022/06/26 12:47:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:47 DEBUG : /: >Getattr: errc=0
2022/06/26 12:47:47 DEBUG : /: Releasedir: fh=0x1
2022/06/26 12:47:47 DEBUG : /: >Releasedir: errc=0
2022/06/26 12:47:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:47 DEBUG : /: >Getattr: errc=0
2022/06/26 12:47:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:47 DEBUG : /: >Getattr: errc=0
2022/06/26 12:47:47 DEBUG : /: Opendir: 
2022/06/26 12:47:47 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2022/06/26 12:47:47 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2022/06/26 12:47:47 DEBUG : /: >Opendir: errc=0, fh=0x1
2022/06/26 12:47:47 DEBUG : /: Statfs: 
2022/06/26 12:47:47 DEBUG : Google drive root '': read info from Shared Drive "Series"
2022/06/26 12:47:47 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2022/06/26 12:47:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:47 DEBUG : /: >Getattr: errc=0
2022/06/26 12:47:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:47 DEBUG : /: >Getattr: errc=0
2022/06/26 12:47:47 DEBUG : /: Opendir: 
2022/06/26 12:47:47 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2022/06/26 12:47:47 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2022/06/26 12:47:47 DEBUG : /: >Opendir: errc=0, fh=0x2
2022/06/26 12:47:47 DEBUG : /: Releasedir: fh=0x1
2022/06/26 12:47:47 DEBUG : /: >Releasedir: errc=0
2022/06/26 12:47:47 DEBUG : /: Releasedir: fh=0x2
2022/06/26 12:47:47 DEBUG : /: >Releasedir: errc=0
2022/06/26 12:47:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:47 DEBUG : /: >Getattr: errc=0
2022/06/26 12:47:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:47 DEBUG : /: >Getattr: errc=0
2022/06/26 12:47:47 DEBUG : /: Opendir: 
2022/06/26 12:47:47 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2022/06/26 12:47:47 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2022/06/26 12:47:47 DEBUG : /: >Opendir: errc=0, fh=0x1
2022/06/26 12:47:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:47 DEBUG : /: >Getattr: errc=0
2022/06/26 12:47:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:47 DEBUG : /: >Getattr: errc=0
2022/06/26 12:47:47 DEBUG : /: Opendir: 
2022/06/26 12:47:47 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2022/06/26 12:47:47 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2022/06/26 12:47:47 DEBUG : /: >Opendir: errc=0, fh=0x2
2022/06/26 12:47:47 DEBUG : /: Releasedir: fh=0x1
2022/06/26 12:47:47 DEBUG : /: >Releasedir: errc=0
2022/06/26 12:47:47 DEBUG : /: Releasedir: fh=0x2
2022/06/26 12:47:47 DEBUG : /: >Releasedir: errc=0
2022/06/26 12:47:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:47 DEBUG : /: >Getattr: errc=0
2022/06/26 12:47:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:47 DEBUG : /: >Getattr: errc=0
2022/06/26 12:47:47 DEBUG : /: Opendir: 
2022/06/26 12:47:47 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2022/06/26 12:47:47 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2022/06/26 12:47:47 DEBUG : /: >Opendir: errc=0, fh=0x1
2022/06/26 12:47:47 DEBUG : /: Releasedir: fh=0x1
2022/06/26 12:47:47 DEBUG : /: >Releasedir: errc=0
2022/06/26 12:47:55 DEBUG : /Desktop.ini: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:55 DEBUG : /Desktop.ini: >Getattr: errc=-2
2022/06/26 12:47:55 DEBUG : /AutoRun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:55 DEBUG : /AutoRun.inf: >Getattr: errc=-2
2022/06/26 12:47:56 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:56 DEBUG : /: >Getattr: errc=0
2022/06/26 12:47:56 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:56 DEBUG : /: >Getattr: errc=0
2022/06/26 12:47:56 DEBUG : /: Opendir: 
2022/06/26 12:47:56 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2022/06/26 12:47:56 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2022/06/26 12:47:56 DEBUG : /: >Opendir: errc=0, fh=0x1
2022/06/26 12:47:56 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:56 DEBUG : /: >Getattr: errc=0
2022/06/26 12:47:56 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/06/26 12:47:56 DEBUG : /: >Getattr: errc=0
2022/06/26 12:47:56 DEBUG : /: Releasedir: fh=0x1
2022/06/26 12:47:56 DEBUG : /: >Releasedir: errc=0
2022/06/26 12:48:14 DEBUG : Google drive root '': Checking for changes on remote""
2

Did you run a scan for the log you shared? There' s really no activity at all in that log.

3

Thanks for your quick response!

I've only been working with rclone for 3 days and still haven't figured out a few things.

How can I run the analysis?

4

While you have the mount going, recreate the issue in Plex and share the log of when you have the issue.

5

Thank you sir!

I'll take a while, I'll edit the message as soon as I have the analysis

EDIT:

Sorry sir, I think the log was too big to copy the analysis

I have uploaded the file here

6

That site you linked to isn't something I can download.

image
image725×515 68.6 KB

7

Sorry!! I'll find another way to expose the log, thanks you again!

8

hi,

gdrive is 60TB and the vfs file cache is limited to 1.5TB,
so trying to understand your statement?

your scan is for small subset of the 60TB?

9

Hello! First of all, thanks for your help!

I really didn't think that was a problem. I've been reading quite a lot to find information before having to open a new topic and from what I could see there are users who have up to 220TB in Drive and limit the cache, I don't know if this is possible

Would it be better not to limit it?

10

with plex, it assumes a fast local file system.
so by default, for each media file, plex will generate thumbnails, previews so plex might scan most/all of the 60TB.

so for plex to scan a media file, it will force rclone to download most/all of the media file into the rclone vfs file cache, and that is why the scan would be slow.

also, gdrive has a hard limit for downloads per day 10 TiB, you might be hitting that limit
https://rclone.org/drive/#drive-stop-on-download-limit

as per
https://github.com/animosity22/homescripts#plex-tweaks

"Enable Thumbnail previews - off: This creates a full read of the file to generate the preview and is set per library that is setup
Perform extensive media analysis during maintenance - off: This is listed under Scheduled Tasks and does a full download of files and is ony used for bandwidth analysis when streaming."

1 Like
11

Thank you again!

I have always had the configurations you mention disabled

This would be just for the first scan, right?

Hopefully, so far I haven't had this problem.

I have TV shows and movies in two different accounts

EDIT:
The problem with all this is that I can't understand why it does a slow scan despite already having the content cached.

I have the option "Perform a partial scan when changes are detected".

"Theoretically" it should be faster. I will try to upload the log record as soon as I can

12

well, there is always a hard limit to the size of the vfs file cache.
in your case, using a 2TiB drive and 1.5TiB for the vfs file cache.

and might want to tweak your command, with these flags.
https://github.com/animosity22/homescripts/blob/master/systemd/rclone-drive.service

--drive-pacer-min-sleep 10ms --drive-pacer-burst 200

and it is very common to pre-load the vfs dir cache, can speed up plex scanning.
to the rclone mount command add the flags,
--rc --rc-addr :5572 --rc-no-auth

and after the mountpoint is live, run
rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5572

then run the plex scan

1 Like
13

Hi again!

Do I need to apply this flag in the rclone.conf file?

I hope you'll excuse me, I'm still learning about rlcone and I can't figure out how to edit this file, the little information that I find is in English and when translated into Spanish the context is a bit difficult to understand

Should I do this every time I need to do a full or partial scan?

14

--drive-pacer-min-sleep 10ms --drive-pacer-burst 200, add to the rclone mount comand.
no need to modify rclone.conf

rclone.conf is a text file.
can be found using rclone config file

each time you run rclone mount, need to run rclone rc vfs/refresh once.

from the link i shared above, might add this flags to your command.
https://github.com/animosity22/homescripts/blob/master/systemd/rclone-drive.service

# Google Drive is a polling remote so this value can be set very high and any changes are detected via polling.
--dir-cache-time 5000h
# I reduce the poll interval down to 10 seconds as this makes changes appear fast the API quotas per day are huge
--poll-interval 10s
1 Like
15

@VBB, hi,
before a plex scan, do you run rclone rc vfs/refresh or do you rely on polling?
what is your current mount command?

16

My Plex mount is read-only with polling disabled and no cache (as you know). Before my daily scan, I always run rclone rc vfs/refresh recursive=true --drive-pacer-burst 200 --drive-pacer-min-sleep 10ms --timeout 30m --user-agent *******

My current mount command is rclone mount --attr-timeout 5000h --dir-cache-time 5000h --drive-pacer-burst 200 --drive-pacer-min-sleep 10ms --no-checksum --poll-interval 0 --rc --read-only --user-agent ******* --vfs-read-chunk-size 1M -v

@RicarSeg If this is your initial Plex scan, things will be slow. That's normal. Make sure you use the Plex Series Agent, not the legacy one. Triple check that all the extra stuff is turned off (intro detection, thumbnails, etc.).

You also never said what exactly you mean by "slow". Maybe you can expand on that a little, so we have an idea.

EDIT: To give you an idea of how fast a scan can be, it takes about five minutes for me to scan my entire library of about 950 TB.

2 Likes
17

No sir, I have not used this flag before

This was my old setup

This would be my new configuration adding the commands that you have recommended, now i just need to try it

I still don't know for sure if the order of the commands negatively affects

18

do you mean command or flags?
--log-file is a flag, not a command.

the order of the flags does not matter.

and not sure you need this
--tpslimit 10 --tpslimit-burst 10

1 Like
19

EDIT!

Buenas tardes! Thank you for joining the help.

Of course this is incredibly fast! It is a significant amount of data! After reading this I'm pretty sure I'm doing something wrong xD

EDIT!

Does it mean that the scans are done manually? At the three dots "Search for files in the library"?

**I meant by this, that the scan of the libraries is not scheduled from time to time, since you must execute these flags before

To get a little idea of ​​how long it takes to scan, This library alone takes me about almost ten minutes to scan, I'm not sure it's something very common when I only have to do the partial scan, right?

image
image471×166 4.87 KB

I use this Plex agent fortv show and all unnecessary things are disabled sir. Sorry for the delay, I have to translate everything little by little

20

Yes, I do everything manually, including a Plex scan once a day.

This should take a couple of seconds. Plex's partial scan does not work on anything other than local drives, in my experience. You want to turn that off and try something like GitHub - l3uddz/plex_autoscan: Script to assist sonarr/radarr with plex imports. Will only scan the folder that has been imported, instead of the whole library section.

1 Like