Plex scan of TV library running really long

Hey, you have cache-db-purge in your mount. that will purge the dir cache everytime you mount. You do not want that.

Yeah… I already removed that.

rclone mount gcrypt: /mnt/disks/media --allow-other --dir-cache-time=48h --cache-info-age=72h --cache-chunk-path=/mnt/user/rc_chunks --cache-db-path=/mnt/user/rc_db --cache-workers=7 --cache-tmp-upload-path=/mnt/user/rclone_upload --cache-tmp-wait-time 60m --buffer-size 0M --attr-timeout=1s --rc --log-level DEBUG --log-file=/mnt/user/data/rclog/rclonecache.log &

thats my mount.

–dir-cache-time is the length to cache dir entries. I setup a cron to do the recursive ls of my media dir every 2 days

With those settings, will rclone still pick up NEW files added from outside of the cache? Or will it take 48 hours for them to show up in rclone? And what about file changes?

finding a new file thats not in the dir cache takes about 5-7 seconds. Im starting to populate my media now that im done testing so im running the ls more often on the new server. It will find the new files it just takes a little longer to find the dir entry.

I noticed that you’re on an older version of the script. Thought I’d link you to the original, it has an added “Time to watch” calculation. Would be cool to see what yours ends up being.
https://github.com/ajkis/scripts/blob/master/plex/plex-library-stats.sh

Ha, nice. Here is my 2 big sections:

Library = Movies
Minutes = 187429
  Hours = 3123
   Days = 130

Library = TV Shows
Minutes = 753709
  Hours = 12561
   Days = 523

Wow that’s long.

Mine is nothing compared to yours. Currently at 16TB.

Library = Movies
Minutes = 31417
  Hours = 523
   Days = 21

Library = TV Shows
Minutes = 72643
  Hours = 1210
   Days = 50

Yeah, my entire ls on the mount happens in subsecond with everything cached:

[felix@gemini gmedia]$ time ls -alR | wc -l
32348

real 0m0.826s
user 0m0.061s
sys 0m0.147s

Good info, guys…

My ls timing on 1st run:
$ time ls -alR | wc -l
266990

real 7m30.164s
user 0m3.860s
sys 0m5.020s

Then second pass:
$ time ls -alR | wc -l
266990

real 0m31.662s
user 0m3.770s
sys 0m4.720s

Unfortunately, the scan is still taking a very long time though… :frowning:

I have been monitoring the scanner log. It’s interesting… It seems like Plex is taking multiple passes through the directories. The first time through, the “Taking directory…” log entries have a “# left” which decreases with every directory it scans. I thought the scan would be done when it reached zero left. However, it seems to be going through everything again. I see this now:

  • Taking directory ‘Doctor Who/Season 10’ (5956) out of the map (0 left)

Any thoughts on how the scanner works? I should try this again with PlexDrive and see what happens…

If you grab the plex-analyze script from the link up a few posts, what is the output at the bottom?

21320 files in library
0 files missing analyzation info

I’m interested in seeing the missing analyzation info.

19.04.2018 18:32:20 PLEX LIBRARY STATS
Media items in Libraries
Library = Movies
Items = 6544

Library = TV Shows
Items = 35857

Time to watch
Library = Movies
Minutes = 686818
Hours = 11446
Days = 476

Library = TV Shows
Minutes = 1226768
Hours = 20446
Days = 851

78094 files in library
1645 files missing analyzation info
1 media_parts marked as deleted
1 metadata_items marked as deleted
0 directories marked as deleted
44095 files missing deep analyzation info.

You have a lot of analysis left to do still.

Use the plex-analyze-cli.py script there as well. That’ll take some time and let that run to analyze the rest of your library. Once that # hits 0, scans should be quick.

my stats ( but i removed time to watch from my own script :stuck_out_tongue: )
20.04.2018 16:03:14 PLEX LIBRARY STATS
Media items in Libraries
Library = .ReadMe
Items = 1

Library = 4K Movies (test)
  Items = 40

Library = Movies
  Items = 6931

Library = TV Shows
  Items = 31283

38274 files in library

Close to 130 TB library atm.

2 Likes

I still having very long running TV scans in Plex. I have run the analyze script several times. Interestingly, I must have some strange files in my library. Plex plays them fine, but it doesn’t show the bitrate metadata. Maybe they are variable bitrate? So, the analyze script includes them every time it runs since it’s selection criteria is the lack of bitrate metadata in the Plex database. If you are interested, here is the mediainfo output for one of them:
Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4
Format settings, CABAC : Yes
Format settings, ReFrames : 15 frames
Codec ID : V_MPEG4/ISO/AVC
Width : 720 pixels
Height : 576 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 25.000 fps
Standard : PAL
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Writing library : x264 core 148 r2643 5c65704
Encoding settings : cabac=1 / ref=15 / deblock=1:0:0 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-2 / threads=9 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=8 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Default : Yes
Forced : No

So, I’m not much better off than I was before. I’m also not totally convinced that the lack of fully analyzed files is what is slowing down the scan. I only have 150 or so files that don’t have the bitrate metadata.

What’s your library stats now? If you are having problems with certain files, you might need to do the plex dance (move a file out, scan library, delete, re-add the file, scan). I’ve seen before where the metadata for a particular file doesn’t quite work until the dance is done.