I'm currently using Animosity's VFS setup and I'm at the step of adding the MergerFS mount as a folder for the library in addition to my local directory. I'm going to remove the local directory after the initial scan is done however that initial scan is taking forever. I added the directory for my smallest library first and it has been scanning for two hours and is still no where near complete for a 42 show library with ~4000 files. Should it be taking this long? Is there a way to speed it up? I have multiple much larger libraries left to add the MergerFS directory for before removing the local NAS directory, at this point it seems like it will take weeks. My total library is about 35TB.
If it is indeed going to take this long is it safe to add the new directory for all my libraries now and just let it scan or will I start hitting API limits? I'm using my own client ID to connect to my Google Drive.
Empty Trash automatically is currently off.
I have Plex Autoscan setup for subsequent updates from Sonarr/Radarr.
I find that Plex adds files a little slower than Emby as I run them side by side.
Depending on the file / complexity of it, I usually see ~3-10 seconds per file that it adds in.
If you check the Plex logs, you can see what it's doing. If it's a file that was already there before and you changed the path, you would see that in the Plex Scanner Log.
Are these all brand new files being added to the library or did you add a new path to the library ? The flow does matter a bit as that defines the path that plex takes.
If you have a new file, it analyzes the file for the mediainfo. If you add a path to the library and move a file, it should see the same file and update the path.
I added a new path to the library with a copy of the all the existing files.
example, for over a year my Plex library has had an Anime library with
/nfs/Media/Anime
After having copied all of the media from my NAS to my Google Drive I edited the library and added the folder
/mnt/media/Anime
They are currently one for one copies of each other but the /mnt/ directory is a MergerFS mount of a scratch directory on my NAS to write to first for Sonarr to download to and an encrypted rclone mount to my Google Drive.
This kicked off a scan and once complete I would remove
/nfs/Media/Anime
from the library. This is how I migrated my 4K libraries to my Google Drive however there are far fewer files in those so when it ran the initial scan after adding the new path it was much quicker so I did not anticipate this issue. Should I have done it a different way? Am I just going to have to wait it out at this point? I’m assuming the scan after I remove the /nfs/ directories will be much faster? I have Plex Autoscan setup for new media scans from Sonarr so I believe those will be much faster as well.
So there were 2 copies of the files under the Library at one time?
I think that makes it analyze the second file so it would take longer.
I recall when I did that, I added the path and removed it from the original path. You can do that logically or physically like by unmounting the mount point or something rather than deleting it.
If you do that and the file size/timestamps are the same, you should see it rename the file part to the new location in the Plex logs, which should be faster.
I tried unmounting the NFS share and then adding the MergerFS mount to the library and it was taking just as long. I think the timestamps are different on the files uploaded to the Google Drive. I've remounted the NFS share for the time being so that users can still watch stuff and disabled plexupdate and unattended upgrades as one of them caused the scan to stop this morning. I guess I'll just let it run for a few days. Thank you for the help and all the scripts and instructions on getting this working.
An update to this. When trying to add the second directory I ran into a couple of problems. Not to mention how long the scan was taking for a large library.
One problem I had was the scan and/or Plex kept crashing in various ways. One time Plex crashed altogether and had to be manually restarted. Another time the scanner crashed and showed it was scanning but tailing the "Plex Media Scanner.log" for over an hour showed nothing was being scanned. Another issue with the scan process one time was the metadata agents were failing, so the media was being scanned and added to the library but every new file was failing to be matched by the agent.
The other main problem I had was at some point the agent on my movie libraries was changed. So when the scan would detect the new duplicate file on the rclone mount, if the file already in the library was matched using the agent that the library was now currently not set to it would not automatically merge the two files as a duplicate and show the 2 in the corner, I just had two separate entries in the library.
After 4-5 days of fighting these issues with trying to scan and manually merge items I removed the rclone mounts from my libraries and put my library back to how it was before attempting to move to the rclone mount. I then backed up the database, copied it from my headless Linux server to a windows machine, and backed up two more times for good measure. I then opened it up using DB Browser for SQ Lite and began running the SQL queries mentioned in this post.
They are as follows
UPDATE section_locations SET root_path= replace(root_path, ‘Old/Directory’, ‘New/Directory’) where root_path like ‘%Old/Directory’%’
In my case old directory was "/nfs/Media" as that is where my NAS share was mounted, and new directory for the rclone mount was "/mnt/media". You will need to adjust accordingly if you choose to follow this method. That should update the folder monitored on all of your libraries (if you edit library and go to add folders, the directory shown here)
Next was
UPDATE metadata_items SET guid= replace(guid, ‘file:///Old/Directory’, ‘file:///New/Directory’) where guid like ‘%file:///Old/Directory%’
In my case this updated 0 items. According to the post on the Plex forums this will only modify items if you have personal media that is not matched with an agent, or if you have unmatched items in your library.
Next
UPDATE media_streams SET url= replace(url, ‘file:///Old/Directory’, ‘file:///New/Directory’) where url like ‘%file:///Old/Directory%’
Not exactly sure what this updated, but it's a field in the DB that relates to the file path of the media.
Finally,
UPDATE media_parts SET file= replace(file, ‘Old/Directory’, ‘New/Directory’) where file like ‘%Old/Directory%’
This updates the actual location of the media file that Plex looks to for the library item. If you choose an item in your library and "Get Info" this is the path displayed there.
Once those 4 changes are done write the changes to the database, copy it back over to your server and since I'm running Plex on a Linux server I had to use "sudo chown plex:plex DBfilenamehere" to give ownership of the file back to the Plex user. I started Plex back up and chose scan libraries and in about 10 minutes it was done with a scan and everything was being read from the rclone mount.
This however did not fix my problem with the Metadata agents. Wanting to get everything back to using the Plex Movie agent instead of TheMovieDB agent I backed up the database again post file changes, and opened it back up in DB Browser and used this post
UPDATE metadata_items SET guid = '' where library_section_id = x and guid like 'com.plexapp.agents.themoviedb%';
You will need to replace the "x" for where library_section_id = to the number for your library section. If you choose browse data with the DB opened you can choose the section_locations table and it will list all of your libraries along with their section ID number. I did this on multiple library sections for probably about ~600 movies total. Wrote the changes, copied the database back to the server and changed ownership back to the Plex user. I then started with the smallest movie library first and chose "refresh all metadata" This forces it to rematch any item that it now sees as "unmatched" because of the database change and only refreshes the metadata for the items that we did not remove the matched agent for with the SQL statement.
If you can you will want to scroll with it as it matches these items, I had ~10 items that it failed to match automatically with the Plex agent. Also any metadata items you had locked will not be affected, and from what I can tell watched status was not affected either. I did discover that if you manually choose a poster on an item Plex then denotes that field as "locked" so I had a number of items that when the refresh occurred the had a blank poster. I had to edit those items again and manually choose the poster.
So all of my media has now been moved and is once again matched with the same agent. I felt this was a far easier way to move the media locations then the Plex recommended, add the duplicate directory, wait for it to scan and merge all the duplicates, then remove the old directory and scan again. Hopefully this helps someone in the future.
*I'm not responsible if you don't backup your database and you accidentally corrupt and/or destroy it somehow