Recommended Dropbox (Formally Google Drive) and Plex Mount Settings

Howto Guides
1064

@Animosity022

i had few questions in regards to your rclone mount settings
i am intending to host the emby/plex server on a vps but the existing storage in vps is quite low . its around 100 gb .i cant really upgrade anytime soon , additonal storage is quite costly
i think considering my huge libraby , the existing space isnt really enough to store the metadata locally , so i had few doubts

  1. is it a good idea to mount 2 remotes together for plex/emby streaming and use one for the plex/emby library and other for storing metadata . i would change the default metadata location to the secondary mount . Will this affect streaming performance in any way if i store metadata and cache off rclone mounts apart from the primary one for library

  2. Secondly what settings would you recommend for mounting remotes for metadata and cache , should i use exactly the same mount settings from your github which i am using for library or you would recommend a different settings for metadata and cache

PS / if you think mounting all these off single google account, it can hit api limit or affect performance due to some google limitations. i can also mount the remotes for metadata and cache off different gsuite accounts , online storage i do have in plenty . Waiting for your suggestions

1065

What is the actual size of your library as huge doesn't mean much.

I wouldn't use cloud storage for metadata as it's small and makes thing slow. Depending on the size of your library, that could generally have things be pretty slow in loading and general use. 4K Movies and 41K TV episodes is about 11GB of local storage for me and I'm about 120TB of storage online.

Those files are small and I've never done any testing with lots of small files. I tend to start with defaults and go from there. In the end, you'd have most of that data stuck in the cache anyway as it's referenced a lot so not sure trying to offload it to the cloud with vfs-cache-mode full would be a great bang for the buck anyway. You'd have to test and see how things work out and what the local storage looks like.

1066

its around 300 TB , i also have songs and all among other things . Maybe i am mistaken , i was told by someone metadata would reach 100 gigs considering the size of my library

thanks for clarifying , if it consumers the whole storage in vps , i would try experimenting with storing metadata in gdrive and play around with flags to see if i can make its speed usable enough ( if not equivalent to local metadata speeds ) . i think cache size would be even lower than metadata so i do better leave that one in vps itself

1067

@Animosity022 i tried using your rclone mount settings on my vps and the vps disk storage of 40gb got over suddenly even before the emby library could complete its scan . After searching a while , i realised that your hardware is high end and its not really a basic vps low on storage like mine so i thought i should probably make some changes . Correct me if i am wrong but with my little knowledge on mount , i understand that vfs cache mode full refers to - a remote folder is mounted on a local folder, then each time a file is loaded from the remote folder, it is first saved to a cache folder, and then forwarded to whatever program opened the remote file .
The vps specs are 40 gb disk space , 2 vcpu , 4 gb ram , 500 mega bits upload speed . I have come up with the following settings. Does it look good enough to you or you would recommend any further change ?

ExecStart=/usr/bin/rclone mount test: /GD \
--log-file /home/user/rclone/rclone.log \
--log-level INFO \
--umask 000 \
--poll-interval 15s
--allow-other \
--fast-list \
--transfers 24 \
--dir-cache-time 72h \
--drive-chunk-size=32M \
--fuse-flag direct_io \
--cache-chunk-total-size 8G \
--cache-chunk-size 8M \
--cache-chunk-no-memory \
--cache-workers=24 \
--buffer-size=32M \
--vfs-cache-mode minimal \
--vfs-read-chunk-size 32M \
--vfs-cache-max-age 1h \
--vfs-read-chunk-size-limit off \
--cache-tmp-upload-path=/home/user/rclone \
--config /home/user/.config/rclone/rclone.conf \
ExecStop=/bin/fusermount -uz /GD
Restart=on-abort
1068

That has is nothing near my settings. If you have a question on my settings and why I use something or what it does, please post here.

If you have a different question, please make a new thread as you have old cache backend settings in there.

1 Like
1069

Can you atleast clear the doubt on vfs cache mode on why you use full mode in your settings and what should be the preferred mode for vps users

1070

I don't use a VPS so you'd have to adjust that based on what the VPS settings are based on your disk space or if not disk space perhaps look for another VPS.

# This is used for caching files to local disk for streaming
--vfs-cache-mode full \

I use that to keep a local copy of files cached on disk for streaming as the change was with the latest release as they changed how cache-mode works and it works similar to the cache backend and works very well if streaming is your use case.

1 Like
1071

First of all, thanks a lot for sharing your work.
I'm trying to move on the cloud / gsuite with unlimited but very aware of name obfuscation and data encryption. I'm looking to animosity's scripts and was wondering what kind of encryption did you use? Is it rclone on the fly encryption or whatever else?
Thanks for any advices

1072

I use encryption in rclone which basically sits on top of the original remote I create and that scrambles the file and directory names along with encrypting the files. This all happens on the fly so if someone looks, they see gibberish but they do see file sizes. It's a very, very low CPU hit for encryption.

1073

Thanks a lot for answering my newbie question ! I've so set up an encrypted drive, automounted on macos and that's satisfying !

Now as I want a full plex configuration, I'll have a deep look in all your scripts

Kind Regards

1 Like
1074

I put my chunk of media up into GDrive, and I tried adding it, but it seems like the first thing that's happening is everything is getting pulled back down, 1 by 1, I'm guessing as part of a media scan. I thought I'd turned most of that off, is that something you just have to live with on the initial add/re-add of content?

In Plex, I've turned off both the media analysis options in the scheduled tasks, I've disabled any generation of anything settings in the library settings. It's kinda a bummer though, as I do like some of those settings. Do you have them all disabled, @Animosity022?

Thanks!

1075

When you add in any library in plex/emby/etc, it does a ffprobe/mediainfo on the file to get the information so it doesn't pull down the whole file, but a few parts of it along the way to build the metadata on the items you are adding to the library. It's best to just let it go and finish as you either pay now or pay later as it has to be done.

Yes, what settings in particular are you talking about? I have extended analysis off as it does full bitrate analysis for a file and is only used for bandwidth limiting, which I do not use. Most of the other stuff can probably be turned on these if you really wanted to but it would download a bit of data from each file to build that information up.

1076

That's how I'd hoped it would work, but from the amount of network and disk traffic I'm seeing when it happens, and rclone reporting the size change in the cache, it sure doesn't look like it's doing an ffprobe on just the header portion of the file.

Settings wise, I was referring to the settings under Library that begin with "Generate" and cover:

  • video preview thumbnails
  • intro video markers
  • chapter thumbnails
    and also "Analyze audio tracks for loudness".

I don't mind letting it run if it comes down to it, just the network and disk cache hit was significant (which I found odd) to the point it was consuming so much time on my disk, Plex started having trouble reaching the database file.

The console scroll is showing a "Multi-thread Copied (new)" for every single file in the mounted folder, which I assumed meant it was pulling the entire file, especially from the amount of space consumer and the amount of transfer that was happening, but I guess I could be wrong there. I'll have to look inside the cache more and see if they seem like they are just partials I guess.
Thanks!

1077

That I have off as it generates a lot of disk space and not really useful imo.

I do use that as it's awesome. Took like 2 weeks to analyze my whole library as I let it do 3-5 hours a night when it launched.

That's for music which I have none of so that's off for me.

They are sparse files so if the OS supports it, it may look like the full file, but it really isn't.

1078

Well, I'd agree with you, but I was watching network traffic and eyeballing the time, it certainly had enough time to pull the file down. If it was a thing where it was just a burst of traffic and done, but it was pulling traffic at 20MB/sec for 15 straight minutes before I killed it. So it seems like it was more than just the first 100MB of each file.

I'll keep tweaking settings, I'm just seeing a much larger amount of traffic when adding a remote mounted library than I thought I would(outside actually streaming the file) so I was trying to figure out why that was happening.

Thanks!

1079

It would depend on what settings you had on. With 100% certainty, the ffprobe/mediainfo only pulls a bit of the file. The other things do grab more of the file depending on what setting you are talking about.

1080

Heh! Well that's the crux of my question. Trying to get those settings figured out/documented so I can know what things will trigger it, and have a better expectation of when that will happen.

Your earlier comment might be something I have to adjust to, maybe move to a situation where I'm more letting overnight processing handle things than the on-demand/as soon as added do things.

1081

Adding a file and having it analysis the file only runs a ffprobe/mediainfo on the file so it's small bit of the file.

Intros only scan the first 5-10 minutes of a TV show.

The thumbnails and and deep analysis for bitrates read the entire file to generate that information.

1082

I think this might got expanded in Plex Media Server 1.19.5.3021:

  • (Intro Detection) Find any intros in the first half of an episode
1083

Most intro's are in the beginning so it scans and finds the intro and stops. That's saying if an intro happens to be up to 50% of the episode, it'll continue to search.

Majority of shows I watch the intros are in the first 5-10 minutes.