Best mount settings for streaming ( Plex )


#83

@Ajki

I understand, still on average you are using a reasonable amount of data.

The 24 hour ban is annoying, and difficult to avoid when you have a large library that’s actively growing. I am currently using node-gdrive-fuse to mount Google Drive instead of rclone, as node-gdrive-fuse supports caching the entire drive structure. It’s performance is not superior to rclone, and it has some critical bugs, but I haven’t been banned for 24 hours since switching. I’ve also played around with the idea if using unionfs for two separate mounts, but it has the usual mod time inconsistency issue, so it’s more of a headache.


#84

Hi,

Just wondering what you are using to generate your stats?

Also with regards to unionfs, could you not just layer Google Drive under ACD?

My current unionfs is Layer 1 - Local Data, Layer 2 - ACD. I have allot of the same data in both, but as Layer 1 takes precedence it will play from my Local Data if it exists. The only question is how unionfs would handle the layer if there was a fault with it, rather than just a non existent file. I would just give it a try :slight_smile:

Wob


#85

I’ll test using unionfs like that. Right now I’m banned from Google Drive so if I mount Google Drive on top of ACD it should fall back to ACD since I already know GDrive won’t stream.


#86

Results of using unionfs:

Note, I am 24h banned from Google Drive atm.

DID NOT WORK - unionfs-fuse -o cow,allow_other /path/to/gdrive/=RO:/path/to/acd/=RO /path/to/dir/

DID WORK - unionfs-fuse -o cow,allow_other /path/to/acd/=RO:/path/to/gdrive/=RO /path/to/dir2/

Looks like Plex is only using the first directory, otherwise they should both stream.


#87

Interesting,

I have Local over ACD and it can definitely play from ACD (Layer 2). Out of interest, when you access /path/to/grdive/ outside of the union fs, what happens, is it blank or do you get an error?


#88

I can cd into the mount and view all of its files, so it is mounted properly at least.


#89

What happens when you attempt to access a file? I would have thought you would not even see files if banned? unionfs is probably trying to play the files, since they exist. I wonder if ACD does the same during a ban. Looks like a check of some sort will be needed.


#90

@jpat0000 care sharing how to install node-drive-fuse

I haven’t been able to make it work lately keep getting errors and it used to work fine

Hold me over till we can cache in rclone


#91

google will show you files during ban but will error if you try and play


#92

@MartinBowling

You’ll have to share what errors you’re getting. A couple problematic bugs with node-gdrive-fuse:

mod time is invalid for files, leads to Plex database corruption/dashboard breaking.

Writing the mount results in the same file appearing many times in the mount, doesn't play well with applications like Sonarr

No cache data cleanup of any kind...

https://github.com/thejinx0r/node-gdrive-fuse/pull/79/commits/3d755b9bfb704de4c5aebb97703a8d5ad964fa4b

Simply make the changes to the two files manually to fix the mod time problem.

For writing I use unionfs-fuse. This is a little tricky actually, because even though you can specify the local disk as the read/write priority, it won’t write to the local disk unless the folder it’s writing to exists already, or the less prioritized mount doesn’t have write enabled. I use automatic download handling software like Sonarr and I want it to be able to delete upgraded copies of my media.

Make local folder for temporary write:

/home/user/tmpwrite

Clone node-gdrive-folder structure:

cp -R --attributes-only /home/user/nodegdrivemount/MYMEDIA/Shows /home/user/tmpwrite/MYMEDIA/

cd /home/user/tmpwrite/MYMEDIA/Shows
find . -type f -size -1M -exec rm {} +
(Do this every once in a while)

Merge em together:

unionfs-fuse -o cow,allow_other /home/user/tmpwrite=RW:/home/user/nodegdrivemount=RW /home/user/unionfsmount

Move your temporarily written files to Google Drive at your leisure:

rclone --no-check-certificate --no-traverse --drive-chunk-size=64M --transfers=6 move /home/user/tmpwrite google:MYMEDIA/Shows --exclude .@**

Finally, the cache problem. I recommend setting the cacheLocation to /dev/shm/nodegdrivecache (your RAM).

Create bash script: cleanup.sh
find /dev/shm/gdrivecache/download -maxdepth 1 -mmin +1 -exec rm -f {} \;

Create a screen to run the script:

screen -S cleanup

Execute every 60 seconds:

watch -n 60 bash cleanup.sh

Adding new stuff to Google Drive is fine with node-gdrive-fuse but moving files around will result in you needing to rebuild the database, simply delete it from /dev/shm (or wherever) and start your mount again if needed.


Google Drive vs. ACD for Plex
Gdrive ban question
Google Drive vs. ACD for Plex
#93

@jpat0000 thanks for the tips! I should have been more specific.

I was getting harmony errors when running it so I
Hadn’t even gotten to my plex setup.

I’ll work through it and use some of your tips
Try some different Node versions


#94

@Ajki Been using your mount script for a few days now, have you noticed it sometimes takes things ages to start playing? But after they start it’s all good?


#95

@toomuchio define ages ?

All i can tell you is that connection to amazon is a bit spastic in general.

For example every now and then I run tree -D -s -h trough all my library, the server in both cases had 0 people streaming and Plex Library was not updating eg same CPU usage.

14.12.2016/17:51
3304 directories, 28933 files 32TB
tree -D -s -h completed in 317 seconds

15.12.2016/14:09
3304 directories, 29000 files 32TB
tree -D -s -h completed in 552 seconds

As you can see the difference is massive hence you cant really claim hmm something is wrong with rclone version and/or mount parameter before you do some proper test just to see how amazon drive is behaving.

At home I was running tons of copy file tests to measure the speed and its spastic as well … not 10% difference but 200% difference within one hour. Sometimes I could copy file within 2 minutes and 20 minutes later it would take 5 minutes while nothing was changed.


#96

Yes it works on all remotes.


#98

I’ve seen it take up to 30 seconds to start playing something but then it’s fine. I think it’s just crappy ACD performance. Otherwise these mount settings have been great for me.

This is my mount script if anybody was wondering @Ajki have you made any further changes? This seems to be working really well.

killall rclone
fusermount -uz …
rclone mount --allow-other --read-only --max-read-ahead=2G --acd-templink-threshold=0 --dir-cache-time=60m --checkers=32 --timeout=30s --contimeout=15s --retries=3 --low-level-retries=1 --stats=0 --quiet acdge:/ … &

killall unionfs-fuse
fusermount -uz …
unionfs-fuse -o allow_other,cow,max_readahead=2000000000 …=RW:…=RO …


#99

The issue with slow starts is due to rclone and not ACD.

I’ll post my setup tomorrow that uses rclone crypt and acd_cli together to make start ups happen in about 3-4 seconds.


#100

Why killall ? this will kill current upload progress as well.
@Stokkes so far the only slow start i noticed is from Plex not rclone eg when I run some movie that is 24GB or more it may take up to 8 to 10 seconds to start if I run the same movie with media play directly a home it will take 3 seconds.


#101

I make sure I never run my mount script while copying, the unmount doesn’t always kill the rclone processes in the background


#102

@Ajki

That is really odd. When I was on Hetzner or even now on SYS, mounting the encrypted drive and pointing Plex to it would cause start times to be 10+ seconds, almost always higher. I’m always using the latest betas too.

What my new setup does is mount the ACD drive using acd_cli and then use rclone to mount the encrypted folder for my media on the local ACD path. This way rclone isn’t talking to ACD directly, acd_cli does and it has been significantly faster, even on a mobile connection its 4-5 seconds to start a video when on LTE.

I have no idea why this is but I can easily recreate it by just remounting directly with rclone and modifying my UnionFS mount options.

I’m definitely up for ideas as adding acd_cli does add an extra layer where things could go wrong.


#103

@toomuchio
Never had problem with unmount, did you ever check with findmnt how many times you mouted as i saw people with multiple rclone mounts-

What you could do is put in script to keep unmounting while mount exist
(not tested but should work )

while [   mountpoint -q -- "/storage/acd" ]; do
             fusermount -uz /storage/acd
done 

@Stokkes Maybe there is difference in performance crypto vs encfs ( iam using the latter )

My mounts:

Whatever video I start is between 3 to 5 seconds and those 24GB between 5 and 10.
The only way to get it between 10 to 15 seconds is if I do burn in subtitles and video is 24GB but thats not rclone but transcoder that needs some time