Best mount settings for streaming ( Plex )

Ajki · February 10, 2017, 10:40pm

Atm Iam a bit busy moving my data from acdDE to acdUK.
Just fyi if you dont pay or your payment method is not accepted Amazon block your upload while you still have full access to your files.

Also i received my new plex server and fiddling with that one as I plan to start it from acdUK and not old drive.
Iam trying out reverse proxy with nginx with http2 and letsencrypt. The nice benefit of running http server is that i installed speedtest mini so my friends can test their connection to the server.

Btw nice find about pragam, just checked my plex db and its a 107K of pages, how is it calulates eg page size vs page count. In new server I have 64GB of ram so i would just double the amount needed. But then again i also installed 2xSSD disk in raid 0 on it so it should be able to handle it. ( until one disk die and crashes all system )

p.s. Iam using gdrive strictly for backups and at home if I play some of the videos directly with media player.

Tank · February 11, 2017, 3:09am

where you hosting this new server @Ajki? I believe SQLite default page size is 1024K. I think in newer versions it is 4096K. I have yet to tweak that PRAGMA setting. But I have found a few other settings that I might try tweaking for SQLite, I just don’t want to mess with anything yet as things are running quite well.

I run NGINX also, but not for Plex. Only for my other micro services to reduce open ports.

@ncw Is it worth using --write-back-cache for rclone mount if the mount is not read-only? If so what is a recommended setting?

peatnik · February 11, 2017, 8:19am

Why are you migrating from DE to UK? Didn’t you just migrate from US to DE?

Ajki · February 11, 2017, 9:56am

Read about it here: Warning: Amazon.DE if you are not from Germany/Austria

sirpilsofd · February 11, 2017, 10:31pm

@Ajki What is your current mount settings and what has performance been like for you?

Thanks

Meowingtons · February 11, 2017, 11:31pm

Just an update on performance since moving to another server
So I moved from an i7 3770k with 1Gbps line to a server with a Intel Xeon with a guaranteed 1Gbps line and Ent HDD’s in RAID 1 and a INIC.
I also began to move to Crypt rather then using encfs + rclone.
Boy oh boy, I’ve gone from peaks of 13MiB/s with averages of 7MiB/s to peaks of 90MiB/s and averages of 50+ MiB/s. So much happier with performance now, dunno if it’s due to Crypt or thanks to a different server but I’m much happier now with performance

ErAzOr · February 12, 2017, 9:01am

@Meowingtons
how much do you pay for a guaranteed 1Gbps line?

peatnik · February 12, 2017, 3:39pm

Not Meowingtons, but two of the best options i know are online.net personal servers and hetzner servers, although the latter have a 20TB upload limit if you’re paying less than 47€ per month for the server. Hetzner might be the best connection you can get when looking for relatively cheap dedicated servers in europe.
But don’t make the mistake of buying a online.net “deal” because they usually come with hidden speed and bandwidth caps, e.g. 200 mbps for the personal line.

Meowingtons · February 12, 2017, 5:20pm

I’m paying €33 euros for this setup
Intel Xeon E3-1245V2
2x Enterprise HDD 3.0 TB (RAID 1)
16GB of RAM
Intel INIC
1Gbps Guaranteed line
20TB of out going traffic (Internal and incoming is unlimited)

toomuchio · February 13, 2017, 12:08am

@Ajki What server are you using to move from EU->UK? I’ve just made the same mistake and need to get my data out of there asap The good thing is I have 50% of it in my Amazon US account already so I’m probably just going to crypt to crypt copy it all over again.

Ajki · February 13, 2017, 8:46am

Iam doing it on my main server since my DE account is blocked for upload.

toomuchio · February 13, 2017, 9:10am

Would a scaleaway vps work do you think? About 26TB of data I need to move :’(

Ajki · February 13, 2017, 1:33pm

Yes and i got average speeds between 400 and 500Mbit for 2.99€/monthly price.

benamira · February 13, 2017, 7:33pm

@ajki what’s your experience when scanning a large section. Like 2000 HD movies for example. everything encrypted on Amazon Drive.

Looking into logs i see tons of errors:
failed to authenticate decrypted block - bad password?
ReadFileHandle.Read error: low level retry 1/10: unexpected EOF

Any ideas?
This results in not able to complete correctly the scan. Attached picture.

Ajki · February 13, 2017, 10:18pm

I was just checking on my new plex server ( after almost 7 days I manage to transfer 40TB from acdDE to acdUK )
rclone v1.35-92-g18c75a8β ( note you need to set -vv flag to see debug in logs )

Rclone mount options:

rclone mount \
           --read-only \
           --allow-other \
           --acd-templink-threshold 0 \
           -vv \
           --debug-fuse \
           --dump-headers \
           --log-file=/home/plex/logs/mountacdcrypt.log \
           acdcrypt: /mnt/acdcrypt &

Unionfs mount options:

unionfs-fuse
-o cow,allow_other,auto_cache
/storage/local=RW:/mnt/acdcrypt=RO
/mnt/unionfs

I did full Library scan 2700+ movies and 13K+ episodes, and bellow are different type of errors in log

2017/02/13 22:53:59 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2017/02/13 22:54:01 DEBUG : fuse: <- Lookup [ID=0x148 Node=0x1 Uid=1000 Gid=1000 Pid=2696] “crypt-check”
2017/02/13 22:54:01 DEBUG : crypt-check: Dir.Lookup
2017/02/13 22:54:01 DEBUG : crypt-check: Dir.Lookup OK
2017/02/13 22:54:01 DEBUG : crypt-check: File.Attr
2017/02/13 22:54:01 DEBUG : fuse: -> [ID=0x148] Lookup 0x3 gen=0 valid=1m0s attr={valid=1m0s ino=1551189674461498721 size=26 mode=-rw-r–r--}
2017/02/13 22:54:01 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2017/02/13 22:54:01 DEBUG : HTTP RESPONSE (req 0xc42011e3c0)
2017/02/13 22:54:01 DEBUG : HTTP/1.1 500 Internal Server Error
Content-Length: 30
Connection: keep-alive
Content-Type: application/vnd.error+json
Date: Mon, 13 Feb 2017 21:54:01 GMT
Server: Amazon-Cloud-Drive
X-Amzn-Requestid: 6ed04bb2-9ae3-4a94-a14a-f71faacc5fd1

2017/02/13 22:54:01 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2017/02/13 22:54:01 DEBUG : pacer: Rate limited, sleeping for 204.764712ms (1 consecutive low level retries)
2017/02/13 22:54:01 DEBUG : pacer: low level retry 1/10 (error )
2017/02/13 22:54:01 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2017/02/13 22:54:01 DEBUG : HTTP REQUEST (req 0xc42022a870)
2017/02/13 22:54:01 DEBUG : GET /cdproxy/templink/8twGWQWZa_H-qLmq1zoTN3uUnfkxJqTP0xyo23vhXNI38XJVg HTTP/1.1
Host: content-eu.drive.amazonaws.com
User-Agent: rclone/v1.35-92-g18c75a8Î˛
Range: bytes=3985561632-

2017/02/13 22:54:01 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

2017/02/13 22:57:26 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2017/02/13 22:57:26 DEBUG : pacer: Rate limited, sleeping for 967.209084ms (2 consecutive low level retries)
2017/02/13 22:57:26 DEBUG : pacer: low level retry 1/10 (error )
2017/02/13 22:57:26 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2017/02/13 22:57:26 DEBUG : HTTP REQUEST (req 0xc420c042d0)
2017/02/13 22:57:26 DEBUG : GET /cdproxy/templink/3b1md-tWMFm6eXlkcVn6Y1B0zN-1Q4LkI19lVe0i9ks38XJVg HTTP/1.1
Host: content-eu.drive.amazonaws.com
User-Agent: rclone/v1.35-92-g18c75a8Î˛
Range: bytes=3804965872-

2017/02/13 22:57:31 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2017/02/13 22:57:31 DEBUG : movies/Babe (1995)/Babe (1995).mkv: ReadFileHandle.Read OK
2017/02/13 22:57:31 DEBUG : fuse: -> [ID=0x9ad] Read 126976
2017/02/13 22:57:31 DEBUG : movies/Babe (1995)/Babe (1995).mkv: ReadFileHandle.Read size 4096 offset 4434046976
2017/02/13 22:57:31 DEBUG : movies/Babe (1995)/Babe (1995).mkv: ReadFileHandle.seek from 4434178048 to 4434046976 (io.Seeker)
2017/02/13 22:57:31 DEBUG : hm2ka8or5qp1pefhtc1e62jn50/nt3r0na3fq7sro4pvmq2hru334/jbhqa01i3eoqaak77n8dcfev8s: Dowloading large object via tempLink
2017/02/13 22:57:31 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2017/02/13 22:57:31 DEBUG : HTTP REQUEST (req 0xc420544960)
2017/02/13 22:57:31 DEBUG : GET /cdproxy/templink/obC0oKRk8ChbcikRCNgzQ_IuwtZh0K8Y2bIIBrPiUgs38XJVg HTTP/1.1
Host: content-eu.drive.amazonaws.com
User-Agent: rclone/v1.35-92-g18c75a8Î˛
Range: bytes=4435117248-

2017/02/13 22:57:31 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2017/02/13 22:57:34 DEBUG : fuse: <- Lookup [ID=0x9af Node=0x1 Uid=1000 Gid=1000 Pid=1754] “.unionfs-fuse”
2017/02/13 22:57:34 DEBUG : .unionfs-fuse: Dir.Lookup
2017/02/13 22:57:34 DEBUG : fuse: -> [ID=0x9af] Lookup error=ENOENT
2017/02/13 22:57:34 DEBUG : fuse: <- Open [ID=0x9b0 Node=0x2 Uid=1000 Gid=1000 Pid=1754] dir=true fl=OpenReadOnly+OpenDirectory+OpenNonblock
2017/02/13 22:57:34 DEBUG : fuse: -> [ID=0x9b0] Open 0x2 fl=0
2017/02/13 22:57:34 DEBUG : fuse: <- Read [ID=0x9b1 Node=0x2 Uid=1000 Gid=1000 Pid=1754] 0x2 4096 @0x0 dir=true fl=0 lock=0 ffl=OpenReadOnly+OpenDirectory+OpenNonblock
2017/02/13 22:57:34 DEBUG : movies: Dir.ReadDirAll
2017/02/13 22:57:34 DEBUG : movies: Dir.ReadDirAll OK with 2770 entries

Most of the errors in log are fuse one:

2017/02/13 23:03:24 DEBUG : fuse: -> [ID=0x192e3] Lookup 0x20c7 gen=20 valid=1m0s attr={valid=1m0s ino=12813857630482816394 size=7033188210 mode=-rw-r–r--}
2017/02/13 23:03:24 DEBUG : fuse: <- Lookup [ID=0x192e4 Node=0x1 Uid=1000 Gid=1000 Pid=1754] “.unionfs-fuse”
2017/02/13 23:03:24 DEBUG : .unionfs-fuse: Dir.Lookup
2017/02/13 23:03:24 DEBUG : fuse: -> [ID=0x192e4] Lookup error=ENOENT

Those are really constantly spaming in log file, like every second …
@ncw any idea what are those

Those are more then likely corrupted files, I dont have any those errors in my log.

peatnik · February 14, 2017, 4:21am

isn’t the scaleway vps for 3€ capped at 200 mbps?

mjf14 · February 14, 2017, 4:33am

Yeah, I was thinking the same thing too?

Ajki · February 14, 2017, 9:16am

Yes its say it capped, but its working in average from 400 to 500Mbit.

Maddux · February 14, 2017, 9:00pm

Hi Guys, after having a bad streaming experience with acd_cli, I’m trying to get rclone mount working with Plex. After reading this thread and adjusting my mount options, I’m still having issues. About every 5 minutes (sometimes less) I get rebuffing. I’ve tried on a DigitalOcean VPS, and on a Mac Mini at home (w/ a 100mb/s connection) and both perform about equally.

Here are my current mount settings:

clone mount
–read-only
–allow-other
–acd-templink-threshold 0
–debug-fuse
–dump-headers
ACD:[cloud mount point] [local mount point] &

The --debug-fuse option spits out this error sometimes when it buffers:
ReadFileHandle.Read error: low level retry 1/10: unexpected EOF

Any ideas?

Ajki · February 14, 2017, 9:03pm

Drop this out as its impacting performance and EOF errors are normal.

you can debug it by adding -vv --debug-fuse --dump-headers --log-file=/path to/logfile.log
Use the latest beta.