Best mount settings for streaming ( Plex )

My library is about 30TB on ACD. I don’t think you need to go 20x, even setting your cache at a million might be sufficient for awhile. If I did my math correctly even 1 million pages in cache is only 1.5GB of RAM. Honestly if you have the RAM to spare, why not?

I’m always on the hunt for more performance and at this point it’s really the mount options or ACD. I guess this is why we’re all here. I could go SSD, but I don’t think it’s needed until I run out of RAM.

@Ajki have you noticed any improvement over google drive vs ACD?

@Tank

Great thanks… I will try with 2 million cache size I have 32GB of ram so lots to spare.

I’ve played with many mount options but I don’t really feel any make any significant difference so I just use ``rclone mount remote: folder --allow-other --read-only`, pretty simple.

For Google Drive, I notice it is significantly more performant for me but I can’t use it without getting banned :frowning:

Atm Iam a bit busy moving my data from acdDE to acdUK.
Just fyi if you dont pay or your payment method is not accepted Amazon block your upload while you still have full access to your files.

Also i received my new plex server and fiddling with that one as I plan to start it from acdUK and not old drive.
Iam trying out reverse proxy with nginx with http2 and letsencrypt. The nice benefit of running http server is that i installed speedtest mini so my friends can test their connection to the server.

Btw nice find about pragam, just checked my plex db and its a 107K of pages, how is it calulates eg page size vs page count. In new server I have 64GB of ram so i would just double the amount needed. But then again i also installed 2xSSD disk in raid 0 on it so it should be able to handle it. ( until one disk die and crashes all system :stuck_out_tongue: )

p.s. Iam using gdrive strictly for backups and at home if I play some of the videos directly with media player.

where you hosting this new server @Ajki? I believe SQLite default page size is 1024K. I think in newer versions it is 4096K. I have yet to tweak that PRAGMA setting. But I have found a few other settings that I might try tweaking for SQLite, I just don’t want to mess with anything yet as things are running quite well.

I run NGINX also, but not for Plex. Only for my other micro services to reduce open ports.

@ncw Is it worth using --write-back-cache for rclone mount if the mount is not read-only? If so what is a recommended setting?

Why are you migrating from DE to UK? Didn’t you just migrate from US to DE?

Read about it here: Warning: Amazon.DE if you are not from Germany/Austria

@Ajki What is your current mount settings and what has performance been like for you?

Thanks

Just an update on performance since moving to another server
So I moved from an i7 3770k with 1Gbps line to a server with a Intel Xeon with a guaranteed 1Gbps line and Ent HDD’s in RAID 1 and a INIC.
I also began to move to Crypt rather then using encfs + rclone.
Boy oh boy, I’ve gone from peaks of 13MiB/s with averages of 7MiB/s to peaks of 90MiB/s and averages of 50+ MiB/s. So much happier with performance now, dunno if it’s due to Crypt or thanks to a different server but I’m much happier now with performance :slight_smile:

@Meowingtons
how much do you pay for a guaranteed 1Gbps line?

Not Meowingtons, but two of the best options i know are online.net personal servers and hetzner servers, although the latter have a 20TB upload limit if you’re paying less than 47€ per month for the server. Hetzner might be the best connection you can get when looking for relatively cheap dedicated servers in europe.
But don’t make the mistake of buying a online.net “deal” because they usually come with hidden speed and bandwidth caps, e.g. 200 mbps for the personal line.

I’m paying €33 euros for this setup
Intel Xeon E3-1245V2
2x Enterprise HDD 3.0 TB (RAID 1)
16GB of RAM
Intel INIC
1Gbps Guaranteed line
20TB of out going traffic (Internal and incoming is unlimited)

@Ajki What server are you using to move from EU->UK? I’ve just made the same mistake and need to get my data out of there asap :frowning: The good thing is I have 50% of it in my Amazon US account already so I’m probably just going to crypt to crypt copy it all over again.

Iam doing it on my main server since my DE account is blocked for upload.

Would a scaleaway vps work do you think? About 26TB of data I need to move :’(

Yes and i got average speeds between 400 and 500Mbit for 2.99€/monthly price.

@ajki what’s your experience when scanning a large section. Like 2000 HD movies for example. everything encrypted on Amazon Drive.

Looking into logs i see tons of errors:
failed to authenticate decrypted block - bad password?
ReadFileHandle.Read error: low level retry 1/10: unexpected EOF

Any ideas?
This results in not able to complete correctly the scan. Attached picture.

I was just checking on my new plex server ( after almost 7 days I manage to transfer 40TB from acdDE to acdUK )
rclone v1.35-92-g18c75a8β ( note you need to set -vv flag to see debug in logs )

Rclone mount options:

rclone mount \
           --read-only \
           --allow-other \
           --acd-templink-threshold 0 \
           -vv \
           --debug-fuse \
           --dump-headers \
           --log-file=/home/plex/logs/mountacdcrypt.log \
           acdcrypt: /mnt/acdcrypt &

Unionfs mount options:

unionfs-fuse
-o cow,allow_other,auto_cache
/storage/local=RW:/mnt/acdcrypt=RO
/mnt/unionfs

I did full Library scan 2700+ movies and 13K+ episodes, and bellow are different type of errors in log

2017/02/13 22:53:59 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2017/02/13 22:54:01 DEBUG : fuse: <- Lookup [ID=0x148 Node=0x1 Uid=1000 Gid=1000 Pid=2696] “crypt-check”
2017/02/13 22:54:01 DEBUG : crypt-check: Dir.Lookup
2017/02/13 22:54:01 DEBUG : crypt-check: Dir.Lookup OK
2017/02/13 22:54:01 DEBUG : crypt-check: File.Attr
2017/02/13 22:54:01 DEBUG : fuse: -> [ID=0x148] Lookup 0x3 gen=0 valid=1m0s attr={valid=1m0s ino=1551189674461498721 size=26 mode=-rw-r–r--}
2017/02/13 22:54:01 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2017/02/13 22:54:01 DEBUG : HTTP RESPONSE (req 0xc42011e3c0)
2017/02/13 22:54:01 DEBUG : HTTP/1.1 500 Internal Server Error
Content-Length: 30
Connection: keep-alive
Content-Type: application/vnd.error+json
Date: Mon, 13 Feb 2017 21:54:01 GMT
Server: Amazon-Cloud-Drive
X-Amzn-Requestid: 6ed04bb2-9ae3-4a94-a14a-f71faacc5fd1

2017/02/13 22:54:01 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2017/02/13 22:54:01 DEBUG : pacer: Rate limited, sleeping for 204.764712ms (1 consecutive low level retries)
2017/02/13 22:54:01 DEBUG : pacer: low level retry 1/10 (error )
2017/02/13 22:54:01 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2017/02/13 22:54:01 DEBUG : HTTP REQUEST (req 0xc42022a870)
2017/02/13 22:54:01 DEBUG : GET /cdproxy/templink/8twGWQWZa_H-qLmq1zoTN3uUnfkxJqTP0xyo23vhXNI38XJVg HTTP/1.1
Host: content-eu.drive.amazonaws.com
User-Agent: rclone/v1.35-92-g18c75a8β
Range: bytes=3985561632-

2017/02/13 22:54:01 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

2017/02/13 22:57:26 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2017/02/13 22:57:26 DEBUG : pacer: Rate limited, sleeping for 967.209084ms (2 consecutive low level retries)
2017/02/13 22:57:26 DEBUG : pacer: low level retry 1/10 (error )
2017/02/13 22:57:26 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2017/02/13 22:57:26 DEBUG : HTTP REQUEST (req 0xc420c042d0)
2017/02/13 22:57:26 DEBUG : GET /cdproxy/templink/3b1md-tWMFm6eXlkcVn6Y1B0zN-1Q4LkI19lVe0i9ks38XJVg HTTP/1.1
Host: content-eu.drive.amazonaws.com
User-Agent: rclone/v1.35-92-g18c75a8β
Range: bytes=3804965872-

2017/02/13 22:57:31 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2017/02/13 22:57:31 DEBUG : movies/Babe (1995)/Babe (1995).mkv: ReadFileHandle.Read OK
2017/02/13 22:57:31 DEBUG : fuse: -> [ID=0x9ad] Read 126976
2017/02/13 22:57:31 DEBUG : movies/Babe (1995)/Babe (1995).mkv: ReadFileHandle.Read size 4096 offset 4434046976
2017/02/13 22:57:31 DEBUG : movies/Babe (1995)/Babe (1995).mkv: ReadFileHandle.seek from 4434178048 to 4434046976 (io.Seeker)
2017/02/13 22:57:31 DEBUG : hm2ka8or5qp1pefhtc1e62jn50/nt3r0na3fq7sro4pvmq2hru334/jbhqa01i3eoqaak77n8dcfev8s: Dowloading large object via tempLink
2017/02/13 22:57:31 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2017/02/13 22:57:31 DEBUG : HTTP REQUEST (req 0xc420544960)
2017/02/13 22:57:31 DEBUG : GET /cdproxy/templink/obC0oKRk8ChbcikRCNgzQ_IuwtZh0K8Y2bIIBrPiUgs38XJVg HTTP/1.1
Host: content-eu.drive.amazonaws.com
User-Agent: rclone/v1.35-92-g18c75a8β
Range: bytes=4435117248-

2017/02/13 22:57:31 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2017/02/13 22:57:34 DEBUG : fuse: <- Lookup [ID=0x9af Node=0x1 Uid=1000 Gid=1000 Pid=1754] “.unionfs-fuse”
2017/02/13 22:57:34 DEBUG : .unionfs-fuse: Dir.Lookup
2017/02/13 22:57:34 DEBUG : fuse: -> [ID=0x9af] Lookup error=ENOENT
2017/02/13 22:57:34 DEBUG : fuse: <- Open [ID=0x9b0 Node=0x2 Uid=1000 Gid=1000 Pid=1754] dir=true fl=OpenReadOnly+OpenDirectory+OpenNonblock
2017/02/13 22:57:34 DEBUG : fuse: -> [ID=0x9b0] Open 0x2 fl=0
2017/02/13 22:57:34 DEBUG : fuse: <- Read [ID=0x9b1 Node=0x2 Uid=1000 Gid=1000 Pid=1754] 0x2 4096 @0x0 dir=true fl=0 lock=0 ffl=OpenReadOnly+OpenDirectory+OpenNonblock
2017/02/13 22:57:34 DEBUG : movies: Dir.ReadDirAll
2017/02/13 22:57:34 DEBUG : movies: Dir.ReadDirAll OK with 2770 entries

Most of the errors in log are fuse one:

2017/02/13 23:03:24 DEBUG : fuse: -> [ID=0x192e3] Lookup 0x20c7 gen=20 valid=1m0s attr={valid=1m0s ino=12813857630482816394 size=7033188210 mode=-rw-r–r--}
2017/02/13 23:03:24 DEBUG : fuse: <- Lookup [ID=0x192e4 Node=0x1 Uid=1000 Gid=1000 Pid=1754] “.unionfs-fuse”
2017/02/13 23:03:24 DEBUG : .unionfs-fuse: Dir.Lookup
2017/02/13 23:03:24 DEBUG : fuse: -> [ID=0x192e4] Lookup error=ENOENT

Those are really constantly spaming in log file, like every second …
@ncw any idea what are those

Those are more then likely corrupted files, I dont have any those errors in my log.

isn’t the scaleway vps for 3€ capped at 200 mbps?

Yeah, I was thinking the same thing too?

Yes its say it capped, but its working in average from 400 to 500Mbit.