Rclone mount setup - Plex server reading from rclone off an Openwrt router!


#1

Hey all. Great forum, awesome software.

I’m running a Plex server off my Nvidia Shield, which is connected to a Samba server 1TB drive that runs off my WRT1900ACS OpenWrt router––def not the most efficient setup but it works. I also run a seedbox on the other drive connected to the router. Please keep this in mind when you look at my config. There’s limited memory and processing power on the router (512MB DDR3) so everything is built with that in mind. Believe it or not, Plex streams well.

I am a little confused between rclone mount and rclone cache. Do all the options listed on the mount manpage apply to the cache manpage, From my understanding it seems the --vfs options are only applicable to non-cache mounts, but it’s not clear.

Anyway, below is my config. Questions right now:

  1. Do all the commands I put in mount apply to cache?
  2. I haven’t tested adding more movies to the drive and triggering a Plex rescan. Not quite sure how I’m going to do that yet, since right now my downloads are on another drive and get uploaded via a separate script, but with this I would have to manually update the library… any suggestions?
  3. Any suggestions to my config below to free up more memory? My memory on the router goes from 30% free to 5% free (linux has great built-in memory management) but I thought with this config no memory would be getting used. Any suggestions? VSZ is 979m 159%
  4. For those who are curious: Plex takes about 5 secs to load the file data then another 10 seconds to buffer the video. Load on the router is around .5 to .9 whenever I stream one movie. Haven’t tried multiple yet. Seeking takes about 10 seconds to jump around.
  5. I’m thinking about transferring this process to the Nvidia SHIELD since it has 3GB memory and I want my router focused on router things. However I can’t find an rclone android app, which means I’m going to have to go hard and root it, etc. Any plans for an android app to avoid this?
[movies]
    type = cache
    remote = remote:/Movies
    plex_url = 
    plex_username = 
    plex_password = 
    chunk_path = /mnt/a/Downloads/cache
    chunk_total_size = 250G
    chunk_size = 15M
    chunk_clean_interval = 15m0s
    chunk_no_memory = true
    db_path = /mnt/a/tools/rclone/dbcache
    db_purge = false
    info_age = 1d
    workers = 3

Here’s the command I run on startup (any way to put these in the config file?)

rclone mount movies: /mnt/a/Movies \
            --no-checksum \
            --no-modtime \
            --allow-other \
            --attr-timeout 10s \
            --tpslimit 8 \
            --sys-log \
            --log-level INFO \
            --daemon \

#2

That looks pretty good as all those commands apply to the cache that you have in your config.

You probably want to add --buffer-size 0 as part of that the cache backend doesn’t need any buffer at all which would help on memory.

Some android info here -> Rclone for Android (new app)

You can run Plex on that stuff assuming that you don’t transcode anything. Once you hit a transcode, things are going to crawl as there isn’t much power to support that.


#3

Thanks. Looks like in a new library scan I ran out of memory and linux killed it:

I will add --buffer-size 0 to it and see if it dies again. Thank you!

Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.167185] kthreadd invoked oom-killer: gfp_mask=0x15080c0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), nodemask=(null),  order=1, oom_score_adj=0
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.179348] COMPACTION is disabled!!!
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.183027] CPU: 1 PID: 2 Comm: kthreadd Not tainted 4.14.63 #0
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.188969] Hardware name: Marvell Armada 380/385 (Device Tree)
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.194917] Function entered at [<c010f500>] from [<c010b1d0>]
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.200775] Function entered at [<c010b1d0>] from [<c061a8a8>]
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.206631] Function entered at [<c061a8a8>] from [<c01a16d8>]
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.212486] Function entered at [<c01a16d8>] from [<c01a097c>]
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.218342] Function entered at [<c01a097c>] from [<c01a15a0>]
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.224196] Function entered at [<c01a15a0>] from [<c01a5380>]
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.230057] Function entered at [<c01a5380>] from [<c011fb4c>]
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.235916] Function entered at [<c011fb4c>] from [<c012100c>]
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.241771] Function entered at [<c012100c>] from [<c0121290>]
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.247626] Function entered at [<c0121290>] from [<c013d97c>]
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.253486] Function entered at [<c013d97c>] from [<c0107968>]
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.259388] Mem-Info:
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.261676] active_anon:107421 inactive_anon:47 isolated_anon:0
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.261676]  active_file:307 inactive_file:312 isolated_file:0
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.261676]  unevictable:0 dirty:0 writeback:0 unstable:0
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.261676]  slab_reclaimable:1138 slab_unreclaimable:4317
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.261676]  mapped:153 shmem:916 pagetables:325 bounce:0
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.261676]  free:4097 free_pcp:227 free_cma:0
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.294238] Node 0 active_anon:429684kB inactive_anon:188kB active_file:984kB inactive_file:1056kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:492kB dirty:0kB writeback:0kB shmem:3664kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.316935] Normal free:16388kB min:16384kB low:20480kB high:24576kB active_anon:429684kB inactive_anon:188kB active_file:1124kB inactive_file:1280kB unevictable:0kB writepending:0kB present:524288kB managed:511928kB mlocked:0kB kernel_stack:904kB pagetables:1300kB bounce:0kB free_pcp:1364kB local_pcp:732kB free_cma:0kB
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.345387] lowmem_reserve[]: 0 0 0
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.348900] Normal: 517*4kB (UME) 865*8kB (UME) 464*16kB (UME) 5*32kB (U) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 16572kB
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.361666] 1578 total pagecache pages
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.365429] 0 pages in swap cache
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.368767] Swap cache stats: add 0, delete 0, find 0/0
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.374012] Free swap  = 0kB
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.376902] Total swap = 0kB
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.379796] 131072 pages RAM
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.382685] 0 pages HighMem/MovableOnly
Wed Jan  2 12:38:52 2019 kern.warn kernel: [15251.386534] 3090 pages reserved
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.389689] [ pid ]   uid  tgid total_vm      rss nr_ptes nr_pmds swapents oom_score_adj name
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.398270] [ 1021]     0  1021      244       19       4       0        0             0 ubusd
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.406920] [ 1027]     0  1027      169        7       3       0        0             0 askfirst
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.415835] [ 1709]     0  1709      261       42       4       0        0             0 logd
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.424399] [ 1726]     0  1726      368       38       4       0        0             0 rpcd
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.432973] [ 1848]     0  1848      379       37       5       0        0             0 netifd
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.441715] [ 1887]     0  1887      304       27       4       0        0             0 odhcpd
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.450455] [ 1942]     0  1942      204       10       3       0        0             0 dropbear
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.459368] [ 1968]     0  1968      170        9       3       0        0             0 hd-idle
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.468193] [ 1980]     0  1980      219       15       3       0        0             0 netserver
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.477194] [ 2091]     0  2091      202       13       3       0        0             0 odhcp6c
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.486026] [ 2124]     0  2124      264        6       3       0        0             0 udhcpc
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.494766] [ 3425]     0  3425      404       32       4       0        0             0 hostapd
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.503592] [ 3715]     0  3715      911      208       4       0        0             0 uhttpd
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.512331] [ 3853]     0  3853      328       20       4       0        0             0 dbus-daemon
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.521506] [ 3869]     0  3869      405       31       4       0        0             0 hostapd
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.530331] [ 4007]     0  4007      643       67       4       0        0             0 smbd
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.538896] [ 4008]     0  4008      654       26       4       0        0             0 nmbd
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.547460] [ 4111]     0  4111      291       48       4       0        0             0 miniupnpd
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.556469] [ 4313]     0  4313      744       53       5       0        0             0 openvpn
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.565312] [ 5274]     0  5274      265       11       3       0        0             0 ntpd
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.573886] [10428]   453 10428     2225     1963       8       0        0             0 dnsmasq
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.582716] [  584]     0   584   204152    99135     210       0        0             0 rclone
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.591465] [18617]     0 18617      853      168       5       0        0             0 smbd
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.600040] [25001]   224 25001     6116     4542      16       0        0             0 transmission-da
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.609566] [ 9226]     0  9226      316       63       4       0        0             0 sh
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.617958] [13937]     0 13937      265        8       3       0        0             0 sleep
Wed Jan  2 12:38:52 2019 kern.err kernel: [15251.626612] Out of memory: Kill process 584 (rclone) score 752 or sacrifice child
Wed Jan  2 12:38:52 2019 kern.err kernel: [15251.634165] Killed process 584 (rclone) total-vm:816608kB, anon-rss:396540kB, file-rss:0kB, shmem-rss:0kB
Wed Jan  2 12:38:52 2019 kern.info kernel: [15251.744174] oom_reaper: reaped process 584 (rclone), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

#5

Thanks. While I wait for my other post to get approved (marked by the spam filters) do you know the proper way to kill the rclone daemon? Whenever I kill the process there is always a problem with fuse:

Wed Jan 2 13:06:43 2019 daemon.err uhttpd[3715]: df: /mnt/a/Movies: Socket not connected


#6

Normally, you’d fusermount -uz

You can also kill the process id.

Yes, if you have no memory, the plex scan opens many, many files at once so you’d quickly run out. The buffer-size 0 may help that but not sure.


#7

But fusermount -uz is only for fuse, the rclone daemon will still run, right? So it sounds like I have to fusermount -uz first and then kill process?

Regarding your comment on transcoding, I see where you’re coming from, but the shield can handle transcodes pretty well. I’m not sure the drive caching will impact it or not, but yes I’ll test it.


#8

fusermount -uz will remove the mount and stop the process.


#9

Thanks, but fusermount -uz only removed the mount, the rclone mount process was still running and I had to kill that after fusermount -uz

Any idea on 2)? Is there any way to get rclone to trigger plex to do a rescan after it detects a new file?

.


#10

fusermount stops the process as well.

felix@gemini:~$ ps -ef | grep rclone | grep Test
felix    14271 14133  0 12:48 pts/1    00:00:00 rclone mount gcrypt: /Test
felix@gemini:~$ fusermount -uz /Test
felix@gemini:~$ ps -ef | grep rclone | grep Test
felix@gemini:~$

as an example.

You’d get plex to scan via a few ways. You can use the integration with Sonarr/Radarr to kick off a scan. You can use something like https://github.com/l3uddz/plex_autoscan to kick off a scan.

You can run the plex CLI and scan after your process moving the files finishes.


#11

You are right, I ran it again and it worked. Weird. Thank you!!!

Any idea on how I get plex to scan from a different system? Ideally, I’d like to run it on my transmission script torrent done, which is a script that runs after the download completes. After the download completes, I upload the file to the cloud, then I want the script to trigger a Plex scan.

It seems like Sonarr/Radarr, Plex Autoscan, and Plex CLI all are required to run off the machine hosting the PMS.


#12

None of them are required to be on the same system. You can just point it to the proper host name/IP and use the right user name/password and it works.


#13

Thanks. I’m using these:

https://support.plex.tv/articles/201638786-plex-media-server-url-commands/