I’m using the values from the first post and although it worked great when I had some media with both scanning and streaming, once I added a large amount of media, I keep getting “Rate Limit Exceeded” errors from google drive. I do have thousands of files in TV shows folder now and when I start the service, after either a “find” or a plex scan, it fails. I can not start the service until after a reboot. Here’s the error I’m getting when trying to remount:
2018/08/21 12:19:45 DEBUG : pacer: Rate limited, sleeping for 1.542017021s (1 consecutive low level retries)
2018/08/21 12:19:45 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/08/21 12:19:45 DEBUG : pacer: Rate limited, sleeping for 2.75577138s (2 consecutive low level retries)
2018/08/21 12:19:45 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/08/21 12:19:46 DEBUG : pacer: Resetting sleep to minimum 10ms on success
2018/08/21 12:19:46 Fatal error: Can not open: /GDcrypt: open /GDcrypt: transport endpoint is not connected
Been looking into it more, and the issue may have been memory related.
I’m running this on a vps with 1GB of ram + 1GB of swap and watching htop while doing a plex scan shows as soon as memory is maxed out, rclone mount dies
That is a good char to check out to see if you are hitting a lot of queries per second. I can see when I was initially seeding, some 403s and such are expected as it just slows it down a bit.
Oh yeah, that would kill it as plex opens up many files so the 256M buffer size would way too big for a 1G of RAM. I’d leave it at the default of 16M. My server has 32GB of memory
The scans are always going to be tight with that low memory so you may want to go ever lower. To get an idea, you can run as root “lsof /mountpoint” and see how many files are actually open.
If you are seeing “Direct Play”, having a bigger buffer would be helpful if you have any latency.
If you are seeing “Direct Stream” or “Transcode”, it wouldn’t matter much as it’s going to transcode up to your throttle settings in Plex. I usually keep my throttle setting at 600 or 900 seconds so 10 or 15 minutes of buffer.
That worked great. So I watched the rclone log while set to debug, and holy moly, does plex access a lot of files during a scan.
With the buffer set to 16MB, scans of large amounts of files work, so do direct play and direct stream.
For reference, plex is running on a DO vps (the cheapest $5 per month one). However, since the bandwidth is limited on DO, I’m uploading content to gdrive from a local vm, set up with the same rclone config as the DO vps.
I uploaded several terabytes over a couple of weeks and it works great.
The vps has 1GB of ram, 1GB of swap (may up it to 2GB) and a single core processor. It can handle one transcoded stream, but barely. I try to do direct play or direct stream only.
You could always set the buffer to 0 for the initial scan. After that set it to a low amount. Ran like that on a 2GB VPS for some time and that was the only way I could scan my 25TB at the time.
@Animosity022 Thanks for the info, really helped me understand a lot of things. This settings and scripts I got at the Unraid forums really made things easy.
Have a odd problem, wondering if anyone had encountered it. It’s to do with Direct Play on Plex. Direct streaming on Plex works fine, either because the file is an MKV or if I turn off direct play in the app settings. However, when direct playing all that happens is a black screen with the loading icon. I’ve played with various settings on buffer and chunk size and limit. It makes no difference. The exact same files direct play when stored locally through the exact same Plex server. In a nutshell, to play files from gdrive I have to direct stream. Any attempt to direct play them causes a black screen with infinite load. However, these same files direct play when accessed locally through that same Plex server. Client - Apple TV 4K , All files are formatted as MP4, with ac3 and mov_text - compatible with direct play.
Please let me know if anyone has encountered this or have found a solution. Thanks.
I’ve just done that. Thanks for the steps. Nothing turns up on the log. It just displays the scanning of other files. No mention whatsoever of the file i’m trying to play. Tried it four to five times. However, plex logs show the file being selected for Direct Play. Is it normal for the log file to be growing at a rapid rate? It 42 mb in less than 2 hours. Since I might not know what i’m looking for, will be helpful if i just posted the log file at the time i start playing? I asking since there are so many lines even within a minute. And will the plex logs be useful to you? Thanks.
Thanks. Found what you mentioned, I hope. It goes on much longer than this while I see a black loading screen on TV. I’ve linked the whole part of the log, while this black screen is going on, if needed. I’ve also kept Plex logs when Direct Playing and Rclone logs when it Direct Streams and works correctly which is all the time with your setup. Thanks.
What’s your actual mount command that you using? The strange part in the log is that you can see it open and close the file quite a bit which is strange. I was seeing that behavior at one point but I could not figure out what parameters were causing it and I haven’t been able to recreate it:
Is there any unionfs or mergerfs or anything else or that just a direct rclone mount? It is basically showing the black screen as the client you are playing from keeps opening and closing the file quite a bit instead of actually just reading the file.
What’s the client that’s direct playing? Can you try also playing from the Plex Media Player from your PC/Laptop? That should direct play just about everything as well as it a good place to validate it.
My apologies. That text file was for the direct streaming - the file was .mkv as seen. This works consistently across all players. Start up is at about 5 sec. I do notice the behavior you notice, the opening and closing, but it’s not felt in playback.
Using Infuse or any other application (laptop native plex client) that doesn’t use Apple TV native player works since they support a wide range of formats and buffer ahead I believe. The specific problem happens when playing Direct Play compatible media using the Plex app in AppleTV. For example when a Direct Play compatible media file which is stored locally is played through the Plex server it plays well - Direct Play. When playing the same media file on an unionfs mount while the media file is local (before upload cron job is run) it plays well - Direct Play. When playing the same file in the same Plex server using the same unionfs mount (after it’s uploaded to gdrive) it goes to black screen. I noticed a bunch of posts on this topic with people using Apple TVs. I figured someone might know what’s up.
Are you able to test without the unionfs mount for the same media? I did that by adding just another mount and adding a test library in plex with just the one specific item / show I want to play.
I basically had the same issue you are talking about and it was able to be detected by the debug log and the player opening and closing the files repeatedly.
If you want to add to the topic I posted in regards to Direct Play, it is here ->
I’ve yet to recreate the issue on my setup as I’ve desperately tried. If you can get the same thing on the rclone mount, I’d be interested in seeing that debug to see if the same pattern shows.
My gut is a particular setting that doesn’t jive with the ATV (4k and regular one) but I can’t figure out which either so have to go through the process of eliminating all the variables.
