I just started using rclone (version 1.5) and running into a problem. All of my remuxes be 1080p/4K work really great when I'm streaming locally, where the server is. However, when I try to stream remotely, it buffers and just doesn't work. I tried it on my phone, LG TV, computer, they all work great locally, but when I go to a friend's house (he has a 500 mbps connection) it gets stuck and just buffers, even on my phone that works locally. If it works locally on my phone, it should work remotely too, correct? I've done speedtest there, and it's around 400 mbps
Another thing is, I also have some TV shows, some are 1080p, some are even 4K, and they work great both locally and remotely! However, the file size on these are only about 1-2GBs. I've been trying to mess around with the buffer and chunk size, but nothing really helps. All TV shows work like it should remotely and locally, but 1080p/4k remuxes only locally. Would be great if I can get some help to get it to work. I'm new to this, so please HELP!
Settings:
gdrive > crypt
Mount script:
rclone mount --allow-other --dir-cache-time 96h --vfs-cache-max-age 48h --vfs-read-chunk-size 20M --vfs-read-chunk-size-limit off --buffer-size 512M gcrypt: X:
I've tried read-chunk-size and buffer-size at different numbers, and chunk-size-limit to 1G, 2G, etc, but still having the same problem. Any help is greatly APPRECIATE!!!
The problem you are referring to is fairly well known - and it mostly not related to rclone settings (ie. the default settings should stream 4K just fine).
The issue is usually related to the video-player software.
The problem is that some players tend to frequently open and close files each time they read a new chunk of data. Opening/closing files via cloud is very slow compared to a harddrive (while seeking in an already open file is fairly quick). On a harddrive this strategy would work fine because access-time for opening a file will be milliseconds, but on a clouddrive it could be a second or more each time. Thus you may end up with so much time spent opening/closing files compared to actually fetching data that the effective throughput is abysmal => very bad stuttering and buffering.
People experiencing this issue will often find that it happens almost no matter the bitrate. It is not a throughput problem. it is not really fixable in rclone itself as rclone will only serve what applications request - and it's the requests here that mess up. The only workaround is using the cache backend, but that is a poor bandaid that does not address the real problem and it may not even work adequately so I don't really recommend that approach.
I would suggest the following:
Test a few alternative video-players. I usually use VLC and have few if any problems. VLC is designed around streaming on networks (although I'm sure many other players will work fine too).
Check if it affects all/most files or just a few spesific ones. Even though VLC works for me 99% of the time I have occasionally seen spesific files that display issues. I tend to chalk that up to quirky interactions between the player and the format but it happens so rarely for me that I have not dug into the issue deeper.
thanks for the reply. But the issue here is not the video-player software you are referring to as I can flawlessly stream using my phone at home. What makes me confused is that while at home my phone can stream 4K remuxes just fine, but when I go to my friend's it buffers and doesn't play at all. Again, his connection is even faster than mine. Even more perplexing is that TV shows play flawlessly both locally and remotely at 1080p/4K, only thing that is different is these files are a lot smaller. Say 1-2GBs per file as supposed to like a 40-60GB for a 4K remux.
Again, I can even use my LGTV to stream whatever files I throw at it locally, same with my phone. So I know it is not the player on the phone, and it is certainly not the internet connection at my friend's location. I mostly use mpc and it plays flawlessly. I don't know what the issue is.
Files stored on gdrive have limited download quota. Especially with large file like remuxes, the the only solution right now is wait 24 hours for the quota reset. 40-60 GiB files are not supposed to be download multiple times. From my experience, you can only watch a remuxes of that size only one or two times without any seeking while playing it (each seeking counts toward the download quota).
What's the upload from your system that's streaming it out? You've hit more on a capacity issue from your internet it sounds like.
What's the bitrate of the media being played to?
That's simply not quite the case. You don't count a full file when you seek around as they are range requests. Google doesn't publish specifically how it works other than you have a 10TB daily quota.
You can test it pretty easy with running a mediainfo on a large file repeatedly and if the whole file counted, you'd evaporate the quota pretty fast.
I can use my TV to stream these files just fine, whenever I want to. I'm the only one using it right now to do some testing. So I don't think I'm over the quota. Matter of fact, I can open up my app on my TV and starting streaming just fine. Then I would go over to my friend's location, it just doesn't work. I also believe gdrive has a very high quota, 10TB/24 hr if I'm not mistaken.
The question would be where is the server the files are being mounted on? If those are on 10Mb/s upload, you can't stream as it goes from Google Drive -> Server -> Player.
Be very spesific - are you saying that you can use your phone to view the video fine at home, but the exact same device and software produces stuttering at another location?
Is this streaming happening though Plex? We need to know exactly the chain of traffic from Google down to the software-player to diagnose this.
At home you may very well have loopback action which won't rely on your upload as the traffic may not actually go out to the net and then back, but just make a U-turn at the router. If you "just" have 10Mbit upload that might not be sufficient for really high bitrate 4K.
I think you may need to dump a lot of more details of your exact setup here because I expect we are missing crucial details.
EDIT: With all due respect I think Monarch's comment is way off the mark. You have a 10TB download quota and that is extremely hard to exhaust. Unless you are re-downloading the same file hundreds of times in a day it is not going to be a quota issue. Besides - that wouldn't produce stuttering but rather rclone errors in the log and no video stream whatsoever.
Yes, I can use the Plex app on my phone to stream perfectly fine at home, but the same phone, with the same app, Plex in this case, at another location produces stuttering. I can even use the Plex app on my LGTV to stream fine at home.
So it seems my upload of 10Mbit is not enough to get the job done. I've always thought that the player is streaming directly from Google Drive, and doesn't "really" involve my internet connection. So basically, even though the files are on google, I would still need a fast connection on my plex server where it's mounted on in order for it to work?
Yes, absolutely.
If using Plex, then the traffic-chain will look like this:
Plex server downloads from Google
Plex server sends data out via upstream (unless it is within the same network in which case most modern routers will be intelligent enough to use loopback to just send it right back on the LAN/WIFI)
Remote device receives data and plays it.
In reality the request starts from the bottom (the player) and then goes all the way up the chain to Google and then the data streams back.
At home you may not be upload-limited at all, but at other locations not on the same LAN as the server it will have to rely on upload capacity. 10Mbit should work most of the time, but is perhaps a little too low for very high bitrate 4K. This all depends on the individual files as bitrates can vary wildly. The most modern formats (like H.265/HVEC) will tend to need less bandwidth than the previous gen. H.264, but newer formats also need newer devices with the hardware support for decoding unless you have a beastly CPU to do it in software.
TLDR: You should check the actual bit-rate of the file(s) you are testing with. You can view this in a lot of places (most media players via details, Some OS'es like Windows 10 via properties and via various free tools).
You would probably start to see problems on streams significantly above 8Mbit/sec, because there is of course some overhead both in data-transfer and the 2-step transfer via Plex in this case.
Go ahead and check that, and test with a video that has a bitrate in the 6-8Mbit range (which should work at a remote location at least in theory).
To put it in very clear terms: If you use Plex to stream the video at a friends house it will NOT transfer data directly from Google, but send it via your Plex server at home. If you didn't use the Plex server then it would however stream the data directly from Google. This may be an acceptable solution in some cases where bandwidth is limited, at the cost of various conveniences.
For reference: Some of the highest quality "4K showcase" material I have in my archive right now has up to 40Mbit streams. (about 17GB for an 55 minutes of video).
This is uncommon in most 4K video as it's a bit overkill for general use, but a fair amount of 4K will probably go above 10Mbit. It's very hard to say what the "average" is because that is completely arbitrary and decided by whoever encoded it.
Got it! It is why smaller files (tv shows) work perfectly fine, they are usually in the 5Mbit. Probably the best way to work around this is to set up rclone, at their location and have it mounted there, and stream directly from Google. Or unless somehow Plex Server can just redirect the data from Google to the remote location, instead of downloading it then upstream it, then it would work? Would something like that be possible (in the future perhaps)?
Well, that solves my confusion. Thank you so much!
Streaming the data directly from Google at the remote location would work fine. Then the only limitation is the local bandwidth (which you say is not at all an issue in this case).
As long as you use a decent video player you can just mount the drive and play it like that. if this is a regular thing and you don't use encryption you could even use "sharing" (Google private) or invite your friend (Google Teamdrive) to do it even more directly. This would allow you to keep your personal access details and admin status private. it really depend on how much you trust your friend I guess
Any data that goes via the Plex server will inherently be limited by the bandwidth the server is on. There just isn't any way to have the data be processed via Plex and also have it send that data out faster than what the upload can support. What you are asking for is extremely unlikely to ever happen as I can't see it being theoretically possible.
If this is a regular use-case you may be best served by sharing your media files with your friend and let him stream it directly at his location. Either via his own Plex - or directly in any good media player via a mount (which is probably a much simpler solution). Then the traffic won't have to go through your network at all and you should be golden.
5mbit stream like you mention should work anywhere via the Plex assuming you have 5Mbit of local bandwidth. 5mbit probably isn't very common for 4K though. Some 720P or maybe even some "medium quality" 1080P will probably fall in that range though (it really depends on the codec as H.265/HVEC is almost 2x as efficient as the still common H.264/AVC)
So another solution is perhaps VPS, to work around the upload bandwidth limit? I don't know much or anything about them. Do they themselves have a transferring/bandwidth limit?
Do you actually mean VPS (Virtual Private Server) or do you mean a VPN (Virtual Private Network).
A VPS will of course fix the bandwidth issue, as if you hosted your Plex server in the cloud then it would have huge bandwidth - but it will necessitate some additional costs from the outgoing bandwidth. Not necessarily a lot all things considered, but it can't realistically be done for free. If you are a relatively light user then it might just be a buck or two a month though.... so it's certainly a viable option. Using Google Cloud as a reference - outgoing bandwidth may be as cheap as 0.01USD pr. GB inside the same continent. So yea - even a 60GB movie won't really cost a lot if it's 60cents to stream it, but it still a cost...
A VPN (simulating that the two devices are in a local LAN) won't help you. It will still need to send the data via your upload, so that won't solve anything in this case.
I've worked a fair amount with Google cloud VM servers, so let me know if you want help on that.
You can basically run a micro-instance for free given certain (very reasonable) limitations, but your upload demands for Plex would necessitate an expense. You can however very easily get a free 1-year trial with 300USD worth of credits to spend, so effectively you can run a test-server for free for a year and see how you like it - then past that point you may need to spend a couple of bucks a month to keep it running. GCP is surprisingly cheap all things considered, and very flexible to your demands (ie. you can upgrade the hardware as needed and you just pay for what you use).
