Cache config for Plex mount

Hey,

I am going now to take log out and paste them in here, I was just making some tests, and I found the following:
Just by making a simple cp in linux from the mount, I get the following speed with the mount options I pasted above:

And when I make an rclone copy, I hit the following speeds:

hmmmm that's weird, I was hitting yesterday 100MiB/s, when running the rclone copy command :confused:

Let me go and get out the logs now

2022/07/08 13:30:52 DEBUG : rclone: Version "v1.58.1" starting with parameters ["rclone" "copy" "-P" "--log-level" "DEBUG" "--log-file" "/home/pedro/rclone.log" "encryptteam:/peliculas/Aguas oscuras (2019)/Aguas oscuras (2019).mkv" "/home/pedro/"]
2022/07/08 13:30:52 DEBUG : Creating backend with remote "encryptteam:/peliculas/Aguas oscuras (2019)/Aguas oscuras (2019).mkv"
2022/07/08 13:30:52 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2022/07/08 13:30:52 DEBUG : Creating backend with remote "team:/ia6p26kv83nbj49srnrjfgrtf8/qspdltkcvt68grulili4hj1k245s84op2i92labau1u808hpb520/qefjuvgge2dia3ah8fet7d954rqrgmmcepoqrbpppkid0d75jomg"
2022/07/08 13:30:54 DEBUG : fs cache: adding new entry for parent of "team:/ia6p26kv83nbj49srnrjfgrtf8/qspdltkcvt68grulili4hj1k245s84op2i92labau1u808hpb520/qefjuvgge2dia3ah8fet7d954rqrgmmcepoqrbpppkid0d75jomg", "team:ia6p26kv83nbj49srnrjfgrtf8/qspdltkcvt68grulili4hj1k245s84op2i92labau1u808hpb520"
2022/07/08 13:30:54 DEBUG : Creating backend with remote "/home/pedro/"
2022/07/08 13:30:54 DEBUG : Aguas oscuras (2019).mkv: Sizes differ (src 38279570850 vs dst 28727181312)
2022/07/08 13:30:54 DEBUG : Aguas oscuras (2019).mkv: Starting multi-thread copy with 4 parts of size 8.913Gi
2022/07/08 13:30:54 DEBUG : Aguas oscuras (2019).mkv: multi-thread copy: stream 4/4 (28709683200-38279570850) size 8.913Gi starting
2022/07/08 13:30:54 DEBUG : Aguas oscuras (2019).mkv: multi-thread copy: stream 2/4 (9569894400-19139788800) size 8.913Gi starting
2022/07/08 13:30:54 DEBUG : Aguas oscuras (2019).mkv: multi-thread copy: stream 1/4 (0-9569894400) size 8.913Gi starting
2022/07/08 13:30:54 DEBUG : Aguas oscuras (2019).mkv: multi-thread copy: stream 3/4 (19139788800-28709683200) size 8.913Gi starting

That's the log I have from the rclone copy command
Although I remember that the log from the mount was pretty much more extensive

oki, I think it is not being a good day to test, because I am getting ridiculous speeds from google drive, on whatever account I try... And my internet speed test is maxing to 1gbps on the internet tests :confused:

Are we trying to troubleshoot a mount or a copy command? That log is from a copy command and you were speaking about a mount.

The copy is a multi threaded operation and the mount isn't so you can't compare the two directly.

What problem are we trying to solve?

I am just trying to troubleshoot why my mount speed is to slow compared to the copy command.
I usually see 10mbps on the mount speed, and 1000mbps on the copy one, which is a lot.
But now it seems I am having some sort of limitation, since I am not able to download things at good speed, not even from GD at chrome, from my personal computer, don't know whats going on at the moment TBH.

Once I see my connection is restored back to normal I can go ahead with the testing, under normal conditions

There's a whole thread about Google Drive / slow speeds.

Rclone mount random slow speeds - Help and Support - rclone forum

Might be worth checking that out, albeit with a mount log file, I can't tell if you are hitting that issue or something else as your API screenshot looks like quota.

Yep, I can get that mount log, however since the google drive speed to my home is slow at the moment, I do not want to bother you with those non realistic logs :sweat_smile:

Hey @Animosity022 @asdffdsa

Here are the logs of the cp command from the mounted drive: 2022/07/09 20:03:01 DEBUG : /: Lookup: name="peliculas" 2022/07/09 20:03:01... - c94e5dc6

It seems my connection is back working as normal, and with rclone copy, I am hitting the 1gbps that I have with my ISP, however with cp command not even hitting 20mbps.

With cp command:

With rclone copy command:

As you can see the difference is huge.
Don't know if you will be able to see anything in the logs, but I am not seeing anything strange :sweat_smile:

Hello,

Have been checking these last days, and it seems with CP command, or when Plex tries to read a file from GD, sometimes, my network maxes, and sometimes it gets stuck in 1.6MiB/s as you can see in the post above.
It seems to be an sporadic behaviour, and no errors showing in the logs as I have shown. As I say, sometimes it goes bad, I restart the movie, or the cp command, and then it goes great, or maybe I need to restart 2 times for it to work.
Any ideas? Thanks in advance

See:

Please read that thread.

Hey man, thanks for that, my issue was related and seems now to be solved.
I have already posted on that thread :slight_smile:

Just wanted to know about if something is being developed according to my comment in that thread:

"At the same time it would be nice, was people was talking before, any option to full cache the entire movie, once it is requested, don't know if placing --vfs-read-ahead 200G would fully cache the entire file while watching it, although it would not be multithread, and would be nice for it to be coded as such.
Or at the same time, to fully download the movie prior to the playback of it, in a multithread way also. People was talking about an rclone version that was doing such, but any idea on how?"

Hey @Animosity022

I had no issues with the speed now, but I came with the following in the logs:

Read: read=0, err=open file failed: googleapi: Error 403: The download quota for this file has been exceeded., downloadQuotaExceeded

However, when I get into google console, I see cuotas are not reached:

Any idea?

That means you ran out of the daily download quota. I way to see that as you just have to wait for the daily reset.

mmm, but thats weird I just watched yesterday 1 single movie :confused:
And why in google console, are the quotas at 0%? Shouldn't they be showing at 100% if all are already consumed?

No as you can't see your daily upload or download quota anywhere from Google unfortunately.

The item you can see in API quota which is impossible to run out of.

So, the item I am just seeing is the quota requests, that rclone makes each time it makes a call for a chunk, right?
And that would almost be impossible to run out off, right?

So, then the limit I reached, is due to? Many GB downloaded a day? Is there anything I can tweak from rclone not to reach this limit? I can not understand why just watching a movie made this limit to be hitted :confused:

I saw in another post @ncw saying the following:

"That message says "the download quota has been exceeded" - that is google's way of telling you that you've downloaded that file too many times and you'll have to wait 24h before you can download it again I think."

Maybe each chunk call is counted also and decreasing the number of calls could fix it?

No. It is not as that's not it. You can't really tune around it as you aren't sure what the "it" is as Google won't tell you.

One item doesn't cause you to hit download quota as there is generally more than that but without seeing your full setup, it's really tough to guess.

Google's download and upload limits are really shared other than the 750GB upload per day. Every other download limit is just a best guess based on what people have seen. Certain people see different things and it depends if it's edu account, shared drive, etc.

Google Support also won't tell you anything other than try after 24 hours and download less.

Yep, I knew about the 750GB upload limit, however I usually tend to bypass it with different group accounts.
On the download size, it's weird, because although I was doing some testing yesterday to see if the IP thing solved my issue, which it did, I do not think I downloaded huge amount of data, just to even hit 500GB of download, and my setup, apart from the IP thing, has not changed. As you can see my rclone config is still the one I pasted above...
So to be honest, no clue. Well... Sorry, my mistake, there is a change indeed, I swapped from edu account to workspace one, don't know if the limits are different in there

There are multiple download limits as most folks believe there is a general overall limit per day, limits per file, shared limits for file, shared drives seem to work different.

It's really impossible for me to guess what non documented limited you hit.

For plex, most folks turn off intro analysis and lots of things to ensure extra stuff doesn't run.