Insane buffering lately?

Right, but if you are buffering, one of those isn't working.

If you can see memory/disk/CPU, it's usually networking, which you really can't do much on.

Located the problem

A system update to ubuntu 16.04 enabled ipv6 .. THAT was shit.. disabled it, rebooted.. working flawlessly now :slight_smile:

Thought i had it solved.. Damn :frowning: stil buffering

How can I test if a drive is not working properly? Lets say its a drive mallfunctioning?

I'm on Arch, but something along these lines:

https://wiki.archlinux.org/index.php/S.M.A.R.T.

1 Like

This morning - Im all alone on the server. Started up a 4K UHD movie - runs PERFECTLY no buffering. LOAD average on the server is 1.55 . No transcoding, subs enabled etc . I can't say how smooth its running, but its flawless.

Checking logs for the mount - empty .. Nothing.. all good.

Its .. really frustrating not knowing what in the world is going on at afternoon, when users are using the server. I dont see any bottleneck on the cpu at all. HW transcoding is enable, and running , testet via intel_gpu_top , running and chewing on the trancodes for those who are transcoding. A lot of memory available, free space on the NVME drives in raid0 .

Im running out of ideas . The ONLY thing I can imagine which causes the problem, is Google throttling the out going traffic at night.

Is there any chance that we might can see that from other users? How can I / we test to see if thats the case?

I tried rolling back to rclone 1.47 and downloaded the exact same file from gdrive . Here is the new speed:

48,473,708,384 100% 95.24MB/s 0:08:05 (xfr#1, to-chk=0/1)

Google isn't throttling anything. Again, if you are out of things, you are most likely having network congestion along the way or out of Hetzner. Not much you can do to fix that.

Ok status - Closed the server, moved server to Hetzner Germany . No problems what so ever. :slight_smile:

hello, are you 100% sure that they use was caused by Hetzner Finland? I'm having a lot of buffering issues since 2 weeks ago and I really don't know what can I dot to fix it

I moved the server to Germany, seems to ve running ok after i rolled back to mergerfs version 2.25.

@Morphy - There was a bugged version of mergerfs that was an issue. If you are on 2.25, that should work fine, but also the latest should work as well:

I'm using this version:

mergerfs -V
mergerfs version: 2.28.1
FUSE library version: 2.9.7-mergerfs_2.28.0
fusermount version: 2.9.9
using FUSE kernel interface version 7.29

with the setting:

async_read=false
root      1356     1  1 Jul11 ?        01:22:28 mergerfs /data/local:/GD /gmedia -o rw,async_read=false,use_ino,allow_other,auto_cache,func.getattr=newest,category.action=all,category.create=ff,func.getattr=newest,dev,suid 

That's the full output of a ps command and my service file looks like:

[felix@gemini system]$ cat gmedia.mount
[Unit]
Description = /gmedia MergerFS mount
After=gmedia-rclone.service
RequiresMountsFor=/data

[Mount]
What = /data/local:/GD
Where = /gmedia
Type = fuse.mergerfs
Options = async_read=false,use_ino,allow_other,auto_cache,func.getattr=newest,category.action=all,category.create=ff,func.getattr=newest

[Install]
WantedBy=multi-user.target
1 Like

Thanks Ani :slight_smile:

I tried the latest version earlier today, but NOT with async_read=false .

I had some buffering going on, so i tried the 2.25 which seemed to solve my buffering.

What does the setting do to mergerfs mount?

Going to test a bit more and thanks for sharing your service details :slight_smile:

This is for the sync_read option so as rclone is by default compiled with sync_read so they need to match.

I use this to allow hard links.

This is similar to allow_other in rclone as it allows other users to access the mount.

This uses some extra memory for caching. Not needed but if you have spare memory, it doesn't hurt.

This is so it always picks the latest file as I move files around at times and this ensures nothing goofy gets cached.

This is to make it always write to the first disk listed in the mount command, which is my case is my local disk. I do this to keep my rclone mount read/write as I upgrade media and I want to be able to delete from the mount.

1 Like

I'm on mergerfs 2.28.1, how do you roll back to 2.25?

And how do you know if it was rolling back to 2.25 or switching to Hetzner Germany what fixed your buffering issues?

@soy_titooo - I'm on mergerfs 2.28.1 as well. If you share your options, we can rule that out of the mix.

Thanks for the help Ani :slight_smile:

@soy_titooo

I can see the mergerfs version in ssh by typing mergerfs -v

When i moved, i could see the difference instantly. I had a bit buffering yesterday, so i went back to 2.25 by downloading , unpack etc.

Stopped my mergerfs service and started it up again. Didnt see any buffering on the movie I had problems with earlier same day.

Hi again Ani .

Just a quick question , I see that you have rw just before async? But I dont see your rw in the service file? :slight_smile:

1 1 Jul11 ? 01:22:28 mergerfs /data/local:/GD /gmedia -o rw,async_read=false,use_ino,allow_other,auto_cache,func.getattr=newest,category.action=all,category.create=ff,func.getattr=newest,dev,suid

Also you have func.getattr=newest in the service two times?

I'm using the rclone mount options that you have on github for gmedia-rclone.service
I have been using them for months without issues (whenever I see you updated them in github I do the same on my server), so I don't think that's the problem.

I'm more inclined to think that it's an issue with the Finnish servers of Hetzner or routing the traffic from those servers through Cloudflare.

It's quite difficult to find out what's wrong as sometimes the playback is perfect, but others it buffers several times within the first minute, which makes everything unwatcheable

I will try disabling cloudflare as my friends in USA don't have any problems despite being way more far than Finaland but those in Europe have problems (including myself). It shouldn't be necessary as it has been working for months, but maybe something regarding bringing the traffic from cloudfare servers in USA back to Europe is causing the buffering issues.

I really don't know, I'm just talking out loud, lol

Thanks @soy_titooo

Well - I did a complete reinstall on the Finnish server - but the problem was still there.

Yesterday I read in :

Saw that they were posting about mergerfs it was buggy, or at least the performance was bad.

So I have for now - tried to roll back to mergerfs 2.25 and havent seen any issues so far.

Still testing and keeping an eye on this :slight_smile: .

Want to try the latest though, but had some minor questions for Ani . Seems like there are double options in his service file as I can see . Was just curious if that is a mistake or not.

I tried to run with his options on 2.25 but the service wouldnt start.

I will run it on a test server later tonight :slight_smile: