Let me know what SSDnode’s says. Because I’ve been seeing this behavior for about 2 months Sometimes I can restart the rclone service, plex, radarr, and sickrage and some days everything will behave other days not so much.
will do. I complained to them in February and this was there response
Hello,
We are currently experiencing Virtuozzo bugs on our KVM platform and are working with Virtruozzo to get it resolved. If you are experiencing issues with the network or managing your server in the dashboard, it is due to these bugs. We apologize for any inconvenience in the meantime.
I then complained again in March and they said
Thank you for writing in.
It looks like the server got throttled due to high resource usage. Can you please adjust the software settings so that it does not use high resources for a prolonged time.
Here is our resource usage FAQ.
https://www.ssdnodes.com/manage/knowledgebase.php?action=displayarticle&id=13
I have removed the throttling and rebooted your server. Can you check again now?
I responded that there is no way that I should be throttled as all I am running is a Plex server and i’m not even using it as a primary as the performance is too poor. They said that they had unthrottled it and now it was speed testing as follows
root@ssdnodes-server1:~# speedtest-cli --server 8706
Retrieving speedtest.net configuration…
Retrieving speedtest.net server list…
Selecting best server based on ping…
Hosted by Cable One, Inc. (Dallas, TX) [1953.54 km]: 6.798 ms
Testing download speed…
Download: 1555.80 Mbit/s
Testing upload speed…
Upload: 656.64 Mbit/s
I have never been able to achieve those results when I test it.
SSDnodes response to my support ticket where I asked them to check on network connectivity and relayed my issues with poor upload performance.
"Hello,
The speed test result depends on a number of factors including the test server that is selected and its location.
Here is the result of 2 tests that I ran almost at the same time and the result shows a lot of change. When we select a server without specifying the location, it selects a server in California whereas your server is in Dallas. Please note the result of the second test that clearly shows the difference.
root@ssdnodes-server1:~# speedtest-cli --server 8706
Retrieving speedtest.net configuration…
Testing from Strasmore (104.225.218.00)…
Retrieving speedtest.net server list…
Selecting best server based on ping…
Hosted by Cable One, Inc. (Dallas, TX) [1953.54 km]: 50.37 ms
Testing download speed…
Download: 65.59 Mbit/s
Testing upload speed…
Upload: 122.88 Mbit/s
root@ssdnodes-server1:~#
root@ssdnodes-server1:~# speedtest-cli --server 11207
Retrieving speedtest.net configuration…
Testing from Strasmore (104.225.218.00)…
Retrieving speedtest.net server list…
Selecting best server based on ping…
Hosted by Sprint (Dallas, TX) [1953.54 km]: 13.984 ms
Testing download speed…
Download: 1044.49 Mbit/s
Testing upload speed…
Upload: 360.19 Mbit/s
root@ssdnodes-server1:~#"
Basically blowing me off
I’m noticing the same symptoms but on a real-Hetzner instance when I switched from plexdrive to rclone-cache.
Sometimes pressing play takes a while, it is prone to stutter more, and even the screen where you press play on can at times timeout / need to retry (which i just find strange)
I ended up emailing Matt, the CEO as I have always found him excellent to deal with.
His response is that it could still be the effects of the Spectre and Meltdown patches and then Virtuozzo playing catchup with their software to deal with the knock on performance issues.
His email below.
Hi Jude,
Thank you for reaching out to me via email. Sorry to hear you’re experiencing issues with your server.
This may be still related to some of the Virtuozzo bugs we’ve been experiencing. We actually received an email from the Virtuozzo CEO earlier today. The email itself is long, but I want to show you one part of it:
“Finally, we continue to work to address the impact of the Spectre and Meltdown hardware-based vulnerabilities. Our priority is to ensure our customers have the latest information on the fixes and potential impact to your business. We know one of the biggest concerns has been around the performance issues associated with these fixes. We have been addressing the vulnerabilities proactively by sending email updates to subscribers. We recently released kernel with retpoline for Virtuozzo 6 which improves performance to near pre-Spectre level. A similar update for Virtuozzo 7 will soon be available. Look for additional email communication soon that will detail the results of this update.”
(We are using Virtuozzo 7).
You mentioned experiencing these issues for the past few months, it would coincide with the Spectre/Meltdown mitigations that were released in January/February. We are working hard to get these resolved.
–
Matt Connor
Founder & CEO
Strasmore, Inc.
I am still dealing with their tech support. They want to be able to reproduce the problems that I am seeing. I’m not sure what the best way to do this would be.
Hi @Linhead. Thats pretty much what I’m seeing on my VPS although not stuttering as much as freezing for extended periods. My local server (running Rclone cache works without fault).
@mjmayer can I suggest that you open a support ticket with SSDnodes and let them know the issues you are having. It might also be helpful to to reference my ticket number 396020
I think that it will add weight to the issues if a number of us are reporting these problems
@Animosity022 I’m not sure how to test for bandwidth issues either.
I just ran a pretty informal test for download speed.
1 Using Midnight Commander GUI file manager. A simple copy from my decrypted Rclone Cache to /tmp. Watching the transfer in iftop I see 5 or 6 connections to GoogleDrive and it very quickly gets to a consistent 330 Mbps and stays there until the transfer (a 2GB file) is complete.
2 I copy a different file from the same directory to /tmp but this time I use the following command.
sudo rclone copy local-crypt:/SSDnodes/ssdnodes-server1.2017-10-29.tar.bz2 /tmp
Again watching in iftop I see one (possibly two but only briefly) active connections to GoogleDrive. The speed fluctuates wildly going up to 200Mbps but then dropping as low as 20 (mostly below 60). This continues until the transfer is complete. I didn’t time the transfer but it took a lot longer for a similarly sized (2GB) file.
Shouldn’t the two copy functions work the same? Is this a bug in Rclone?
Sorry as I wasn’t quite clear.
I’m good on how to test speeds of a file copy but if the provider is throttling your VPS, how would you know that without them telling you? That would be seen as inconsistent speeds and odd things happening which it looks multiple users of that service are reporting.
I have a bit more control over my connection so I can validate via ntopng connections and what else is consuming my connection. I can run the config I shared streaming multiple 4K streams without stuttering or freezes in the frames.
Right but don’t you think its odd that using rclone copy i’m only getting one download stream from GoogleDrive and then with copy in Midnight Commander I’m getting four or five times that?
I’ve all but given up on my SSD Nodes VPS. They are overselling their servers & performance over time deteriorates then when you complain they do a fix which works for a bit then the performance starts to decline all over again. I have never seen the network bandwidth remotely approach the promised 1Gbps. I paid up front for 12 months of an X-Large VPS but will not be renewing asI think the performance is crap & their overselling deceitful.
Who are you planning to use instead?
Not strictly a VPS but a seedbox with Bytesized Hosting. There is a nice web interface for setting up Plex Sonarr, Deluge etc & you are spared most of the detail of server admin but best of all the performance is great & support exceptionally good https://bytesized-hosting.com/plans
@jasanson - if someone else can chime in, I thought rclone copy just runs parallel copies so if you copy 1 file, only a single connection is used. If you are using Midnight Commander or something else, I would think it might read ahead based on your cache-workers configured assuming you are using a cached mount so you’d see more connections.
Here is the response from SSDNode regarding networking performance. I started streaming a movie shortly after this change and it played with little delay or stuttering. But correlation is not causation.
Hi Michael,
Here are Virtuozzo’s own test results showing the performance impact of implementing the security fixes:
We are running their latest code with the fixes included. They have promised improvements in the future that, >apparently, walk back most of that performance loss. We will continue to deploy updates as we receive them from >Virtuozzo.
We have been working very hard on these issues in recent months. That has included an ongoing Skype chat between >myself and other SSD Nodes people with Sergey Maximov, author of the article I referred to above, and senior >Virtuozzo people up to and including their CTO.
I have checked the configuration of your VM and made an adjustment to the configuration of your network interface (it >was previously set to emulate an Intel e1000 adapter, it is now set to virtio which will likely perform better).
Here are the speedtest results from your VM:
Retrieving speedtest.net configuration…
Testing from Strasmore (ip omitted)…
Retrieving speedtest.net server list…
Retrieving information for the selected server…
Hosted by Sprint (Dallas, TX) [1953.54 km]: 10.214 ms
Testing download speed…
Download: 1207.82 Mbit/s
Testing upload speed…
Upload: 325.30 Mbit/sWhile these figures vary over time due to network load, on this instance your VM achieved a download speed of >1.2GBps and upload of 325MBps. That is a very good result.
We have no visibility as to the workings of Google drive so we can’t comment on any issues arising from use of that.
I hope this is helpful.
Daniel.
@mjmayer thanks for that. I’ll contact SSDnodes and request a change in virtual adapter so that I can test that.
I went to watch another movie tonight and it took forever to start and buffer. From the network traffic, it looks like rclone isn’t even trying to pull down the data. Restart rclone and plex. Then everything seems ok again. I’m still not convinced this issue has much to do with the hosting provider. I guess its time to turn verbose logging back on and see what I can see.
SSDnodes made similar adjustments to my VPS. Speedtests are faster than they were and it seems like playback of 10-15Mbit media is more stable but it still can’t handle higher bitrate content (20 - 30 Mbit).
SSDnodes is going to work with me on testing from another location but in the meantime I’m going to switch this VPS back to using Plexdrive. Plexdrive seems much more stable for higher bitrate content playing off of a VPS - for me at least.
My local server will happily play 30 - 40Mbit content using Rclone Cache or Plexdrive.