Slow transfers in unRAID

Suspected Bug
1

I recently built an unRAID box and am having a bizarre issue with rclone. I have a 400Mbps downstream connection that I can easily saturate pulling from an unencrypted Google Drive on macOS or Ubuntu and pushing to unRAID via SMB or NFS, respectively. When running within unRAID, though, each transfer is only able to top out at 20Mbps?!

Even more strange is that if I fire up a vanilla Ubuntu docker image it has the same, slow, results, but if I setup an Ubuntu VM on the unRAID box it’s able to run at full speed!

I’ve had this behavior using both rclone 1.42 and 1.43, both installed via a plugin and downloaded directly to the machine.

My use case is syncing from Google Drive to unRAID to have a local copy of everything, and while I find myself hitting occasional rate limits, this behavior occurs even if I simply tell it to copy a single file.

For the life of me I can’t figure out what might be up. I suspect it might be a kernel issue in unRAID given that the full isolation of the VM fixes is, but I’m basically pulling that out of my butt.

2

Can you share what you are using for the command line and some output with -vv set?

One thing that GD is a bit annoying is the limiting needs a bit better documentation so the defaults sometimes are a bit much and cause rate limiting, which makes things much slower.

3

Here’s the copy command from unRAID:

root@Brontostorus:~# rclone --version
rclone v1.43.1
- os/arch: linux/amd64
- go version: go1.11
root@Brontostorus:~# rclone copy gmedia:"Media/Movies/2 Fast 2 Furious (2003)/2 Fast 2 Furious (2003).1080p.bluray.mkv" . -vvv
2018/09/08 09:50:06 DEBUG : rclone: Version "v1.43.1" starting with parameters ["rcloneorig" "--config" "/boot/config/plugins/rclone/.rclone.conf" "copy" "gmedia:Media/Movies/2 Fast 2 Furious (2003)/2 Fast 2 Furious (2003).1080p.bluray.mkv" "." "-vvv"]
2018/09/08 09:50:06 DEBUG : Using config file from "/boot/config/plugins/rclone/.rclone.conf"
2018/09/08 09:50:08 DEBUG : 2 Fast 2 Furious (2003).1080p.bluray.mkv: Couldn't find file - need to transfer
2018/09/08 09:51:08 INFO  :
Transferred:   	  139.996M / 29.114 GBytes, 0%, 2.240 MBytes/s, ETA 3h40m49s
Errors:                 0
Checks:                 0 / 0, -
Transferred:            0 / 1, 0%
Elapsed time:      1m2.5s
Transferring:
 *      2 Fast 2 Furious (2003).1080p.bluray.mkv:  0% /29.114G, 2.361M/s, 3h29m28s

2018/09/08 09:52:08 INFO  :
Transferred:   	  279.996M / 29.114 GBytes, 1%, 2.285 MBytes/s, ETA 3h35m21s
Errors:                 0
Checks:                 0 / 0, -
Transferred:            0 / 1, 0%
Elapsed time:      2m2.5s
Transferring:
 *      2 Fast 2 Furious (2003).1080p.bluray.mkv:  0% /29.114G, 2.299M/s, 3h34m7s

2018/09/08 09:53:08 INFO  :
Transferred:   	  420.996M / 29.114 GBytes, 1%, 2.307 MBytes/s, ETA 3h32m21s
Errors:                 0
Checks:                 0 / 0, -
Transferred:            0 / 1, 0%
Elapsed time:      3m2.5s
Transferring:
 *      2 Fast 2 Furious (2003).1080p.bluray.mkv:  1% /29.114G, 2.348M/s, 3h28m35s

Here’s the copy from an Ubuntu VM running in unRAID:

john@plex-beta:~$ rclone --version
rclone v1.43.1
- os/arch: linux/amd64
- go version: go1.11
john@plex-beta:~$ rclone copy gmedia:"Media/Movies/2 Fast 2 Furious (2003)/2 Fast 2 Furious (2003).1080p.bluray.mkv" . -vvv
2018/09/08 15:57:37 DEBUG : rclone: Version "v1.43.1" starting with parameters ["rclone" "copy" "gmedia:Media/Movies/2 Fast 2 Furious (2003)/2 Fast 2 Furious (2003).1080p.bluray.mkv" "." "-vvv"]
2018/09/08 15:57:37 DEBUG : Using config file from "/home/john/.rclone.conf"
2018/09/08 15:57:40 DEBUG : 2 Fast 2 Furious (2003).1080p.bluray.mkv: Couldn't find file - need to transfer
2018/09/08 15:58:40 INFO  :
Transferred:   	    2.034G / 29.114 GBytes, 7%, 33.295 MBytes/s, ETA 13m52s
Errors:                 0
Checks:                 0 / 0, -
Transferred:            0 / 1, 0%
Elapsed time:      1m2.5s
Transferring:
 *      2 Fast 2 Furious (2003).1080p.bluray.mkv:  6% /29.114G, 35.142M/s, 13m9s

2018/09/08 15:59:40 INFO  :
Transferred:   	    4.140G / 29.114 GBytes, 14%, 34.586 MBytes/s, ETA 12m19s
Errors:                 0
Checks:                 0 / 0, -
Transferred:            0 / 1, 0%
Elapsed time:      2m2.5s
Transferring:
 *      2 Fast 2 Furious (2003).1080p.bluray.mkv: 14% /29.114G, 35.892M/s, 11m52s

2018/09/08 16:00:40 INFO  :
Transferred:   	    6.211G / 29.114 GBytes, 21%, 34.837 MBytes/s, ETA 11m13s
Errors:                 0
Checks:                 0 / 0, -
Transferred:            0 / 1, 0%
Elapsed time:      3m2.5s
Transferring:
 *      2 Fast 2 Furious (2003).1080p.bluray.mkv: 21% /29.114G, 35.472M/s, 11m1s

The Ubuntu transfer is ~15x faster!

4

So for the dumb questions, but to make sure, the unRAID is connected at 1000 full as well and isn’t hitting any CPU bottlenecks or anything like that? I’m not familiar with unRAID myself as that does seem awfully slow.

5

Yup, running at full speed.

root@Brontostorus:~# ethtool eth0
Settings for eth0:
	Supported ports: [ TP ]
	Supported link modes:   10baseT/Half 10baseT/Full
	                        100baseT/Half 100baseT/Full
	                        1000baseT/Full
	Supported pause frame use: Symmetric
	Supports auto-negotiation: Yes
	Supported FEC modes: Not reported
	Advertised link modes:  10baseT/Half 10baseT/Full
	                        100baseT/Half 100baseT/Full
	                        1000baseT/Full
	Advertised pause frame use: Symmetric
	Advertised auto-negotiation: Yes
	Advertised FEC modes: Not reported
	Speed: 1000Mb/s
	Duplex: Full
	Port: Twisted Pair
	PHYAD: 1
	Transceiver: internal
	Auto-negotiation: on
	MDI-X: on (auto)
	Supports Wake-on: pumbg
	Wake-on: g
	Current message level: 0x00000007 (7)
			       drv probe link
	Link detected: yes

Is there anyone out there with an unRAID installation that could verify with a similar test? It’s using Waseh's rclone plugin, which just adds a wrapper for the config file (though same results when downloaded directly).

Is there any low level debugging I can do, maybe with a beta or debug build?

6

Well I guess it’ll remain a mystery. Tried the same test after upgrading to unRAID 6.6.0 and speeds seems to be what I’d expect.