Performance issue with linux kernel 5.4.0 & rclone mount

Help and Support
1

I ran into a performance issue with rclone v1.50.2 & the new linux kernel 5.4.0

  • rclone mount (google drive, crypt)
  • measure transfer speed of a plain file copy operation (either cp/time, dd, pv)
  • linux kernel 5.3.8 or lower: speed well beyond 300mbit/s up to 900mbit/s (gigabit line)
  • linux kernel 5.4.0: speed max 20mbit/s

same mount command/params, just rebooted with another kernel several times, same result. using "rclone copy" instead of the mount always works as expected with up to 900mbit/s

Can anyone else confirm this behaviour?

Ubuntu 19.04 disco

rclone v1.50.2
- os/arch: linux/amd64
- go version: go1.13.4

rclone mount driveCrypt: /mnt/drive -vv --progress --read-only --allow-other --rc --umask 0222 --attr-timeout 1s --dir-cache-time 672h --vfs-cache-max-age 675h --vfs-read-chunk-size 8M --buffer-size 0 --poll-interval 1m0s --bind MY_IPv6_IP --vfs-read-chunk-size-limit 64M --tpslimit 7 --tpslimit-burst 10 --use-mmap

1 Like
2

There is some discussion of a go and linux 5.4 kernel issue: https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.2-Go-Register-Corrupt

I'm not sure what you are seeing is related and I thought it only came into effect with go-tip, but you've compiled your rclone with go1.13.4

So assuming that that isn't related it looks like a FUSE problem since rclone copy does not show the problem.

Are there any errors in the log file? It might be worth running with -vv to see if there are any clues.

Also give the latest beta a go which has the latest fuse libraries and a number of fuse tweaks.

3

I had some issues with the newer kernels and my goal was stability so I just moved back to Debian and ran the 4.9 kernel for now until everything gets fixed up.

There is a fun part of run bleeding edge, but the problem is finding bugs and I have no need for that. I saw a few posts here and there regarding TCP performance and perhaps that's what you are hitting.

4

I'm seeing similar issues but haven't dug into it any deeper than my plex acting like it's starved for data. Spin for a long time, play a second, Spin for a long time, repeat.

I roll back to kernel 5.3.12 and it's fine.

Edit: I just wanted to let you know that other people that I work with are also seeing the same behavior...rolling back to 5.3.x also fixes it for them.

5

The latest beta has fixed the issue on Linux 5.4

1 Like
6

I ran the same tests again.
There is no noticeable performance loss with the latest beta and linux kernel 5.4.3. :+1:

7

Interesting! What go version is the latest beta on? I wonder if the go team fixed something...

8

rclone v1.50.2-090-g0ecb8bc2-beta

  • os/arch: linux/amd64
  • go version: go1.13.5
9

I don't see anything of particular significance in the changelog: https://github.com/golang/go/issues?q=milestone%3AGo1.13.5

:confused:

10

I guess the answer is in what happened in the linux kernel...

Can you try the rclone version v1.50.2 you tried before to see if it is the linux kernel changes or the rclone changes?

11

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.