HTTP2 Testing with Google Drive

We had this bug with HTTP2 sometime back and if you have a lot of load and use Google Drive, we could use some help in testing and seeing if this is resolved.

If you can write any comments to that issue.

You can turn on http2 by just adding:

--drive-disable-http2=false

To your mount command.

HTTP2 should speed up things so it be nice to get it turned back on by default.

1 Like

After I enable it, what exactly I'm looking for?

If it's working, you wouldn't see anything.

Under load or sometimes randomly, folks would see:

stream error: stream ID 57; INTERNAL_ERROR

In your rclone logs and stuff would lock up (for me anyway) and things would basically go very bad from that point as IO would get hung and never return.

If things are working, you really should not see much difference. You may want to check for better performance as you were trying to throttle things before. HTTP2 should be faster/more efficient than HTTP.

I can change my mount in 12 hours, and then I'll have results in ~8 hours.

I think I'm the biggest rclone user around here so if it works for me it should work for everyone...

I'm curious what you mean by "biggest rclone user around here" :wink:

I will try to test this when I get a chance to unmount.

@Animosity022 Is this the same issue we all tried to figure out back then? The one that would crash the mount. I recall testing several versions, and I thought it was a bug in Go at the time. Has anything changed?

@VBB - That would be the one. We had a few fixes go into the GO libraries and wanted to retest as we think it might be fixed. @ncw did mention to use the latest beta as well but not sure exactly why on that comment.

OK great. I'll help test this by using the flag you mentioned, since I'm always on the latest beta anyway. Will report back.

Oh, just so you are using the latest go runtime.

Which right now seems to be 1.14.2, and I think it was 1.12.x when we were testing back then, no?

That's a negative. Plex scan just locked up about twenty minutes in. Rclone window says "... ReadFileHandle.Read error: low level retry 1/10: stream error: stream ID 2173; INTERNAL_ERROR". Sorry, didn't enable logging, but it's clear that it's not working with HTTP2 :frowning:

So I was just playing with rclone serve http, and I tried enabling and disabling http2 and testing the speed of a transfer between 1 server to another server.

Using http2 didn't cause errors, but made no significant difference on performance given the same chunk size ( both maxed the uploading server 1 gbps bandwidth )

More tests using just rclone mount:

rclone mount 1: /root/testmount --vfs-read-chunk-size 10M --no-modtime -vv (20 seconds = 50.6 MB/s)

rclone mount 1: /root/testmount --vfs-read-chunk-size 10M --no-modtime -vv --drive-disable-http2=false ( 20 seconds = 51.2 MB/s )

rclone mount 1: /root/testmount --no-modtime -vv --drive-disable-http2=false ( 18.5 seconds = 57.8 MB/s )

rclone mount 1: /root/testmount --no-modtime -vv ( 14 seconds = 71.9 MB/s )

benchmark command used:

dd if="/root/testmount/testfile" of=/dev/null bs=1M count=1024 iflag=nocache conv=fdatasync status=progress

So unless I'm doing something wrong, it doesn't seem http2 is affecting performance that much? Would love to see data showing how much and where the performance gains are good. Of course this isn't real world test yet.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.