Datacenter limits packet rate

Help and Support
1

Is it possible to use jumbo frames with rclone if so how?

My datacenter limits me to 80pps before null routing me so I need to find a way around it.

There are about 7 gdrive vfs mounts and a FTP.

Current mount command --allow-other --allow-non-empty --log-file /opt/rclone/logs/mlibrarygdriverclone1.log -v --dir-cache-time 96h --vfs-cache-max-age 12h --vfs-read-chunk-size 32M --vfs-read-chunk-size-limit 5G --timeout 1h gdrive1: /gdrives/GD_1 --user-agent='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)

2

Jumbo frames is a server related LAN / network configuration and not much to do with rclone.

You'd have to enable on your server/computer and ensure that all your things along the way support jumbo frames.

I doubt your going to get jumbo frames working all the way to Google though so I personally wouldn't waste the effort.

What operating system are you using?

3

Ubuntu 18.04 I did see someone mentioned it with Amazon drive?

4

Are you sure it's 80 packets per second? That's almost nothing as 10Mbs is like 15,000 pps.

You can enable jumbo frames on Ubuntu by using something like:

https://linuxconfig.org/how-to-enable-jumbo-frames-in-linux

You also need to enable it on any switches you have and any router and assuming your ISP supports it.

You can't use rclone with Amazon Drive unfortunately as the access was pulled by Amazon quite some time back.

5

pps, as in packets pr second? With a typical MTU of 1500bytes that's just 120KB/sec bandwidth... that can't be right, can it?

As Animosity says it's very unlikely that jumbo packets will make it further than your local network, and unless they can get past your datacenter it won't matter. I don't think that is very commonly used or supported in general infrastructure these days so it seems like a very unlikely solution to the problem.

6

Using --bwlimit will limit the number of pps. 80pps must be wrong though... 83k pps would be 1 Gbit/s (roughly) so you could use --bwlimit 100M to stay well under it.

7

I have set the bandwidth limit to 57 MBps but I can still see that it is using surges to around 960mbps if I try to play something on plex?

My mount command rclone mount --allow-other --allow-non-empty --bwlimit 57M --log-file /opt/rclone/logs/mlibrarygdriverclone1.log -v --dir-cache-time 96h --vfs-cache-max-age 12h --vfs-read-chunk-size 128M --buffer-size 3G --vfs-read-chunk-size-limit 5G --timeout 1h gdrive1: /gdrives/GD_1 --user-agent='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36': &

8

That is 57 MByte/s, you want --bwlimit 7.1M if you want 57 Mbit/s

9

Yes it is correct the server is on a gigabit connection.

10

57Mbyte/s ~= 450 Mbit/s

So 960 Mbit/s is a big high :frowning: It isn't an exact science bandwidth limiting, so I'd expect the long term average to be no higher than 57Mbyte/s. If you add --stats 1m to the mount then it will show what bandwidth rclone thinks it is using every minute in the log which might help to debug what is going on.

I suspect it is the very big 3G buffer which is the problem - I think rclone may be reading that before the bandwidth limiter...

11

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.