I had some force closed connections issues. Basically OneDrive was closing the connection, forcing rclone to reconnect and slowed everything down. I always use the --disable-http2 flag for that reason and have experienced a way more stable connection.
1 Like
I was copying a couple of media files that I know cause freezing (one one of them it happened repeatedly this morning) from OneDrive to the server storage itself. The purpose was going to be to play the files from the server, to see if the connectivity between OneDrive and server was to blame.
During the rclone copy, I noticed the transfer rate was generally around 35 to 40MB/s, but occasioanlly dropped to 10MB/s. I thought this would cause freezing (if freezing is a network connectivity issue which I assume at this point it is) on some media files, however it would have to persist for several minutes as I have restored the vfs-read-ahead 2G flag.
Never mind, the killing blow came near the end of one of the files; the rate went down to 5MB/s and it stayed there for 10-15 minutes, before going back up again. No amount of vfs-read-ahead that I can possibly use will cure that.
There is no mention of anything odd in the logs. So it seems the rclone transfer randomly drops to below the required rate for these files, hence the freezing happens - this makes good sense.
Now, since no mention exists in logs, can I assume this is not throttling but just some connectivity issue? And if so, is there any combination of rclone mount settings I have not yet tried that II can use to mitigate it?
So after oh-so-many variations in the rclone config and tests, I have concluded that
is the answer.
Just to clarify, there seem to be no indications whatsoever in the logs about any throttling (no pacer messages or anything like that).
The issue was media files over a certain bitrate freeze up randomly.
From running some copying commands, I noticed that while the transfer starts at a bitrate that can support these files (let's say 10 to 15Mbytes/s), it often drops down to below 5, which is not suitable for these big files. I did see it stay at such low levels for maybe 20 minutes at some point, which would indeed mean that any cache I may have set up would have been used up by that point.
OneDrive works fine for files below a certain bitrate. So far I am seeing that anything 4Mbyte/s and below works fine, so I have moved everything above that to alternative storage (iDrive which seems to be working perfectly so far).
Thank you to all who helped debug this issue!
1 Like
It might be a network routing issue. I know my home has great routing to Azure and I easily get like 40MBps, yet when I tested doing some offsite backups for a hosted server, I noticed it never goes above 20MBps and sometimes dip into low 4-5MBps as you see.
Also, it does seem OneDrive is not optimized for high speed throughput as I've never got even close to saturate my gbit line.
1 Like
This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.