My upload to Backblaze B2 is consistently very slow. I see speeds around 100-150 kilobytes per second although I have very reliable 50 mbps upload / 50 mbps download connection. I am located in the Czech Republic. Is this normal behavior? See the attached screenshot of running rclone with -v option.
What’s the command line you are running?
rclone version shows?
The command line was:
rclone -v sync /media/extHD/_sklad/backup/frantisek_borg/bin b2:borgbackup-frantisek/bin
Rclone version is 1.36 on Xubuntu.
That version is about 2 years old. Can you retry on the latest version? 1.46
Thanks.
You could try this which tests your bandwidth to backblaze directly
https://www.backblaze.com/speedtest/
Here is what I got on my 50Mbit/s down / 20 Mbit/s up connection in the UK
For me, this test shows over 50 / 50 mbits, i.e. the full speed of my connection.
I have updated to the latest version. Some files are now uploaded at around 1.5 megabytes/s (see the new screenshot), which is however still very slow (it should be over 6 megabytes/s). I am not sure if this was also the case before. Most files are still uploaded at only 100-150 megabytes/s (as with the old version).
Running lots of little files or larger files?
If I understand the rclone log correctly, in screenshot 1 it’s a single large file (very slow upload), in screenshot 2 it’s 7 files with total size of 53 MB (slightly faster upload).
I wonder if the slow is the checksum going on.
Can you run it with -vv on it and see if the debug log helps out?
Here is the screenshot of the output with -vv. Also note the processor load on my PC doesn’t show any increase during the upload.
That looks like the Backblaze side is really throwing errors on you. Not much to do other than try their support or another node? I’m not familiar with Backblaze but doesn’t seem to be local or on the rclone side from that error message.
That makes it more likely that it is a problem with b2 being overloaded.
Those 503 errors are indicative the b2 server is too busy.
Can you try at different times of day to see if that make a difference?
I tried uploading a 1GB file on a 1 Gbit/s connection - the lowest speed was about 240 kByte/s whereas it peaked at about 8MByte/s for an average of <2 MByte/s so very variable.
Unfortunately, this occures constantly for at least a month. Time of day does not matter.
It might be worth asking backblaze support about it - see what they say.
I’m having similar issues with Backblaze B2 from Western Australia, but I don’t seem to have any 500 errors from my testing.
I’m running rclone v1.46 on Windows Server 2008 R2, using rclone sync with default settings on a folder full of folders of photos and videos with file size ranging from 100’s of kilobytes to 2 gigabytes. I’m not able to get more than 200 kBytes/s out of my sync, even after running for almost 24 hours.
My server has an ethernet connection into the modem/router on a 50Gbit connection and I get great readings from the Backblaze speed test (and speedtest.net for that matter):
I’ve run it with -vv and still nothing stands out:
I have more images to post but the forum is limiting me to 1 image per post and a maximum of 3 posts per topic as a new user.
I’ve tried playing with the --b2-upload-cutoff size, --b2-chunk-size, and even set --transfers 1 but I only get worse performance.
Can someone spot an error on my part and/or offer me suggestions? I have a movie folder with much larger files once this Photo folder is uploaded.
Lots of small files (eg images) will take longer to upload - try increasing --transfers to 32. (Note that rclone may allocate buffers of 32 * 96M so make sure you have enough memory!)
I note your ping time is quite large which means that it will take transfers a while to get going and take a long time between command and response, all of which are working against you for small files. Bigger files should be better.
1 Like
Thanks for pointing me in the right direction, and the speedy response (I really appreciate it). Increasing the number of transfers does increase the speed and I’m now getting 1Mb/s (5 times as fast).
I’ll continue to monitor and tinker to see if things get better or worse.
1 Like