B2 - Very slow and single-file uploading. Can't figure out if it's my ISP or B2

Help and Support
1

What is the problem you are having with rclone?

Recently, my weekly upload from my backup server to B2 has been taking days to complete. It seems to be a combination of a slow upload speed, and more importantly, only a single file being uploaded at once. I tried updating rclone from 1.52.3 to the latest 1.53.1 with no resolve.

I'm trying to determine if it's something with my config (which hasn't changed), something with B2, or if my ISP is throttling me? I have FiOS 400/400Mbps.

I have one large bucket that I sync weekly, it's currently about 4.5TB with 90278 objects. About 80-100GB per week is added.

What is your rclone version (output from rclone version)

rclone v1.53.1
- os/arch: linux/amd64
- go version: go1.15

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Ubuntu 20.04.1, 64 bit

Which cloud storage system are you using? (eg Google Drive)

B2

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync /custom_mounts/ backup01_crypt:my_unique_bucket_name --transfers=16 --checkers=24 --checksum --log-level DEBUG --stats=1m --exclude /exclude_dirs/** --log-file=/tmp/upload.log

The rclone config contents with secrets removed.

[backup01]
type = b2
account = XXX
key = XXX
endpoint = 

[backup01_crypt]
type = crypt
remote = backup01:my_unique_bucket_name
filename_encryption = standard
password = XXX
password2 = XXX
directory_name_encryption = true

A log from the command with the -vv flag

It runs at zero for about 15 mins...

2020/10/06 16:07:12 INFO  : 
Transferred:             0 / 84.416 GBytes, 0%, 0 Bytes/s, ETA -
Checks:             25161 / 25161, 100%
Transferred:            0 / 7, 0%
Elapsed time:      12m1.0s
Transferring:
 * backup_proxmox/dump/vz…10_05-06_00_02.vma.zst:  0% /36.275G, 0/s, -
 * backup_proxmox/dump/vz…10_05-07_05_10.vma.zst:  0% /31.063G, 0/s, -
 * backup_proxmox/dump/vz…10_05-08_03_43.vma.zst:  0% /3.311G, 0/s, -
 * backup_servers/docker0…_docker_volumes.tar.gz:  0% /3.545G, 0/s, -
 * media/library/Movies/filename1.mp4:             0% /1.893G, 0/s, -
 * media/library/Movies/filename2.mp4:             0% /1.775G, 0/s, -
 * media/library/Movies/filename3.mkv:             0% /6.555G, 0/s, -

...then will eventually switch to a single file uploading at a really slow speed. I'm used to all files being uploaded at a total speed of about 15-30Mbps.

2020/10/06 16:13:12 INFO  : 
Transferred:        3.719M / 84.416 GBytes, 0%, 3.529 kBytes/s, ETA 41w3d5h28m41s
Checks:             26782 / 26782, 100%
Transferred:            0 / 7, 0%
Elapsed time:      18m1.0s
Transferring:
 * backup_proxmox/dump/vz…10_05-06_00_02.vma.zst:  0% /36.275G, 0/s, -
 * backup_proxmox/dump/vz…10_05-07_05_10.vma.zst:  0% /31.063G, 0/s, -
 * backup_proxmox/dump/vz…10_05-08_03_43.vma.zst:  0% /3.311G, 0/s, -
 * backup_servers/docker0…_docker_volumes.tar.gz:  0% /3.545G, 99.726k/s, 10h20m33s
 * media/library/Movies/filename1.mp4:             0% /1.893G, 0/s, -
 * media/library/Movies/filename2.mp4:             0% /1.775G, 0/s, -
 * media/library/Movies/filename3.mkv:             0% /6.555G, 0/s, -
2

almost same problem v1.52.3 or v1.53.1
single file:
rclone -v --transfers 2 copy /media/USB/Filmler/Kurtlar.Vadisi.Vatan.2017.HDTvRip.1080p.x264.AC3.by.cideli37.TORK.mkv remote:xxxxxx
2020/10/06 22:00:02 INFO :
Transferred: 0 / 2.237 GBytes, 0%, 0 Bytes/s, ETA -
Transferred: 0 / 1, 0%
Elapsed time: 1m0.9s
Transferring:

  • Kurtlar.Vadisi.Vatan.2…3.by.cideli37.TORK.mkv: 0% /2.237G, 0/s, -

2020/10/06 22:01:02 INFO :
Transferred: 23.188M / 2.237 GBytes, 1%, 198.998 kBytes/s, ETA 3h14m29s
Transferred: 0 / 1, 0%
Elapsed time: 2m0.9s
Transferring:

  • Kurtlar.Vadisi.Vatan.2…3.by.cideli37.TORK.mkv: 1% /2.237G, 117.268k/s, 5h30m3s

2020/10/06 22:02:02 INFO :
Transferred: 23.188M / 2.237 GBytes, 1%, 132.413 kBytes/s, ETA 4h52m18s
Transferred: 0 / 1, 0%
Elapsed time: 3m0.9s
Transferring:

  • Kurtlar.Vadisi.Vatan.2…3.by.cideli37.TORK.mkv: 1% /2.237G, 2.439k/s, 264h20m2s

2020/10/06 22:03:02 INFO :
Transferred: 31.094M / 2.237 GBytes, 1%, 133.045 kBytes/s, ETA 4h49m53s
Transferred: 0 / 1, 0%
Elapsed time: 4m0.9s
Transferring:

  • Kurtlar.Vadisi.Vatan.2…3.by.cideli37.TORK.mkv: 1% /2.237G, 41.300k/s, 15h33m53s

2020/10/06 22:04:02 INFO :
Transferred: 31.094M / 2.237 GBytes, 1%, 106.375 kBytes/s, ETA 6h2m34s
Transferred: 0 / 1, 0%
Elapsed time: 5m0.9s
Transferring:

  • Kurtlar.Vadisi.Vatan.2…3.by.cideli37.TORK.mkv: 1% /2.237G, 880/s, 747h56m35s

2020/10/06 22:05:02 INFO :
Transferred: 35.062M / 2.237 GBytes, 2%, 99.922 kBytes/s, ETA 6h25m18s
Transferred: 0 / 1, 0%
Elapsed time: 6m0.9s
Transferring:

  • Kurtlar.Vadisi.Vatan.2…3.by.cideli37.TORK.mkv: 1% /2.237G, 15.406k/s, 41h39m3s

2020/10/06 22:06:02 INFO :
Transferred: 35.062M / 2.237 GBytes, 2%, 85.624 kBytes/s, ETA 7h29m39s
Transferred: 0 / 1, 0%
Elapsed time: 7m0.9s
Transferring:

  • Kurtlar.Vadisi.Vatan.2…3.by.cideli37.TORK.mkv: 1% /2.237G, 328/s, 2001h27m50s

2020/10/06 22:07:02 INFO :
Transferred: 41M / 2.237 GBytes, 2%, 87.591 kBytes/s, ETA 7h18m24s
Transferred: 0 / 1, 0%
Elapsed time: 8m0.9s
Transferring:

  • Kurtlar.Vadisi.Vatan.2…3.by.cideli37.TORK.mkv: 1% /2.237G, 46.358k/s, 13h48m20s

2020/10/06 22:08:02 INFO :
Transferred: 42.875M / 2.237 GBytes, 2%, 81.406 kBytes/s, ETA 7h51m19s
Transferred: 0 / 1, 0%
Elapsed time: 9m0.9s
Transferring:

  • Kurtlar.Vadisi.Vatan.2…3.by.cideli37.TORK.mkv: 1% /2.237G, 5.149k/s, 124h10m4s
3

What does this say?

4

@ncw - See below.

Download 89.76 Mbit/s

Upload 52.56 Mbit/s

Ping 100.00 ms

Jitter 97.04ms

Here's where I am after an hour. Still not uploading two of the seven files, even though --transfers=16.

2020/10/06 17:48:20 INFO  : 
Transferred:        3.012G / 84.416 GBytes, 4%, 883.130 kBytes/s, ETA 1d2h50m55s
Checks:             69761 / 69761, 100%
Transferred:            0 / 7, 0%
Elapsed time:     1h2m0.9s
Transferring:
 * backup_proxmox/dump/vz…10_05-06_00_02.vma.zst:  0% /36.275G, 0/s, -
 * backup_proxmox/dump/vz…10_05-07_05_10.vma.zst:  0% /31.063G, 0/s, -
 * backup_proxmox/dump/vz…10_05-08_03_43.vma.zst:  3% /3.311G, 238.745k/s, 3h54m43s
 * backup_servers/docker0…_docker_volumes.tar.gz: 55% /3.545G, 290.728k/s, 1h35m10s
 * media/library/Movies/filename2.mp4:            21% /1.775G, 306.443k/s, 1h19m48s
 * media/library/Movies/filename3.mkv:             3% /6.555G, 287.713k/s, 6h25m55s
 * media/library/Movies/filename1.mp4:            19% /1.893G, 281.176k/s, 1h34m41s
5

Bandwidth looks OK.

What does your computer look like? CPU, disk etc?

Remember rclone calculates a checksum before transferring the data also.

I think ISP throttling is a possibility. You could try a VPN?

6

It's a VM with 2CPU and 8GB of RAM. Only at about 50% usage on both.

I'm going to contact Verizon to see about throttling. My question to you is, even if I'm being throttled, why are all my files not being uploaded at once? If I have transfers=16, shouldn't it attempt to upload 16 files at once, even if the upload is throttled?

7

Worth an ask.

It should yes, but if there aren't files ready to go then it will transfer less. Your logs above look like there are only 7 files that need transferrring.

Note also

This delay at the start is caused by rclone calculating the sha1sum of the file before upload.

You can turn this off so the uploads start immediately with --b2-disable-checksum at the cost of those files no longer having sha1 checksums.

What disk does it have? Is it SSD or HDD? Remember that you are reading --transfers files at once which can be hard work for HDD. Check to see how the disk is doing with vmstat 5 (let it run for 30 seconds).

8

Ugh, maybe it is my ISP (I'm dreading talking to their tech support). I created three 1GB test files.

-rwxr-xr-x 1 bbbackup bbbackup 1073741824 Oct  7 10:01 test1.img
-rwxr-xr-x 1 bbbackup bbbackup 1073741824 Oct  7 10:04 test2.img
-rwxr-xr-x 1 bbbackup bbbackup 1073741824 Oct  7 09:58 test3.img

Then ran the upload script. It transferred super-fast and finished in a few minutes.

2020/10/07 10:24:12 INFO  :  
Transferred:        3.001G / 3.001 GBytes, 100%, 20.801 MBytes/s, ETA 0s
Checks:             69602 / 69602, 100%
Transferred:            3 / 3, 100%
Elapsed time:       3m0.3s

I appreciate your response. You can close this out.

1 Like
9

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.