SLOWNESS - BackBlaze B2 and Windows PCs

Help and Support
1

What is the problem you are having with rclone?

SLOWNESS

Windows based systems (64 bit, Windows 10 & 11). We moved from a LAN setup with a Windows based file server to the Cloud using BackBlaze B2 with WinFSP and Rclone. In moving files, renaming files / folders and accessing file we have noticed a slowness we did not have previously. We would like assistance please in having faster access.

Are there Cloud platforms that has shown faster response times than B2. One that would allow functionality more in line with a LAN and a windows based file server.

Simply mostly fyi – our offices are located in Western NY, and mostly our Internet speeds are rather high. Our PC are not running other programs or services during our file access of B2.

Thank you so very much; Mark mfinch@themjaco.com

Run the command 'rclone version' and share the full output of the command.

rclone v1.61.1

- os/version: Microsoft Windows 11 Home 21H2 (64 bit)

- os/kernel: 10.0.22000.1455 (x86_64)

- os/type: windows

- os/arch: amd64

- go/version: go1.19.4

- go/linking: static

- go/tags: cmount

Are you on the latest version of rclone? Yes

Which cloud storage system are you using? (eg Google Drive)

BackBlaze B2

The command you were trying to run (eg rclone copy /tmp remote:tmp)


**rclone.exe --vfs-cache-mode full --transfers 32 mount BackBlaze:HUB-MJA K:**

The rclone config contents with secrets removed.


**[BackBlaze]**

**type = b2**

**account = (REMOVED)**

**key = (REMOVED)**

A log from the command with the -vv flag


**C:\Users\markf\Documents\rClone>rclone.exe --vfs-cache-mode full --transfers 32 -vv mount** **BackBlaze:HUB-MJA K:**

**2023/01/26 12:47:34 DEBUG : rclone: Version "v1.61.1" starting with parameters ["rclone.exe" "--vfs-cache-mode" "full" "--transfers" "32" "-vv" "mount" "BackBlaze:HUB-MJA" "K:"]**

**2023/01/26 12:47:34 DEBUG : Creating backend with remote "BackBlaze:HUB-MJA"**

**2023/01/26 12:47:34 DEBUG : Using config file from "C:\\Users\\markf\\AppData\\Roaming\\rclone\\rclone.conf"**

**2023/01/26 12:47:34 INFO : B2 bucket HUB-MJA: poll-interval is not supported by this remote**

**2023/01/26 12:47:34 DEBUG : vfs cache: root is "C:\\Users\\markf\\AppData\\Local\\rclone"**

**2023/01/26 12:47:34 DEBUG : vfs cache: data root is "\\\\?\\C:\\Users\\markf\\AppData\\Local\\rclone\\vfs\\BackBlaze\\HUB-MJA"**

**2023/01/26 12:47:34 DEBUG : vfs cache: metadata root is "\\\\?\\C:\\Users\\markf\\AppData\\Local\\rclone\\vfsMeta\\BackBlaze\\HUB-MJA"**

**2023/01/26 12:47:34 DEBUG : Creating backend with remote "C:/Users/markf/AppData/Local/rclone/vfs/BackBlaze/HUB-MJA"**

**2023/01/26 12:47:34 DEBUG : fs cache: renaming cache item "C:/Users/markf/AppData/Local/rclone/vfs/BackBlaze/HUB-MJA" to be canonical "//?/C:/Users/markf/AppData/Local/rclone/vfs/BackBlaze/HUB-MJA"**

**2023/01/26 12:47:34 DEBUG : Creating backend with remote "C:/Users/markf/AppData/Local/rclone/vfsMeta/BackBlaze/HUB-MJA"**

**2023/01/26 12:47:34 DEBUG : fs cache: renaming cache item "C:/Users/markf/AppData/Local/rclone/vfsMeta/BackBlaze/HUB-MJA" to be canonical "//?/C:/Users/markf/AppData/Local/rclone/vfsMeta/BackBlaze/HUB-MJA"**

**2023/01/26 12:47:34 DEBUG : Network mode mounting is disabled**

**2023/01/26 12:47:34 DEBUG : Mounting on "K:" ("BackBlaze HUB-MJA")**

**2023/01/26 12:47:34 DEBUG : B2 bucket HUB-MJA: Mounting with options: ["-o" "attr_timeout=1" "-o" "uid=-1" "-o" "gid=-1" "--FileSystemName=rclone" "-o" "volname=BackBlaze HUB-MJA"]**

**2023/01/26 12:47:34 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item Windows (C) - Shortcut.lnk not removed, freed 0 bytes**

**2023/01/26 12:47:34 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item _QUOTED JOBS/Advance Energy - Budgetary - Niagara County 8-22/ADVANCED ENERGY - POlishing & Joints 8-2022.pdf not removed, freed 0 bytes**

**2023/01/26 12:47:34 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item Microsoft Office Word 2007.lnk not removed, freed 0 bytes**

**2023/01/26 12:47:34 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item SECURITY FOLDER/SecurityDocument .docx not removed, freed 0 bytes**

**2023/01/26 12:47:34 INFO : vfs cache: cleaned: objects 4 (was 4) in use 0, to upload 0, uploading 0, total size 508.234Ki (was 508.234Ki)**

**2023/01/26 12:47:34 DEBUG : B2 bucket HUB-MJA: Init:**

**2023/01/26 12:47:34 DEBUG : B2 bucket HUB-MJA: >Init:**

**2023/01/26 12:47:34 DEBUG : /: Statfs:**

**2023/01/26 12:47:34 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0**

**2023/01/26 12:47:34 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:34 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:34 DEBUG : /: Readlink:**

**2023/01/26 12:47:34 DEBUG : /: >Readlink: linkPath="", errc=-40**

**The service rclone has been started.**

**2023/01/26 12:47:34 DEBUG : /: Statfs:**

**2023/01/26 12:47:34 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0**

**2023/01/26 12:47:34 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:34 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:34 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:34 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:34 DEBUG : /: Opendir:**

**2023/01/26 12:47:34 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx**

**2023/01/26 12:47:34 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>**

**2023/01/26 12:47:34 DEBUG : /: >Opendir: errc=0, fh=0x0**

**2023/01/26 12:47:34 DEBUG : /: Releasedir: fh=0x0**

**2023/01/26 12:47:34 DEBUG : /: >Releasedir: errc=0**

**2023/01/26 12:47:34 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:34 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:34 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:34 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:34 DEBUG : /: Opendir:**

**2023/01/26 12:47:34 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx**

**2023/01/26 12:47:34 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>**

**2023/01/26 12:47:34 DEBUG : /: >Opendir: errc=0, fh=0x0**

**2023/01/26 12:47:34 DEBUG : /: Releasedir: fh=0x0**

**2023/01/26 12:47:34 DEBUG : /: >Releasedir: errc=0**

**2023/01/26 12:47:34 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:34 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:34 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:34 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:34 DEBUG : /: Opendir:**

**2023/01/26 12:47:34 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx**

**2023/01/26 12:47:34 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>**

**2023/01/26 12:47:34 DEBUG : /: >Opendir: errc=0, fh=0x0**

**2023/01/26 12:47:34 DEBUG : /: Releasedir: fh=0x0**

**2023/01/26 12:47:34 DEBUG : /: >Releasedir: errc=0**

**2023/01/26 12:47:37 DEBUG : /autorun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:51 DEBUG : /autorun.inf: >Getattr: errc=-2**

**2023/01/26 12:47:51 DEBUG : /autorun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:51 DEBUG : /autorun.inf: >Getattr: errc=-2**

**2023/01/26 12:47:51 DEBUG : /autorun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:51 DEBUG : /autorun.inf: >Getattr: errc=-2**

**2023/01/26 12:47:51 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:51 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:51 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:51 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:51 DEBUG : /: Opendir:**

**2023/01/26 12:47:51 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx**

**2023/01/26 12:47:51 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>**

**2023/01/26 12:47:51 DEBUG : /: >Opendir: errc=0, fh=0x0**

**2023/01/26 12:47:51 DEBUG : /: Releasedir: fh=0x0**

**2023/01/26 12:47:51 DEBUG : /: >Releasedir: errc=0**

**2023/01/26 12:47:51 DEBUG : /AutoRun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:51 DEBUG : /AutoRun.inf: >Getattr: errc=-2**

**2023/01/26 12:47:57 DEBUG : /_QUOTED JOBS/Advance Energy - Budgetary - Niagara County 8-22: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:57 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:57 DEBUG : /: Opendir:**

**2023/01/26 12:47:57 DEBUG : /_PRE-BID/AMAZON - TOWN OF NIAGARA 3-22: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx**

**2023/01/26 12:47:57 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>**

**2023/01/26 12:47:57 DEBUG : /: >Opendir: errc=0, fh=0x0**

**2023/01/26 12:47:57 DEBUG : /: Releasedir: fh=0x0**

**2023/01/26 12:47:57 DEBUG : /: >Releasedir: errc=0**

**2023/01/26 12:47:57 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:57 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:57 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:57 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:57 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:57 DEBUG : /: Opendir:**

**2023/01/26 12:47:57 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:57 DEBUG : /: Opendir:**

**2023/01/26 12:47:57 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx**

**2023/01/26 12:47:57 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx**

**2023/01/26 12:47:57 DEBUG : /: Opendir:**

**2023/01/26 12:47:57 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx**

**2023/01/26 12:47:57 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>**

**2023/01/26 12:47:57 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>**

**2023/01/26 12:47:57 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>**

**2023/01/26 12:47:57 DEBUG : /: >Opendir: errc=0, fh=0x0**

**2023/01/26 12:47:57 DEBUG : /: >Opendir: errc=0, fh=0x1**

**2023/01/26 12:47:57 DEBUG : /: >Opendir: errc=0, fh=0x2**

**2023/01/26 12:47:57 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:57 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: Releasedir: fh=0x0**

**2023/01/26 12:47:57 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: Releasedir: fh=0x1**

**2023/01/26 12:47:57 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:57 DEBUG : /: >Releasedir: errc=0**

**2023/01/26 12:47:57 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:57 DEBUG : /: >Releasedir: errc=0**

**2023/01/26 12:47:57 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:57 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:57 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: Releasedir: fh=0x2**

**2023/01/26 12:47:57 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: Opendir:**

**2023/01/26 12:47:57 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:57 DEBUG : /: >Releasedir: errc=0**

**2023/01/26 12:47:57 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:57 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:57 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:57 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx**

**2023/01/26 12:47:57 DEBUG : /: Opendir:**

**2023/01/26 12:47:57 DEBUG : /: Opendir:**

**2023/01/26 12:47:57 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF**

**2023/01/26 12:47:57 DEBUG : /: >Getattr: errc=0**

**2023/01/26 12:47:57 DEBUG : /: Opendir:**

**2023/01/26 12:47:57 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>**
2

hello and welcome to the forum,

if you have move the files to cloud, then that is going to be slower than having the files local.
did you expect something different?
can you post the output of a speedtest, including ping

rclone has to calculate checksums
rclone will calculate the SHA1 checksum of the input before uploading

local to b2 or b2 to local or what?
B2 does not support server-side move, so rclone has to do copy and delete

rclone has two caches, might want to pre-cache the vfs dir cache.
https://forum.rclone.org/t/status-about-using-rclone-for-music-storage-playback-in-2021-access-times-improved/27648/34
to the mount command, add --rc --rc-no-auth
and once the mount is running, run rclone rc vfs/refresh recursive=true

could experiment with
--buffer-size=0 --network-mode

in all my testing, nothing is faster than wasabi, also, does not charge for api calls.

B2 support the S3 api, might test with that

3

Hi Everyone and thank you so very much for the help. I am currently testing with;
rclone.exe --vfs-cache-mode full --vfs-read-ahead 500M --buffer-size 1024M --transfers 75 --network-mode --b2-disable-checksum mount BackBlaze:SECRET K: --rc --rc-no-auth
We do use this setup from multiple locations, from my particular location the results of speedtest with ping;
DOWNLOAD Mbps 32.51 UPLOAD Mbps 7.59 Ping ms 11 116 9
As for the question of did you expect something different then slower than the LAN the answer is me personally no I did not, others and hence the reaching out for help is yes.
Again I greatly appreciate any help.
Thank you much, Mark.

4

I just ran; rclone rc vfs/refresh recursive=true after Rclone was started and it appears - very brief testing - that directory listing speeds are greatly improved. Still some hesitation periodically and from some research on Windows File Explorer it appears there might some issues? Have others experienced such and how was it corrected. Thank you again for any additional input.

5

did you try my suggeston from up above?
--buffer-size=0 --network-mode

and i would test without --vfs-read-ahead 500M

6

HI thank you for the latest reply - greatly appreciated.
I now have; rclone.exe --vfs-cache-mode full --buffer-size=0 --transfers 75 --network-mode --b2-disable-checksum mount BackBlaze:SECRET K: --rc --rc-no-auth
followed by (after a 15 second delay); rclone rc vfs/refresh recursive=true

7

hopefully, those changes will make a difference?

8

Oh I hope they do. I am performing some testing and will request one of the users to try the new settings. I imagine at that time I will get some further input.

9

ok, good.

and not sure your exact case, but might test on wasabi

10

Hi Again. For now we like to stick with B2 as we are configured that way. What we have is a Windows mount point (via Rclone) of B2 we call the K drive. All our files are located on the K drive. I just ran a test where I moved approximately 2,400 files located in about 210 folders into another folder and it took 31 minutes. Any comments if this is good, bad or indifferent speed wise would be greatly appreciated. Thank you, Mark.

11

it is more about the total amount of data transferred, than the total number of dir/files.

b2 does not support move, so that would be the two-step, the debug log should show

  1. copy source dest - note: the copy is server-side
  2. delete source
12

The total data volume was about 3.00 GB. Thank you.

13

very, very small amount of data.
as for the exact reason, the move took 31 minutes, would look into debug log.

without a debug log, just a guess,
might be an issue rclone comparing the source to dest and a large number of api calls.
as i understand it,
rclone copy can use --fast-list
rclone mount cannot use --fast-list

for a one-time transfer, where the dest is empty, rclone move, can use

--no-check-dest can be used with `move` or `copy` 
and it causes rclone not to check the destination at all when copying files.
14

Hi, thank you for the information on move / copy. I have yet to test. I did however have one of my users try the settings of; rclone.exe --vfs-cache-mode full --buffer-size=0 --transfers 75 --network-mode --b2-disable-checksum mount BackBlaze:HUB-MJA K: --rc --rc-no-auth on Friday and they indicated oh this is much better, very good. Today they were not so happy...... After the Rclone command is executed we do perform a; rclone rc vfs/refresh recursive=true I have now set -vv and redirected to a file which I plan to upload to; https://github.com/ as I created an account. Once I upload my debug how do I get the data to you please my userid is mfinch65. I was planning on starting Rclone on my PC and letting it run overnight and then sending up the data and then tomorrow asking my user to also run in debug mode and also list his complaints and then sending along the debug file for hopefully how to fix things. That you again your support of our issue it is greatly appreciated. Regards, Mark.

15

Hi again, sorry to be such a persistent SOB and I thank you for your help. I have a 325Meg log / debug file that shows me using Rclone and also performing moves. How might I get this to you or what could I look for in the DEBUG file and present to you for hopefully problem resolution. Thank you so very much, Mark. In summary what I am looking for is the fastest possible speeds using Rclone as a windows mount point to BackBlaze B2 Cloud. Regards, Mark.

16

I have created another Application Key in B2 that I use exclusively. Additionally my Rclone command is now; start "Rclone for HUB-MJA" /min rclone.exe --vfs-cache-mode full --buffer-size=0 --checkers 256 --transfers 256 --fast-list --progress --drive-chunk-size 128M --max-backlog 999999 --log-level ERROR --log-file="MyErrorLog.txt" --network-mode --b2-disable-checksum mount BackBlaze:HUB-MJA K: --rc --rc-no-auth Any other suggestions. Thank you, Mark.

17
  • --fast-list does nothing on a mount.
  • --checkers, afaik, does nothing on a mount.
  • --drive-chunk-size is for gdrive, not b2
  • --max-backlog 999999, what is the logic for that?

might consider https://rclone.org/donate
note: i do not profit from donations/sponsoring.

18

Thank you for the response. I will check with my management about a donation. Based on the reply it seems things are as good as they will get? / I am not seeing any errors. Any other suggestions greatly appreciated. Thank you, Mark.

1 Like
19

WINDOWS FILE EXPLORER - From searching this forum it seems Windows File Explorer and Rclone can cause some slowness. Are there recommended setting for Windows File Explorer to get the fastest responses possible.

20

basically, dumb down explorer, as much as possible, no icons, no previews, etc..
this has been discussed in the forum many times...

there are two types of caches, the vfs dir cache and the vfs file cache.
to speed up explorer, want to pre-load the vfs dir cache.

https://forum.rclone.org/t/status-about-using-rclone-for-music-storage-playback-in-2021-access-times-improved/27648/34