tik0
February 6, 2025, 2:49pm
1
What is the problem you are having with rclone?
The copy via smb does not show the same performance as other smb clients (e.g., smbclient
).
With my 10 Gbit/s connection I can almost max out the file transfer rate using smbclient
and an optimized smb.conf
configuration (an unoptimized configuration only reaches about 40%).
Using rclone, I only managed to utilize my connection by about 70% using the command provided below.
I assume, that the smb backend go-smb2
could be further optimized?
But by reading the docs, it seems to be the case that no go-smb2
options are exposed to the user.
Are there other possibilities to optimize the smb transfer rate?
Run the command 'rclone version' and share the full output of the command.
rclone v1.68.2
- os/version: ubuntu 24.04 (64 bit)
- os/kernel: 6.8.0-51-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.23.3
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
Local Samba (SMB) server.
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
rclone copy --multi-thread-streams 1024 --ignore-checksum --progress remote:share$/file /tmp
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
$ rclone config redacted
[remote]
type = smb
host = XXX
user = XXX
pass = XXX
hide_special_share = false
A log from the command that you were trying to run with the -vv
flag
Not of interest?
asdffdsa
(jojothehumanmonkey)
February 6, 2025, 2:57pm
2
welcome to the forum,
tik0:
hide_special_share
might try increaing --transfers
and --checkers
tik0:
--ignore-checksum
afaik, that does nothing on smb remote.
check out my howto guide
https://forum.rclone.org/t/how-to-access-smb-samba-with-rclone/42754
tik0:
I assume, that the smb backend go-smb2
could be further optimized?
But by reading the docs, it seems to be the case that no go-smb2
options are exposed to the user.
Are there other possibilities to optimize the smb transfer rate?
sure. And some people are working on it.
Already included in beta:
master
← jgiannuzzi:smb-better-connection-pool
opened 12:05PM - 23 Jan 25 UTC
#### What is the purpose of this change?
The goal is to improve the effic… iency of the connection pooling in the `smb` backend.
The following changes were made to achieve it:
* Lower pacer `minSleep` to establish new connections faster
* Use `Echo` requests to check whether connections are working (required an upgrade of `go-smb2` to include https://github.com/CloudSoda/go-smb2/pull/25)
* Only remount shares when needed
* Use context for connection establishment
* When returning a connection to the pool, only check the ones that encountered errors
* Close connections in parallel
#### Was the change discussed in an issue or in the forum before?
No
#### Checklist
- [x] I have read the [contribution guidelines](https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#submitting-a-new-feature-or-bug-fix).
- [x] I have added tests for all changes in this PR if appropriate.
- [x] I have added documentation for the changes if appropriate.
- [x] All commit messages are in [house style](https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#commit-messages).
- [x] I'm done, this Pull Request is ready for review :-)
Work in progress:
master
← jgiannuzzi:smb-chunked-uploads
opened 02:05PM - 30 Jan 25 UTC
#### What is the purpose of this change?
#### Was the change discussed in… an issue or in the forum before?
#### Checklist
- [x] I have read the [contribution guidelines](https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#submitting-a-new-feature-or-bug-fix).
- [x] I have added tests for all changes in this PR if appropriate.
- [x] I have added documentation for the changes if appropriate.
- [x] All commit messages are in [house style](https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#commit-messages).
- [ ] I'm done, this Pull Request is ready for review :-)
Jump in if you think you have some ideas. I think initial smb implementation was more or less proof of concept and everything else depends on community.
BTW - I know it is off topic really but what were key changes you made to achieve it?
tik0
February 7, 2025, 7:48am
5
Thank you for the fast reply. The key options in my smb.conf
are as follows:
max connections=128
large readwrite = yes
aio write size = 1
aio read size = 1
write cache size = 2097152
min receivefile size = 16384
getwd cache = yes
Furthermore, I disabled the SMB's transport encryption on the NAS (it is covered by VPN for remote access anyways.)
I am new to GO, but I will have a look on the PRs to get into it.
1 Like
system
(system)
Closed
March 9, 2025, 7:49am
6
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.