SMB signing rclone mount

Finn_Jaeger · February 21, 2024, 5:28pm

What is the problem you are having with rclone?

Rclone mount connects to SMB shares using signing on according to the synology GUI, regular SMB mounts do not use signing, i am hoping for some more performance and less CPU overhead by turning it off , this is a 10/25GbitE network.

Run the command 'rclone version' and share the full output of the command.

rclone v1.65.2
- os/version: Microsoft Windows 11 Pro 23H2 (64 bit)
- os/kernel: 10.0.22631.3155 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.21.6
- go/linking: static
- go/tags: cmount

Yes, latest version

Which cloud storage system are you using? (eg Google Drive)

SMB on a synology nas

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone.exe mount onix:proj X: --vfs-cache-mode full --cache-dir D:\rclone --vfs-cache-max-age 36h0m0s --vfs-cache-min-free-space 20G --no-modtime

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[onix]
type = smb
host = XXX
user = XXX
pass = XXX
### Double check the config for sensitive info before posting publicly

A log from the command that you were trying to run with the `-vv` flag

C:\Users\finn>C:\Users\finn\Downloads\rclone-v1.65.2-windows-amd64\rclone-v1.65.2-windows-amd64\rclone.exe mount onix:proj X: --vfs-cache-mode full --cache-dir D:\rclone --vfs-cache-max-age 36h0m0s --vfs-cache-min-free-space 20G --no-modtime -vv
2024/02/21 18:29:53 DEBUG : rclone: Version "v1.65.2" starting with parameters ["C:\\Users\\finn\\Downloads\\rclone-v1.65.2-windows-amd64\\rclone-v1.65.2-windows-amd64\\rclone.exe" "mount" "onix:proj" "X:" "--vfs-cache-mode" "full" "--cache-dir" "D:\\rclone" "--vfs-cache-max-age" "36h0m0s" "--vfs-cache-min-free-space" "20G" "--no-modtime" "-vv"]
2024/02/21 18:29:53 DEBUG : Creating backend with remote "onix:proj"
2024/02/21 18:29:53 DEBUG : Using config file from "C:\\Users\\finn\\AppData\\Roaming\\rclone\\rclone.conf"
2024/02/21 18:29:53 INFO  : smb://finn@onix:445/proj/: poll-interval is not supported by this remote
2024/02/21 18:29:53 DEBUG : vfs cache: root is "D:\\rclone"
2024/02/21 18:29:53 DEBUG : vfs cache: data root is "\\\\?\\D:\\rclone\\vfs\\onix\\proj"
2024/02/21 18:29:53 DEBUG : vfs cache: metadata root is "\\\\?\\D:\\rclone\\vfsMeta\\onix\\proj"
2024/02/21 18:29:53 DEBUG : Creating backend with remote "D:/rclone/vfs/onix/proj"
2024/02/21 18:29:53 DEBUG : fs cache: renaming cache item "D:/rclone/vfs/onix/proj" to be canonical "//?/D:/rclone/vfs/onix/proj"
2024/02/21 18:29:53 DEBUG : Creating backend with remote "D:/rclone/vfsMeta/onix/proj"
2024/02/21 18:29:53 DEBUG : fs cache: renaming cache item "D:/rclone/vfsMeta/onix/proj" to be canonical "//?/D:/rclone/vfsMeta/onix/proj"
2024/02/21 18:29:53 DEBUG : Network mode mounting is disabled
2024/02/21 18:29:53 DEBUG : vfs cache RemoveNotInUse (maxAge=129600000000000, emptyOnly=false): item XXX_development/pipetest/040/src/plate01/virginialottery_040_plate01_acescg_v000_exr/virginialottery_040_plate01_acescg_v000.1015.exr not removed, freed 0 bytes
2024/02/21 18:29:53 DEBUG : vfs cache RemoveNotInUse (maxAge=129600000000000, emptyOnly=false): item XXX_development/pipetest/040/src/plate01/virginialottery_040_plate01_acescg_v000_exr/virginialottery_040_plate01_acescg_v000.1017.exr not removed, freed 0 bytes
2024/02/21 18:29:53 DEBUG : vfs cache RemoveNotInUse (maxAge=129600000000000, emptyOnly=false): item XXX_development/pipetest/transfer/writespeedtest/virginialottery_040_plate01_acescg_v000.1000.exr not removed, freed 0 bytes
2024/02/21 18:29:53 DEBUG : vfs cache RemoveNotInUse (maxAge=129600000000000, emptyOnly=false): item XXX_development/pipetest/transfer/writespeedtest/virginialottery_040_plate01_acescg_v000.1044.exr not removed, freed 0 bytes
2024/02/21 18:29:53 DEBUG : vfs cache RemoveNotInUse (maxAge=129600000000000, emptyOnly=false): item XXX_development/pipetest/transfer/writespeedtest/virginialottery_040_plate01_acescg_v000.1046.exr not removed, freed 0 bytes
2024/02/21 18:29:53 DEBUG : vfs cache RemoveNotInUse (maxAge=129600000000000, emptyOnly=false): item XXX_development/pipetest/transfer/writespeedtest/virginialottery_040_plate01_acescg_v000.1041.exr not removed, freed 0 bytes
2024/02/21 18:29:53 DEBUG : vfs cache RemoveNotInUse (maxAge=129600000000000, emptyOnly=false): item XXX_development/pipetest/040/src/plate01/virginialottery_040_plate01_acescg_v000_exr/virginialottery_040_plate01_acescg_v000.1012.exr not removed, freed 0 bytes
2024/02/21 18:29:53 DEBUG : Mounting on "X:" ("onix proj")
2024/02/21 18:29:53 DEBUG : vfs cache RemoveNotInUse (maxAge=129600000000000, emptyOnly=false): item XXX_development/pipetest/040/src/plate01/virginialottery_040_plate01_acescg_v000_exr/virginialottery_040_plate01_acescg_v000.1018.exr not removed, freed 0 bytes
2024/02/21 18:29:53 DEBUG : vfs cache RemoveNotInUse (maxAge=129600000000000, emptyOnly=false): item XXX_development/pipetest/040/src/plate01/virginialottery_040_plate01_acescg_v000_exr/virginialottery_040_plate01_acescg_v000.1041.exr not removed, freed 0 bytes
2024/02/21 18:29:53 DEBUG : vfs cache RemoveNotInUse (maxAge=129600000000000, emptyOnly=false): item XXX_development/pipetest/040/src/plate01/virginialottery_040_plate01_acescg_v000_exr/virginialottery_040_plate01_acescg_v000.1060.exr not removed, freed 0 bytes
2024/02/21 18:29:53 DEBUG : vfs cache RemoveNotInUse (maxAge=129600000000000, emptyOnly=false): item XXX_development/pipetest/transfer/writespeedtest/virginialottery_040_plate01_acescg_v000.0998.exr not removed, freed 0 bytes

asdffdsa · February 21, 2024, 6:36pm

the synbox has settings to control that.

the client is a windows machine, which has native support for mounting smb/samba.
so why use rclone mount?

might check out my how-to guide comparing native mount versus os mount.
https://forum.rclone.org/t/how-to-access-smb-samba-with-rclone/42754

Finn_Jaeger · February 21, 2024, 7:03pm

I have disabled smb signing on the syno, usually all smb clients mount without signing this is why i am so confused by it.

So my usecase is a bit odd, but basically we have quiet a lot of software that does not play well with the lower performance of smb VFS with a cache.

Thefoundry nuke is extremely slow writing to network shares and reading from them especially in windows this is known, using vfs/rclone completely mitigates this issue for us

nuke has a cache feature where the source file is copied to fast local storage (localization) so reading is not the big issue here

blackmagic Resolve and autodesk flame both do not have localization, they only have caching which creates a new rendered file which takes a long time.

both of those tools repeately read the same file maybe hundrets of times a day, that file never changes. (think video editing). the files
however are usually between 50mb-1.5gb a FRAME (3d renderc EXRs) in abera maybe 150MB/frame which still ends up being at 25fps 3.7GB/s which is exeeding even our 25Gbit/s infrastructure. a single NVME however has no issues playing this back, and the most beefy machnes have a 16TB 8x nvme raid0 for caching.

bascially i am looking at rclone mount with a large 4-8tb nvme per workstation as a way out of having to upgrade the nas itself to full nvme and run 100g bitE fibre everywhere. local access is just so much better for what we do, no matter how beefy the nas is.

would something like webdav or NFS be a better choice vs smb? or maybe even sftp? or minio server on the syno and use S3?

kapitainsky · February 21, 2024, 7:07pm

I would definitely prefer NFS over SMB but I am not on Windows so not sure how well it works in your environment. If you want to try NFS then do this with the latest rclone beta (v.1.66) as some problems were only fixed in this version.

Finn_Jaeger · February 21, 2024, 7:08pm

cool
yea i will
try this with nfs, nfs is not a problem we are a very small environment (think 10 users) , essy to get that going

I should add we are a mixed environment win/linux/mac

asdffdsa · February 21, 2024, 7:15pm

in each case, the hard limit seems to be the nas itself.
so once the files are in the rclone mount, might not make much of a practical difference.

Finn_Jaeger · February 21, 2024, 7:23pm

i have observed a maximum of around 300MB/s on a 25Gbir connection, pulling the same files via smb directly gives me around 1.6GB/s and there is very little cpu usesge on direct pull vs a lot with rclone. just trying to minimize that first-pull-time if possible

system · March 22, 2024, 7:24pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.