Need help for slow samba share speed over share in linux

chchia · June 22, 2020, 5:58am

What is the problem you are having with rclone?

When i mount the gdrive in ubuntu, i am having below problem:

copy file within ubuntu from remote mounted gdrive folder to local disk inside ubuntu i get 50MB/s (i have a 500m/500m fiber), this is working perfectly.
i share the remote mounted gdrive folder over samba, and access from Windows 10, i can browse the file, but when i want to copy the file over samba to windows 10, i get speed 2~3MB/s. which is some thing wrong.
i create another samba share which not pointing to remote mount, and copy file from windows 10, i get 80~90 MB/s, which is normal.

What is your rclone version (output from `rclone version`)

any version, stable or beta.

Which OS you are using and how many bits (eg Windows 7, 64 bit)

ubuntu 20.04 lts 64 bits
windows 10 64 bits

Which cloud storage system are you using? (eg Google Drive)

google drive

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

mounting remote gdrive with these command:
--fast-list --allow-other --transfers=10 --checkers=10 --buffer-size 64M --tpslimit=10 --default-permissions --copy-links --no-gzip-encoding --no-check-certificate --vfs-cache-mode writes --drive-chunk-size 32M --vfs-read-chunk-size 4M --vfs-read-chunk-size-limit 512M --vfs-read-wait 60ms

sorry no log file, because it is working but just with slow speed with samba share over mount.

i had tryied to play around with samba with option like tcp_nodelay etc but doesnt seems to help, can any one advise me what could be the solutions? thank you.

Animosity022 · June 22, 2020, 10:09am

Lots of that does nothing on a mount.

Try just making:

--allow-other --log-level DEBUG --log-file /tmp/rclone.log

and share the log when you are copying and se can take a look.

chchia · June 22, 2020, 1:51pm

thank you so much, i am now using the rclone serve webdav to resolve my issues.

i have to say that serve webdav is more faster compared to samba in my cases, i am happyly with this webdav function.

mouting webdav in synology also fixed one of my previously reported issues here

i can now transfer file to webdav mount using sylogy file station and get it uploaded to gdrive on the fly.

thank you so much.

mindtripper · August 18, 2020, 2:00pm

Hi!

I've got the exact same issue.
Mounting cloud storage with FUSE then sharing the mount with SAMBA.
Windows accessing the share results in 1-2MB/s transfer speeds. Accessing the FUSE mount directly from Linux results in ~80MB/s.

This is the debug log from the transfer:

2020/08/18 15:53:28 DEBUG : : Statfs:
2020/08/18 15:53:28 DEBUG : : >Statfs: stat={Blocks:274877906944 Bfree:271203839704 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2020/08/18 15:53:28 DEBUG : : Statfs:
2020/08/18 15:53:28 DEBUG : : >Statfs: stat={Blocks:274877906944 Bfree:271203839704 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2020/08/18 15:53:28 DEBUG : ANNONYMIZED_FILE: ReadFileHandle.seek from 16646144 to 14548992 (fs.RangeSeeker)
2020/08/18 15:53:28 DEBUG : ANNONYMIZED_FILE: ChunkedReader.RangeSeek from 17035264 to 14548992 length -1
2020/08/18 15:53:28 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at -1 length 4096 chunkOffset 14548992 chunkSize 134217728
2020/08/18 15:53:28 DEBUG : ANNONYMIZED_FILE: ChunkedReader.openRange at 14548992 length 134217728
2020/08/18 15:53:28 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 14553088 length 8192 chunkOffset 14548992 chunkSize 134217728
2020/08/18 15:53:28 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 14561280 length 16384 chunkOffset 14548992 chunkSize 134217728
2020/08/18 15:53:28 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 14577664 length 32768 chunkOffset 14548992 chunkSize 134217728
2020/08/18 15:53:28 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 14610432 length 65536 chunkOffset 14548992 chunkSize 134217728
2020/08/18 15:53:28 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 14675968 length 131072 chunkOffset 14548992 chunkSize 134217728
2020/08/18 15:53:28 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 14807040 length 262144 chunkOffset 14548992 chunkSize 134217728
2020/08/18 15:53:28 DEBUG : &{ANNONYMIZED_FILE (r)}: >Read: read=131072, err=<nil>
2020/08/18 15:53:28 DEBUG : &{ANNONYMIZED_FILE (r)}: Read: len=131072, offset=13500416
2020/08/18 15:53:28 DEBUG : &{ANNONYMIZED_FILE (r)}: Read: len=131072, offset=15597568
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 15069184 length 524288 chunkOffset 14548992 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 15593472 length 1048576 chunkOffset 14548992 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ReadFileHandle.seek from 14680064 to 13500416 (fs.RangeSeeker)
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.RangeSeek from 16642048 to 13500416 length -1
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at -1 length 4096 chunkOffset 13500416 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.openRange at 13500416 length 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 13504512 length 8192 chunkOffset 13500416 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 13512704 length 16384 chunkOffset 13500416 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 13529088 length 32768 chunkOffset 13500416 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 13561856 length 65536 chunkOffset 13500416 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 13627392 length 131072 chunkOffset 13500416 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 13758464 length 262144 chunkOffset 13500416 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : &{ANNONYMIZED_FILE (r)}: >Read: read=131072, err=<nil>
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 14020608 length 524288 chunkOffset 13500416 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ReadFileHandle.seek from 13631488 to 15597568 (fs.RangeSeeker)
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.RangeSeek from 14544896 to 15597568 length -1
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at -1 length 4096 chunkOffset 15597568 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.openRange at 15597568 length 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 15601664 length 8192 chunkOffset 15597568 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 15609856 length 16384 chunkOffset 15597568 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 15626240 length 32768 chunkOffset 15597568 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 15659008 length 65536 chunkOffset 15597568 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 15724544 length 131072 chunkOffset 15597568 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 15855616 length 262144 chunkOffset 15597568 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : &{ANNONYMIZED_FILE (r)}: >Read: read=131072, err=<nil>
2020/08/18 15:53:29 DEBUG : &{ANNONYMIZED_FILE (r)}: Read: len=131072, offset=16646144
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: waiting for in-sequence read to 16646144 for 5ms
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: aborting in-sequence read wait, off=16646144
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: failed to wait for in-sequence read to 16646144
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 16117760 length 524288 chunkOffset 15597568 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 16642048 length 1048576 chunkOffset 15597568 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ReadFileHandle.seek from 15728640 to 16646144 (fs.RangeSeeker)
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.RangeSeek from 17690624 to 16646144 length -1
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at -1 length 4096 chunkOffset 16646144 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.openRange at 16646144 length 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 16650240 length 8192 chunkOffset 16646144 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 16658432 length 16384 chunkOffset 16646144 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 16674816 length 32768 chunkOffset 16646144 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 16707584 length 65536 chunkOffset 16646144 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 16773120 length 131072 chunkOffset 16646144 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 16904192 length 262144 chunkOffset 16646144 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : &{ANNONYMIZED_FILE (r)}: >Read: read=131072, err=<nil>
2020/08/18 15:53:29 DEBUG : &{ANNONYMIZED_FILE (r)}: Flush:
2020/08/18 15:53:29 DEBUG : &{ANNONYMIZED_FILE (r)}: >Flush: err=<nil>
2020/08/18 15:53:29 DEBUG : &{ANNONYMIZED_FILE (r)}: Flush:
2020/08/18 15:53:29 DEBUG : &{ANNONYMIZED_FILE (r)}: >Flush: err=<nil>
2020/08/18 15:53:29 DEBUG : &{ANNONYMIZED_FILE (r)}: Release:
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ReadFileHandle.Release closing
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 17166336 length 524288 chunkOffset 16646144 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : Series/: ReadDirAll:
2020/08/18 15:53:29 DEBUG : Series/: >ReadDirAll: item=33, err=<nil>
2020/08/18 15:53:29 DEBUG : Series_Temp/: ReadDirAll:
2020/08/18 15:53:29 DEBUG : Series_Temp/: >ReadDirAll: item=6, err=<nil>
2020/08/18 15:53:29 DEBUG : ANNONYMIZED_FILE: ChunkedReader.Read at 17690624 length 1048576 chunkOffset 16646144 chunkSize 134217728
2020/08/18 15:53:29 DEBUG : &{ANNONYMIZED_FILE (r)}: >Release: err=<nil>
2020/08/18 15:53:30 DEBUG : /: Lookup: name="Series"
2020/08/18 15:53:30 DEBUG : /: >Lookup: node=Series/, err=<nil>
2020/08/18 15:53:30 DEBUG : Series/: Attr:
2020/08/18 15:53:30 DEBUG : Series/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2020/08/18 15:53:30 DEBUG : /: Lookup: name="Series"
2020/08/18 15:53:30 DEBUG : /: >Lookup: node=Series/, err=<nil>
2020/08/18 15:53:30 DEBUG : Series/: Attr:

Any idea what the problem can be?

Animosity022 · August 18, 2020, 2:19pm

What version are you running?
What's your mount command?
What Storage provider are you using?
Can you share a full debug log with -vv?
What Samba command are you using?

mindtripper · August 18, 2020, 2:26pm

Alpine Linux using;
rclone v1.51.0-082-g0c6f14c6-beta
rclone mount --allow-other --log-level DEBUG --log-file /tmp/rclone.log
JottaCloud
Log is forthcoming....
Default samba config with the following share information
[share name]
comment = JottaCloud
path = /path/path
valid users = root user
public = no
writable = no
printable = no
create mask = 0765

mindtripper · August 18, 2020, 2:36pm

It will be painful to anonymize the complete logfile. What do you need from it?
Alternately I can share it with you only. If so. How?

Animosity022 · August 18, 2020, 2:39pm

You can PM if you want.

It's hard to say exactly what to look for as it can be many things.

If you see a lot of open/closes, you want to try out the new VFS beta with the new cache mode as that would help.

I don't think it's a good idea to put rclone on Samba at all as I'd just put rclone on whatever host needs it, but that's me.

I've seen a few posts on SMB bugs so could be something there.

mindtripper · August 18, 2020, 2:44pm

First off. Thank you for looking into this!
Check you inbox.

I can try the latest beta.

I'm doing what you suggest about putting rclone everywhere. This works, but I'd like to centralize things.

asdffdsa · August 18, 2020, 2:56pm

can you post your rclone config file, redacting password and ids.

mindtripper · August 19, 2020, 7:43am

Below is what I can share.

[jotta-open]
type = jottacloud
....

[jotta-crypt]
type = crypt
remote = jotta-open:Private
filename_encryption = standard
directory_name_encryption = true
....

mindtripper · August 19, 2020, 7:53am

Using smbclient on the Alpine Linux hosting the share, copying a file from share to /tmp results in this after about 10 seconds:

smb: \Remote\Folder\> get File.Name.Ext /tmp/File.Name.Ext parallel_read returned NT_STATUS_IO_TIMEOUT
smb: \Remote\Folder\> getting file \Remote\Folder\File.Name.Ext of size 5863869928 as /tmp/File.Name.Ext SMBecho failed (NT_STATUS_CONNECTION_DISCONNECTED). The connection is disconnected now

mindtripper · August 19, 2020, 8:17am

Tried the latest version.
rclone v1.52.3
- os/arch: linux/amd64
- go version: go1.14.7

Same result.

And tried the the latest beta version.
rclone v1.52.3-309-g47d08ac1-beta
- os/arch: linux/amd64
- go version: go1.15

Same result.

I's like to note that using mergerFS works fine. This filesystem tool also uses FUSE.
[ext 4 mountpoint -> samba share] -> windows client works beautifully.
[ext 4 mountpoint -> mergerfs -> samba share] -> windows client works beautifully.
[ext 4 mountpoint + rclone -> mergerfs -> samba share] -> windows client has the issue.
[rclone -> samba share] -> windows client has the issue.

system · October 19, 2020, 4:18am

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.