Slow transfer rate to nfs share mounted as local drive

[vincent-rg]

What is the problem you are having with rclone?

Low transfert rate with rclone copy of a file to a mounted nfs share

Run the command 'rclone version' and share the full output of the command.

rclone v1.63.1
- os/version: Microsoft Windows 10 Pro 22H2 (64 bit)
- os/kernel: 10.0.19045.3324 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.20.6
- go/linking: static
- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

local

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy d:\50MB.bin f:\temp --progress -vv --local-no-preallocate

note: if I don't use --local-no-preallocate, destination file will be padded with 0x00 up to a modulo 8192 size.
found this workaround on the forum.

The rclone config contents with secrets removed.

my config file contains only 2 webdav remotes, irrelevant for the purpose of this test.

A log from the command that you were trying to run with the -vv flag

D:\apps\rclone-v1.63.1-windows-amd64>rclone copy d:\50MB.bin f:\temp --progress -vv --local-no-preallocate
2023/09/10 12:43:37 DEBUG : rclone: Version "v1.63.1" starting with parameters ["rclone" "copy" "d:\\50MB.bin" "f:\\temp" "--progress" "-vv" "--local-no-preallocate"]
2023/09/10 12:43:37 DEBUG : Creating backend with remote "d:\\50MB.bin"
2023/09/10 12:43:37 DEBUG : Using config file from "C:\\Users\\vincent\\AppData\\Roaming\\rclone\\rclone.conf"
2023/09/10 12:43:37 DEBUG : local: detected overridden config - adding "{lUE4q}" suffix to name
2023/09/10 12:43:37 DEBUG : fs cache: adding new entry for parent of "d:\\50MB.bin", "local{lUE4q}://?/d:/"
2023/09/10 12:43:37 DEBUG : Creating backend with remote "f:\\temp"
2023/09/10 12:43:37 DEBUG : local: detected overridden config - adding "{lUE4q}" suffix to name
2023/09/10 12:43:37 DEBUG : fs cache: renaming cache item "f:\\temp" to be canonical "local{lUE4q}://?/f:/temp"
2023-09-10 12:43:37 DEBUG : 50MB.bin: Need to transfer - File not found at Destination
2023-09-10 12:45:02 DEBUG : 50MB.bin: md5 = 1699992acbe557095d85941d51e1a36c OK
2023-09-10 12:45:02 DEBUG : 50MB.bin.zopumuq2.partial: renamed to: 50MB.bin
2023-09-10 12:45:02 INFO  : 50MB.bin: Copied (new)
Transferred:       47.790 MiB / 47.790 MiB, 100%, 578.612 KiB/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:      1m24.7s
2023/09/10 12:45:02 INFO  :
Transferred:       47.790 MiB / 47.790 MiB, 100%, 578.612 KiB/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:      1m24.7s

2023/09/10 12:45:02 DEBUG : 5 go routines active

For the same file to the same locally mounted nfs share, robocopy reports a transfer rate of around 7MB/s (yes it's still slow but it's a raspberry pi 2 with 100MB ethernet link and hard drive on usb2 port).

I tried to find something similar, no luck so far.
I'd be glad to provide more informations or run other tests.

Vincent

[kapitainsky]

Any reason why you use rclone to copy files locally?

Clearly your test shows that it is not working very well. All mix - Windows, mounted NFS (which is not Windows native thing) and rclone does not sound like good solution for me.

I would switch to SFTP or SMB on raspberry pi and use appropriate rclone remotes until this specific issue is investigated and fixed - if somebody feels like willing to spend time on it.

[vincent-rg]

Hi,

Actually, I was running some tests on how to backup files on a local NFS share.
I am already using it under linux to encrypt and transfer VM backups from this NFS share to a cloud storage.
I thought I would use it too to backup my laptop files to this NFS share.
I just ended using rclone to encrypt and transfer my laptop files directly to the cloud storage.

I thought I would give a try to a SMB share, windows seems to handle this better.

I not a major drawback for me, just thought I would report this, just in case

Vincent

[kapitainsky]

You are correct - something is not right here. It should not be so slow and --local-no-preallocate story also points into something not working.

I am only afraid that it is such edge case that it might not be fixed quickly.

[vincent-rg]

Surely it's an edge case

I've just setup an SMB share on my raspberry, it works way better from windows!