ERROR : sftp://url:22//: Statfs failed: your remote may not have the required df utility: failed to run "df -k /": : ssh: command df -k / failed

rvm1975 · October 31, 2023, 3:51pm

What is the problem you are having with rclone?

Everything works fine but console output filled with error "ERROR : sftp://url:22//: Statfs failed: your remote may not have the required df utility: failed to run "df -k /": : ssh: command df -k / failed"

SSH server is Azure SFTP for blob storage

Run the command 'rclone version' and share the full output of the command.

rclone v1.64.2

os/version: Microsoft Windows 11 Enterprise 22H2 (64 bit)
os/kernel: 10.0.22621.2428 (x86_64)
os/type: windows
os/arch: amd64
go/version: go1.21.3
go/linking: static
go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

SFTP

.\rclone mount --vfs-cache-mode writes --dir-cache-time 2s --poll-interval 1s blob:/ s:

The rclone config contents with secrets removed.

[blob]
type = sftp
host = XXXX.blob.core.windows.net
user = XXXX.input.YYYY
pass = ZZZZ
disable_hashcheck = true

A log from the command with the `-vv` flag

2023/10/31 17:43:46 DEBUG : rclone: Version "v1.64.2" starting with parameters ["C:\\TEMP\\rclone-v1.64.2-windows-amd64\\rclone.exe" "mount" "-vv" "--vfs-cache-mode" "writes" "--dir-cache-time" "2s" "--poll-interval" "1s" "blob:/" "s:"]
2023/10/31 17:43:46 DEBUG : Creating backend with remote "blob:/"
2023/10/31 17:43:46 DEBUG : Using config file from "C:\\TEMP\\rclone-v1.64.2-windows-amd64\\rclone.conf"
2023/10/31 17:43:47 DEBUG : URL:22: New connection 10.25.160.106:64363->20.60.197.68:22 to "SSH-2.0-AzureSSH_1.0.0"
2023/10/31 17:43:47 DEBUG : URL:22: Shell type "shellTypeNotSupported" from config
2023/10/31 17:43:47 DEBUG : URL:22: Using root directory "/"
2023/10/31 17:43:47 INFO  : URL:22: poll-interval is not supported by this remote
2023/10/31 17:43:47 DEBUG : vfs cache: root is "C:\\Users\\XXXX\\AppData\\Local\\rclone"
2023/10/31 17:43:47 DEBUG : vfs cache: data root is "\\\\?\\C:\\Users\\XXXX\\AppData\\Local\\rclone\\vfs\\blob"
2023/10/31 17:43:47 DEBUG : vfs cache: metadata root is "\\\\?\\C:\\Users\\XXXX\\AppData\\Local\\rclone\\vfsMeta\\blob"
2023/10/31 17:43:47 DEBUG : Creating backend with remote "C:/Users/XXXX/AppData/Local/rclone/vfs/blob//"
2023/10/31 17:43:47 DEBUG : fs cache: renaming cache item "C:/Users/XXXX/AppData/Local/rclone/vfs/blob//" to be canonical "//?/C:/Users/XXXX/AppData/Local/rclone/vfs/blob"
2023/10/31 17:43:47 DEBUG : Creating backend with remote "C:/Users/XXXX/AppData/Local/rclone/vfsMeta/blob//"
2023/10/31 17:43:47 DEBUG : fs cache: renaming cache item "C:/Users/XXXX/AppData/Local/rclone/vfsMeta/blob//" to be canonical "//?/C:/Users/XXXX/AppData/Local/rclone/vfsMeta/blob"
2023/10/31 17:43:47 DEBUG : Network mode mounting is disabled
2023/10/31 17:43:47 DEBUG : Mounting on "s:" ("blob")
2023/10/31 17:43:47 DEBUG : URL:22: Mounting with options: ["-o" "attr_timeout=1" "-o" "uid=-1" "-o" "gid=-1" "--FileSystemName=rclone" "-o" "volname=blob"]
2023/10/31 17:43:47 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item JOB1111.TXT not removed, freed 0 bytes
2023/10/31 17:43:47 INFO  : vfs cache: cleaned: objects 1 (was 1) in use 0, to upload 0, uploading 0, total size 1.055Ki (was 1.055Ki)
2023/10/31 17:43:47 DEBUG : URL:22: Init:
2023/10/31 17:43:47 DEBUG : URL:22: >Init:
2023/10/31 17:43:47 DEBUG : /: Statfs:
2023/10/31 17:43:47 DEBUG : URL:22: Server does not have the VFS statistics extension, trying shell command instead
2023/10/31 17:43:47 DEBUG : URL:22: Shell path "/"
2023/10/31 17:43:47 DEBUG : URL:22: About path "/"
2023/10/31 17:43:47 DEBUG : URL:22: About using shell command for shell type "shellTypeNotSupported"
2023/10/31 17:43:47 DEBUG : URL:22: Running remote command: df -k /
2023/10/31 17:43:47 DEBUG : URL:22: About shell command for shell type "shellTypeNotSupported" failed (set option shell_type to override): failed to run "df -k /": : ssh: command df -k / failed
2023/10/31 17:43:47 ERROR : URL:22: Statfs failed: your remote may not have the required df utility: failed to run "df -k /": : ssh: command df -k / failed
2023/10/31 17:43:47 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:8796093022207 Bfree:8796093022207 Bavail:8796093022207 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2023/10/31 17:43:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/10/31 17:43:47 DEBUG : /: >Getattr: errc=0
2023/10/31 17:43:47 DEBUG : /: Readlink:
2023/10/31 17:43:47 DEBUG : /: >Readlink: linkPath="", errc=-40
2023/10/31 17:43:47 DEBUG : /: Getxattr: name="non-existant-a11ec902d22f4ec49003af15282d3b00"
2023/10/31 17:43:47 DEBUG : /: >Getxattr: errc=-40, value=""
The service rclone has been started.
2023/10/31 17:43:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/10/31 17:43:47 DEBUG : /: >Getattr: errc=0
2023/10/31 17:43:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/10/31 17:43:47 DEBUG : /: >Getattr: errc=0
2023/10/31 17:43:47 DEBUG : /: Opendir:
2023/10/31 17:43:47 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/10/31 17:43:47 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/10/31 17:43:47 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/10/31 17:43:47 DEBUG : /: Statfs:
2023/10/31 17:43:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/10/31 17:43:47 DEBUG : /: >Getattr: errc=0
2023/10/31 17:43:47 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2023/10/31 17:43:47 DEBUG : /: Releasedir: fh=0x0
2023/10/31 17:43:47 DEBUG : /: >Releasedir: errc=0
2023/10/31 17:43:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/10/31 17:43:47 DEBUG : /: >Getattr: errc=0
2023/10/31 17:43:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/10/31 17:43:47 DEBUG : /: >Getattr: errc=0
2023/10/31 17:43:47 DEBUG : /: Opendir:
2023/10/31 17:43:47 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/10/31 17:43:47 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/10/31 17:43:47 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/10/31 17:43:47 DEBUG : /: Releasedir: fh=0x0
2023/10/31 17:43:47 DEBUG : /: >Releasedir: errc=0
2023/10/31 17:43:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/10/31 17:43:47 DEBUG : /: >Getattr: errc=0
2023/10/31 17:43:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/10/31 17:43:47 DEBUG : /: >Getattr: errc=0
2023/10/31 17:43:47 DEBUG : /: Opendir:
2023/10/31 17:43:47 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/10/31 17:43:47 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/10/31 17:43:47 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/10/31 17:43:47 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/10/31 17:43:47 DEBUG : /: >Getattr: errc=0
2023/10/31 17:43:47 DEBUG : /: Releasedir: fh=0x0
2023/10/31 17:43:47 DEBUG : /: >Releasedir: errc=0
2023/10/31 17:43:50 DEBUG : /autorun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/10/31 17:43:50 DEBUG : /autorun.inf: >Getattr: errc=-2
2023/10/31 17:43:50 DEBUG : /autorun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/10/31 17:43:50 DEBUG : /autorun.inf: >Getattr: errc=-2
2023/10/31 17:43:50 DEBUG : /autorun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/10/31 17:43:50 DEBUG : /autorun.inf: >Getattr: errc=-2
2023/10/31 17:43:50 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/10/31 17:43:50 DEBUG : /: >Getattr: errc=0
2023/10/31 17:43:50 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/10/31 17:43:50 DEBUG : /: >Getattr: errc=0
2023/10/31 17:43:50 DEBUG : /: Opendir:
2023/10/31 17:43:50 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/10/31 17:43:50 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/10/31 17:43:50 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/10/31 17:43:50 DEBUG : /: Releasedir: fh=0x0
2023/10/31 17:43:50 DEBUG : /: >Releasedir: errc=0
2023/10/31 17:43:50 DEBUG : /AutoRun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/10/31 17:43:50 DEBUG : /AutoRun.inf: >Getattr: errc=-2
2023/10/31 17:43:54 DEBUG : : forgetting directory cache
2023/10/31 17:43:54 DEBUG : results: forgetting directory cache
2023/10/31 17:43:54 DEBUG : network_drive: forgetting directory cache
2023/10/31 17:43:54 DEBUG : network_drive: forgetting directory cache
2023/10/31 17:43:54 DEBUG : results: forgetting directory cache

kapitainsky · October 31, 2023, 4:02pm

As per docs here and here. It would indicate that Azure SFTP does not have required features. SFTP should work regardless - but you might not have correct info about available free space etc.

rvm1975 · October 31, 2023, 6:07pm

Could you be so kind and tell what exactly I should search on mentioned pages. There are no word "Azure" on both pages.
Also per description of "Shell access" - "Rclone tries to auto-detect what type of shell is used on the server" and should set " Value none can be set to avoid any attempts at executing shell commands, e.g. if this is not allowed on the server."

kapitainsky · October 31, 2023, 6:12pm

search for what you see in your error message:

df

It is not supported by your SFTP server - not the end of the world.

Either somebody can investigate what Azure SFTP is doing/using and maybe make rclone compatible. Or report it to Microsoft - they often do things "differently". Somebody has to bend and become compliant:)

rclone expects SFTP server to provide some basic standard functionality - if somebody created SFTP server not doing it... then you have your ERROR message.

Other solution if you use Azure - use native protocol - rclone supports it.

rvm1975 · November 1, 2023, 11:19am

Thank you for explanation.
Actually I am complaining about error messages behavior.
Even if you set shell_type = none you will get tons of messages like

2023/11/01 13:15:34 ERROR : URL:22//: Statfs failed: not supported with shell type "none"

ncw · November 1, 2023, 10:44pm

Hmm, I think that is a misfeature.

I think rclone needs a special error to be returned from the About call so the VFS doesn't report that error and the VFS falls back to reporting what it usually does.

Can you make a GitHub issue about this and I'll look at fixing it.

Thanks

system · December 1, 2023, 10:45pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.