Using mount on macOS only reports 997 MB available on the remote

Help and Support
1

What is the problem you are having with rclone?

I configured a remote to use my NAS as a remote. I'm using SFTP. My intention is to mount the NAS so I can add some files to it from macOS's UI. However, when mounting the remote, Finder reports only 997MB
is available on the mounted directory. I have over 30TBs of storage free on this NAS, and the mounted folder is not constrained by any settings (i.e. I did not enable a storage quota).

Screenshot 2023-09-10 at 4.06.54 PM
Screenshot 2023-09-10 at 4.06.54 PM2064×1208 140 KB

Run the command 'rclone version' and share the full output of the command.

rclone v1.61.0

Which cloud storage system are you using? (eg Google Drive)

SFTP

The command you were trying to run (eg rclone copy /tmp remote:tmp)

DATE=`date +%Y_%m_%d_%H_%M_%S`
NAME="contentadmin_ikkokiri_mount"
LOG_NAME=$NAME\_$DATE.log
LOG_FILE="/Users/andyibanez/.logs/rclone/$LOG_NAME"
rclone mount contentadmin_ikkokiri: /Users/andyibanez/.mount/contentadmin_ikkokiri --vfs-cache-mode full --log-level=NOTICE --log-file=$LOG_FILE  --buffer-size 256M --dir-cache-time 72h --cache-dir /Users/andyibanez/.temp/rclone_cache --cache-tmp-upload-path /Users/andyibanez/.temp/rclone_cache --vfs-read-ahead 20G --vfs-read-chunk-size 128M --vfs-cache-max-size=50G

The rclone config contents with secrets removed.

[contentadmin_ikkokiri]
type = sftp
host = REMOVED
user = THE USER
pass = REMOVED
shell_type = unix
md5sum_command = none
sha1sum_command = none

A log from the command that you were trying to run with the -vv flag

(I changed the debug level to DEBUG, which should be equivalent to -vv.)

2023/09/10 16:05:08 DEBUG : rclone: Version "v1.61.0" starting with parameters ["rclone" "mount" "contentadmin_ikkokiri:" "/Users/andyibanez/.mount/contentadmin_ikkokiri" "--vfs-cache-mode" "full" "--log-level=DEBUG" "--log-file=/Users/andyibanez/.logs/rclone/contentadmin_ikkokiri_mount_2023_09_10_16_05_08.log" "--buffer-size" "256M" "--dir-cache-time" "72h" "--cache-dir" "/Users/andyibanez/.temp/rclone_cache" "--cache-tmp-upload-path" "/Users/andyibanez/.temp/rclone_cache" "--vfs-read-ahead" "20G" "--vfs-read-chunk-size" "128M" "--vfs-cache-max-size=50G"]
2023/09/10 16:05:08 DEBUG : Creating backend with remote "contentadmin_ikkokiri:"
2023/09/10 16:05:08 DEBUG : Using config file from "/Users/andyibanez/.config/rclone/rclone.conf"
2023/09/10 16:05:09 DEBUG : sftp://contentadmin@kokiri.taila1803.ts.net:22/: New connection 100.70.202.16:60736->100.105.213.7:22 to "SSH-2.0-OpenSSH_8.2"
2023/09/10 16:05:09 DEBUG : sftp://contentadmin@kokiri.taila1803.ts.net:22/: Shell type "unix" from config
2023/09/10 16:05:09 DEBUG : sftp://contentadmin@kokiri.taila1803.ts.net:22/: Relative path resolved to "/"
2023/09/10 16:05:09 DEBUG : sftp://contentadmin@kokiri.taila1803.ts.net:22/: Using root directory "/"
2023/09/10 16:05:09 INFO  : sftp://contentadmin@kokiri.taila1803.ts.net:22/: poll-interval is not supported by this remote
2023/09/10 16:05:09 DEBUG : vfs cache: root is "/Users/andyibanez/.temp/rclone_cache"
2023/09/10 16:05:09 DEBUG : vfs cache: data root is "/Users/andyibanez/.temp/rclone_cache/vfs/contentadmin_ikkokiri"
2023/09/10 16:05:09 DEBUG : vfs cache: metadata root is "/Users/andyibanez/.temp/rclone_cache/vfsMeta/contentadmin_ikkokiri"
2023/09/10 16:05:09 DEBUG : Creating backend with remote "/Users/andyibanez/.temp/rclone_cache/vfs/contentadmin_ikkokiri/"
2023/09/10 16:05:09 DEBUG : fs cache: renaming cache item "/Users/andyibanez/.temp/rclone_cache/vfs/contentadmin_ikkokiri/" to be canonical "/Users/andyibanez/.temp/rclone_cache/vfs/contentadmin_ikkokiri"
2023/09/10 16:05:09 DEBUG : Creating backend with remote "/Users/andyibanez/.temp/rclone_cache/vfsMeta/contentadmin_ikkokiri/"
2023/09/10 16:05:09 DEBUG : fs cache: renaming cache item "/Users/andyibanez/.temp/rclone_cache/vfsMeta/contentadmin_ikkokiri/" to be canonical "/Users/andyibanez/.temp/rclone_cache/vfsMeta/contentadmin_ikkokiri"
2023/09/10 16:05:09 DEBUG : Mounting on "/Users/andyibanez/.mount/contentadmin_ikkokiri" ("contentadmin_ikkokiri")
2023/09/10 16:05:09 DEBUG : Adding "-o modules=iconv,from_code=UTF-8,to_code=UTF-8-MAC" for macOS
2023/09/10 16:05:09 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2023/09/10 16:05:09 DEBUG : sftp://contentadmin@kokiri.taila1803.ts.net:22/: Mounting with options: ["-o" "attr_timeout=1" "-o" "fsname=contentadmin_ikkokiri:" "-o" "subtype=rclone" "-o" "max_readahead=131072" "-o" "atomic_o_trunc" "-o" "daemon_timeout=600" "-o" "volname=contentadmin_ikkokiri" "-o" "noappledouble" "-o" "modules=iconv,from_code=UTF-8,to_code=UTF-8-MAC"]
2023/09/10 16:05:09 DEBUG : sftp://contentadmin@kokiri.taila1803.ts.net:22/: Init: 
2023/09/10 16:05:09 DEBUG : sftp://contentadmin@kokiri.taila1803.ts.net:22/: >Init: 
2023/09/10 16:05:09 DEBUG : /: Statfs: 
2023/09/10 16:05:09 DEBUG : sftp://contentadmin@kokiri.taila1803.ts.net:22/: Server has VFS statistics extension
2023/09/10 16:05:09 DEBUG : sftp://contentadmin@kokiri.taila1803.ts.net:22/: About path "/"
2023/09/10 16:05:09 DEBUG : /: Statfs: 
2023/09/10 16:05:09 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:596382 Bfree:243623 Bavail:243623 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2023/09/10 16:05:09 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:596382 Bfree:243623 Bavail:243623 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2023/09/10 16:05:09 DEBUG : /: Statfs: 
2023/09/10 16:05:09 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:596382 Bfree:243623 Bavail:243623 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2023/09/10 16:05:09 DEBUG : /: Statfs: 
2023/09/10 16:05:09 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:596382 Bfree:243623 Bavail:243623 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2023/09/10 16:05:09 DEBUG : /: Statfs: 
2023/09/10 16:05:09 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:596382 Bfree:243623 Bavail:243623 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2023/09/10 16:05:09 DEBUG : /: Access: mask=00
2023/09/10 16:05:09 DEBUG : /: >Access: errc=0
2023/09/10 16:05:09 DEBUG : /: Access: mask=00
2023/09/10 16:05:09 DEBUG : /: >Access: errc=0
2023/09/10 16:05:09 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/09/10 16:05:09 DEBUG : /: >Getattr: errc=0
(these repeat ad-infinitum)

I'm dealing with a situation in which I cannot use rclone copy or move directly, so I need to be able get the mount to report the right size.

1 Like
2

Please update to the latest version first - v1.63.1 now

Could you run:

  1. rclone about contentadmin_ikkokiri:

  2. sftp user@host
    and then
    df -h

and report results.

3

Thanks, I have updated it for now, though I don't think it includes any specific fixes for this.

Running rclone about contentadmin_ikkokiri:

rclone about contentadmin_ikkokiri:/
Total:   2.275 GiB
Used:    1.343 GiB
Free:    954.484 MiB

sftp> df -h
    Size     Used    Avail   (root)    %Capacity
   2.3GB    1.3GB    838MB    954MB          59%

I don't understand why :melting_face: but this points me to the right direction that it appears to be Synology SFTP issue instead of rclone, so thanks for the tips!

1 Like
4

Good thinking:) As it exactly proves that rclone only dully reports what is provided with:)

1 Like
5

Also when you find what the problem was would be nice if you share it. Can be helpful for others with similar issues.

6

Yeah I’m researching like a madman, but no luck. I have discarded it being an account issue, as this an admin account, and a storage quota issue, as I can’t assign storage quotas to admins (on Synology anyway).

7

Can you SSH to the nas? If so ssh in and try du -h from the command line.

I suspect it has more than one disk and df via sftp is only reporting the size of one of them.

It might be that you can cd into another disk and get df to give a different answer when using sftp also.

8

Yeah, my NAS has 8 16TB hard drives, one 700GB storage SSD, and an external hard drive. Nothing as small as 3GB though. Doing df -h after SSHhing into the NAS:

user@nas:/$ df -h
Filesystem              Size  Used Avail Use% Mounted on
/dev/md0                2.3G  1.4G  842M  63% /
devtmpfs                 16G     0   16G   0% /dev
tmpfs                    16G  248K   16G   1% /dev/shm
tmpfs                    16G   24M   16G   1% /run
tmpfs                    16G     0   16G   0% /sys/fs/cgroup
tmpfs                    16G  1.8M   16G   1% /tmp
/dev/mapper/cachedev_0   84T   53T   31T  64% /volume1
/dev/usb1p1              13T   11T  2.6T  80% /volumeUSB1/usbshare

It looks like when I sftp into it or use rclone it reports the size for /dev/md0. No clue what it is.

EDIT: cd'ing into the main storage pool, volume1, I see the same output as above.

Thank you guys for trying to help, even though I'm sure this escapes the realm of rclone at this point.

9

I just had a look at the rclone source code and it is defaulting to using the SFTP equivalent of df on /. This is probably because the SFTP server doesn't support RealPath so rclone isn't managing to look up where the home directory is. Or maybe the SFTP server is chrooted.

This does suggest a workaround though/

If you know exactly where in the SFTP path your home directory is (as in the directory that you get to with just contentadmin_ikkokiri:) then you could put it in, and rclone should report the size properly.

Eg something like

rclone mount contentadmin_ikkokiri:/volume1/ ...

Try with this first

rclone about contentadmin_ikkokiri:/volume1/

I don't know exactly what that path is though, and it may not work due to the way the SFTP is set up on the NAS (it might be chrooted).

1 Like
10

Interesting it's taking a while to run this, but I will report back.

1 Like