Mounting (nfsmount) massive directory stalls

o1o1oo1 · October 19, 2024, 7:16am

Hello.

The case: huge Google Drive directory (100K+ files, one directory), used to work fine with Macfuse, but because of MacOS (Sequoia, that breaks things) numerous flaws I need to migrate to NFS.

rclone nfsmount drive: /Photos --config /.rclone.conf --log-file /clone.log --cache-dir /.cache/rclone --vfs-cache-mode full --vfs-cache-max-size 200G --vfs-read-chunk-streams 4 --volname Photos --noapplexattr --allow-other --dir-cache-time 9999h --poll-interval 1h --attr-timeout 9999h --allow-non-empty --log-level ERROR --drive-pacer-burst 200 --drive-pacer-min-sleep 10ms

The above command just stalls with no errors. Would you have any idea how to make it work?

kapitainsky · October 19, 2024, 7:30am

I won't comment about nfsmount but have you tried FUSE-T instead of MacFUSE for normal mount? It does not need any kext and is also using NFS - but one provided by Apple.

kapitainsky · October 19, 2024, 10:36am

And regarding nfsmount, as per docs:

Note that --nfs-cache-handle-limit controls the maximum number of cached file handles stored by the nfsmount caching handler. This should not be set too low or you may experience errors when trying to access files. The default is 1000000, but consider lowering this limit if the server's system resource usage causes problems.

I am not sure how many handlers are "consumed" per file but I would try to increase this value.

Please note that nfsmount development is still work in progress and at least on macOS it has known bugs:

github.com/rclone/rclone

Microsoft Office apps can't write on an S3 mount when mounted with nfsmount on MacOS 14.5

opened 06:15PM - 24 Jul 24 UTC

alexbet

bug OS: Mac Serving

#### What is the problem you are having with rclone? I started using the nfsmou…nt option to mount Amazon S3 drives because it is very problematic to give the proper permissons for the kernel extensions for MacFuse when dealing with remote computers. At first, I thought that the nfs mount worked perfectly. It does, but then I realized that Microsoft Office apps Word, Excel, Powerpoint, can NOT write on the mounted drive. They can only read. All other apps on my Mac can write on the mounted drive, just not the Microsoft apps, which is very weird. I tried playing with these options: --vfs-cache-mode --dir-perms --file-perms But nothing worked. This makes using Rclone very problematic for us because most of us use MS Office daily. The way we go around it now is to, save the file locally, and then move it to the mounted drive, or vice versa. Definitely not ideal! #### What is your rclone version (output from `rclone version`) 1.67.0 #### Which OS you are using and how many bits (e.g. Windows 7, 64 bit) Mac OS Sonoma 14.5, 64 bit, M chip #### Which cloud storage system are you using? (e.g. Google Drive) Amazon S3 #### The command you were trying to run (e.g. `rclone copy /tmp remote:tmp`) rclone nfsmount s3:7c-users/acamastra ~$USER/rClone/files/user-files -vv \ --vfs-cache-mode full \ --vfs-cache-max-size 10G \ --vfs-cache-max-age 9999h \ --dir-cache-time 9999h \ --vfs-refresh \ --vfs-fast-fingerprint \ --volname Antoinettes-Files \ #### A log from the command with the `-vv` flag (e.g. output from `rclone -vv copy /tmp remote:tmp`) I tried to open the file "test.docx" and write to it, but couldn't. Here is the log file: [Terminal Saved Output.txt](https://github.com/user-attachments/files/16366136/Terminal.Saved.Output.txt) #### How to use GitHub * Please use the 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to show that you are affected by the same issue. * Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue. * Subscribe to receive notifications on status change and new comments.

Hence from my experience I would recommend FUSE-T on macOS instead.

o1o1oo1 · October 20, 2024, 8:11am

I've just tried Fuse-T. Works, but is unstable (Finder has a tendency to hang). I'll try with Rclone native NFS and more handlers...

Thank you for the help.

kapitainsky · October 20, 2024, 8:51am

most likely it hangs as updating directory every few minutes. Add:

--vfs-refresh --dir-cache-time 9999h

to load all data structure on mount start in the background (give it some time to see results). It is polling remote so also no need to invalidate it on timer.

o1o1oo1 · October 21, 2024, 12:18pm

Thank you. For other reasons I'm on latest Sequoia beta, so MacFuse does not load (known boot loop), Fuse-T works on and off and extremely slow and NFS serve also has hiccup. I guess I've got to wait for promised native Apple Fuse API (no kernel ext.).

system · November 20, 2024, 12:19pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.