When a rclone mount is done by a windows service as the local user SYSTEM, Administrator can't create anything in the mount point. Creating a new folder results in creating 4 folders that actually don't really exist. I know it's a permission issue as discussed in the Github issue #3216. Based on rclone mount documentation #windows-caveats, using nssm.exe would be a workaround but I don't know how.
Thank you for helping. That happens with Rclone 1.49.5 and WinFSP 1.4.19016. I just upgraded Rclone to latest 1.51.0 but it's still happens. What surprises me is that it used to work fine with the windows service mounting the rclone resource. I could browse the rclone mount in Windows Explorer as Administrator without any issues a few months back. Looks like it just started to happen.
I did. Uninstall 1.4, reboot, Install latest 1.6, stopped the windows service (created with nssm.exe), started the service again, opened Windows Explorer, created a New folder and bam, 4 folders again...
Don't know what to do.
seems to me that dragondisk is not creating the empty folder, but ceph itself is.
s3 does not any concept of a folder.
ceph is not s3 and perhaps it has a concept of a folder and allowing empty folders to exist.
different software 'creates' a s3 'folder' in different ways.
TLDR: ncw is awesome!
recently, i ran into a problem.
rclone had one definition of a folder
cloudberry/wasabi had a different definition of a folder
rclone considered that 'folder' created with cloudberry/wasabi to be a file with no name and of zero size.
ncw saved the day, he created a beta that fixed the problem, that will be merged in v1.52.
you can read all about it here.
Thanks for these clues. I will downgrade the packages on the RGW and try again. This is the only change I can see for now besides any Windows updates that might have broken WinFSP. As I said previously, creating empty folders was working fine until recently and Ceph RGWs handles this well whatever the client.
First of all, let me agree with this statement : "TLDR: ncw is awesome!"
For a moment I thought I was hitting the same one as you hit as I had seen this messages ERROR : : Entry doesn't belong in directory "" many times in the past using the linux version of rlcone. But... Trying the latest rclone-beta-latest-windows-amd64.zip did not help with the 4 folders being created. I'm left with downgrading yum packages on one of our RGWs.
You guys have a nice day.
Well well well... I could isolate one of the RGWs and downgrade the packages to the version that I know was working fine. Then I forced Windows network traffic to go through this RGWs (entry in hosts file) and it appears that the Ceph version is not the cause. Creating a folder still results in showing 4 folders, with actually none created.
What I don't understand is why this is now happening as I don't remember ever encountering this issue before, when I made all my tests of the rclone mount in September (Rclone download) and the windows user attached to the rclone mount service was already the .\SYSTEM user.
When mounting from the CLI with no user privileges elevation (Windows+R + 'cmd') I can create a folder with no issues (virtually as the folder is not actually created until a file is dropped in it) and it doesn't turn into 4 folders being created (or not created actually).
When mounting from the CLI with Administrator privileges elevation ('cmd' run as Administrator) then it doesn't work and a folder creation turns into 4 folders... Same as if the mount was started by the service under the .\SYSTEM account.
I'm not sure how to solve this. Should I use a specific user to run the windows service?
This is what Bill Ziss (the WinFSP developer) said about the 4 directory problem
This is almost always a permission issue. Windows Explorer has this weird behavior where if it gets "Access Denied" during directory creation it retries it 4 times! In order for this problem to be resolved you will have to fix the permissions that the file system reports.
This could perhaps be a permission problem of the underlying s3 system, but I still suspect some problem with the actual WinFSP config.
Note that s3 doesn't really support empty directories without making directory markers and rclone doesn't make directory markers.
This means you can create empty directories in the mount but they will disappear after the directory cache time expires.
Some clients do create directory markers (which are 0 length files ending in /).
BTW I run the integration tests with rclone against CEPH+RGQ both swift and s3 interfaces.
As an experiment open a cmd window as a normal user and run the rclone mount command. See if it works like that.