Cannot access files when directory listing is not allowed in one of parent directories

Suspected Bug
1

What is the problem you are having with rclone?

It is often the case that some directories do contain child files/directories but not allow listing of their content. The most common examples are Unix folders with permission 711 (or 701) but not limited to that (I personally use such a webdav server application).
If I access such directories via rclone mount or rclone serve webdav, it shows nothing (as expected) because we can't access them.
However, even if I access some contents that actually exist in such directories, rclone says they're not found.
This seems due to rclone accesses files hierarchically from the root.
Is there any option to fix this, or is this a bug?

Run the command 'rclone version' and share the full output of the command.

1.67.0, 1.68.2 (latest)

rclone v1.68.2

  • os/version: Microsoft Windows 11 Home 24H2 (64 bit)
  • os/kernel: 10.0.26100.2605 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.23.3
  • go/linking: static
  • go/tags: cmount

rclone v1.67.0

  • os/version: Microsoft Windows 11 Home 24H2 (64 bit)
  • os/kernel: 10.0.26100.2605 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.22.4
  • go/linking: static
  • go/tags: cmount

(I also tried the latest in linux)
rclone v1.68.2

  • os/version: linuxmint 22 (64 bit)
  • os/kernel: 6.8.0-50-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.23.3
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

sftp, local, webdav

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone serve webdav --addr :8888 /
rclone mount --addr :8888 / M:
rclone serve webdav --baseurl /test --addr :8888 --sftp-ssh "ssh someserver" ssh:/

The rclone config contents with secrets removed.

(no specific config required for examples above, except basic ssh)
[ngc]
type = sftp
host = localhost
ssh = ssh dummy
shell_type = unix
md5sum_command = md5 -r
sha1sum_command = none

A log from the command with the -vv flag

In this case, /my/perm701 has permission 701 but /my/perm701/this_folder_exists and any of it's content have 755. Everything is owned by root.

rclone serve webdav -vv --addr :8888 /
2024/12/18 16:03:56 DEBUG : rclone: Version "v1.68.2" starting with parameters ["./rclone" "serve" "webdav" "-vv" "--addr" ":8888" "/"]
2024/12/18 16:03:56 DEBUG : Creating backend with remote "/"
2024/12/18 16:03:56 DEBUG : Using config file from "/home/myname/.config/rclone/rclone.conf"
2024/12/18 16:03:56 INFO  : Local file system at /: poll-interval is not supported by this remote
2024/12/18 16:03:56 NOTICE: Local file system at /: WebDav Server started on [http://[::]:8888/]
2024/12/18 16:04:01 DEBUG : /my/perm701/this_folder_exists/child: OpenFile: flags=O_RDONLY, perm=----------
2024/12/18 16:04:01 NOTICE: lib: Can't follow symlink without -L/--copy-links
2024/12/18 16:04:01 NOTICE: bin: Can't follow symlink without -L/--copy-links
2024/12/18 16:04:01 NOTICE: lib32: Can't follow symlink without -L/--copy-links
2024/12/18 16:04:01 NOTICE: libx32: Can't follow symlink without -L/--copy-links
2024/12/18 16:04:01 NOTICE: sbin: Can't follow symlink without -L/--copy-links
2024/12/18 16:04:01 NOTICE: lib64: Can't follow symlink without -L/--copy-links
2024/12/18 16:04:01 ERROR : my/perm701: failed to open directory "my/perm701": open /my/perm701: permission denied
2024/12/18 16:04:01 DEBUG : /my/perm701/this_folder_exists/child: >OpenFile: fd=<nil>, err=file does not exist
2024/12/18 16:04:01 INFO  : //my/perm701/this_folder_exists/child: GET from 192.168.1.212:57646
2

(Self-reply)

I've found a partial solution (works at least for my specific case).
I changed the implementation of my server application to create folders for every single digit of the ID.
For example, instead of placing all files 000000, 000001, ....., 999999 at a single directory (that doesn't allow file listing), each file can be accessed with a path like localhost:8888/1/2/3/4/5/6.
But this is still not applicable for normal local filesystems. (we may be able to create a kind of "reverse proxy" for this but don't like such overhead)

3

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.