Strange unionfs-fuse permissions with rclone mount as RO directory

Help and Support
#1

I have about 10 TB of media files in ACD, some of which have fairly wonky permissions and I’d just like to chown and chmod the entire thing recursively so that they’re consistent. (I haven’t yet deciphered how ACD actually decides what umask to apply to new files since they’re all uniform on my local drive before uploading but perms are seemingly random after upload.)

What’s the best way to do this without having to download all 10 TB and re-upload it? If I do it directly on an rclone mount point, rclone starts downloading everything without any indication of the process of the download or any ETA. If I do this on a fuse mount, same issue. Is there a way to use server-side copying to help with this?

#2

Not sure I understand the issue. Rclone doesn’t set permissions or ownership on ACD and as far as I know ACD doesnt support it either.

How are you viewing the permissions? How did you upload the files in the first place?

What exactly are you doing here?

#3

Sorry, I wasn’t very clear… For example, I have a folder where the folder itself and all files inside are umask 777. If I rclone sync that folder and then hop over to the rclone mount point to the just uploaded folder and do an ls -l, it lists the files as writable by me and some seemingly random permissions for everybody else. Sometimes the umask is 666, sometimes it’s 600, sometimes it’s 430, sometimes it’s something else. There is certainly some sort of permissions being set in ACD that appear on the rclone mount.

Does that make any sense?

#4

rclone mount sets the permissions of all directories to 0777 and all files to 0666 modified by your umask which would normally be 002, so directories will be 0775 and files 0664.

So I don’t understand why you would see anything other than 775 and 664 in an rclone mount.

What is the command line you are using to start the rclone mount?

#5

Thanks for the reply. I think I’ve narrowed this down to something funky that fuse is inferring from the rclone mount perms.

Here are my rclone mount and fuse commands (I’ve tried to remove almost all options so I can diagnose):

rclone mount acd_enc_media: /mnt/acd_media &

unionfs-fuse -o cow,allow_other /mnt/local_media=RW:/mnt/acd_media=RO /mnt/media/

My rclone mount and fuse mount show the same perms on this file…

rclone mount point:

plex@caprica:/mnt/acd_media/Movies/SomeMovie (2016)$ ls -l
total 4068841
-rw-rw-r-- 1 plex plex 4166492429 Feb 14 19:04 SomeMovie (2016) - WEBDL-720p.mkv

fuse mount point:

plex@caprica:/mnt/media/Movies/SomeMovie (2016)$ ls -l
total 4068841
-rw-rw-r-- 1 plex plex 4166492429 Feb 14 19:04 SomeMovie (2016) - WEBDL-720p.mkv

But if I try to remove the file at fuse mount point, I get permission denied (even if sudo):

plex@caprica:/mnt/media/Movies/SomeMovie (2016)$ rm *
rm: cannot remove 'SomeMovie (2016) - WEBDL-720p.mkv': Permission denied

Try to remove the same file at rclone mount point, no problem:

plex@caprica:/mnt/acd_media/Movies/SomeMovie (2016)$ rm *
plex@caprica:/mnt/acd_media/Movies/SomeMovie (2016)$

Any ideas? I have tried this with a fuse mount of a non-rclone crypt mount and it works as expected.

#6

Isn’t that what you’ve specified in the command line with /mnt/acd_media=RO ?

unionfs-fuse is refusing to delete stuff out of /mnt/adc_media?

I’ve never used unionfs-fuse so that is just a guess!

#7

The way fuse works (or is supposed to work…) is that any writes in the fused mount go to the RW dir, and any reads come from the RO dir. If a file is marked for deletion in the RW dir, it gets “whiteout” and should disappear from the fused mount dir.

Lots of folks rclone move then sync around this to allow for deleted files to be removed from the cloud drive:

# Move new local files to the cloud and sync away files marked for deletion by UnionFS.
#su - user -c "rclone move localfiles/ amazoncrypt: --exclude=.unionfs/** --log-file=/home/user/CloudCopy.log -v"
#su - user -c "rclone sync unionfolder/ amazoncrypt: --exclude=.unionfs/** --log-file=/home/user/CloudSync.log -v"

# Clean up files marked as deleted by UnionFS as well as empty folders left by rclone move.
#su - user -c "find localfiles/.unionfs/* -exec rm -rf {} \;"
#su - user -c "find localfiles/* -type d -empty -delete"
#8

OK…

Another idea - check the permissions on the parent directories of the file you are trying to delete. Unix needs write permissions on the directory to delete a file within it…

#9

Good thinking- shows as writable in both dirs for this user, but different umask:

plex@caprica:~$ ls -l /mnt/media/Movies/
total 0
drwxrwxr-x 1 plex plex 0 Feb 14 19:30 SomeMovie (2016)

plex@caprica:~$ ls -l /mnt/acd_media/Movies/
total 0
drwxrwxr-x 1 plex plex 0 Feb 14 19:30 SomeMovie (2016)

Those dir permissions are the same all the way up to /mnt/ for both.

#10

Those permissions look OK.

I presume you are doing your tests as the plex user?

What happens if you run the failing strace rm SomeMove? Can you post the output?

#11

Correct, doing everything as the plex user (including the mounting).

Here’s the strace:

plex@caprica:/mnt/media/Movies/X2 (2003)$ strace rm *
execve("/bin/rm", ["rm", "X2 (2003) - Bluray-720p.mkv"], [/* 20 vars */]) = 0
brk(NULL)                               = 0x60f000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff6affd7000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=89783, ...}) = 0
mmap(NULL, 89783, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff6affc1000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\t\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1864888, ...}) = 0
mmap(NULL, 3967392, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7ff6af9eb000
mprotect(0x7ff6afbaa000, 2097152, PROT_NONE) = 0
mmap(0x7ff6afdaa000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1bf000) = 0x7ff6afdaa000
mmap(0x7ff6afdb0000, 14752, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7ff6afdb0000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff6affc0000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff6affbf000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff6affbe000
arch_prctl(ARCH_SET_FS, 0x7ff6affbf700) = 0
mprotect(0x7ff6afdaa000, 16384, PROT_READ) = 0
mprotect(0x60d000, 4096, PROT_READ)     = 0
mprotect(0x7ff6affd9000, 4096, PROT_READ) = 0
munmap(0x7ff6affc1000, 89783)           = 0
brk(NULL)                               = 0x60f000
brk(0x630000)                           = 0x630000
open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=3583552, ...}) = 0
mmap(NULL, 3583552, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff6af680000
close(3)                                = 0
ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0
newfstatat(AT_FDCWD, "X2 (2003) - Bluray-720p.mkv", {st_mode=S_IFREG|0664, st_size=7016633494, ...}, AT_SYMLINK_NOFOLLOW) = 0
geteuid()                               = 1001
newfstatat(AT_FDCWD, "X2 (2003) - Bluray-720p.mkv", {st_mode=S_IFREG|0664, st_size=7016633494, ...}, AT_SYMLINK_NOFOLLOW) = 0
faccessat(AT_FDCWD, "X2 (2003) - Bluray-720p.mkv", W_OK) = 0
unlinkat(AT_FDCWD, "X2 (2003) - Bluray-720p.mkv", 0) = -1 EACCES (Permission denied)
open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=2995, ...}) = 0
read(3, "# Locale name alias data base.\n#"..., 4096) = 2995
read(3, "", 4096)                       = 0
close(3)                                = 0
open("/usr/share/locale/en_GB.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_GB.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_GB/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en_GB.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en_GB.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en_GB/LC_MESSAGES/coreutils.mo", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=14622, ...}) = 0
mmap(NULL, 14622, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff6affd3000
close(3)                                = 0
open("/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=619, ...}) = 0
mmap(NULL, 619, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff6affd2000
close(3)                                = 0
write(2, "rm: ", 4rm: )                     = 4
write(2, "cannot remove 'X2 (2003) - Blura"..., 43cannot remove 'X2 (2003) - Bluray-720p.mkv') = 43
open("/usr/share/locale/en_GB.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_GB.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_GB/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en_GB.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en_GB.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en_GB/LC_MESSAGES/libc.mo", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=3537, ...}) = 0
mmap(NULL, 3537, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff6affd1000
close(3)                                = 0
open("/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, ": Permission denied", 19: Permission denied)     = 19
write(2, "\n", 1
)                       = 1
lseek(0, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
close(0)                                = 0
close(1)                                = 0
close(2)                                = 0
exit_group(1)                           = ?
+++ exited with 1 +++
#12

After doing some digging, here’s a post from oct about my exact same issue. It looks like users other than root don’t have enough permissions to delete items straight off of the rclone mount. I’ve asked the author of that post to reply with his solution.

Any thoughts about the best way to change perms on rclone’s mount?

#13

I figured this out, thanks for the help @ncw! The .union-fs folder had been rclone copied to ACD during a previous sync, which meant that the fuse drive was using the RO perms in the acd_media folder and wouldn’t overwrite any new whiteout files.

For future reference, if anybody is doing an rclone copy or clone sync from a fuse mount point, be sure to exclude the hidden .union-fs folder, or you will lock yourself into an unwritable whiteout list that fuse won’ be able to update.

#14

Ran into this exact situation and now I have an unwritable whiteout list, any suggestions on how to correct the matter? Thanks in advance.

#15

Do you have a ton of files that have been marked for deletion? If not, then just go into your local mount of the cloud storage (NOT the union mount) and delete the whole .union-fs folder. Then make sure to exclude the .union-fs folder on your next sync and you should be good to go.

Otherwise, if you have a lot of files that were marked for deletion you can do something clever like a script to go through the .union-fs folder and delete the corresponding files in your cloud mount.

#16