Remove executable bit from all files

What is the problem you are having with rclone?

Whenever I look at any of my unionfs/rclone folders, every single file is set with the executable bit. I've looked at my configs and I don't see anything that should be allowing that. The systemd services all have a umask of 002, yet it's still 777 across the board. How can I fix this so that the files are 664, like I would expect them to be?

What is your rclone version (output from rclone version)

1.52.2

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Ubuntu 18.04.5 TLS 64bit

Which cloud storage system are you using? (eg Google Drive)

Google Drive with Team Drives

The command you were trying to run (eg rclone copy /tmp remote:tmp)

/usr/bin/mergerfs \
  -o category.create=ff,async_read=false,cache.files=partial \
  -o dropcacheonclose=true,use_ino,minfreespace=0 \
  -o xattr=nosys,statfs_ignore=ro,allow_other,umask=002,noatime \
  /mnt/local=RW:/mnt/remote=NC:/mnt/sharedrives/td_*=NC /mnt/unionfs

The rclone config contents with secrets removed.

[google]
type = drive
client_id = .apps.googleusercontent.com
client_secret = 
scope = drive
token = {"access_token":"","token_type":"Bearer","refresh_token":"","expiry":"2020-10-09T01:59:49.236015285+03:00"}
root_folder_id = 0AB2EKp
server_side_across_configs = true

[td_4K_movies]
type = drive
client_id = .apps.googleusercontent.com
client_secret = 
scope = drive
server_side_across_configs = true
team_drive = -0NUk9PVA
service_account_file = /opt/service-account/1.json

[td_4K_tv]
type = drive
client_id = .apps.googleusercontent.com
client_secret = 
scope = drive
server_side_across_configs = true
team_drive = k9PVA
service_account_file = /opt/service-account/2.json

[td_audiobooks]
type = drive
client_id = .apps.googleusercontent.com
client_secret = 
scope = drive
server_side_across_configs = true
team_drive = dxUk9PVA
service_account_file = /opt/service-account/3.json

[td_books]
type = drive
client_id = .apps.googleusercontent.com
client_secret = 
scope = drive
server_side_across_configs = true
team_drive = uUk9PVA
service_account_file = /opt/service-account/4.json

[td_comics]
type = drive
client_id = .apps.googleusercontent.com
client_secret = 
scope = drive
server_side_across_configs = true
team_drive = 9uUk9PVA
service_account_file = /opt/service-account/5.json

[td_movies]
type = drive
client_id = .apps.googleusercontent.com
client_secret = 
scope = drive
server_side_across_configs = true
team_drive = Uk9PVA
service_account_file = /opt/service-account/6.json

[td_music]
type = drive
client_id = .apps.googleusercontent.com
client_secret = 
scope = drive
server_side_across_configs = true
team_drive = NUk9PVA
service_account_file = /opt/service-account/7.json

[td_tv]
type = drive
client_id = .apps.googleusercontent.com
client_secret = 
scope = drive
server_side_across_configs = true
team_drive = Uk9PVA
service_account_file = /opt/service-account/8.json
offspring@CloudSeedbox:/mnt/unionfs$ ls -alh
total 72M
drwxrwxr-x 6 offspring docker    4.0K Oct  7 23:38 .
drwxr-xr-x 6 root      root      4.0K Jul 21 09:41 ..
-rwxrwxr-x 1 offspring offspring  71M Jul 29 16:45 17.05.02.01_MSM_Windows.zip
-rwxrwxr-x 1 offspring offspring    0 Jun 17 15:26 4K_movies_mounted.bin
-rwxrwxr-x 1 offspring offspring    0 Jun 17 15:27 4K_tv_mounted.bin
-rwxrwxr-x 1 offspring offspring    0 Jun 13 23:48 audiobooks_mounted.bin
drwxrwxr-x 1 offspring offspring    0 Jul 10 12:36 backup
drwxrwxr-x 1 offspring offspring    0 Jun 28 20:56 backups
-rwxrwxr-x 1 offspring offspring    0 Jun 13 23:10 books_mounted.bin
-rwxrwxr-x 1 offspring offspring    0 Jun 13 23:06 comics_mounted.bin
-rwxrwxr-x 1 offspring offspring  40K Oct  8 12:25 docker-compose.yml

hello and welcome to the forum,

can you post the rclone mount command(s)?

https://rclone.org/commands/rclone_mount/

--dir-perms FileMode                     Directory permissions (default 0777)
--file-perms FileMode                    File permissions (default 0666)

Can we see a ls -al on the rclone folder and the mount command for rclone and we can look at the mergerfs next.

Here's the rclone folder's ls -alh:

drwxrwxr-x  6 offspring docker    4.0K Oct  7 23:38 unionfs

And here's the rclone command:

ExecStart=/usr/bin/rclone mount \
  --user-agent='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36' \
  --config=/home/offspring/.config/rclone/rclone.conf \
  --allow-other \
  --allow-non-empty \
  --rc \
  --rc-addr=localhost:5572 \
  --fast-list \
  --drive-skip-gdocs \
  --vfs-read-chunk-size=64M \
  --vfs-read-chunk-size-limit=2048M \
  --buffer-size=64M \
  --poll-interval=1m \
  --dir-cache-time=168h \
  --timeout=10m \
  --drive-chunk-size=64M \
  --umask=002 \
  --syslog \
  -v \
  google: /mnt/remote
ExecStop=/bin/fusermount -uz /mnt/remote

I'd remove that as it allows for over mounting and you could be hiding something.

Does nothing on a mount and can be removed.

So your rclone mount looks right on the umask, correct? I see the w bit missing for other in that output.

Yeah, it looks like it should be fine but all of the files all have +x which is what I don't understand.

I'll go remove those two flags as well, so thanks for that pointer. Out of curiosity, what exactly does "over mounting" mean?

Edit: googled it and found an answer from you from 2017, which makes total sense in regards to hiding things. Thanks for that past response :).

Directories always have an execute bit so you can cd into them. Can you ls -al a file?

-rwxrwxr-x 1 offspring offspring  71M Jul 29 16:45 17.05.02.01_MSM_Windows.zip
-rwxrwxr-x 1 offspring offspring    0 Jun 17 15:26 4K_movies_mounted.bin
-rwxrwxr-x 1 offspring offspring    0 Jun 17 15:27 4K_tv_mounted.bin
-rwxrwxr-x 1 offspring offspring    0 Jun 13 23:48 audiobooks_mounted.bin

Can you run a mount with debug and share the same output?

felix@gemini:~/test/test$ ls -al
total 0
-rw-rw-r-- 1 felix felix 0 Oct  8 19:44 one
-rw-rw-r-- 1 felix felix 0 Oct  8 19:50 two
felix@gemini:~/test/test$

and my log

felix@gemini:~$ rclone mount gcrypt: /home/felix/test -vv --umask 002
2020/10/08 19:58:48 DEBUG : rclone: Version "v1.53.1" starting with parameters ["rclone" "mount" "gcrypt:" "/home/felix/test" "-vv" "--umask" "002"]
2020/10/08 19:58:48 DEBUG : Creating backend with remote "gcrypt:"
2020/10/08 19:58:48 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2020/10/08 19:58:48 DEBUG : Creating backend with remote "GD:crypt"
2020/10/08 19:58:48 DEBUG : Encrypted drive 'gcrypt:': Mounting on "/home/felix/test"
2020/10/08 19:58:48 DEBUG : : Root:
2020/10/08 19:58:48 DEBUG : : >Root: node=/, err=<nil>
2020/10/08 19:58:53 DEBUG : /: Attr:
2020/10/08 19:58:53 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/10/08 19:58:54 DEBUG : /: ReadDirAll:
2020/10/08 19:58:54 DEBUG : /: >ReadDirAll: item=4, err=<nil>
2020/10/08 19:58:54 DEBUG : /: Lookup: name="Movies"
2020/10/08 19:58:54 DEBUG : /: >Lookup: node=Movies/, err=<nil>
2020/10/08 19:58:54 DEBUG : Movies/: Attr:
2020/10/08 19:58:54 DEBUG : Movies/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/10/08 19:58:54 DEBUG : /: Lookup: name="TV"
2020/10/08 19:58:54 DEBUG : /: >Lookup: node=TV/, err=<nil>
2020/10/08 19:58:54 DEBUG : TV/: Attr:
2020/10/08 19:58:54 DEBUG : TV/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/10/08 19:58:54 DEBUG : /: Lookup: name="mounted"
2020/10/08 19:58:54 DEBUG : /: >Lookup: node=mounted, err=<nil>
2020/10/08 19:58:54 DEBUG : mounted: Attr:
2020/10/08 19:58:54 DEBUG : mounted: >Attr: a=valid=1s ino=0 size=243 mode=-rw-rw-r--, err=<nil>
2020/10/08 19:58:54 DEBUG : /: Lookup: name="test"
2020/10/08 19:58:54 DEBUG : /: >Lookup: node=test/, err=<nil>
2020/10/08 19:58:54 DEBUG : test/: Attr:
2020/10/08 19:58:54 DEBUG : test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/10/08 19:58:55 DEBUG : /: Attr:
2020/10/08 19:58:55 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/10/08 19:58:55 DEBUG : test/: Attr:
2020/10/08 19:58:55 DEBUG : test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/10/08 19:58:55 DEBUG : test/: ReadDirAll:
2020/10/08 19:58:56 DEBUG : test/: >ReadDirAll: item=2, err=<nil>
2020/10/08 19:58:56 DEBUG : test/: Lookup: name="one"
2020/10/08 19:58:56 DEBUG : test/: >Lookup: node=test/one, err=<nil>
2020/10/08 19:58:56 DEBUG : test/one: Attr:
2020/10/08 19:58:56 DEBUG : test/one: >Attr: a=valid=1s ino=0 size=0 mode=-rw-rw-r--, err=<nil>
2020/10/08 19:58:56 DEBUG : test/: Lookup: name="two"
2020/10/08 19:58:56 DEBUG : test/: >Lookup: node=test/two, err=<nil>
2020/10/08 19:58:56 DEBUG : test/two: Attr:
2020/10/08 19:58:56 DEBUG : test/two: >Attr: a=valid=1s ino=0 size=0 mode=-rw-rw-r--, err=<nil>
2020/10/08 19:58:56 DEBUG : test/: Attr:
2020/10/08 19:58:56 DEBUG : test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/10/08 19:58:56 DEBUG : test/: ReadDirAll:
2020/10/08 19:58:56 DEBUG : test/: >ReadDirAll: item=2, err=<nil>
2020/10/08 19:59:48 DEBUG : Google drive root 'crypt': Checking for changes on remote
2020/10/08 20:00:48 DEBUG : Google drive root 'crypt': Checking for changes on remote
2020/10/08 20:01:48 DEBUG : Google drive root 'crypt': Checking for changes on remote

In your mount command you have:

I think this is the important part, because (normally) that will result in files having mode 775, which is rwxrwxr-x, which matches what you see with ls.

So, depending on what you want these might work:

rw-rw-rw- --> permission 666 --> umask 0111
rw-rw-r-- --> permission 664 --> umask 0113

The leading 0 in the umask is important.

It's not sure if this will remove the x permission from directories though. If it does then you might want to remove the umask value and use dir-perms and file-perms instead.

That's actually not correct as it works a bit different.

felix@gemini:~/test$ umask 002
felix@gemini:~/test$ touch one
felix@gemini:~/test$ umask 000
felix@gemini:~/test$ touch two
felix@gemini:~/test$ ls -al
total 4
drwxrwxr-x  2 felix felix   28 Oct  8 21:39 .
drwxr-xr-x 24 felix felix 4096 Oct  8 20:39 ..
-rw-rw-r--  1 felix felix    0 Oct  8 21:39 one
-rw-rw-rw-  1 felix felix    0 Oct  8 21:39 two

If you have a umask of 002, all the files will be created with 664 (666-002) permissions and dir with 775 (777-002) permissions.

You've missed a subtle point. It's important to remember that umask modifies the permission mask requested by the program, and touch requests 0666 as the permissions, and so the umask would then make that 0664. If you call creat(2) with mode 0777 then it'd be different. That's how files are created.

For example:

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

main()
{
  umask(002);
  creat("x",0777);
}

The result is:

-rwxrwxr-x 1 sweh sweh 0 Oct  8 22:18 x

Here 0777 AND NOT(002) == 0775 ==> rwxrwxr-x

We can see that touch creates files with 0666 permissions:

$ strace touch foo 2>&1 | grep 666
open("foo", O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK, 0666) = 3

which explains your output.

FWIW, you'll see this in practice when compiling files:

$ rm a.out ; umask 002 ; gcc x.c ; ls -l a.out
-rwxrwxr-x 1 sweh sweh 8408 Oct  8 22:26 a.out
$ rm a.out ; umask 022 ; gcc x.c ; ls -l a.out
-rwxr-xr-x 1 sweh sweh 8408 Oct  8 22:26 a.out
$ rm a.out ; umask 077 ; gcc x.c ; ls -l a.out
-rwx------ 1 sweh sweh 8408 Oct  8 22:26 a.out

That's because the result of gcc is created with permissions 0777, the same as my example program.

But this isn't the same as how modes are faked when reading filesystems that don't have unix permission modes.

Now I don't know what protocol rclone mount follows when faking permissions. But the original umask of 002 matches very well "0777 AND NOT(002)" and so is very suggestive.

I think my point was umask 002 makes a file look like 664 on output, which is what rclone does

felix@gemini:~$ rclone mount gcrypt: /home/felix/test -vv --umask 002
2020/10/08 19:58:48 DEBUG : rclone: Version "v1.53.1" starting with parameters ["rclone" "mount" "gcrypt:" "/home/felix/test" "-vv" "--umask" "002"]
2020/10/08 19:58:48 DEBUG : Creating backend with remote "gcrypt:"
2020/10/08 19:58:48 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2020/10/08 19:58:48 DEBUG : Creating backend with remote "GD:crypt"
2020/10/08 19:58:48 DEBUG : Encrypted drive 'gcrypt:': Mounting on "/home/felix/test"
2020/10/08 19:58:54 DEBUG : /: Lookup: name="mounted"
2020/10/08 19:58:54 DEBUG : /: >Lookup: node=mounted, err=<nil>
2020/10/08 19:58:54 DEBUG : mounted: Attr:
2020/10/08 19:58:54 DEBUG : mounted: >Attr: a=valid=1s ino=0 size=243 mode=-rw-rw-r--, err=<nil>

You said

Which is not right.

No.

The umask command modifies how files are created. The umask command does not modify output.

$ ls -l x ; umask 0 ; ls -l x ; umask 0777 ; ls -l x
-rwxrwxr-x 1 sweh sweh 0 Oct  8 22:18 x*
-rwxrwxr-x 1 sweh sweh 0 Oct  8 22:18 x*
-rwxrwxr-x 1 sweh sweh 0 Oct  8 22:18 x*

Your examples, earlier, showed the umask command.

However the umask flag to mount affects output. These are two different things. Even though they're both called "umask" they are two different things. Related, but different.

I may be wrong about the mount, which is why I said "suggestive" and "may". But nothing you've written demonstrates you think it does. You provided unix commands that only affect creation. Not how rclone uses it for mounts.

I think you are confusing what happens on a cloud remote vs a local file system.

That statement is accurate on a cloud remote. On a mounted rlcone remote it is all display as there is no concept of permissions on my mounted Google Drive so it modifies the output.

Same remote as with a different umask.

felix@gemini:~$ rclone mount gcrypt: /home/felix/test --umask 222 -vv
2020/10/08 22:35:15 DEBUG : rclone: Version "v1.53.1" starting with parameters ["rclone" "mount" "gcrypt:" "/home/felix/test" "--umask" "222" "-vv"]
2020/10/08 22:35:15 DEBUG : Creating backend with remote "gcrypt:"
2020/10/08 22:35:15 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2020/10/08 22:35:15 DEBUG : Creating backend with remote "GD:crypt"
2020/10/08 22:35:16 DEBUG : Encrypted drive 'gcrypt:': Mounting on "/home/felix/test"
2020/10/08 22:35:26 DEBUG : /: >Lookup: node=Movies/, err=<nil>
2020/10/08 22:35:26 DEBUG : Movies/: Attr:
2020/10/08 22:35:26 DEBUG : Movies/: >Attr: attr=valid=1s ino=0 size=0 mode=dr--r----x, err=<nil>
2020/10/08 22:35:26 DEBUG : /: Lookup: name="TV"
2020/10/08 22:35:26 DEBUG : /: >Lookup: node=TV/, err=<nil>
2020/10/08 22:35:26 DEBUG : TV/: Attr:
2020/10/08 22:35:26 DEBUG : TV/: >Attr: attr=valid=1s ino=0 size=0 mode=dr--r----x, err=<nil>
2020/10/08 22:35:26 DEBUG : /: Lookup: name="mounted"
2020/10/08 22:35:26 DEBUG : /: >Lookup: node=mounted, err=<nil>
2020/10/08 22:35:26 DEBUG : mounted: Attr:
2020/10/08 22:35:26 DEBUG : mounted: >Attr: a=valid=1s ino=0 size=243 mode=-r--r-----, err=<nil>
2020/10/08 22:35:26 DEBUG : /: Lookup: name="test"
2020/10/08 22:35:26 DEBUG : /: >Lookup: node=test/, err=<nil>

Here is the constrast between a local Linux disk and a mounted rclone remote.

felix@gemini:~/test$ ls -al
total 1
-r--r----- 1 felix felix 243 May 19  2019 mounted
dr--r----x 1 felix felix   0 Jun 17  2018 Movies
dr--r----x 1 felix felix   0 Oct  8 19:44 test
dr--r----x 1 felix felix   0 Apr 18  2017 TV
felix@gemini:~/test$ umask 000
felix@gemini:~/test$ touch blah
ls -al
felix@gemini:~/test$ ls -al
total 1
-r--r----- 1 felix felix   0 Oct  8 22:36 blah
-r--r----- 1 felix felix 243 May 19  2019 mounted
dr--r----x 1 felix felix   0 Jun 17  2018 Movies
dr--r----x 1 felix felix   0 Oct  8 19:44 test
dr--r----x 1 felix felix   0 Apr 18  2017 TV
felix@gemini:~/test$ cd
felix@gemini:~$ touch test2
felix@gemini:~$ ls -al test2
-rw-rw-rw- 1 felix felix 0 Oct  8 22:36 test2

Umask in rclone is purely display unlike a local disk when it's all about file creation.

I agree. It's what I've been saying. What I was disagreeing with was your use of the unix command line umask command to demonstrate how it works. Which is what you did. Because the two concepts are totally different.

umask on a mount is synthesising permissions on file read. umask on the command line is modifying permissions on file creation.

Gotcha. I'll try to be more careful on wording on the future as I see your point as well as that could also create confusion.

Good discussion always breeds better results so thanks for the feedback as well!

1 Like

With all this said, I don't have a google team drive, but I do have a normal drive.

In testing, "folders" end up with 0777 AND NOT (umask) and files end up with 0666 AND NOT (umask).

Which matches a lot of other Unix remote file systems (eg smbfs to mount Windows shares).

So I think rclone is doing the right thing. Unless there's something odd with teams drives!

So I would look at the mergefs layer that's above rclone and look at the umask value there. I don't think this is an rclone issue but a mergefs issue.

In his command though, he's running --umask 002 and getting the wrong output so either his rclone command he shared is wrong or he's not on the rclone drive.

He said he's running:

Which if he ran that, he'd get:

felix@gemini:~$ rclone mount gcrypt: /home/felix/test -vv --umask 002
2020/10/08 19:58:54 DEBUG : mounted: >Attr: a=valid=1s ino=0 size=243 mode=-rw-rw-r--, err=<nil>

and he's getting

which tells me, he's either not showing me the right rclone mount point or he's not running the same command that he posted.

I was trying to make sure rclone was good before going up a layer to mergerfs.

in the original post the OP is running mergefs on two "local" directories, of which one directory is rclone mounted. The mergefs layer could be hiding things and synthesising its own set of permissions.

I don't think we actually saw the underlying rclone mounted filesystem, just the merged one.

I'd definitely like to see

ls -l /mnt/remote/4k* /mnt/sharedrives/td_*/4k*