Chtimes Error On Local Mounted Copy

What is the problem you are having with rclone?

When copying to a cifs mounted directory, RClone Remote Control will successfully copy the file but return a 500 error with "error": "chtimes /mnt2/transfer/folder/1.txt: operation not permitted"

The permission on the directory is 775 with the rclone user being part of the group but not the owner

The error only occurs on first time copy. Using "--no-update-modtime" ensures that the error does not occur if the file is copied again (with the same data)

Hoping for an rclone solution rather than changing permissions on the mounted directory

What is your rclone version (output from rclone version)

1.51

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Centos 7

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone rc operations/copyfile srcFs="/usr/local/redacted" srcRemote="1.txt" dstFs="test:/mnt2/transfer/folder" dstRemote="1.txt"

The rclone config contents with secrets removed.

[test]
type = local
set_modtime = false

A log from the command with the -vv flag

$ rclone rcd --rc-no-auth --no-update-modtime --config=/usr/local/redacted/configs/redacted/rclone.config -vv
2020/07/09 09:49:01 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone" "rcd" "--rc-no-auth" "--no-update-modtime" "--config=/usr/local/redacted/configs/redacted/rclone.config" "-vv"]
2020/07/09 09:49:01 NOTICE: Serving remote control on http://127.0.0.1:5572/
2020/07/09 09:49:16 DEBUG : rc: "operations/copyfile": with parameters map[dstFs:test:/mnt2/transfer/folder dstRemote:1.txt srcFs:/usr/local/redacted srcRemote:1.txt]
2020/07/09 09:49:16 DEBUG : Using config file from "/usr/local/redacted/configs/redacted/rclone.config"
2020/07/09 09:49:16 DEBUG : 1.txt: Need to transfer - File not found at Destination
2020/07/09 09:49:16 ERROR : 1.txt: Failed to copy: chtimes /mnt2/transfer/folder/1.txt: operation not permitted
2020/07/09 09:49:16 ERROR : rc: "operations/copyfile": error: chtimes /mnt2/transfer/folder/1.txt: operation not permitted

Those are normally permissions errors.

Can you share the ls -al on the file and id of the user running it?

[username@xxx-4 folder]$ ls -l /mnt2/transfer/folder/
total 116736
-rwxrwxr-x. 1 adminuser usersgroup        3 Jul  9 09:49 1.txt
[username@xxx-4 folder]$ groups rcloneuser
rcloneuser: xxxx usersgroup xxxx

Find the requested information above.The names have been changed but the data is correct.

Can you replicate the error just doing a normal file copy?

That's a permissions error on the CIFS side.

Unable to reproduce the error to a non-mounted local copy.

Changing the owner/permissions of the mount is a "last resort"solution (but not out of the question). Was mostly wondering if there was a way to have RClone not update the modification time at all (similiar to set_modtime = false which seems to be aimed directly at sftp)

Sorry as I wasn't clear.

Can you do a rclone move -vv from the command prompt without the rcd ?

Here is the result with the file already existing in the destination:

[rcloneuser@xxx-4 ~]$ rclone move /usr/local/redacted/1.txt /mnt2/transfer/folder/1.txt -vv
2020/07/09 15:08:51 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone" "move" "/usr/local/redacted/1.txt" "/mnt2/transfer/folder/1.txt" "-vv"]
2020/07/09 15:08:51 DEBUG : Using config file from "/home/rcloneuser/.config/rclone/rclone.conf"
2020/07/09 15:08:51 Failed to create file system for "/mnt2/transfer/folder/1.txt": is a file not a directory
[rcloneuser@xxx-4 ~]$ rclone move /usr/local/redacted/1.txt /mnt2/transfer/folder/ -vv
2020/07/09 15:08:59 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone" "move" "/usr/local/redacted/1.txt" "/mnt2/transfer/folder/" "-vv"]
2020/07/09 15:08:59 DEBUG : Using config file from "/home/rcloneuser/.config/rclone/rclone.conf"
2020/07/09 15:08:59 DEBUG : 1.txt: Modification times differ by 1177h26m56.971729223s: 2020-05-21 13:24:05.720178777 -0700 PDT, 2020-07-09 14:51:02.691908 -0700 PDT
2020/07/09 15:08:59 DEBUG : 1.txt: MD5 = e2665feebe35bc97aff1b329c87b87e7 OK
2020/07/09 15:08:59 ERROR : 1.txt: Failed to set modification time: chtimes /mnt2/transfer/folder/1.txt: operation not permitted
2020/07/09 15:08:59 DEBUG : 1.txt: Unchanged skipping
2020/07/09 15:08:59 ERROR : 1.txt: Couldn't delete: remove /usr/local/redacted/1.txt: permission denied
2020/07/09 15:08:59 ERROR : Attempt 1/3 failed with 2 errors and: remove /usr/local/redacted/1.txt: permission denied
2020/07/09 15:08:59 DEBUG : 1.txt: Modification times differ by 1177h26m56.971729223s: 2020-05-21 13:24:05.720178777 -0700 PDT, 2020-07-09 14:51:02.691908 -0700 PDT
2020/07/09 15:08:59 DEBUG : 1.txt: MD5 = e2665feebe35bc97aff1b329c87b87e7 OK
2020/07/09 15:08:59 ERROR : 1.txt: Failed to set modification time: chtimes /mnt2/transfer/folder/1.txt: operation not permitted
2020/07/09 15:08:59 DEBUG : 1.txt: Unchanged skipping
2020/07/09 15:08:59 ERROR : 1.txt: Couldn't delete: remove /usr/local/redacted/1.txt: permission denied
2020/07/09 15:08:59 ERROR : Attempt 2/3 failed with 2 errors and: remove /usr/local/redacted/1.txt: permission denied
2020/07/09 15:08:59 DEBUG : 1.txt: Modification times differ by 1177h26m56.971729223s: 2020-05-21 13:24:05.720178777 -0700 PDT, 2020-07-09 14:51:02.691908 -0700 PDT
2020/07/09 15:08:59 DEBUG : 1.txt: MD5 = e2665feebe35bc97aff1b329c87b87e7 OK
2020/07/09 15:08:59 ERROR : 1.txt: Failed to set modification time: chtimes /mnt2/transfer/folder/1.txt: operation not permitted
2020/07/09 15:08:59 DEBUG : 1.txt: Unchanged skipping
2020/07/09 15:08:59 ERROR : 1.txt: Couldn't delete: remove /usr/local/redacted/1.txt: permission denied
2020/07/09 15:08:59 ERROR : Attempt 3/3 failed with 2 errors and: remove /usr/local/redacted/1.txt: permission denied
2020/07/09 15:08:59 Failed to move with 2 errors: last error was: remove /usr/local/redacted/1.txt: permission denied

Here is the result with the file not already existing in the destination. The file copied correctly but RClone still returned an error.

[rcloneuser@xxx-4 ~]$ rclone move /usr/local/redacted/1.txt /mnt2/transfer/folder/ -vv
2020/07/09 15:10:07 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone" "move" "/usr/local/redacted/1.txt" "/mnt2/transfer/folder/" "-vv"]
2020/07/09 15:10:07 DEBUG : Using config file from "/home/rcloneuser/.config/rclone/rclone.conf"
2020/07/09 15:10:07 DEBUG : 1.txt: Need to transfer - File not found at Destination
2020/07/09 15:10:07 DEBUG : 1.txt: Can't move: rename /usr/local/redacted/1.txt /mnt2/transfer/folder/1.txt: invalid cross-device link: trying copy
2020/07/09 15:10:07 DEBUG : 1.txt: Can't move, switching to copy
2020/07/09 15:10:07 ERROR : 1.txt: Failed to copy: chtimes /mnt2/transfer/folder/1.txt: operation not permitted
2020/07/09 15:10:07 ERROR : 1.txt: Not deleting source as copy failed: chtimes /mnt2/transfer/folder/1.txt: operation not permitted
2020/07/09 15:10:07 ERROR : Attempt 1/3 failed with 1 errors and: chtimes /mnt2/transfer/folder/1.txt: operation not permitted
2020/07/09 15:10:07 DEBUG : 1.txt: Modification times differ by 1177h46m1.822324746s: 2020-05-21 13:24:05.720178777 -0700 PDT, 2020-07-09 15:10:07.542503523 -0700 PDT
2020/07/09 15:10:07 DEBUG : 1.txt: MD5 = e2665feebe35bc97aff1b329c87b87e7 OK
2020/07/09 15:10:07 ERROR : 1.txt: Failed to set modification time: chtimes /mnt2/transfer/folder/1.txt: operation not permitted
2020/07/09 15:10:07 DEBUG : 1.txt: Unchanged skipping
2020/07/09 15:10:07 ERROR : 1.txt: Couldn't delete: remove /usr/local/redacted/1.txt: permission denied
2020/07/09 15:10:07 ERROR : Attempt 2/3 failed with 2 errors and: remove /usr/local/redacted/1.txt: permission denied
2020/07/09 15:10:07 DEBUG : 1.txt: Modification times differ by 1177h46m1.818593723s: 2020-05-21 13:24:05.720178777 -0700 PDT, 2020-07-09 15:10:07.5387725 -0700 PDT
2020/07/09 15:10:07 DEBUG : 1.txt: MD5 = e2665feebe35bc97aff1b329c87b87e7 OK
2020/07/09 15:10:07 ERROR : 1.txt: Failed to set modification time: chtimes /mnt2/transfer/folder/1.txt: operation not permitted
2020/07/09 15:10:07 DEBUG : 1.txt: Unchanged skipping
2020/07/09 15:10:07 ERROR : 1.txt: Couldn't delete: remove /usr/local/redacted/1.txt: permission denied
2020/07/09 15:10:07 ERROR : Attempt 3/3 failed with 2 errors and: remove /usr/local/redacted/1.txt: permission denied
2020/07/09 15:10:07 Failed to move with 2 errors: last error was: remove /usr/local/redacted/1.txt: permission denied

This functionality works correctly on a NFS mount (currently using CIFS). As mentioned before, this is obviously a permission-based issue but checking if there is a way to turn off modifications for RClone

From my understanding:

That's for the SFTP backend so it wouldn't do anything for local.

No update modtime is when you make a copy and two files are the same and the mod times are different and it won't update it.

I don't believe what you are asking exists but in your use case, I probably wouldn't even use rclone as I'd probably use rsync. I only use rclone when I have cloud needs.

I'm understanding the problem to be that rclone can't set modtimes on your destination which is a cifs mounted directory.

There isn't a way of disabling the setting of modtimes on a write to the local backend at the moment.

To double check this is the problem try rclone touch to set a mod time, and compare that with what the unix command touch does both running as the rclone user which is part of the group but not the owner.

You need to have write access to the file to set the modification time

$ echo hello > file.txt
$ sudo chown root.root file.txt
$ ls -l --full-time
total 4
-rw-rw-r-- 1 root root 6 2020-07-10 15:51:31.721277902 +0100 file.txt
$ touch file.txt 
touch: cannot touch 'file.txt': Permission denied
$ sudo chgrp ncw file.txt
$ touch file.txt 
$ sudo chmod 644 file.txt 
$ ls -l --full-time
total 4
-rw-r--r-- 1 root ncw 6 2020-07-10 15:53:02.853866988 +0100 file.txt
$ touch file.txt 
touch: cannot touch 'file.txt': Permission denied
$ sudo chmod 664 file.txt 
$ ls -l --full-time
total 4
-rw-rw-r-- 1 root ncw 6 2020-07-10 15:53:02.853866988 +0100 file.txt
$ touch file.txt 
$ 

I can't think of a way of writing a file which then has the permissions meaning you can't touch it afterwards. So I suspect there is a setting in your CIFS which needs tweaking.

What permissions do the files have when rclone has finished writing them?

Here is the result of the rclone touch command:

[rcloneuser@xxx-4 ~]$ rclone touch /mnt2/transfer/folder/1.txt
2020/07/10 08:53:19 ERROR : Attempt 1/3 failed with 1 errors and: touch: couldn't set mod time: chtimes /mnt2/transfer/folder/1.txt: operation not permitted
2020/07/10 08:53:19 ERROR : Attempt 2/3 failed with 1 errors and: touch: couldn't set mod time: chtimes /mnt2/transfer/folder/1.txt: operation not permitted
2020/07/10 08:53:19 ERROR : Attempt 3/3 failed with 1 errors and: touch: couldn't set mod time: chtimes /mnt2/transfer/folder/1.txt: operation not permitted
2020/07/10 08:53:19 Failed to touch: touch: couldn't set mod time: chtimes /mnt2/transfer/folder/1.txt: operation not permitted

After the file is copied (as it was above), the file permissions are: owner = adminuser, group = usersgroup, permissions = 0775. This matches the values on the mount setup:

sudo mount -t cifs //xxxx-xxxx/users/transfer -o username=redacted,password=redacted /mnt2/transfer -o file_mode=0775,dir_mode=0775,uid=1001,gid=1002

Can you try the Unix touch command too?

The Unix touch command works as expected.

touch -m /mnt2/transfer/folder/1.txt
returns touch: setting times of ‘/mnt2/transfer/tslaughter/1.txt’: Operation not permitted

My assumption of what is happening:

  • RClone copies the file to the CIFs mount.
  • The CIFs mount updates the permissions of the file to admingroup/usersgroup
  • By the time RClone attempts to update the modification time, the RClone user no longer owns the file which causes the error.

Based on the man page for utime (which essentially is what Go calls from what I can tell), the user must be the owner of the file to update the modification time. (Tried to add the link to man page but is not allowed)

It does not seem like there will be a RClone related solution unless a config is added in the future to not update the modification time.

Ok that makes sense.

Adding a flag to rclone would be quite easy. It would be a local backend flag.

Do you want to have a go at adding this? Happy to talk you through it .

I might be able to take a crack at adding the new flag.

I am trying to use RClone (and this potential new feature) for a project with an upcoming deadline. If a pull request is submitted this week, when would the update be released (or when is the next scheduled release)?

Great!

PR's go into the the latest beta first and then on into the next release. I think 1.53 would be 2-4 weeks away from now.

Are there any tips for getting started on this? I see there is a page on GitHub which I can follow: https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md . Are there any tips on where the new flag should live or things like that?

This needs to be a local backend flag, so you'll need to add another flag like this one

And then add it to the Options structure

And then check for it in SetModTime and don't set the modtime

I suggest we call the option no_set_modtime which will make a flag --local-no-set-modtime. This will potentially cause a problem with syncing if rclone can't set modtimes so this flag may need to alter the Precision to return fs.ModTimeNotSupported to signal to the rest of rclone that time's can't be set so shouldn't be used for syncing

You shouldn't need to edit any other files the docs will be auto generated from the option help.

Hello @ncw

Created a PR based on the suggestions you provided. The flag works on our local tests.

Do you happen to have a timeline for the next RClone release (hopefully including this fix)?

1 Like

I've merged this to master now - thank you! - which means it will be in the latest beta in 15-30 mins and released in v1.53.

1.53 should be about 2 weeks away

Thank you for the update and for all the help on this!

1 Like