Failed to run "md5sum /volume1/dms_backup/test1.txt": Permission denied

Help and Support
1

What is the problem you are having with rclone?

Hi at all,

whenever I want to transmit a file from the rclone host (SFTP) to a remote NAS (Synology) I encounter this error:
failed to run "md5sum /volume1/dms_backup/test1.txt": Permission denied

Run the command 'rclone version' and share the full output of the command.

rclone --version
rclone v1.61.0
- os/version: debian 11.5 (64 bit)
- os/kernel: 5.15.60-2-pve (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.19.4
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

SFTP on Synology NAS

The command you were trying to run (eg rclone copy /tmp remote:tmp)

sudo rclone copy -vv --retries 1 --config /root/.config/rclone/rclone.conf /mnt/paperless_ngx/backup/test1.txt nas_dms:dms_backup --sftp-path-override /volume1/dms_backup

I also tried variations of the command to ensure it has nothing to do with the path values (e.g. because the docs state to use a leading slash for the remote when the remote is a Synology):

# With leading slash for remote path
sudo rclone copy -vv --retries 1 --config /root/.config/rclone/rclone.conf /mnt/paperless_ngx/backup/test1.txt nas_dms:/dms_backup --sftp-path-override /volume1/dms_backup
# Without path override
sudo rclone copy -vv --retries 1 --config /root/.config/rclone/rclone.conf /mnt/paperless_ngx/backup/test1.txt nas_dms:dms_backup
sudo rclone copy -vv --retries 1 --config /root/.config/rclone/rclone.conf /mnt/paperless_ngx/backup/test1.txt nas_dms:/dms_backup

On the NAS itself I tried to manually execute the md5sum command to ensure there is no problem with the user in question:

sudo -H -u paperless bash -c 'md5sum /volume1/dms_backup/somefile'
c7d226536fee164edd75392207f6b89d  /volume1/dms_backup/somefile

The rclone config contents with secrets removed.

[nas_dms]
type = sftp
host = 192.168.2.102
user = paperless
pass = ...
shell_type = unix
md5sum_command = md5sum
sha1sum_command = sha1sum

A log from the command with the -vv flag

$ sudo rclone copy -vv --retries 1 --config /root/.config/rclone/rclone.conf /mnt/paperless_ngx/backup/test1.txt nas_dms:dms_backup --sftp-path-override /volume1/dms_backup
2023/03/18 16:02:29 DEBUG : rclone: Version "v1.61.0" starting with parameters ["rclone" "copy" "-vv" "--retries" "1" "--config" "/root/.config/rclone/rclone.conf" "/mnt/paperless_ngx/backup/test1.txt" "nas_dms:dms_backup" "--sftp-path-override" "/volume1/dms_backup"]2023/03/18 16:02:29 DEBUG : Creating backend with remote "/mnt/paperless_ngx/backup/test1.txt"
2023/03/18 16:02:29 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/03/18 16:02:29 DEBUG : fs cache: adding new entry for parent of "/mnt/paperless_ngx/backup/test1.txt", "/mnt/paperless_ngx/backup"
2023/03/18 16:02:29 DEBUG : Creating backend with remote "nas_dms:dms_backup"
2023/03/18 16:02:29 DEBUG : nas_dms: detected overridden config - adding "{GUxpu}" suffix to name
2023/03/18 16:02:29 DEBUG : sftp://paperless@192.168.2.102:22/dms_backup: New connection 192.168.2.31:60524->192.168.2.102:22 to "SSH-2.0-OpenSSH_8.2"
2023/03/18 16:02:29 DEBUG : sftp://paperless@192.168.2.102:22/dms_backup: Shell type "unix" from config
2023/03/18 16:02:29 DEBUG : sftp://paperless@192.168.2.102:22/dms_backup: Relative path resolved to "/dms_backup"
2023/03/18 16:02:29 DEBUG : sftp://paperless@192.168.2.102:22/dms_backup: Using root directory "/dms_backup"
2023/03/18 16:02:29 DEBUG : fs cache: renaming cache item "nas_dms:dms_backup" to be canonical "nas_dms{GUxpu}:dms_backup"
2023/03/18 16:02:29 DEBUG : test1.txt: Need to transfer - File not found at Destination
2023/03/18 16:02:29 DEBUG : sftp://paperless@192.168.2.102:22/dms_backup: Shell path redirected to "/volume1/dms_backup/test1.txt" with option path_override
2023/03/18 16:02:29 DEBUG : sftp://paperless@192.168.2.102:22/dms_backup: Running remote command: md5sum /volume1/dms_backup/test1.txt
2023/03/18 16:02:29 ERROR : test1.txt: Failed to calculate dst hash: failed to calculate md5 hash: failed to run "md5sum /volume1/dms_backup/test1.txt": Permission denied, please try again.: Process exited with status 1
2023/03/18 16:02:29 ERROR : test1.txt: corrupted on transfer: md5 hash differ "d41d8cd98f00b204e9800998ecf8427e" vs ""
2023/03/18 16:02:29 INFO  : test1.txt: Removing failed copy
2023/03/18 16:02:29 ERROR : Attempt 1/1 failed with 2 errors and: corrupted on transfer: md5 hash differ "d41d8cd98f00b204e9800998ecf8427e" vs ""
2023/03/18 16:02:29 INFO  : 
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 2 (retrying may help)
Elapsed time:         0.3s

2023/03/18 16:02:29 DEBUG : 13 go routines active
2023/03/18 16:02:29 DEBUG : sftp://paperless@192.168.2.102:22/dms_backup: Closing 1 unused connections
2023/03/18 16:02:29 Failed to copy with 2 errors: last error was: corrupted on transfer: md5 hash differ "d41d8cd98f00b204e9800998ecf8427e" vs ""
2

hi,

in this post, i show how to get rclone to calculate the md5 on a synbox.
https://forum.rclone.org/t/on-some-sftp-servers-e-g-synology-the-paths-are-different-for-ssh-and-sftp-so-the-hashes-cant-be-calculated-properly/36506

if i intentionally set --sftp-path-override to the wrong path, i get the same error as you.

not an expert with linux but
another possible issue, that you run sudo to get md5sum to work.
i would think user paperless, needs permission to access the files without root.

3

Hi, thanks for supporting. I'll check this. Concerning the path: Maybe this is due to that I am using a shared folder. Maybe this is a symlink or whatever under the hood. Would that matter? I'll check that.

4

shared folder should not make a difference.
this works on a shared folder.

rclone copy file.ext nas_dms:dms_backup --sftp-path-override=/volume1/dms_backup -vv 
2023/03/18 13:00:58 DEBUG : Setting --config "C:\\data\\rclone\\rclone.conf" from environment variable RCLONE_CONFIG="C:\\data\\rclone\\rclone.conf"
2023/03/18 13:00:58 DEBUG : rclone: Version "v1.61.1" starting with parameters ["C:\\data\\rclone\\rclone.exe" "copy" "C:\\data\\rclone\\file.ext" "nas_dms:dms_backup" "--sftp-path-override=/volume1/dms_backup" "-vv"]
2023/03/18 13:00:58 DEBUG : Creating backend with remote "C:\\data\\rclone\\file.ext"
2023/03/18 13:00:58 DEBUG : Using config file from "C:\\data\\rclone\\rclone.conf"
2023/03/18 13:00:58 DEBUG : fs cache: adding new entry for parent of "C:\\data\\rclone\\file.ext", "//?/C:/data/rclone"
2023/03/18 13:00:58 DEBUG : Creating backend with remote "nas_dms:dms_backup"
2023/03/18 13:00:58 DEBUG : nas_dms: detected overridden config - adding "{GUxpu}" suffix to name
2023/03/18 13:00:58 DEBUG : sftp://user99@100.115.37.34:22/dms_backup: New connection 100.109.123.23:49400->100.115.37.34:22 to "SSH-2.0-OpenSSH_8.2"
2023/03/18 13:00:58 DEBUG : sftp://user99@100.115.37.34:22/dms_backup: Shell type "unix" from config
2023/03/18 13:00:58 DEBUG : sftp://user99@100.115.37.34:22/dms_backup: Relative path resolved to "/dms_backup"
2023/03/18 13:00:58 DEBUG : sftp://user99@100.115.37.34:22/dms_backup: Using root directory "/dms_backup"
2023/03/18 13:00:58 DEBUG : fs cache: renaming cache item "nas_dms:dms_backup" to be canonical "nas_dms{GUxpu}:dms_backup"
2023/03/18 13:00:58 DEBUG : file.ext: Need to transfer - File not found at Destination
2023/03/18 13:00:58 DEBUG : sftp://user99@100.115.37.34:22/dms_backup: Shell path redirected to "/volume1/dms_backup/file.ext" with option path_override
2023/03/18 13:00:58 DEBUG : sftp://user99@100.115.37.34:22/dms_backup: Running remote command: md5sum /volume1/dms_backup/file.ext
2023/03/18 13:00:58 DEBUG : sftp://user99@100.115.37.34:22/dms_backup: Remote command result: c039965eb7fcc80be96c2a578bf8bace  /volume1/dms_backup/file.ext
2023/03/18 13:00:58 DEBUG : file.ext: Parsed hash: c039965eb7fcc80be96c2a578bf8bace
2023/03/18 13:00:58 DEBUG : file.ext: md5 = c039965eb7fcc80be96c2a578bf8bace OK
2023/03/18 13:00:58 INFO  : file.ext: Copied (new)
2023/03/18 13:00:58 INFO  : 
Transferred:   	   16.940 KiB / 16.940 KiB, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         0.4s

2023/03/18 13:00:58 DEBUG : 12 go routines active
2023/03/18 13:00:58 DEBUG : sftp://user99@100.115.37.34:22/dms_backup: Closing 1 unused connections
5

One more thing: Can you tell which DSM version it is that you are running?

One more thing after the one thing: Do you know which exact command is run by rclone to get the hash? I mean the complete one like ssh user@host ... 'md5sum file'?

Tried to read it from the GitHub repo but had no luck so far.

Thanks!!!

6

DSM 7.1-42661 Update 3

not an expert but rclone opens a session and then run the exact command as shown in the rclone debug log.
https://github.com/rclone/rclone/blob/97276ce765e40852e631ac008b4105f84e17a495/backend/sftp/sftp.go#L1407

https://github.com/rclone/rclone/blob/97276ce765e40852e631ac008b4105f84e17a495/backend/sftp/sftp.go#L1664

maybe somebody else knows for sure

7

Thanks again.

I think I figured it out. But I don't think I like it...

First things first: It is not a problem of the user not being able to run the md5sum command:

paperless@nas3:/$ md5sum /volume1/dms_backup/somefile
c7d226536fee164edd75392207f6b89d  /volume1/dms_backup/somefile

BUT: To be able to get to this point I recognized that my user paperless had no shell configured.

cat /etc/passwd
[...]
paperless:x:1027:100::/var/services/homes/paperless:/sbin/nologin
[...]

Changing that to
paperless:x:1027:100::/var/services/homes/paperless:/bin/sh
made the rclone command succeed.

But that implies one of the following two things:
@asdffdsa : Is your user within the admin group? admin users are the only group that are allowed to get SSH access. So they might also be the only ones that get a shell. I saw no other option within the Synology config that looked like it would enable a shell for non admin users :frowning: Or did you somewhen enable the shell for your user? Otherwise I really cannot imagine how your user got a shell...

8

yes

good question, i would post that to the synology forum.

9

I checked a few posts out there and they all state that SSH access is only granted to admin users and that way the shell /bin/sh is only set for them not for unpriv'ed users. Soooo thats maybe worth a mention in the docs that next to the plain SFTP permissions a real shell is neccessary (at least that this is a thing on synology NASes)?

10

perhaps use AllowUsers in /etc/ssh/sshd_config

11

From what I read that might not be ideal too. This config file might be overridden (like other system files are) whenever the Synology boots up. And I read about default permissions of user data that are set to 777 on file system level so a shell would mean access to all files... Did not validate that. But either way that would be a not supported workaround. And I learned the hard way to use a Synology only the intended way :wink: So I guess the more stable option would be to ... Turn off hash checking or to use a completely different approach... again... Maybe SMB/CIFS... Since FTP(S) in conjunction with my Synology is also out of scope I am slowly running out of options here :joy:

12

yes, i read that also, but might be worth a try

never!

agreed!

rclone serve sftp

13

Thanks for the hint. Does that basically mean to run rclone serve sftp on the Synology NAS?

14

correct, the rclone sftp server will caclulate the hash without using a shell.

on the client, with remote nas_dms:, set shell_type = none