Sftp server_command help

What is the problem you are having with rclone?

I have files on the server that is only readable by either root or a specific user. Rclone cannot sync these simply because the permissions does not allow it.
I would like to use sudo or similar when launching the sftp server server side.

I found the server_command - and I have used that but its not working at all.
When I run the command manually:

sftp -s "sudo /usr/libexec/openssh/sftp-server" bbs@cephnas.root.dom:/mnt/storage

It works just fine, meaning that I get a sftp connection.

So either I am doing something wrong and the arguments for server_command is not as I expect - or server_command is not working.

It would be great if I either can get an example of a working server_command or something else.

I looked at the commit where support was added, but the commit unfortunately did not include any tests or examples.

Run the command 'rclone version' and share the full output of the command.

rclone v1.56.1

  • os/version: debian 11.5 (64 bit)
  • os/kernel: 5.15.107+truenas (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.16.8
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

None - I am trying to rclone a filesystem from a remote server to the local server

The command you were trying to run (eg rclone copy /tmp remote:tmp)

/usr/bin/rclone sync --progress --stats-one-line --create-empty-src-dirs --stats=30s --transfers 4 --checkers 4 --config /home/admin/.config/rclone/rclone.conf cephnas:/mnt/storage /mnt/tank/backup

The rclone config contents with secrets removed.

[cephnas]
type = sftp
server_command =  "sudo /usr/libexec/openssh/sftp-server"
host = cephnas.root.dom
user = bbs
use_insecure_cipher = true
key_file = /home/admin/.ssh/id_rsa
md5sum_command = md5 -r
sha1sum_command = sha1 -r

A log from the command with the -vv flag

2023/08/31 18:40:09 DEBUG : rclone: Version "v1.56.1" starting with parameters ["/usr/bin/rclone" "sync" "-vv" "--progress" "--stats-one-line" "--create-empty-src-dirs" "--stats=30s" "--transfers" "4" "--checkers" "4" "--config" "/home/admin/.config/rclone/rclone.conf" "cephnas:/mnt/storage" "/mnt/tank/backup"]
2023/08/31 18:40:09 DEBUG : Creating backend with remote "cephnas:/mnt/storage"
2023/08/31 18:40:09 DEBUG : Using config file from "/home/admin/.config/rclone/rclone.conf"
2023/08/31 18:40:09 DEBUG : sftp://bbs@cephnas.root.dom:22//mnt/storage: New connection 192.168.0.200:49476->192.168.2.200:22 to "SSH-2.0-OpenSSH_8.0"
2023/08/31 18:40:09 DEBUG : pacer: low level retry 1/10 (error couldn't initialise SFTP: EOF)
2023/08/31 18:40:09 DEBUG : pacer: Rate limited, increasing sleep to 200ms
2023/08/31 18:40:09 DEBUG : sftp://bbs@cephnas.root.dom:22//mnt/storage: New connection 192.168.0.200:49488->192.168.2.200:22 to "SSH-2.0-OpenSSH_8.0"
2023/08/31 18:40:09 DEBUG : pacer: low level retry 2/10 (error couldn't initialise SFTP: EOF)
2023/08/31 18:40:09 DEBUG : pacer: Rate limited, increasing sleep to 400ms
2023/08/31 18:40:09 DEBUG : sftp://bbs@cephnas.root.dom:22//mnt/storage: New connection 192.168.0.200:49498->192.168.2.200:22 to "SSH-2.0-OpenSSH_8.0"
2023/08/31 18:40:09 DEBUG : pacer: low level retry 3/10 (error couldn't initialise SFTP: EOF)
2023/08/31 18:40:09 DEBUG : pacer: Rate limited, increasing sleep to 800ms
2023/08/31 18:40:10 DEBUG : sftp://bbs@cephnas.root.dom:22//mnt/storage: New connection 192.168.0.200:49504->192.168.2.200:22 to "SSH-2.0-OpenSSH_8.0"
2023/08/31 18:40:10 DEBUG : pacer: low level retry 4/10 (error couldn't initialise SFTP: EOF)
2023/08/31 18:40:10 DEBUG : pacer: Rate limited, increasing sleep to 1.6s
2023/08/31 18:40:11 DEBUG : sftp://bbs@cephnas.root.dom:22//mnt/storage: New connection 192.168.0.200:49516->192.168.2.200:22 to "SSH-2.0-OpenSSH_8.0"
2023/08/31 18:40:11 DEBUG : pacer: low level retry 5/10 (error couldn't initialise SFTP: EOF)
2023/08/31 18:40:11 DEBUG : pacer: Rate limited, increasing sleep to 2s
2023/08/31 18:40:12 DEBUG : sftp://bbs@cephnas.root.dom:22//mnt/storage: New connection 192.168.0.200:49528->192.168.2.200:22 to "SSH-2.0-OpenSSH_8.0"
2023/08/31 18:40:12 DEBUG : pacer: low level retry 6/10 (error couldn't initialise SFTP: EOF)
2023/08/31 18:40:14 DEBUG : sftp://bbs@cephnas.root.dom:22//mnt/storage: New connection 192.168.0.200:49544->192.168.2.200:22 to "SSH-2.0-OpenSSH_8.0"
2023/08/31 18:40:14 DEBUG : pacer: low level retry 7/10 (error couldn't initialise SFTP: EOF)
2023/08/31 18:40:16 DEBUG : sftp://bbs@cephnas.root.dom:22//mnt/storage: New connection 192.168.0.200:43026->192.168.2.200:22 to "SSH-2.0-OpenSSH_8.0"
2023/08/31 18:40:16 DEBUG : pacer: low level retry 8/10 (error couldn't initialise SFTP: EOF)
2023/08/31 18:40:18 DEBUG : sftp://bbs@cephnas.root.dom:22//mnt/storage: New connection 192.168.0.200:43038->192.168.2.200:22 to "SSH-2.0-OpenSSH_8.0"
2023/08/31 18:40:18 DEBUG : pacer: low level retry 9/10 (error couldn't initialise SFTP: EOF)
2023/08/31 18:40:20 DEBUG : sftp://bbs@cephnas.root.dom:22//mnt/storage: New connection 192.168.0.200:43050->192.168.2.200:22 to "SSH-2.0-OpenSSH_8.0"
2023/08/31 18:40:20 DEBUG : pacer: low level retry 10/10 (error couldn't initialise SFTP: EOF)
2023/08/31 18:40:20 Failed to create file system for "cephnas:/mnt/storage": NewFs: couldn't initialise SFTP: EOF

Update your rclone to the latest version first. It is waste of everybody's time to investigate issues in some old releases.

I got it working.
server_command should not be i quotes.

So a working config for me is:

[cephnas]
type = sftp
server_command = sudo /usr/libexec/openssh/sftp-server
host = cephnas.root.dom
user = bbs
use_insecure_cipher = true
key_file = /home/admin/.ssh/id_rsa
md5sum_command = md5 -r
sha1sum_command = sha1 -r

I think it would be awesome if the documentation was improved with an example - and also a note that commands should not be enclosed in qoutes.

It would be awesome if you found that to be awesome enough to submit a PR to update it :slight_smile:

I can do that - is the docs also on github?

1 Like

Yes sir!

Click on the pencil and it'll work through a PR request.

There you go - a pull request is created.

If you want to add example to the --sftp-server-command flag description

--sftp-server-command

Specifies the path or command to run a sftp server on the remote host.

The subsystem option is ignored when server_command is defined.

Properties:

Config: server_command
Env Var: RCLONE_SFTP_SERVER_COMMAND
Type: string
Required: false

please note that this part of help is auto generated (you can see flags section is enclosed in {{< rem autogenerated options start" - DO NOT EDIT - instead edit fs.RegInfo in backend/sftp/sftp.go then run make backenddocs" >}}) from source in

Thank you for PR but sorry I was too slow with detailed explanation:) Your change should go to another file.

I see - so I did it the entirely wrong way.

I do not have go installed anywhere - so I cannot just run make backenddocs - or is there any other way I can contribute this extra docs without actually having to install go on a system somewhere?

Would it be acceptable if I added a section below - similar to the Limitations section?

You do not have to run anything. This is only comment how to generate html file (it happens during release).

Your edit should go to sftp.go file - you will see how it is formatted when you edit it.

okay - I will do that then - sorry for the confusion :slight_smile:

I hope thats better - I was unsure how to embed markdown format inside the documentation.

Cool - now looks perfect:)

Is there someone I can ping - to get the PR approved and merged?

No rush:) Even when merged it wont appear in docs until the next version release. But you can see merged beta docs here - https://tip.rclone.org/

all new PRs are reviewed pretty much daily and processed approved or commented.

Cool - thanks

Is it true that if I prefix a line with 4 spaces, then it turns into markdown?

I took at look at one of the other examples - and that seem to be the case

Frankly I am not sure myself:) But I would do exactly what you said - check other flags:)

I think it is 4 spaces?

Thats also what I added - I copied the space from one of the other examples :slight_smile: I just thought it was 3 - but I updated my comment above.

That should also be added to the "contributing" documentation - how to include markdown as part of the automatic generated docomentation :slight_smile: