Connect to SFTP with connection string not working

Help and Support
1

What is the problem you are having with rclone?

I'm unable to connect through sftp with a connection string. Creating the config and then connecting works ok. The documentation for SFTP says:

If you don't specify pass , key_file , or key_pem or ask_password then rclone will attempt to contact an ssh-agent.

It seems that with the connection string even if the "pass" parameter is set it is trying to use ssh-agent anyway. I guess the connection string code is not setting the parameters correctly for the SFTP backend. Tried to look at the code but don't where the connection string parsing is (also don't know much about go)

Run the command 'rclone version' and share the full output of the command.

rclone v1.63.1

  • os/version: ubuntu 22.04 (64 bit)
  • os/kernel: 6.2.0-32-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.20.6
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

SFTP

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Command to create config:

rclone config create DESTINATION sftp host=vm-ubuntu-2 user=username pass=$(rclone obscure password) -vv

Command to use stored config:

rclone ls DESTINATION:/storage/go-sqlcmd -vv

Command using connection string:

rclone lsf ":sftp,host=vm-ubuntu-2:/storage/go-sqlcmd,user=username,pass=$(rclone obscure password)" -vv

The rclone config contents with secrets removed.

[DESTINATION]
type = sftp
host = vm-ubuntu-2
user = username
pass = password
shell_type = unix

A log from the command that you were trying to run with the -vv flag

Logs from create config:

rclone config create DESTINATION sftp host=vm-ubuntu-2 user=username pass=$(rclone obscure password) -vv

2023/09/06 10:57:50 DEBUG : rclone: Version "v1.63.1" starting with parameters ["rclone" "config" "create" "DESTINATION" "sftp" "host=vm-ubuntu-2" "user=username" "pass=password" "-vv"]
2023/09/06 10:57:50 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/09/06 10:57:50 DEBUG : Saving config "user" in section "DESTINATION" of the config file
2023/09/06 10:57:50 DEBUG : Saving config "pass" in section "DESTINATION" of the config file
2023/09/06 10:57:50 DEBUG : Saving config "host" in section "DESTINATION" of the config file
2023/09/06 10:57:50 DEBUG : DESTINATION: config in: state="", result=""
2023/09/06 10:57:50 DEBUG : DESTINATION: config out: out=<nil>, err=<nil>

[DESTINATION]
type = sftp
user = username
pass = *** ENCRYPTED ***
host = vm-ubuntu-2

Logs using stored config:

rclone ls DESTINATION:/storage/go-sqlcmd -vv

2023/09/06 11:03:21 DEBUG : rclone: Version "v1.63.1" starting with parameters ["rclone" "ls" "DESTINATION:/storage/go-sqlcmd" "-vv"]
2023/09/06 11:03:21 DEBUG : Creating backend with remote "DESTINATION:/storage/go-sqlcmd"
2023/09/06 11:03:21 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/09/06 11:03:21 DEBUG : sftp://username@vm-ubuntu-2:22//storage/go-sqlcmd: New connection 192.168.10.132:46200->192.168.10.172:22 to "SSH-2.0-OpenSSH_8.9p1 Ubuntu-3ubuntu0.3"
2023/09/06 11:03:22 DEBUG : sftp://username@vm-ubuntu-2:22//storage/go-sqlcmd: Shell type "unix" from config
2023/09/06 11:03:22 DEBUG : sftp://username@vm-ubuntu-2:22//storage/go-sqlcmd: Using root directory "/storage/go-sqlcmd"
   273199 NOTICE.md
 19226624 sqlcmd
 26304671 sqlcmd_debug
2023/09/06 11:03:22 DEBUG : 13 go routines active
2023/09/06 11:03:22 DEBUG : sftp://username@vm-ubuntu-2:22//storage/go-sqlcmd: Closing 1 unused connections

Logs using connection string:

rclone lsf ":sftp,host=vm-ubuntu-2:/storage/go-sqlcmd,user=username,pass=$(rclone obscure password)" -vv
2023/09/06 11:08:25 DEBUG : rclone: Version "v1.63.1" starting with parameters ["rclone" "lsf" ":sftp,host=vm-ubuntu-2:/storage/go-sqlcmd,user=username,pass=password" "-vv"]
2023/09/06 11:08:25 DEBUG : Creating backend with remote ":sftp,host=vm-ubuntu-2:/storage/go-sqlcmd,user=username,pass=password"
2023/09/06 11:08:25 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/09/06 11:08:25 DEBUG : :sftp: detected overridden config - adding "{lbiaJ}" suffix to name
2023/09/06 11:08:25 Failed to create file system for ":sftp,host=vm-ubuntu-2:/storage/go-sqlcmd,user=username,pass=password": couldn't connect to ssh-agent: SSH agent requested but SSH_AUTH_SOCK not-specified
2

i believe the issue is with the host

for example, here i use the same format as your command and rclone failed in the same way.

rclone ls ":sftp,host=127.0.0.1:/files,port=2022,user=user,pass=$(rclone obscure pass),shell_type=unix"
Failed to create file system for ":sftp,host=127.0.0.1:/files,port=2022,user=user,pass=C7eSeN4NKK6KMeHdFwEnZZXJ478,shell_type=unix": couldn't connect to ssh-agent: SSH agent requested but SSH_AUTH_SOCK not-specified

and here are two working examples, each using different formatting

rclone ls ":sftp,port=2022,user=user,pass=$(rclone obscure pass),shell_type=unix,host=127.0.0.1:/files"
        0 file.ext

rclone ls ":sftp,host=127.0.0.1,port=2022,user=user,pass=$(rclone obscure pass),shell_type=unix:/files"
        0 file.ext

so you might try

":sftp,user=username,pass=$(rclone obscure password),host=vm-ubuntu-2:/storage/go-sqlcmd"

or

":sftp,host=vm-ubuntu-2,user=username,pass=$(rclone obscure password):/storage/go-sqlcmd"
3

Ok, tested it and it works.
I thought the path was part of the host option, like:

:backend,host=address:/path,options1=value,option2=value,...

When it seems to be

:backend,host=address,options1=value,option2=value,...:/path

Don't understand why the developers didn't use an URL for the connection string something like:

backend://address/path?option1=value&option2=value&...
4

good.

i am not the author, i am sure there could be other reasons.

a remote can be created using a number of methods.
--- config file
--- connection string
--- environment variables
--- rc interface

each method shares the same parameters making it easy to switch between them.
host is the host address, path is the path.

imho, your suggestion breaks that by having a format unique to connection strings.
in effect, merging host and path