I keep failing to connect to a specific sftp

Hi all.

I've been using rclone in order to copy files from multiple client SFTP and until now it worked perfectly.
Our new client got me perplexed and on top of it, every 3 wrong tries it locks my account :frowning:

The command I'm using is:

rclone copy client_sftp:Inbox client_s3:prefix --log-level DEBUG --use-json-log --transfers 16 --config rclone.conf --sftp-use-insecure-cipher --sftp-user username --sftp-pass pass --sftp-host mftprod.host.com

the logs are:

{"level":"debug","msg":"Version \"v1.56.2\" starting with parameters [\"rclone\" REDACTED ","object":"rclone","objectType":"string","source":"cmd/cmd.go:408","time":"2023-02-07T23:31:14.123196+02:00"}
{"level":"debug","msg":"Creating backend with remote \"REDACTED\"","source":"fs/newfs.go:28","time":"2023-02-07T23:31:14.123474+02:00"}
{"level":"debug","msg":"Using config file from \"/REDACTED/rclone.conf\"","source":"config/config.go:356","time":"2023-02-07T23:31:14.124071+02:00"}
{"level":"debug","msg":"detected overridden config - adding \"{wkom7}\" suffix to name","object":"REDACTED","objectType":"string","source":"fs/newfs.go:45","time":"2023-02-07T23:31:14.124203+02:00"}
{"level":"debug","msg":"New connection REDACTED to \"SSH-2.0-Maverick_SSHD\"","object":"REDACTED","objectType":"*sftp.Fs","source":"sftp/sftp.go:330","time":"2023-02-07T23:31:15.137979+02:00"}
{"level":"debug","msg":"low level retry 1/10 (error couldn't initialise SFTP: EOF)","object":"pacer","objectType":"string","source":"fs/pacer.go:90","time":"2023-02-07T23:31:15.652606+02:00"}
{"level":"debug","msg":"Rate limited, increasing sleep to 200ms","object":"pacer","objectType":"string","source":"fs/pacer.go:46","time":"2023-02-07T23:31:15.652682+02:00"}
.
.
.
.
.
{"level":"debug","msg":"low level retry 4/10 (error couldn't connect SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain)","object":"pacer","objectType":"string","source":"fs/pacer.go:90","time":"2023-02-07T23:31:25.228663+02:00"}
.
.
.
{"level":"debug","msg":"low level retry 10/10 (error couldn't connect SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain)","object":"pacer","objectType":"string","source":"fs/pacer.go:90","time":"2023-02-07T23:31:37.045686+02:00"}
2023/02/07 23:31:37 Failed to create file system for "REDACTED": NewFs: couldn't connect SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain

the conf file is pretty redundant:

[client_s3]
type = s3
provider = AWS
env_auth = true

[client_sftp]
type = sftp
host =
user =
md5sum_command = none
sha1sum_command = none

I would appreciate any suggestion or an idea how to debug this issue.

Thanks!!!

hello and welcome to the forum,

please update to latest stable, v1.61.1 and test again.
https://rclone.org/downloads/#script-download-and-install

Thanks, I will do it the minute my account will get unlocked.
A quick question, does the logs from running lsjson instead of copy are suffice? (with the sftp authorization)

thanks!!!!

After updating to 1.61.1

{"level":"debug","msg":"Version \"v1.61.1\" starting with parameters [\"rclone\" \"lsjson\" \"client_sftp:Inbox\" \"--retries\" \"1\" \"--log-level\" \"DEBUG\" \"--use-json-log\" \"--transfers\" \"16\" \"--config\" \"rclone.conf\" \"--sftp-use-insecure-cipher\" \"--sftp-user\" \"REDACTED\" \"--sftp-pass\" \"REDACTED\" \"--sftp-host\" \"mftprod.REDACTED.com\"]","object":"rclone","objectType":"string","source":"cmd/cmd.go:414","time":"2023-02-08T09:05:17.703038+02:00"}
{"level":"debug","msg":"Creating backend with remote \"REDACTED:Inbox\"","source":"fs/newfs.go:27","time":"2023-02-08T09:05:17.703369+02:00"}
{"level":"debug","msg":"Using config file from \"REDACTED/rclone.conf\"","source":"config/config.go:356","time":"2023-02-08T09:05:17.706409+02:00"}
{"level":"debug","msg":"detected overridden config - adding \"{wkom7}\" suffix to name","object":"REDACTED","objectType":"string","source":"fs/newfs.go:47","time":"2023-02-08T09:05:17.706538+02:00"}
{"level":"debug","msg":"New connection REDACTED:65299-\u003eREDACTED:22 to \"SSH-2.0-Maverick_SSHD\"","object":"sftp://REDACTED@mftprod.REDACTED.com:22/Inbox","objectType":"*sftp.Fs","source":"sftp/sftp.go:459","time":"2023-02-08T09:05:19.919756+02:00"}
{"level":"debug","msg":"low level retry 1/10 (error couldn't initialise SFTP: EOF)","object":"pacer","objectType":"string","source":"fs/pacer.go:90","time":"2023-02-08T09:05:20.512123+02:00"}
{"level":"debug","msg":"Rate limited, increasing sleep to 200ms","object":"pacer","objectType":"string","source":"fs/pacer.go:46","time":"2023-02-08T09:05:20.512351+02:00"}
{"level":"debug","msg":"New connection REDACTED:65300-\u003eREDACTED:22 to \"SSH-2.0-Maverick_SSHD\"","object":"sftp://REDACTED@mftprod.REDACTED.com:22/Inbox","objectType":"*sftp.Fs","source":"sftp/sftp.go:459","time":"2023-02-08T09:05:22.19103+02:00"}
{"level":"debug","msg":"low level retry 2/10 (error couldn't initialise SFTP: EOF)","object":"pacer","objectType":"string","source":"fs/pacer.go:90","time":"2023-02-08T09:05:23.931835+02:00"}
{"level":"debug","msg":"Rate limited, increasing sleep to 400ms","object":"pacer","objectType":"string","source":"fs/pacer.go:46","time":"2023-02-08T09:05:23.931952+02:00"}
{"level":"debug","msg":"New connection REDACTED:65301-\u003eREDACTED:22 to \"SSH-2.0-Maverick_SSHD\"","object":"sftp://REDACTED@mftprod.REDACTED.com:22/Inbox","objectType":"*sftp.Fs","source":"sftp/sftp.go:459","time":"2023-02-08T09:05:27.429262+02:00"}
{"level":"debug","msg":"low level retry 3/10 (error couldn't initialise SFTP: EOF)","object":"pacer","objectType":"string","source":"fs/pacer.go:90","time":"2023-02-08T09:05:30.373791+02:00"}
...
{"level":"debug","msg":"low level retry 10/10 (error couldn't connect SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain)","object":"pacer","objectType":"string","source":"fs/pacer.go:90","time":"2023-02-08T09:05:49.049397+02:00"}
2023/02/08 09:05:49 Failed to create file system for "REDACTED:Inbox": NewFs: couldn't connect SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain

please notice that after 3 retries we get locked so after the 3rd retry the error message isn't relevant to the issue.

I can successfully connect via filezilla, can I use it to debug the issue? (If I'm only missing a flag maybe I can deduce it from filezilla logs)

Thank you very much for the time and patience.

That is not the most helpful error message ever, but I think it means that the remote end just shut the TCP connection. This is usually caused by some kind of firewall I think.

Is that possible?

This thread may be useful

Rclone will make multiple connections to the backend which might be a problem though I don't see that in your log.

Thanks @ncw

I'm guessing that if I can connect using FileZilla then it's not a firewall issue ... is there any "identification" mechanism such as user agent in sftp?

I'll read the thread and see if it helps, Thank you!

Probably not. Check the port is correct.

There is in the underlying SSH protocol.

You can change what rclone uses with --user-agent I think. I doubt it will have any effect though.

You don't have any Auth in here. Are you using an ssh agent? Or is there a password line? If so make sure it is obscured with rclone obscure

Thanks,

It seems the port is correct.
I passed the password obscured both via the CLI and configurations, same result.

I investigated further:

  • It doesn't matter if I use an invalid password or a valid one, I get the same log output.
  • I can ssh to the machine and it works fine.

@ncw How can I debug rclone? I have small experience with go and hashicorp's systems.

Is there a starters guide for debugging? couldn't find one on github

Don't think there is. Which os? I'm using Visual Studio Code with the go extension (golang.go), and really happy with that. Here's some pointers to rclone-specific configuration I use:

Edit: And to debug specific commands, add arguments to rclone like this in launch.json:

"args": [
    "serve", "http", "C:\\Temp", "-vv"
]

Thanks @albertony it actually wet smoother than I thought :slight_smile:
I passed the same args I passed to cli and I'm debugging the issue.

I how I'll get a better idea

Ok after debugging I found the issue.

The password ended with an '&'.
obscuring the password without "" dropped the & character.

Thank you all for your patience

Ah ha! Shell escaping strikes again.

Well done for finding the issue.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.