Issues connecting to FTP with RClone

Help and Support
1

What is the problem you are having with rclone?

I'm trying to conect to my suppliers FTP server with user and password. I can connect perfectly using normal FTP tools like MacOS' Finder, Filezilla, etc, but for some reason I can't get RClone to connect to the server.

What is your rclone version (output from rclone version)

Tried from both mac

RClone rclone v1.55.1
- os/type: darwin
- os/arch: amd64
- go/version: go1.16.3
- go/linking: dynamic
- go/tags: none

and linux

rclone v1.55.1
- os/type: linux
- os/arch: amd64
- go/version: go1.16.3
- go/linking: static
- go/tags: none

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone ls CMM:

The rclone config contents with secrets removed.

type = ftp
host = ftp.myprovider.com
user = ***
port = 21
pass = ***

A log from the command with the -vv flag

> $ rclone ls CMM: -vv --dry-run
2021/06/17 10:38:46 DEBUG : Using config file from "/Users/alex/.config/rclone/rclone.conf"
2021/06/17 10:38:46 DEBUG : rclone: Version "v1.55.1" starting with parameters ["rclone" "ls" "CMM:" "-vv" "--dry-run"]
2021/06/17 10:38:46 DEBUG : Creating backend with remote "CMM:"
2021/06/17 10:38:46 DEBUG : ftp://ftp.myprovider.com:21: Connecting to FTP server
2021/06/17 10:38:55 ERROR : : error listing: dial tcp 212.***.***.***:24614: connect: operation timed out
2021/06/17 10:38:55 DEBUG : 4 go routines active
2021/06/17 10:38:55 Failed to ls with 2 errors: last error was: dial tcp 212.***.***.***:24614: connect: operation timed out

Thank you for any help! RClone works like magic with every other connection I've tried and I'm a bit stumped about why its not working.

2

hi,

can you post the output of
rclone ls CMM: -vv --dump bodies

also, many servers that offer ftp also offer sftp and in most cases, that would be a better choice.

3

Thanks for the suggestion, I hate using FTP in any situation. Unfortunately I'm connecting to the classic "Enterprise" what uses tools from the dark age...

I ran the command with the debug flags you suggested. It seems like an issue with the Go FTP client not supporting the "229 Entering Extended Passive Mode (|||18608|)" command?

> $ rclone ls CMM: -vv --dump bodies
2021/06/17 16:58:13 DEBUG : Using config file from "/Users/alex/.config/rclone/rclone.conf"
2021/06/17 16:58:13 DEBUG : rclone: Version "v1.55.1" starting with parameters ["rclone" "ls" "CMM:" "-vv" "--dump" "bodies"]
2021/06/17 16:58:13 DEBUG : Creating backend with remote "CMM:"
2021/06/17 16:58:13 DEBUG : ftp://ftp.myprovider.com:21: Connecting to FTP server
2021/06/17 16:58:13 DEBUG : FTP Rx: "220 (vsFTPd 2.0.5)"
2021/06/17 16:58:13 DEBUG : FTP Tx: "USER ***"
2021/06/17 16:58:13 DEBUG : FTP Rx: "331 Please specify the password."
2021/06/17 16:58:13 DEBUG : FTP Tx: PASS *****
2021/06/17 16:58:13 DEBUG : FTP Rx: "230 Login successful."
2021/06/17 16:58:13 DEBUG : FTP Tx: "FEAT"
2021/06/17 16:58:13 DEBUG : FTP Rx: "211-Features:"
2021/06/17 16:58:14 DEBUG : FTP Rx: " EPRT"
2021/06/17 16:58:14 DEBUG : FTP Rx: " EPSV"
2021/06/17 16:58:14 DEBUG : FTP Rx: " MDTM"
2021/06/17 16:58:14 DEBUG : FTP Rx: " PASV"
2021/06/17 16:58:14 DEBUG : FTP Rx: " REST STREAM"
2021/06/17 16:58:14 DEBUG : FTP Rx: " SIZE"
2021/06/17 16:58:14 DEBUG : FTP Rx: " TVFS"
2021/06/17 16:58:14 DEBUG : FTP Rx: "211 End"
2021/06/17 16:58:14 DEBUG : FTP Tx: "TYPE I"
2021/06/17 16:58:14 DEBUG : FTP Rx: "200 Switching to Binary mode."
2021/06/17 16:58:14 DEBUG : FTP Tx: "EPSV"
2021/06/17 16:58:14 DEBUG : FTP Rx: "229 Entering Extended Passive Mode (|||18608|)"

<Execution pauses for over 10 seconds>

2021/06/17 16:59:14 DEBUG : FTP Tx: "NOOP"
2021/06/17 16:59:14 DEBUG : FTP Rx: "200 NOOP ok."
2021/06/17 16:59:14 ERROR : : error listing: dial tcp 212.***.***.***:18608: i/o timeout
2021/06/17 16:59:14 DEBUG : 4 go routines active
2021/06/17 16:59:14 Failed to ls with 2 errors: last error was: dial tcp 212.***.***.***:18608: i/o timeout
4

rclone supports passive only, not active

not sure what the problem as i do not use ftp.
the documention mentions --contimeout; perhaps try to tweak that.

5

This looks like a firewall or network issue; in passive mode the client (rclone) attempts to connect to the server on a port the server has requested (in the second example it was port 18608). rclone attempts to connect to that... and fails.

So something is blocking that second connection.

6

rclone can act as a sftp server using rclone serve sftp.
if you can open a port on the router in front of the server, then you can run rclone serve sftp.

7

OK fixed! Thanks for all of the help!

The issue seems to be ftp.myprovider.com's firewall blocking the ports used by the Extended Passive Mode. Seeing that the port was received after issuing the EPSV command, I searched the rclone documentation and found the option --ftp-disable-epsv. Running the command with this flag fixed the connection.

Thanks everybody for pointing me in the right direction!

> $ rclone ls CMM: --ftp-disable-epsv -vv --dump bodies 
2021/06/17 17:55:30 DEBUG : Using config file from "/Users/alex/.config/rclone/rclone.conf"
2021/06/17 17:55:30 DEBUG : rclone: Version "v1.55.1" starting with parameters ["rclone" "ls" "CMM:" "--ftp-disable-epsv" "-vv" "--dump" "bodies"]
2021/06/17 17:55:30 DEBUG : Creating backend with remote "CMM:"
2021/06/17 17:55:30 DEBUG : CMM: detected overridden config - adding "{suApg}" suffix to name
2021/06/17 17:55:30 DEBUG : ftp://ftp.myprovider.com:21: Connecting to FTP server
2021/06/17 17:55:30 DEBUG : FTP Rx: "220 (vsFTPd 2.0.5)"
2021/06/17 17:55:30 DEBUG : FTP Tx: "USER ****"
2021/06/17 17:55:30 DEBUG : FTP Rx: "331 Please specify the password."
2021/06/17 17:55:30 DEBUG : FTP Tx: PASS *****
2021/06/17 17:55:31 DEBUG : FTP Rx: "230 Login successful."
2021/06/17 17:55:31 DEBUG : FTP Tx: "FEAT"
2021/06/17 17:55:31 DEBUG : FTP Rx: "211-Features:"
2021/06/17 17:55:31 DEBUG : FTP Rx: " EPRT"
2021/06/17 17:55:31 DEBUG : FTP Rx: " EPSV"
2021/06/17 17:55:31 DEBUG : FTP Rx: " MDTM"
2021/06/17 17:55:31 DEBUG : FTP Rx: " PASV"
2021/06/17 17:55:31 DEBUG : FTP Rx: " REST STREAM"
2021/06/17 17:55:31 DEBUG : FTP Rx: " SIZE"
2021/06/17 17:55:31 DEBUG : FTP Rx: " TVFS"
2021/06/17 17:55:31 DEBUG : FTP Rx: "211 End"
2021/06/17 17:55:31 DEBUG : FTP Tx: "TYPE I"
2021/06/17 17:55:31 DEBUG : FTP Rx: "200 Switching to Binary mode."
2021/06/17 17:55:31 DEBUG : fs cache: renaming cache item "CMM:" to be canonical "CMM{suApg}:"
2021/06/17 17:55:31 DEBUG : FTP Tx: "PASV"
2021/06/17 17:55:31 DEBUG : FTP Rx: "227 Entering Passive Mode (212,230,255,41,131,43)"
2021/06/17 17:55:31 DEBUG : FTP Tx: "LIST"
2021/06/17 17:55:31 DEBUG : FTP Rx: "150 Here comes the directory listing."
2021/06/17 17:55:32 DEBUG : FTP Rx: "226 Directory send OK."
        0 xxxx-20210616.txt
    60708 yyyy-20210616.txt
        0 zzzz-20210616_deg.txt

Unfortunately I have no control over the server, so cannot ask them to open ports, upgrade to a secure protocol or any other true fix, but this will have to do!

8

My guess is they have a "dynamic" firewall with "helper" functionality; it sees the PASV command response and temporarily opens the necessary port so you can connect to it (Or they may have NATting from their border to the internal machine, especially if there's a cluster behind it, or a load balancer or.... many reasons). Linux iptables can do this, for example, with the ftp helper.

Since EPSV is newer and the output format is different (no IP address) it doesn't know how to parse it and so isn't able to allow your connection through.

9

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.