Rclone FTP support

I have another bug…
use of closed network connection
https://paste.ee/p/rO89x

I have to restart rclone and it only works for a while…

Found the problem - I had not specified the protocol in the URL, using only speedtest.tele2.net. Adding ftp:// in front of it fixed it. :slight_smile: I should’ve checked the example in the documentation - I had looked at the SFTP example and I had assumed that if I specify the remote type as FTP, rclone will add the protocol automatically.

Thank you for your help!

Do you also experience the same issue as I do?

use of closed network connection

Quick question, does this support TLS (FTPES)?

Thanks

That will depend on the FTP library https://github.com/jlaffaye/ftp

I don’t think it supports TLS. It probably wouldn’t be too difficult to add as go has very good TLS support.

1 Like

Hi @ncw, I would also love to see explicit TLS on rclone ftp :heart_eyes: Direct upload a directory from ftp to gdrive would be awesome!

1 Like

@ncw Don’t forget to add FTP to https://rclone.org/overview/. :slight_smile:

1 Like

I hope this is the correct place to post this, but I have a particular FTP scenario that doesn’t work and I feel it’s common enough to address. I’m trying to connect to my friend’s Filezilla server (because he’s a sad little Windows addict) but I can’t for the life of me get it to connect for ls, lsd, mount. Using a Filezilla client it works just fine, same for command line ftp in both Windows and Linux, as you’d expect from one of the biggest names in ftp. So… Here’s my commands:

rclone -vvv ls Rez:
rclone -vvv ls Rez:/
rclone mount -vvv --umask 0 --allow-non-empty --allow-other Rez:/ /mnt/Rez

All of which had the same result:

root@automation:~# rclone -vvv ls Rez:
2017/08/02 09:47:47 DEBUG : rclone: Version “v1.37” starting with parameters [“rclone” “-vvv” “ls” “Rez:”]
2017/08/02 09:47:47 DEBUG : ftp://ftp.server.com:21: Connecting to FTP server
2017/08/02 09:47:48 INFO : ftp://ftp.server.com:21: Modify window not supported
2017/08/02 09:48:48 ERROR : : error listing: dial tcp 199.59.170.116:6907: i/o timeout
2017/08/02 09:48:48 Failed to ls: dial tcp 199.59.170.116:6907: i/o timeout
root@automation:~# rclone -vvv ls Rez:
2017/08/02 10:04:43 DEBUG : rclone: Version “v1.37” starting with parameters [“rclone” “-vvv” “ls” “Rez:”]
2017/08/02 10:04:43 DEBUG : ftp://ftp.server.com:21: Connecting to FTP server
2017/08/02 10:04:44 INFO : ftp://ftp.server.com:21: Modify window not supported
2017/08/02 10:05:44 ERROR : : error listing: dial tcp 199.59.170.116:6953: i/o timeout
2017/08/02 10:05:44 Failed to ls: dial tcp 199.59.170.116:6953: i/o timeout

Here’s what the config looks like:

[Rez]
type = ftp
host = ftp.server.com
user = duvrazh
port = 21
pass = mybiglongencryptedpassword

I noticed in an earlier example yours says url instead of host, and if I include ftp:// I get the error too many colons in address. Do I need to manually change the config file?

Of course I’ve substituted the words ftp and server for the actual addresses but there doesn’t seem to be any funnybusiness going on. Honestly I expected a Filezilla server that doesn’t use encryption to kind of “just work”. It works for all the other methods of connecting I can muster. I’m scratching my beard so far. Was hoping you wouldn’t mind sharing your opinion with me @ncw - and thank you in advance.

So that gives a 1 minute timeout.

That makes me think that it is probably active vs passive mode FTP problem. The FTP library should auto detect which mode to use though. Or maybe that range of passive ports isn’t passing through someone’s firewall.

I think debugging this might be a job for wireshark… If you try to capture a session, I’d be interested to see what happens in that 60s pause. Also to see what kind of connection the client asked for - did it use EPSV?

Can you replicate the problem locally if you run the FileZilla FTP server yourself? FileZiller works on Windows, Mac and Linux IIRC.

Also are you behind a NAT box? Can you try on a server which is directly connected to the Internet to see if that makes any difference.

Your config looks fine. We changed the config to line up with sftp just before the release, so lost the url bit.

1 Like

The target only has port 21 open, I’m guessing the library is probably reading that passive is capable but isn’t smart enough to know that if those additional ports aren’t open to restrict data and communications to 21. I did try it on a direct box, a VM out in the void. No change.

If I run Filezilla locally it works just fine, no firewall between initiator and listener. Additionally, I am behind a NAT box, Sophos XG. Unfortunately I can’t bypass that because my wife or kids will slay me in my sleep, but I feel like the VM was an acceptable substitute.

Also I’ve never used wireshark. It doesn’t look like a tool that can be understood on the first use.

I don’t think the FTP library I’m using supports active mode (which is mostly broken behind NAT anyway) which might be what your other programs are using.

You could try using passive mode with your other client just to confirm that it is the passive mode that is the problem.

Can you (or your friend with the FTP server) configure the ports for passive mode and let them through the firewall? That should hopefully make it work.

OK that is good to know.

It is a bit fearsome, but it is an excellent tool for debugging network problems.

Any update on FTPS support? curlftpfs is so broken this might be the last port of call to mount it.

Alas the underlying library I use for FTP doesn’t support SFTP :frowning: