Rclone FTP support

Dev Discussions
1

I saw that there is FTP support in the works… So I installed the latest beta,
rclone-v1.36-114-gcdacf026β-linux-amd64

I mounted the drive using:
rclone mount --read-only --allow-non-empty --allow-other --max-read-ahead 1G --checkers 16 ftp:/PathToDIR /home/plex/.acd &
(are there any settings for FTP? I used this for acd.)

It works well, playing a 8GB file via plex, but I get a few errors:

ReadFileHandle.Release error: 550 Action not taken - really often. About 2-3 times a minute, when it’s reading from the FTP.

and this:

ReadFileHandle.Read error: low level retry 1/10: unexpected EOF i get this only a few times, mostly when I try to seek, but I can live without this.

The ftp is from my ISP, they offer a cloud drive that can be accessed via FTP and via WebDAV.

BTW, it works sooooo good, even with these errors!
I also tried curlftpfs but you can’t even compare rclone ftp to it!

1 Like
2

Thanks for having a go with the FTP support. I’m very pleased it worked that well :smile:

Can you paste a bit more log with -vv leading up to those errors? I’d like to know what it is doing at that time.

That is normally to do with dropped connections. If it is to do with seek then maybe there is a problem there.

Have you got more logs around those errors too?

4

! SFTP is not the same thing as FTP(S)! This topic should be only for the newly implemented FTP!

5

Ok sorry - No need to shout. There’s always one. Rather than being rude. Can I ask what the advantage to using FTP(S) would be over SFTP.

6

Here is the log:
https://pastebin.com/dkdaTa21

Please let me know if you need more logs OR if you need some info about the FTP server, I can ask my ISP.

I mount the ftp via rclone and then decrypt it using encfs I modified some encrypted strings to ‘movie’ and ‘big.buck.bunny’ to be easier to read and also deleted 7000+ lines that had repeating, error less data.

7

My ISP does not support SFTP :slight_smile: so I can’t tell you if one or the other is better or not.
I am sorry I was so harsh.

8

Hi, no problem. I’m using sftp between all of my servers so will probably stick with it, seems to work well.

9

Yes stick with sftp - it is a secure modern protocol, unlike FTP :wink:

10

It looks like that “550 action not taken” error is returned when you close a stream early.

I’ll investigate futher…

11

That’s what I was thinking of too, but I do not close the stream, and it appears mostly when the file starts downloading/streaming, bcause, as you can see, after a while I get no errors.

12

The stream is closed every time you seek (which happens mostly at the start).

Try this beta (uploaded in 15-30 mins) and see if it makes a difference

https://beta.rclone.org/v1.36-116-ge172f00e/

It should mask the error on close for a stream that has been partially transferred.

13

I hope this is the right location - I’m also trying out the FTP support, but I cannot seem to be able to connect to any FTP server. Example from the log using the test.talia.net FTP server set up as a remote:

2017/05/19 15:33:39 DEBUG : rclone: Version "v1.36-114-gcdacf026" starting with parameters ["rclone" "-vv" "lsd" "talia:"]
2017/05/19 15:33:39 DEBUG : test.talia.net: Connecting to FTP server
2017/05/19 15:33:40 ERROR : test.talia.net: Error while Dialing :21: dial tcp :21: connectex: No connection could be made because the target machine actively refused it.
2017/05/19 15:33:40 Failed to create file system for "talia:": NewFs: ftpConnection Dial: dial tcp :21: connectex: No connection could be made because the target machine actively refused it.

I’ve tried 4 servers - two public/anonymous and two private. I can connect to them normally using FTP client software or even the ftp command in the same command prompt, so I think the problem is on my end.

I’m using 64-bit Windows 10. I’ve checked the Windows firewall settings and added rclone as an exception. What else can I try?

14

I no longer have that 550 error…

But I have some other issue.
I think when I access the FTP multiple times, eg. plex scanning, it kicks me out…
I can no longer access anything and I have to close rclone and make the connection again.
This start when I start to get unexpected EOF errors.
I think this is the error:

getsockopt: connection refused

https://pastebin.com/TnEmHV5t

(I think you could close the connection competly and open it again, just like when rclone starts.)

15

It’s a server issue, try to connect to it using filezilla ftp client. Or just try ftp://test.talia.net - it won’t work.

16

Huh. Apparently it has gone down. However, here’s another server: ftp://speedtest.tele2.net

Works in FileZilla and browser, does not work in rclone - I get the exact same messages.

17

Works for me - this is what my config looks like. The password is a made up email address.

[speedtest]
type = ftp
username = anonymous
password = WfbyAlPorIIb4uKVE8DFx5wvIN-UHdhMwPbL2tda
url = ftp://speedtest.tele2.net
18

Good!

I can see what has happened there - the FTP control channel has got out of sync somehow.

2017/05/19 14:25:20 ERROR : movies/big.buck.bunny2/big.buck.bunny2.mp4: Dir.Lookup error: short response: 227 Entering Passive Mode (62,231,125,2,173,173)

This was probably caused by the download failing.

2017/05/19 14:25:17 ERROR : movies/BBB/BBB.mp4: ReadFileHandle.Read error: low level retry 1/10: unexpected EOF

So what I should do is toss away the control channel if a download fails (I already do this on upload)…

Try this beta https://beta.rclone.org/v1.36-120-ga9d29c22/ (uploaded in 15-30 mins)

19

I downloaded 119 half an hour ago, even on that it works much better! I can seek the movie with this version.

I saw in the log that it re-connected to the FTP a few times.

It is not as reliable as acd mount but damn, it works so good compared to curlftpfs!
rclone might become the best FTP mount tool for linux!

I’ll be back with feedback on v1.36-120!

Are the mount parameters OK? Should I change them? Are there any special ones for FTP?

v1.36-120 logs:

https://pastebin.com/VaG0Bxt2

Off-topic, is it just me or does PLEX read the subtitle from drive line by line? Instead of caching it in memory?

20

:slight_smile:[quote=“easy90rider, post:19, topic:2320”]
I saw in the log that it re-connected to the FTP a few times.
[/quote]

I think that is probably to be expected.

:slight_smile: Actually I’m amazed how well it works. FTP is a really old and crusty protocol.

They look fine to me.

21

You really saved me! :slight_smile:

I just started using ACD with rclone and now that they banned it, I remembered I could connect to it via my ISP’s cloud drive, and I did. My only problem was that it could be accessed only via FTP and WebDAV, but only FTP is reliable. Funny thing is that I provided feedback for their FTP server solution too, they didn’t enable REST…

Since I installed v1.36-120, I no longer had major issues. It is not as stable as ACD was, but it works really well!

I will let you know if I see other errors.

Do you plan to support (or does it already) FTPS/FTPES?