Rclone to FTP server

t3amz1 · October 2, 2021, 7:37am

What is the problem you are having with rclone?

I would like to send stuff from remote server (debian server) to my ftp server but I got this error , I googled with no luck

2021/10/02 07:30:50 DEBUG : ftp://ipserver:port/MUSIC: Connecting to FTP server
2021/10/02 07:30:50 DEBUG : pacer: low level retry 1/10 (error EOF)
2021/10/02 07:30:50 DEBUG : pacer: Rate limited, increasing sleep to 20ms
2021/10/02 07:30:50 DEBUG : pacer: low level retry 2/10 (error EOF)
2021/10/02 07:30:50 DEBUG : pacer: Rate limited, increasing sleep to 40ms
2021/10/02 07:30:50 DEBUG : pacer: low level retry 3/10 (error EOF)
2021/10/02 07:30:50 DEBUG : pacer: Rate limited, increasing sleep to 80ms
2021/10/02 07:30:50 DEBUG : pacer: low level retry 4/10 (error EOF)
2021/10/02 07:30:50 DEBUG : pacer: Rate limited, increasing sleep to 160ms
2021/10/02 07:30:50 DEBUG : pacer: low level retry 5/10 (error EOF)
2021/10/02 07:30:50 DEBUG : pacer: Rate limited, increasing sleep to 320ms
2021/10/02 07:30:50 DEBUG : pacer: low level retry 6/10 (error EOF)
2021/10/02 07:30:50 DEBUG : pacer: Rate limited, increasing sleep to 640ms
2021/10/02 07:30:50 DEBUG : pacer: low level retry 7/10 (error EOF)
2021/10/02 07:30:50 DEBUG : pacer: Rate limited, increasing sleep to 1.28s
2021/10/02 07:30:51 DEBUG : pacer: low level retry 8/10 (error EOF)
2021/10/02 07:30:51 DEBUG : pacer: Rate limited, increasing sleep to 2s
2021/10/02 07:30:52 DEBUG : pacer: low level retry 9/10 (error EOF)
2021/10/02 07:30:54 DEBUG : pacer: low level retry 10/10 (error EOF)
2021/10/02 07:30:54 Failed to create file system for "MP3:/MUSIC": NewFs: failed to make FTP connection to "ipserver:port": EOF

What is your rclone version (output from `rclone version`)

--> last one

Which cloud storage system are you using? (eg Google Drive)

FTP server (glftpd)

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone sync /fodlerpath/data/ MP3:/MUSIC --vvv

The rclone config contents with secrets removed.

[MP3]
type = ftp
host = ipserver
user = username
port = port
pass = passw
tls = true
explicit_tls = false

I tried tls false/true, explicit_tls false/true even no_check_certificate = true or false

The error is the same I don't know if is correlated to that conf

Thanks in advance for the reply !

asdffdsa · October 2, 2021, 12:45pm

hello and welcome to the forum,

run rclone version and post the output

to get a more detailed debug log, add
--dump bodies --retries=1 --low-level-retries=1
post the full debug log.

t3amz1 · October 2, 2021, 5:22pm

hello thanks for the reply!
Version :
rclone version
rclone v1.56.2

os/version: debian 9.13 (64 bit)
os/kernel: 4.19.123-feral (x86_64)
os/type: linux
os/arch: amd64
go/version: go1.16.8
go/linking: static
go/tags: none

OUTPUT:[ ~] rclone sync /private/qBittorrent/data/ mp3m:/MUSIC --dump bodies --retries=1 --low-level-retries=1
2021/10/02 17:19:41 NOTICE: Automatically setting -vv as --dump is enabled
2021/10/02 17:19:41 DEBUG : rclone: Version "v1.56.2" starting with parameters ["rclone" "sync" "/private/qBittorrent/data/" "mp3m:/MUSIC" "--dump" "bodies" "--retries=1" "--low-level-retries=1"]
2021/10/02 17:19:41 DEBUG : Creating backend with remote "/private/qBittorrent/data/"
2021/10/02 17:19:41 DEBUG : Using config file from "/path/.config/rclone/rclone.conf"
2021/10/02 17:19:41 DEBUG : Creating backend with remote "mp3m:/MUSIC"
2021/10/02 17:19:41 DEBUG : ftps://ipserver:port/MUSIC: Connecting to FTP server
2021/10/02 17:19:41 DEBUG : FTP Tx: "QUIT"
2021/10/02 17:19:41 DEBUG : pacer: low level retry 1/1 (error EOF)
2021/10/02 17:19:41 DEBUG : pacer: Rate limited, increasing sleep to 20ms
2021/10/02 17:19:41 Failed to create file system for "mp3m:/MUSIC": NewFs: failed to make FTP connection to "ipserver:port": EOF

I am try to running rclone on the seedbox and send stuff to my ftp (i can log with any client without a problem)

asdffdsa · October 2, 2021, 5:31pm

hi,
i do not know why you are experiencing that issue.
ftp with rclone is a finicky thing.
might try --dump headers --retries=1 --low-level-retries=1

fwiw,
most seedbox providers support sftp, which has much better rclone support.
also, sftp, support modtime and depending on the server, supports hash checksums to verify file transfer.

i suggest to use sftp or some other protocol.
if that is not possible, we have experts at rclone+ftp.

t3amz1 · October 2, 2021, 5:33pm

Hello! I need to use glftpd as home ftp server cuz it support some script that i need to run

t3amz1 · October 2, 2021, 5:41pm

copy or sync is the same , I think somehow it can't connect to my ftp , now it's stucked without any respose but with filezilla,flashfxp or rushftp it works.

Maybe SSL problem?

asdffdsa · October 2, 2021, 5:44pm

from that seedbox, you can access the local ftp server using filezilla?

t3amz1 · October 2, 2021, 5:45pm

yes I can connect

t3amz1 · October 2, 2021, 5:50pm

ftp://ipserver:port/MUSIC: Connecting to FTP server
2021/10/02 17:42:43 DEBUG : FTP Tx: "QUIT"
2021/10/02 17:42:43 DEBUG : pacer: low level retry 1/1 (error read tcp ipserverseedbox:port->ipserver:port: i/o timeout)
2021/10/02 17:42:43 DEBUG : pacer: Rate limited, increasing sleep to 20ms
2021/10/02 17:42:43 Failed to create file system for "mp3m:/MUSIC": NewFs: failed to make FTP connection to "ipserver:port": read tcp ipseedbox:port->ipserver:port: i/o timeout

asdffdsa · October 2, 2021, 5:54pm

sorry i do not know,

rclone only support passive, not active connections

tho @ivandeex might now...

t3amz1 · October 2, 2021, 5:55pm

how can i set passive or active? I mean my ftp is in passive mode

btw Thanks for your time

asdffdsa · October 2, 2021, 5:59pm

perhaps the log from one of the other ftp clients will indicate passive/active.

if you setup the local firewall, i would imagine that you had to choose a set of rules, which would be different between active and passive.

as a test, on that local firewall(s), whitelist the ip address of the seedbox.

t3amz1 · October 2, 2021, 6:10pm

I thought was easy , just setting up rclone and done but seems not :S

asdffdsa · October 2, 2021, 6:13pm

perhaps, as a test, try another local ftp server.

on local, run rclone serve ftp - which is rclone's internal ftp server
on seedbox, run rclone sync to that local rclone ftp server

t3amz1 · October 2, 2021, 6:23pm

rclone serve ftp

Error unknow protocol

asdffdsa · October 2, 2021, 6:27pm

what seedbox provider are you using?

did you whitelist the seedbox ip address, that would make debugging easier...

for any command you post, need to add -vv for debug output.
and try to enclose it with three backticks so it looks like

rclone lsd seedboxio_ftp_internet: --dump=bodies -vv
2021/10/02 13:40:17 DEBUG : rclone: Version "v1.56.0" starting with parameters ["c:\\data\\rclone\\scripts\\rclone.exe" "lsd" "seedboxio_ftp_internet:" "--dump=bodies"]
2021/10/02 13:40:17 DEBUG : Creating backend with remote "seedboxio_ftp_internet:"
2021/10/02 13:40:17 DEBUG : Using config file from "C:\\data\\rclone\\scripts\\seedboxio.conf"
2021/10/02 13:40:17 DEBUG : ftp://xxx.seedbox.io:21: Connecting to FTP server
2021/10/02 13:40:17 DEBUG : FTP Rx: "220 ProFTPD Server (seedbox.io)"
2021/10/02 13:40:17 DEBUG : FTP Tx: "USER xxx"
2021/10/02 13:40:17 DEBUG : FTP Rx: "331 Password required for xxx"
2021/10/02 13:40:17 DEBUG : FTP Tx: PASS *****
2021/10/02 13:40:17 DEBUG : FTP Rx: "230 User xxx logged in"
2021/10/02 13:40:17 DEBUG : FTP Tx: "FEAT"
2021/10/02 13:40:17 DEBUG : FTP Rx: "211-Features:"
2021/10/02 13:40:17 DEBUG : FTP Rx: " AUTH TLS"
2021/10/02 13:40:17 DEBUG : FTP Rx: " CCC"
2021/10/02 13:40:17 DEBUG : FTP Rx: " CLNT"
2021/10/02 13:40:17 DEBUG : FTP Rx: " EPRT"
2021/10/02 13:40:17 DEBUG : FTP Rx: " EPSV"
2021/10/02 13:40:17 DEBUG : FTP Rx: " HOST"
2021/10/02 13:40:17 DEBUG : FTP Rx: " LANG zh-CN;zh-TW;en-US;bg-BG;es-ES;fr-FR;it-IT;ja-JP;ko-KR;ru-RU"
2021/10/02 13:40:17 DEBUG : FTP Rx: " MDTM"
2021/10/02 13:40:17 DEBUG : FTP Rx: " MFF modify;UNIX.group;UNIX.mode;"
2021/10/02 13:40:17 DEBUG : FTP Rx: " MFMT"
2021/10/02 13:40:17 DEBUG : FTP Rx: " MLST modify*;perm*;size*;type*;unique*;UNIX.group*;UNIX.groupname*;UNIX.mode*;UNIX.owner*;UNIX.ownername*;"
2021/10/02 13:40:17 DEBUG : FTP Rx: " PBSZ"
2021/10/02 13:40:17 DEBUG : FTP Rx: " PROT"
2021/10/02 13:40:17 DEBUG : FTP Rx: " REST STREAM"
2021/10/02 13:40:17 DEBUG : FTP Rx: " SITE COPY"
2021/10/02 13:40:17 DEBUG : FTP Rx: " SITE MKDIR"
2021/10/02 13:40:17 DEBUG : FTP Rx: " SITE RMDIR"
2021/10/02 13:40:17 DEBUG : FTP Rx: " SITE SYMLINK"
2021/10/02 13:40:17 DEBUG : FTP Rx: " SITE UTIME"
2021/10/02 13:40:17 DEBUG : FTP Rx: " SIZE"
2021/10/02 13:40:17 DEBUG : FTP Rx: " SSCN"
2021/10/02 13:40:17 DEBUG : FTP Rx: " TVFS"
2021/10/02 13:40:17 DEBUG : FTP Rx: " UTF8"
2021/10/02 13:40:17 DEBUG : FTP Rx: "211 End"
2021/10/02 13:40:17 DEBUG : FTP Tx: "TYPE I"
2021/10/02 13:40:17 DEBUG : FTP Rx: "200 Type set to I"
2021/10/02 13:40:17 DEBUG : FTP Tx: "OPTS UTF8 ON"
2021/10/02 13:40:17 DEBUG : FTP Rx: "200 UTF8 set to on"
2021/10/02 13:40:17 DEBUG : FTP Tx: "EPSV"
2021/10/02 13:40:17 DEBUG : FTP Rx: "229 Entering Extended Passive Mode (|||31467|)"
2021/10/02 13:40:18 DEBUG : FTP Tx: "MLSD"
2021/10/02 13:40:18 DEBUG : FTP Rx: "150 Opening BINARY mode data connection for MLSD"
2021/10/02 13:40:18 DEBUG : FTP Rx: "226 Transfer complete"
          -1 2021-09-30 13:09:30        -1 files
          -1 2021-03-31 15:23:01        -1 watch

t3amz1 · October 2, 2021, 6:36pm

i have a seedbox on feralhosting, I would have that output....

I installed rclone on local machine where glftpd is running but that cmd doesnt works

rclone serve ftp

Error unknow protocol

I can connect to my glftpd server even with my phone...

asdffdsa · October 2, 2021, 6:42pm

as i wrote, rclone support of ftp servers is flaky and i am not an expert.
if you want, just wait until somebody else stops by or we can try to figure it out.
up to you...

please understand i cannot see into your computer.
so if you run a rclone command, need to see a full debug log enclosed with three backticks as i showed up above.

t3amz1 · October 2, 2021, 6:47pm

yes I did :

rclone sync /private/qBittorrent/data/ mp3m:/MUSIC --ftp-tls

2021/10/02 18:46:10 DEBUG : pacer: low level retry 1/10 (error EOF)
2021/10/02 18:46:10 DEBUG : pacer: Rate limited, increasing sleep to 20ms
2021/10/02 18:46:10 DEBUG : pacer: low level retry 2/10 (error EOF)
2021/10/02 18:46:10 DEBUG : pacer: Rate limited, increasing sleep to 40ms
2021/10/02 18:46:10 DEBUG : pacer: low level retry 3/10 (error EOF)
2021/10/02 18:46:10 DEBUG : pacer: Rate limited, increasing sleep to 80ms
2021/10/02 18:46:10 DEBUG : pacer: low level retry 4/10 (error EOF)
2021/10/02 18:46:10 DEBUG : pacer: Rate limited, increasing sleep to 160ms
2021/10/02 18:46:10 DEBUG : pacer: low level retry 5/10 (error EOF)
2021/10/02 18:46:10 DEBUG : pacer: Rate limited, increasing sleep to 320ms
2021/10/02 18:46:11 DEBUG : pacer: low level retry 6/10 (error EOF)
2021/10/02 18:46:11 DEBUG : pacer: Rate limited, increasing sleep to 640ms
2021/10/02 18:46:11 DEBUG : pacer: low level retry 7/10 (error EOF)
2021/10/02 18:46:11 DEBUG : pacer: Rate limited, increasing sleep to 1.28s
2021/10/02 18:46:12 DEBUG : pacer: low level retry 8/10 (error EOF)
2021/10/02 18:46:12 DEBUG : pacer: Rate limited, increasing sleep to 2s
2021/10/02 18:46:13 DEBUG : pacer: low level retry 9/10 (error EOF)
2021/10/02 18:46:15 DEBUG : pacer: low level retry 10/10 (error EOF)
2021/10/02 18:46:15 Failed to create file system for "mp3m:/MUSIC": NewFs: failed to make FTP connection

t3amz1 · October 2, 2021, 6:49pm

I really don't know may I'll change seedbox provider but I haven't any ideas to how to fix it