An issue about FTP backend in 2 different systems

kadogo · February 28, 2021, 6:15pm

What is the problem you are having with rclone?

With the latest bug related to mega-webdav I wanted to use their FTP and it work correctly on my laptop but not on my nas. I guess it's something related to the system but I cannot find what so I wanted a bit of help because I cannot upgrade my nas for now.

If you need more info let me know, thanks for your help.

What is your rclone version (output from `rclone version`)

LAPTOP

$ rclone version
rclone v1.54.0
- os/arch: linux/amd64
- go version: go1.15.7

NAS

$ rclone version
rclone v1.54.0
- os/arch: linux/amd64
- go version: go1.15.7

Which OS you are using and how many bits (eg Windows 7, 64 bit)

LAPTOP

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Debian
Description:	Debian GNU/Linux 10 (buster)
Release:	10
Codename:	buster

NAS

$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 9.9 (stretch)
Release:        9.9
Codename:       stretch

Which cloud storage system are you using? (eg Google Drive)

I'm using FTP with megacmd command

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

LAPTOP

$ rclone lsd mega-ftp:/
          -1 2020-10-19 22:38:00        -1 XXX

NAS

$ rclone lsd mega-ftp:/
2021/02/28 19:10:40 ERROR : ftp://127.0.0.1:4990/XXX/YYY: Timeout when waiting for List
2021/02/28 19:10:40 ERROR : : error listing: Timeout when waiting for List
2021/02/28 19:10:40 Failed to lsd with 2 errors: last error was: Timeout when waiting for List

The rclone config contents with secrets removed.

I copy pasted my config from my laptop to my nas, I remove the rest that are not related to the mega ftp backend. I used alias because mega-ftp give a path for the ftp and I didn't found how to use it directly.

[mega-ftp-co]
type = ftp
host = 127.0.0.1
user = anonymous
pass = anonymous
port = 4990

[mega-ftp]
type = alias
remote = mega-ftp-co:/XXX/YYY/

A log from the command with the `-vv` flag

NAS

$ rclone lsd mega-ftp:/ -vvv
2021/02/28 19:13:17 DEBUG : rclone: Version "v1.54.0" starting with parameters ["rclone" "lsd" "mega-ftp:/" "-vvv"]
2021/02/28 19:13:17 DEBUG : Using config file from "$HOME/.config/rclone/rclone.conf"
2021/02/28 19:13:17 DEBUG : Creating backend with remote "mega-ftp:/"
2021/02/28 19:13:17 DEBUG : Creating backend with remote "mega-ftp-co:/XXX/YYY"
2021/02/28 19:13:17 DEBUG : ftp://127.0.0.1:4990/XXX/YYY: Connecting to FTP server
2021/02/28 19:13:17 DEBUG : fs cache: renaming cache item "mega-ftp:/" to be canonical "mega-ftp-co:/XXX/YYY"
2021/02/28 19:13:17 ERROR : ftp://127.0.0.1:4990/XXX/YYY: Timeout when waiting for List
2021/02/28 19:13:17 ERROR : : error listing: Timeout when waiting for List
2021/02/28 19:13:17 DEBUG : 3 go routines active
2021/02/28 19:13:17 Failed to lsd with 2 errors: last error was: Timeout when waiting for List

asdffdsa · February 28, 2021, 6:21pm

is that laptop and NAS on the same lan, behind the router?

kadogo · February 28, 2021, 6:23pm

The same lan and I can direct connect on it with my lan ip.

I forgot to say that on the NAS I can make the connection with lftp

ncw · March 1, 2021, 9:59am

Can you do your command with -vv --dump bodies so we can see the FTP transactions?

kadogo · March 1, 2021, 10:03am

Sure I add it below

LAPTOP

$ rclone lsd mega-ftp:/ -vv --dump bodies
2021/03/01 11:05:59 DEBUG : rclone: Version "v1.54.0" starting with parameters ["rclone" "lsd" "mega-ftp:/" "-vv" "--dump" "bodies"]
2021/03/01 11:05:59 DEBUG : Using config file from "$HOME/.config/rclone/rclone.conf"
2021/03/01 11:05:59 DEBUG : Creating backend with remote "mega-ftp:/"
2021/03/01 11:05:59 DEBUG : Creating backend with remote "mega-ftp-co:/XXX/YYY"
2021/03/01 11:05:59 DEBUG : ftp://127.0.0.1:4990/XXX/YYY: Connecting to FTP server
2021/03/01 11:05:59 DEBUG : FTP Rx: "220 Wellcome to FTP MEGA Server"
2021/03/01 11:05:59 DEBUG : FTP Tx: "USER anonymous"
2021/03/01 11:05:59 DEBUG : FTP Rx: "331 User name okay, need password"
2021/03/01 11:05:59 DEBUG : FTP Tx: PASS *****
2021/03/01 11:05:59 DEBUG : FTP Rx: "230 User logged in, proceed"
2021/03/01 11:05:59 DEBUG : FTP Tx: "FEAT"
2021/03/01 11:05:59 DEBUG : FTP Rx: "211-Features:"
2021/03/01 11:05:59 DEBUG : FTP Rx: " SIZE"
2021/03/01 11:05:59 DEBUG : FTP Rx: " PROT"
2021/03/01 11:05:59 DEBUG : FTP Rx: " EPSV"
2021/03/01 11:05:59 DEBUG : FTP Rx: " PBSZ"
2021/03/01 11:05:59 DEBUG : FTP Rx: " UTF8 ON"
2021/03/01 11:05:59 DEBUG : FTP Rx: "211 End"
2021/03/01 11:05:59 DEBUG : FTP Tx: "TYPE I"
2021/03/01 11:05:59 DEBUG : FTP Rx: "200 OK"
2021/03/01 11:05:59 DEBUG : FTP Tx: "OPTS UTF8 ON"
2021/03/01 11:05:59 DEBUG : FTP Rx: "200 All good"
2021/03/01 11:05:59 DEBUG : FTP Tx: "EPSV"
2021/03/01 11:05:59 DEBUG : FTP Rx: "229 Entering Extended Passive Mode (|||1504|)"
2021/03/01 11:05:59 DEBUG : FTP Tx: "LIST /XXX"
2021/03/01 11:05:59 DEBUG : FTP Rx: "150 Here comes the directory listing"
2021/03/01 11:05:59 DEBUG : FTP Rx: "226 Closing data connection. Requested file action successful."
2021/03/01 11:05:59 DEBUG : fs cache: renaming cache item "mega-ftp:/" to be canonical "mega-ftp-co:/XXX/YYY"
2021/03/01 11:05:59 DEBUG : FTP Tx: "EPSV"
2021/03/01 11:05:59 DEBUG : FTP Rx: "229 Entering Extended Passive Mode (|||1504|)"
2021/03/01 11:05:59 DEBUG : FTP Tx: "LIST /XXX/YYY"
2021/03/01 11:05:59 DEBUG : FTP Rx: "150 Here comes the directory listing"
2021/03/01 11:05:59 DEBUG : FTP Rx: "226 Closing data connection. Requested file action successful."
          -1 2020-10-19 22:38:00        -1 A
2021/03/01 11:05:59 DEBUG : 2 go routines active

NAS

$ rclone lsd mega-ftp:/ -vv --dump bodies
2021/03/01 11:00:37 DEBUG : rclone: Version "v1.54.0" starting with parameters ["rclone" "lsd" "mega-ftp:/" "-vv" "--dump" "bodies"]
2021/03/01 11:00:37 DEBUG : Using config file from "$HOME/.config/rclone/rclone.conf"
2021/03/01 11:00:37 DEBUG : Creating backend with remote "mega-ftp:/"
2021/03/01 11:00:37 DEBUG : Creating backend with remote "mega-ftp-co:/XXX/YYY"
2021/03/01 11:00:37 DEBUG : ftp://127.0.0.1:4990/XXX/YYY: Connecting to FTP server
2021/03/01 11:00:37 DEBUG : FTP Rx: "220 Wellcome to FTP MEGA Server"
2021/03/01 11:00:37 DEBUG : FTP Tx: "USER anonymous"
2021/03/01 11:00:37 DEBUG : FTP Rx: "331 User name okay, need password"
2021/03/01 11:00:37 DEBUG : FTP Tx: PASS *****
2021/03/01 11:00:37 DEBUG : FTP Rx: "230 User logged in, proceed"
2021/03/01 11:00:37 DEBUG : FTP Tx: "FEAT"
2021/03/01 11:00:37 DEBUG : FTP Rx: "211-Features:"
2021/03/01 11:00:37 DEBUG : FTP Rx: " SIZE"
2021/03/01 11:00:37 DEBUG : FTP Rx: " PROT"
2021/03/01 11:00:37 DEBUG : FTP Rx: " EPSV"
2021/03/01 11:00:37 DEBUG : FTP Rx: " PBSZ"
2021/03/01 11:00:37 DEBUG : FTP Rx: " UTF8 ON"
2021/03/01 11:00:37 DEBUG : FTP Rx: "211 End"
2021/03/01 11:00:37 DEBUG : FTP Tx: "TYPE I"
2021/03/01 11:00:37 DEBUG : FTP Rx: "200 OK"
2021/03/01 11:00:37 DEBUG : FTP Tx: "OPTS UTF8 ON"
2021/03/01 11:00:37 DEBUG : FTP Rx: "200 All good"
2021/03/01 11:00:37 DEBUG : FTP Tx: "EPSV"
2021/03/01 11:00:37 DEBUG : FTP Rx: "229 Entering Extended Passive Mode (|||1513|)"
2021/03/01 11:00:37 DEBUG : FTP Tx: "LIST /XXX"
2021/03/01 11:00:37 DEBUG : FTP Rx: "150 Here comes the directory listing"
2021/03/01 11:00:37 DEBUG : FTP Rx: "226 Closing data connection. Requested file action successful."
2021/03/01 11:00:37 DEBUG : fs cache: renaming cache item "mega-ftp:/" to be canonical "mega-ftp-co:/XXX/YYY"
2021/03/01 11:00:37 ERROR : ftp://127.0.0.1:4990/XXX/YYY: Timeout when waiting for List
2021/03/01 11:00:37 ERROR : : error listing: Timeout when waiting for List
2021/03/01 11:00:37 DEBUG : 3 go routines active
2021/03/01 11:00:37 Failed to lsd with 2 errors: last error was: Timeout when waiting for List

I added the laptop too in case it can help

ncw · March 1, 2021, 11:39am

kadogo:

2021/03/01 11:00:37 DEBUG : FTP Rx: "226 Closing data connection. Requested file action successful."
2021/03/01 11:00:37 DEBUG : fs cache: renaming cache item "mega-ftp:/" to be canonical "mega-ftp-co:/XXX/YYY"
2021/03/01 11:00:37 ERROR : ftp://127.0.0.1:4990/XXX/YYY: Timeout when waiting for List
2021/03/01 11:00:37 ERROR : : error listing: Timeout when waiting for List

This is strange...

The default timeout should be 5 minutes, but that didn't take 5 minutes.

Have you got RCLONE_TIMEOUT set in your environment variables maybe?

kadogo · March 1, 2021, 11:42am

Nice caught, I had t set as RCLONE_TIMEOUT=0 so I thought it meant "unlimited". Passing it to 300s make it work...

Sorry I forgot that they were set. You want that I keep it open in case the behavior for RCLONE_TIMEOUT=0 is not right?

ncw · March 1, 2021, 12:09pm

Great

Yes setting it to 0 means use a 0 timeout which isn't that useful.

I fixed this in the latest beta (uploaded in 30 mins if GitHub actions are working again)

kadogo · March 2, 2021, 10:33am

Sorry for the wait. I jut test the latest beta and it works with timeout 0.

I close this one, thanks for your help ^^

system · March 5, 2021, 10:33am

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.

An issue about FTP backend in 2 different systems

What is the problem you are having with rclone?

What is your rclone version (output from rclone version)

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Which cloud storage system are you using? (eg Google Drive)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

The rclone config contents with secrets removed.

A log from the command with the -vv flag

What is your rclone version (output from `rclone version`)

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

A log from the command with the `-vv` flag