Rclone adding a space before paths

Robert_Timis · July 20, 2022, 12:13pm

For some weird reason Rclone is parsing FTP output wrong and adds a space, rendering ls ouput a mess and sync unusable.
Using the ftp command on linux or filezilla nothing seems out of place.

rclone v1.59.0

os/version: arch (64 bit)
os/kernel: 5.18.12-arch1-1 (x86_64)
os/type: linux
os/arch: amd64
go/version: go1.18.3
go/linking: dynamic
go/tags: none

Also tested on Windows

Remote type:
FTP

The config is the default FTP with nothing special added.

This is the FTP output:
drwxrwxrwx 2 ---- staff 256 Mar 21 2019 ANAGRAFICA
drwxrwxrwx 2 ---- staff 256 Apr 16 2019 APP
-rw-rw-rw- 1 ---- staff 2506 Dec 17 2013 CaratteristicheUM_temp.xml
-rw-rw-rw- 1 ---- staff 136 Dec 17 2013 FuoriRange.txt
-rw-rw---- 1 ---- sapsys 35074 Mar 13 2019 PM_GV2.CSV
-rw-rw---- 1 ---- sapsys 30047 Jul 19 23:00 PM_GV2.TXT
-rw-rw-rw- 1 ---- staff 435 Dec 28 2015 config.xml

This is the Rclone output:
rclone -vv --dump bodies ls -l ftpDemetra:SESTOSG
2022/07/20 14:08:35 DEBUG : rclone: Version "v1.59.0" starting with parameters ["rclone" "-vv" "--dump" "bodies" "ls" "-l" "ftpDemetra:SESTOSG"]
2022/07/20 14:08:35 DEBUG : Creating backend with remote "ftpDemetra:SESTOSG"
2022/07/20 14:08:35 DEBUG : Using config file from "/home/rob/.config/rclone/rclone.conf"
2022/07/20 14:08:35 DEBUG : ftp://----:21/SESTOSG: Connecting to FTP server
2022/07/20 14:08:35 DEBUG : FTP Rx: "220 --- FTP server (Version 4.2 Fri Apr 6 19:34:30 CDT 2018) ready."
2022/07/20 14:08:35 DEBUG : FTP Tx: "USER ---"
2022/07/20 14:08:35 DEBUG : FTP Rx: "331 Password required for ftppm."
2022/07/20 14:08:35 DEBUG : FTP Tx: PASS *****
2022/07/20 14:08:35 DEBUG : FTP Rx: "230-Last unsuccessful login: Wed Jul 20 14:06:04 2022 on ftp from ::ffff:10.200.40.50"
2022/07/20 14:08:35 DEBUG : FTP Rx: "230-Last login: Wed Jul 20 14:07:11 2022 on ftp from ::ffff:10.200.40.50"
2022/07/20 14:08:35 DEBUG : FTP Rx: "230 User ---- logged in."
2022/07/20 14:08:35 DEBUG : FTP Tx: "FEAT"
2022/07/20 14:08:35 DEBUG : FTP Rx: "500 'FEAT': command not understood."
2022/07/20 14:08:35 DEBUG : FTP Tx: "TYPE I"
2022/07/20 14:08:35 DEBUG : FTP Rx: "200 Type set to I."
2022/07/20 14:08:35 DEBUG : FTP Tx: "EPSV"
2022/07/20 14:08:36 DEBUG : FTP Rx: "229 Entering Extended Passive Mode (|||62755|)"
2022/07/20 14:08:36 DEBUG : FTP Tx: "LIST ."
2022/07/20 14:08:36 DEBUG : FTP Rx: "150 Opening data connection for /bin/ls."
2022/07/20 14:08:36 DEBUG : FTP Rx: "total 200\n-rwxr----- 1 ---- staff 255 Apr 4 2019 .profile\ndrwx------ 2 ---- staff 256 Apr 9 2019 .ssh\ndrwxrwxrwt 4 ---- sapsys 4096 May 18 2019 ALTOMONTE\ndrwxrwxrwt 4 ---- sapsys 4096 May 27 13:00 AZOTATI\ndrwxrwxrwt 4 ---- sapsys 4096 May 18 2019 BUSSI\ndrwxrwxrwt 4 ---- sapsys 4096 Mar 13 2019 CANDELA\ndrwxrwxr-x 2 ---- sapsys 36864 Jul 19 20:00 ESTRAZIONE_MAT\ndrwxrwxrwx 2 ---- staff 256 Sep 29 2021 GUIDE\ndrwxrwxrwt 4 ---- staff 256 Oct 10 2018 IDROELETTRICO\ndrwxrwxrwt 5 ---- sapsys 4096 May 18 2019 MARGHERALEVANTE\ndrwxrwxrwt 4 ---- sapsys 4096 Jul 20 13:27 SESTOSG\ndrwxrwxrwt 4 ---- sapsys 4096 Apr 10 2019 SIMERICRICHI\ndrwxrwxrwt 4 ---- sapsys 8192 Nov 19 2021 SIRACUSA\ndrwxrwxrwt 4 ---- sapsys 4096 Apr 10 2019 TERNI\ndrwxrwxrwt 4 ---- sapsys 4096 May 27 13:00 TORVISCOSA\ndrwxrwxrwt 3 ---- sapsys 8192 Jul 20 14:01 UPLOAD\ndrwxr-xr-x 2 root system 256 Nov 6 2013 lost+found\ndrwxrwxr-x 2 ---- staff 256 Apr 11 2019 test_1104\n"
2022/07/20 14:08:36 DEBUG : FTP Rx: "226 Transfer complete."
2022/07/20 14:08:36 DEBUG : FTP Tx: "EPSV"
2022/07/20 14:08:36 DEBUG : FTP Rx: "229 Entering Extended Passive Mode (|||62756|)"
2022/07/20 14:08:36 DEBUG : FTP Tx: "LIST SESTOSG"
2022/07/20 14:08:36 DEBUG : FTP Rx: "150 Opening data connection for /bin/ls."
2022/07/20 14:08:36 DEBUG : FTP Rx: "total 160\ndrwxrwxrwx 2 ---- staff 256 Mar 21 2019 ANAGRAFICA\ndrwxrwxrwx 2 ---- staff 256 Apr 16 2019 APP\n-rw-rw-rw- 1 ---- staff 2506 Dec 17 2013 CaratteristicheUM_temp.xml\n-rw-rw-rw- 1 ---- staff 136 Dec 17 2013 FuoriRange.txt\n-rw-rw---- 1 ---- sapsys 35074 Mar 13 2019 PM_GV2.CSV\n-rw-rw---- 1 ---- sapsys 30047 Jul 19 23:00 PM_GV2.TXT\n-rw-rw-rw- 1 ---- staff 435 Dec 28 2015 config.xml\n"
2022/07/20 14:08:36 DEBUG : FTP Rx: "226 Transfer complete."
2506 CaratteristicheUM_temp.xml
136 FuoriRange.txt
35074 PM_GV2.CSV
435 config.xml
30047 PM_GV2.TXT
2022/07/20 14:08:36 DEBUG : FTP Tx: "EPSV"
2022/07/20 14:08:36 DEBUG : ftp://----:21/SESTOSG: Connecting to FTP server
2022/07/20 14:08:36 DEBUG : FTP Rx: "229 Entering Extended Passive Mode (|||62757|)"
2022/07/20 14:08:36 DEBUG : FTP Tx: "LIST SESTOSG/ APP"
2022/07/20 14:08:36 DEBUG : FTP Rx: "220 ---- FTP server (Version 4.2 Fri Apr 6 19:34:30 CDT 2018) ready."
2022/07/20 14:08:36 DEBUG : FTP Tx: "USER ----"
2022/07/20 14:08:36 DEBUG : FTP Rx: "150 Opening data connection for /bin/ls."
2022/07/20 14:08:36 DEBUG : FTP Rx: "SESTOSG/:\ntotal 160\ndrwxrwxrwx 2 ---- staff 256 Mar 21 2019 ANAGRAFICA\ndrwxrwxrwx 2 ---- staff 256 Apr 16 2019 APP\n-rw-rw-rw- 1 ---- staff 2506 Dec 17 2013 CaratteristicheUM_temp.xml"
2022/07/20 14:08:36 DEBUG : FTP Rx: "\n-rw-rw-rw- 1 ---- staff 136 Dec 17 2013 FuoriRange.txt\n-rw-rw---- 1 ---- sapsys 35074 Mar 13 2019 PM_GV2.CSV\n-rw-rw---- 1 ---- sapsys 30047 Jul 19 23:00 PM_GV2.TXT\n-rw-rw-rw- 1 ---- staff 435 Dec 28 2015 config.xml\n"
2022/07/20 14:08:36 DEBUG : FTP Rx: "226 Transfer complete."
2506 APP/ CaratteristicheUM_temp.xml
136 APP/ FuoriRange.txt
35074 APP/ PM_GV2.CSV
435 APP/ config.xml
30047 APP/PM_GV2.TXT

I'm pretty sure that the issue comes from the date format, as you can see when the year is showed instead of a date a space will be added messing up the parsing.

Thanks!

Robert_Timis · July 20, 2022, 12:15pm

2022/07/20 14:08:36 DEBUG : ftp://----:21/SESTOSG: Connecting to FTP server
2022/07/20 14:08:36 DEBUG : FTP Tx: "EPSV"
2022/07/20 14:08:36 DEBUG : FTP Rx: "331 Password required for ----."
2022/07/20 14:08:36 DEBUG : FTP Tx: PASS *****
2022/07/20 14:08:36 DEBUG : FTP Rx: "229 Entering Extended Passive Mode (|||62758|)"
2022/07/20 14:08:36 DEBUG : FTP Rx: "230-Last unsuccessful login: Wed Jul 20 14:06:04 2022 on ftp from ::ffff:10.200.40.50"
2022/07/20 14:08:36 DEBUG : FTP Rx: "230-Last login: Wed Jul 20 14:08:29 2022 on ftp from ::ffff:10.200.40.50"
2022/07/20 14:08:36 DEBUG : FTP Rx: "230 User --- logged in."
2022/07/20 14:08:36 DEBUG : FTP Tx: "FEAT"
2022/07/20 14:08:36 DEBUG : FTP Tx: "LIST SESTOSG/ APP/ APP"

PS: down the line this happens, it starts recursing on non existant paths

Robert_Timis · July 20, 2022, 12:19pm

rclone lsf -R ftpDemetra:SESTOSG
ANAGRAFICA/
APP/
CaratteristicheUM_temp.xml
FuoriRange.txt
PM_GV2.CSV
config.xml
PM_GV2.TXT
APP/ ANAGRAFICA/
APP/ APP/
APP/ CaratteristicheUM_temp.xml
APP/ FuoriRange.txt
APP/ PM_GV2.CSV
APP/ config.xml
APP/PM_GV2.TXT
APP/ ANAGRAFICA/ ANAGRAFICA/
APP/ ANAGRAFICA/ APP/
APP/ ANAGRAFICA/ CaratteristicheUM_temp.xml
APP/ ANAGRAFICA/ FuoriRange.txt
APP/ ANAGRAFICA/ PM_GV2.CSV
APP/ ANAGRAFICA/ config.xml
APP/ ANAGRAFICA/PM_GV2.TXT
APP/ ANAGRAFICA/ APP/ ANAGRAFICA/
APP/ ANAGRAFICA/ APP/ APP/
APP/ ANAGRAFICA/ APP/ CaratteristicheUM_temp.xml
APP/ ANAGRAFICA/ APP/ FuoriRange.txt
APP/ ANAGRAFICA/ APP/ PM_GV2.CSV
APP/ ANAGRAFICA/ APP/ config.xml
APP/ ANAGRAFICA/ APP/PM_GV2.TXT
ANAGRAFICA/ ANAGRAFICA/
ANAGRAFICA/ APP/
ANAGRAFICA/ CaratteristicheUM_temp.xml
ANAGRAFICA/ FuoriRange.txt
ANAGRAFICA/ PM_GV2.CSV
ANAGRAFICA/ config.xml
ANAGRAFICA/PM_GV2.TXT
APP/ ANAGRAFICA/ APP/ APP/ ANAGRAFICA/
APP/ ANAGRAFICA/ APP/ APP/ APP/
APP/ ANAGRAFICA/ APP/ APP/ CaratteristicheUM_temp.xml
APP/ ANAGRAFICA/ APP/ APP/ FuoriRange.txt
APP/ ANAGRAFICA/ APP/ APP/ PM_GV2.CSV
APP/ ANAGRAFICA/ APP/ APP/ config.xml
APP/ ANAGRAFICA/ APP/ APP/PM_GV2.TXT
APP/ APP/ ANAGRAFICA/
APP/ APP/ APP/
APP/ APP/ CaratteristicheUM_temp.xml
APP/ APP/ FuoriRange.txt
APP/ APP/ PM_GV2.CSV
APP/ APP/ config.xml
APP/ APP/PM_GV2.TXT
ANAGRAFICA/ APP/ ANAGRAFICA/
ANAGRAFICA/ APP/ APP/
ANAGRAFICA/ APP/ CaratteristicheUM_temp.xml
ANAGRAFICA/ APP/ FuoriRange.txt
ANAGRAFICA/ APP/ PM_GV2.CSV
ANAGRAFICA/ APP/ config.xml
ANAGRAFICA/ APP/PM_GV2.TXT
APP/ ANAGRAFICA/ APP/ APP/ APP/ ANAGRAFICA/
APP/ ANAGRAFICA/ APP/ APP/ APP/ APP/
APP/ ANAGRAFICA/ APP/ APP/ APP/ CaratteristicheUM_temp.xml
APP/ ANAGRAFICA/ APP/ APP/ APP/ FuoriRange.txt
APP/ ANAGRAFICA/ APP/ APP/ APP/ PM_GV2.CSV
APP/ ANAGRAFICA/ APP/ APP/ APP/ config.xml
APP/ ANAGRAFICA/ APP/ APP/ APP/PM_GV2.TXT
APP/ ANAGRAFICA/ ANAGRAFICA/ ANAGRAFICA/
APP/ ANAGRAFICA/ ANAGRAFICA/ APP/
APP/ ANAGRAFICA/ ANAGRAFICA/ CaratteristicheUM_temp.xml
APP/ ANAGRAFICA/ ANAGRAFICA/ FuoriRange.txt
APP/ ANAGRAFICA/ ANAGRAFICA/ PM_GV2.CSV
APP/ ANAGRAFICA/ ANAGRAFICA/ config.xml
APP/ ANAGRAFICA/ ANAGRAFICA/PM_GV2.TXT
APP/ APP/ APP/ ANAGRAFICA/
APP/ APP/ APP/ APP/
APP/ APP/ APP/ CaratteristicheUM_temp.xml
APP/ APP/ APP/ FuoriRange.txt
APP/ APP/ APP/ PM_GV2.CSV
APP/ APP/ APP/ config.xml
APP/ APP/ APP/PM_GV2.TXT
ANAGRAFICA/ APP/ ANAGRAFICA/ ANAGRAFICA/
ANAGRAFICA/ APP/ ANAGRAFICA/ APP/
ANAGRAFICA/ APP/ ANAGRAFICA/ CaratteristicheUM_temp.xml
ANAGRAFICA/ APP/ ANAGRAFICA/ FuoriRange.txt
ANAGRAFICA/ APP/ ANAGRAFICA/ PM_GV2.CSV
ANAGRAFICA/ APP/ ANAGRAFICA/ config.xml
ANAGRAFICA/ APP/ ANAGRAFICA/PM_GV2.TXT
ANAGRAFICA/ APP/ APP/ ANAGRAFICA/
ANAGRAFICA/ APP/ APP/ APP/
ANAGRAFICA/ APP/ APP/ CaratteristicheUM_temp.xml
ANAGRAFICA/ APP/ APP/ FuoriRange.txt
ANAGRAFICA/ APP/ APP/ PM_GV2.CSV
ANAGRAFICA/ APP/ APP/ config.xml
ANAGRAFICA/ APP/ APP/PM_GV2.TXT
APP/ ANAGRAFICA/ APP/ APP/ APP/ ANAGRAFICA/ ANAGRAFICA/
APP/ ANAGRAFICA/ APP/ APP/ APP/ ANAGRAFICA/ APP/
APP/ ANAGRAFICA/ APP/ APP/ APP/ ANAGRAFICA/ CaratteristicheUM_temp.xml
APP/ ANAGRAFICA/ APP/ APP/ APP/ ANAGRAFICA/ FuoriRange.txt
APP/ ANAGRAFICA/ APP/ APP/ APP/ ANAGRAFICA/ PM_GV2.CSV
APP/ ANAGRAFICA/ APP/ APP/ APP/ ANAGRAFICA/ config.xml
APP/ ANAGRAFICA/ APP/ APP/ APP/ ANAGRAFICA/PM_GV2.TXT
APP/ ANAGRAFICA/ ANAGRAFICA/ ANAGRAFICA/ ANAGRAFICA/
APP/ ANAGRAFICA/ ANAGRAFICA/ ANAGRAFICA/ APP/
APP/ ANAGRAFICA/ ANAGRAFICA/ ANAGRAFICA/ CaratteristicheUM_temp.xml
APP/ ANAGRAFICA/ ANAGRAFICA/ ANAGRAFICA/ FuoriRange.txt
APP/ ANAGRAFICA/ ANAGRAFICA/ ANAGRAFICA/ PM_GV2.CSV
APP/ ANAGRAFICA/ ANAGRAFICA/ ANAGRAFICA/ config.xml
APP/ ANAGRAFICA/ ANAGRAFICA/ ANAGRAFICA/PM_GV2.TXT
APP/ ANAGRAFICA/ APP/ APP/ APP/ APP/ ANAGRAFICA/
APP/ ANAGRAFICA/ APP/ APP/ APP/ APP/ APP/
APP/ ANAGRAFICA/ APP/ APP/ APP/ APP/ CaratteristicheUM_temp.xml
APP/ ANAGRAFICA/ APP/ APP/ APP/ APP/ FuoriRange.txt
APP/ ANAGRAFICA/ APP/ APP/ APP/ APP/ PM_GV2.CSV
APP/ ANAGRAFICA/ APP/ APP/ APP/ APP/ config.xml
APP/ ANAGRAFICA/ APP/ APP/ APP/ APP/PM_GV2.TXT

It starts doing this

Animosity022 · July 20, 2022, 12:28pm

If you put backtics before and after a block, it will mark it as a code block as it's hard to understand/see what you are talking about because it's just formatting on the forums.

A backtick is this `

Put 3 before and 3 after

like
this

and paste in the text and we can see what you are talking about.

Robert_Timis · July 20, 2022, 12:30pm

[quote="Robert_Timis, post:1, topic:32006"]
This is the FTP output:

drwxrwxrwx 2 ---- staff 256 Mar 21 2019 ANAGRAFICA
drwxrwxrwx 2 ---- staff 256 Apr 16 2019 APP
-rw-rw-rw- 1 ---- staff 2506 Dec 17 2013 CaratteristicheUM_temp.xml
-rw-rw-rw- 1 ---- staff 136 Dec 17 2013 FuoriRange.txt
-rw-rw---- 1 ---- sapsys 35074 Mar 13 2019 PM_GV2.CSV
-rw-rw---- 1 ---- sapsys 30047 Jul 19 23:00 PM_GV2.TXT
-rw-rw-rw- 1 ---- staff 435 Dec 28 2015 config.xml

This is the Rclone output:

rclone -vv --dump bodies ls -l ftpDemetra:SESTOSG
2022/07/20 14:08:35 DEBUG : rclone: Version "v1.59.0" starting with parameters ["rclone" "-vv" "--dump" "bodies" "ls" "-l" "ftpDemetra:SESTOSG"]
2022/07/20 14:08:35 DEBUG : Creating backend with remote "ftpDemetra:SESTOSG"
2022/07/20 14:08:35 DEBUG : Using config file from "/home/rob/.config/rclone/rclone.conf"
2022/07/20 14:08:35 DEBUG : ftp://----:21/SESTOSG: Connecting to FTP server
2022/07/20 14:08:35 DEBUG : FTP Rx: "220 --- FTP server (Version 4.2 Fri Apr 6 19:34:30 CDT 2018) ready."
2022/07/20 14:08:35 DEBUG : FTP Tx: "USER ---"
2022/07/20 14:08:35 DEBUG : FTP Rx: "331 Password required for ftppm."
2022/07/20 14:08:35 DEBUG : FTP Tx: PASS *****
2022/07/20 14:08:35 DEBUG : FTP Rx: "230-Last unsuccessful login: Wed Jul 20 14:06:04 2022 on ftp from ::ffff:10.200.40.50"
2022/07/20 14:08:35 DEBUG : FTP Rx: "230-Last login: Wed Jul 20 14:07:11 2022 on ftp from ::ffff:10.200.40.50"
2022/07/20 14:08:35 DEBUG : FTP Rx: "230 User ---- logged in."
2022/07/20 14:08:35 DEBUG : FTP Tx: "FEAT"
2022/07/20 14:08:35 DEBUG : FTP Rx: "500 'FEAT': command not understood."
2022/07/20 14:08:35 DEBUG : FTP Tx: "TYPE I"
2022/07/20 14:08:35 DEBUG : FTP Rx: "200 Type set to I."
2022/07/20 14:08:35 DEBUG : FTP Tx: "EPSV"
2022/07/20 14:08:36 DEBUG : FTP Rx: "229 Entering Extended Passive Mode (|||62755|)"
2022/07/20 14:08:36 DEBUG : FTP Tx: "LIST ."
2022/07/20 14:08:36 DEBUG : FTP Rx: "150 Opening data connection for /bin/ls."
2022/07/20 14:08:36 DEBUG : FTP Rx: "total 200\n-rwxr----- 1 ---- staff 255 Apr 4 2019 .profile\ndrwx------ 2 ---- staff 256 Apr 9 2019 .ssh\ndrwxrwxrwt 4 ---- sapsys 4096 May 18 2019 ALTOMONTE\ndrwxrwxrwt 4 ---- sapsys 4096 May 27 13:00 AZOTATI\ndrwxrwxrwt 4 ---- sapsys 4096 May 18 2019 BUSSI\ndrwxrwxrwt 4 ---- sapsys 4096 Mar 13 2019 CANDELA\ndrwxrwxr-x 2 ---- sapsys 36864 Jul 19 20:00 ESTRAZIONE_MAT\ndrwxrwxrwx 2 ---- staff 256 Sep 29 2021 GUIDE\ndrwxrwxrwt 4 ---- staff 256 Oct 10 2018 IDROELETTRICO\ndrwxrwxrwt 5 ---- sapsys 4096 May 18 2019 MARGHERALEVANTE\ndrwxrwxrwt 4 ---- sapsys 4096 Jul 20 13:27 SESTOSG\ndrwxrwxrwt 4 ---- sapsys 4096 Apr 10 2019 SIMERICRICHI\ndrwxrwxrwt 4 ---- sapsys 8192 Nov 19 2021 SIRACUSA\ndrwxrwxrwt 4 ---- sapsys 4096 Apr 10 2019 TERNI\ndrwxrwxrwt 4 ---- sapsys 4096 May 27 13:00 TORVISCOSA\ndrwxrwxrwt 3 ---- sapsys 8192 Jul 20 14:01 UPLOAD\ndrwxr-xr-x 2 root system 256 Nov 6 2013 lost+found\ndrwxrwxr-x 2 ---- staff 256 Apr 11 2019 test_1104\n"
2022/07/20 14:08:36 DEBUG : FTP Rx: "226 Transfer complete."
2022/07/20 14:08:36 DEBUG : FTP Tx: "EPSV"
2022/07/20 14:08:36 DEBUG : FTP Rx: "229 Entering Extended Passive Mode (|||62756|)"
2022/07/20 14:08:36 DEBUG : FTP Tx: "LIST SESTOSG"
2022/07/20 14:08:36 DEBUG : FTP Rx: "150 Opening data connection for /bin/ls."
2022/07/20 14:08:36 DEBUG : FTP Rx: "total 160\ndrwxrwxrwx 2 ---- staff 256 Mar 21 2019 ANAGRAFICA\ndrwxrwxrwx 2 ---- staff 256 Apr 16 2019 APP\n-rw-rw-rw- 1 ---- staff 2506 Dec 17 2013 CaratteristicheUM_temp.xml\n-rw-rw-rw- 1 ---- staff 136 Dec 17 2013 FuoriRange.txt\n-rw-rw---- 1 ---- sapsys 35074 Mar 13 2019 PM_GV2.CSV\n-rw-rw---- 1 ---- sapsys 30047 Jul 19 23:00 PM_GV2.TXT\n-rw-rw-rw- 1 ---- staff 435 Dec 28 2015 config.xml\n"
2022/07/20 14:08:36 DEBUG : FTP Rx: "226 Transfer complete."
2506 CaratteristicheUM_temp.xml
136 FuoriRange.txt
35074 PM_GV2.CSV
435 config.xml
30047 PM_GV2.TXT
2022/07/20 14:08:36 DEBUG : FTP Tx: "EPSV"
2022/07/20 14:08:36 DEBUG : ftp://----:21/SESTOSG: Connecting to FTP server
2022/07/20 14:08:36 DEBUG : FTP Rx: "229 Entering Extended Passive Mode (|||62757|)"
2022/07/20 14:08:36 DEBUG : FTP Tx: "LIST SESTOSG/ APP"
2022/07/20 14:08:36 DEBUG : FTP Rx: "220 ---- FTP server (Version 4.2 Fri Apr 6 19:34:30 CDT 2018) ready."
2022/07/20 14:08:36 DEBUG : FTP Tx: "USER ----"
2022/07/20 14:08:36 DEBUG : FTP Rx: "150 Opening data connection for /bin/ls."
2022/07/20 14:08:36 DEBUG : FTP Rx: "SESTOSG/:\ntotal 160\ndrwxrwxrwx 2 ---- staff 256 Mar 21 2019 ANAGRAFICA\ndrwxrwxrwx 2 ---- staff 256 Apr 16 2019 APP\n-rw-rw-rw- 1 ---- staff 2506 Dec 17 2013 CaratteristicheUM_temp.xml"
2022/07/20 14:08:36 DEBUG : FTP Rx: "\n-rw-rw-rw- 1 ---- staff 136 Dec 17 2013 FuoriRange.txt\n-rw-rw---- 1 ---- sapsys 35074 Mar 13 2019 PM_GV2.CSV\n-rw-rw---- 1 ---- sapsys 30047 Jul 19 23:00 PM_GV2.TXT\n-rw-rw-rw- 1 ---- staff 435 Dec 28 2015 config.xml\n"
2022/07/20 14:08:36 DEBUG : FTP Rx: "226 Transfer complete."
2506 APP/ CaratteristicheUM_temp.xml
136 APP/ FuoriRange.txt
35074 APP/ PM_GV2.CSV
435 APP/ config.xml
30047 APP/PM_GV2.TXT

Down the line this happens:

rclone lsf -R ftpDemetra:SESTOSG
ANAGRAFICA/
APP/
CaratteristicheUM_temp.xml
FuoriRange.txt
PM_GV2.CSV
config.xml
PM_GV2.TXT
APP/ ANAGRAFICA/
APP/ APP/
APP/ CaratteristicheUM_temp.xml
APP/ FuoriRange.txt
APP/ PM_GV2.CSV
APP/ config.xml
APP/PM_GV2.TXT
APP/ ANAGRAFICA/ ANAGRAFICA/
APP/ ANAGRAFICA/ APP/
APP/ ANAGRAFICA/ CaratteristicheUM_temp.xml
APP/ ANAGRAFICA/ FuoriRange.txt
APP/ ANAGRAFICA/ PM_GV2.CSV
APP/ ANAGRAFICA/ config.xml
APP/ ANAGRAFICA/PM_GV2.TXT
APP/ ANAGRAFICA/ APP/ ANAGRAFICA/
APP/ ANAGRAFICA/ APP/ APP/
APP/ ANAGRAFICA/ APP/ CaratteristicheUM_temp.xml
APP/ ANAGRAFICA/ APP/ FuoriRange.txt
APP/ ANAGRAFICA/ APP/ PM_GV2.CSV
APP/ ANAGRAFICA/ APP/ config.xml
APP/ ANAGRAFICA/ APP/PM_GV2.TXT
ANAGRAFICA/ ANAGRAFICA/
ANAGRAFICA/ APP/
ANAGRAFICA/ CaratteristicheUM_temp.xml
ANAGRAFICA/ FuoriRange.txt
ANAGRAFICA/ PM_GV2.CSV
ANAGRAFICA/ config.xml
ANAGRAFICA/PM_GV2.TXT
APP/ ANAGRAFICA/ APP/ APP/ ANAGRAFICA/
APP/ ANAGRAFICA/ APP/ APP/ APP/
APP/ ANAGRAFICA/ APP/ APP/ CaratteristicheUM_temp.xml
APP/ ANAGRAFICA/ APP/ APP/ FuoriRange.txt
APP/ ANAGRAFICA/ APP/ APP/ PM_GV2.CSV
APP/ ANAGRAFICA/ APP/ APP/ config.xml
APP/ ANAGRAFICA/ APP/ APP/PM_GV2.TXT
APP/ APP/ ANAGRAFICA/
APP/ APP/ APP/
APP/ APP/ CaratteristicheUM_temp.xml
APP/ APP/ FuoriRange.txt
APP/ APP/ PM_GV2.CSV
APP/ APP/ config.xml
APP/ APP/PM_GV2.TXT
ANAGRAFICA/ APP/ ANAGRAFICA/
ANAGRAFICA/ APP/ APP/
ANAGRAFICA/ APP/ CaratteristicheUM_temp.xml
ANAGRAFICA/ APP/ FuoriRange.txt
ANAGRAFICA/ APP/ PM_GV2.CSV
ANAGRAFICA/ APP/ config.xml
ANAGRAFICA/ APP/PM_GV2.TXT
APP/ ANAGRAFICA/ APP/ APP/ APP/ ANAGRAFICA/
APP/ ANAGRAFICA/ APP/ APP/ APP/ APP/
APP/ ANAGRAFICA/ APP/ APP/ APP/ CaratteristicheUM_temp.xml
APP/ ANAGRAFICA/ APP/ APP/ APP/ FuoriRange.txt
APP/ ANAGRAFICA/ APP/ APP/ APP/ PM_GV2.CSV
APP/ ANAGRAFICA/ APP/ APP/ APP/ config.xml
APP/ ANAGRAFICA/ APP/ APP/ APP/PM_GV2.TXT
APP/ ANAGRAFICA/ ANAGRAFICA/ ANAGRAFICA/
APP/ ANAGRAFICA/ ANAGRAFICA/ APP/
APP/ ANAGRAFICA/ ANAGRAFICA/ CaratteristicheUM_temp.xml
APP/ ANAGRAFICA/ ANAGRAFICA/ FuoriRange.txt
APP/ ANAGRAFICA/ ANAGRAFICA/ PM_GV2.CSV
APP/ ANAGRAFICA/ ANAGRAFICA/ config.xml
APP/ ANAGRAFICA/ ANAGRAFICA/PM_GV2.TXT
APP/ APP/ APP/ ANAGRAFICA/
APP/ APP/ APP/ APP/
APP/ APP/ APP/ CaratteristicheUM_temp.xml
APP/ APP/ APP/ FuoriRange.txt
APP/ APP/ APP/ PM_GV2.CSV
APP/ APP/ APP/ config.xml
APP/ APP/ APP/PM_GV2.TXT
ANAGRAFICA/ APP/ ANAGRAFICA/ ANAGRAFICA/
ANAGRAFICA/ APP/ ANAGRAFICA/ APP/
ANAGRAFICA/ APP/ ANAGRAFICA/ CaratteristicheUM_temp.xml
ANAGRAFICA/ APP/ ANAGRAFICA/ FuoriRange.txt
ANAGRAFICA/ APP/ ANAGRAFICA/ PM_GV2.CSV
ANAGRAFICA/ APP/ ANAGRAFICA/ config.xml
ANAGRAFICA/ APP/ ANAGRAFICA/PM_GV2.TXT
ANAGRAFICA/ APP/ APP/ ANAGRAFICA/
ANAGRAFICA/ APP/ APP/ APP/
ANAGRAFICA/ APP/ APP/ CaratteristicheUM_temp.xml
ANAGRAFICA/ APP/ APP/ FuoriRange.txt
ANAGRAFICA/ APP/ APP/ PM_GV2.CSV
ANAGRAFICA/ APP/ APP/ config.xml
ANAGRAFICA/ APP/ APP/PM_GV2.TXT
APP/ ANAGRAFICA/ APP/ APP/ APP/ ANAGRAFICA/ ANAGRAFICA/
APP/ ANAGRAFICA/ APP/ APP/ APP/ ANAGRAFICA/ APP/
APP/ ANAGRAFICA/ APP/ APP/ APP/ ANAGRAFICA/ CaratteristicheUM_temp.xml
APP/ ANAGRAFICA/ APP/ APP/ APP/ ANAGRAFICA/ FuoriRange.txt
APP/ ANAGRAFICA/ APP/ APP/ APP/ ANAGRAFICA/ PM_GV2.CSV
APP/ ANAGRAFICA/ APP/ APP/ APP/ ANAGRAFICA/ config.xml
APP/ ANAGRAFICA/ APP/ APP/ APP/ ANAGRAFICA/PM_GV2.TXT
APP/ ANAGRAFICA/ ANAGRAFICA/ ANAGRAFICA/ ANAGRAFICA/
APP/ ANAGRAFICA/ ANAGRAFICA/ ANAGRAFICA/ APP/
APP/ ANAGRAFICA/ ANAGRAFICA/ ANAGRAFICA/ CaratteristicheUM_temp.xml
APP/ ANAGRAFICA/ ANAGRAFICA/ ANAGRAFICA/ FuoriRange.txt
APP/ ANAGRAFICA/ ANAGRAFICA/ ANAGRAFICA/ PM_GV2.CSV
APP/ ANAGRAFICA/ ANAGRAFICA/ ANAGRAFICA/ config.xml
APP/ ANAGRAFICA/ ANAGRAFICA/ ANAGRAFICA/PM_GV2.TXT
APP/ ANAGRAFICA/ APP/ APP/ APP/ APP/ ANAGRAFICA/
APP/ ANAGRAFICA/ APP/ APP/ APP/ APP/ APP/
APP/ ANAGRAFICA/ APP/ APP/ APP/ APP/ CaratteristicheUM_temp.xml
APP/ ANAGRAFICA/ APP/ APP/ APP/ APP/ FuoriRange.txt
APP/ ANAGRAFICA/ APP/ APP/ APP/ APP/ PM_GV2.CSV
APP/ ANAGRAFICA/ APP/ APP/ APP/ APP/ config.xml
APP/ ANAGRAFICA/ APP/ APP/ APP/ APP/PM_GV2.TXT

Sorry first time posting

Robert_Timis · July 20, 2022, 12:31pm

I want to use sync ultimately but it starts outputting a lot of non-existing paths from the wrong parsing.
If I had to guess it has something to do with the times that come in different.
eg: 2019 vs 19:00

Robert_Timis · July 20, 2022, 12:44pm

So i went and downloaded the source code and fixed the problem by adding this:

diff --git a/backend/ftp/ftp.go b/backend/ftp/ftp.go
index d92678f08..46393643e 100644
--- a/backend/ftp/ftp.go
+++ b/backend/ftp/ftp.go
@@ -736,7 +736,7 @@ func (f *Fs) List(ctx context.Context, dir string) (entries fs.DirEntries, err e
        for i := range files {
                object := files[i]
                f.entryToStandard(object)
-               newremote := path.Join(dir, object.Name)
+               newremote := path.Join(dir, strings.TrimSpace(object.Name))
                switch object.Type {
                case ftp.EntryTypeFolder:
                        if object.Name == "." || object.Name == ".." {

ncw · July 20, 2022, 9:24pm

What FTP server is this?

It doesn't support "MLSD" which has a more sensible output which is the problem...

The listings look normal though

total 200
-rwxr----- 1 ---- staff 255 Apr 4 2019 .profile
drwx------ 2 ---- staff 256 Apr 9 2019 .ssh
drwxrwxrwt 4 ---- sapsys 4096 May 18 2019 ALTOMONTE
drwxrwxrwt 4 ---- sapsys 4096 May 27 13:00 AZOTATI
drwxrwxrwt 4 ---- sapsys 4096 May 18 2019 BUSSI
drwxrwxrwt 4 ---- sapsys 4096 Mar 13 2019 CANDELA
drwxrwxr-x 2 ---- sapsys 36864 Jul 19 20:00 ESTRAZIONE_MAT
drwxrwxrwx 2 ---- staff 256 Sep 29 2021 GUIDE
drwxrwxrwt 4 ---- staff 256 Oct 10 2018 IDROELETTRICO
drwxrwxrwt 5 ---- sapsys 4096 May 18 2019 MARGHERALEVANTE
drwxrwxrwt 4 ---- sapsys 4096 Jul 20 13:27 SESTOSG
drwxrwxrwt 4 ---- sapsys 4096 Apr 10 2019 SIMERICRICHI
drwxrwxrwt 4 ---- sapsys 8192 Nov 19 2021 SIRACUSA
drwxrwxrwt 4 ---- sapsys 4096 Apr 10 2019 TERNI
drwxrwxrwt 4 ---- sapsys 4096 May 27 13:00 TORVISCOSA
drwxrwxrwt 3 ---- sapsys 8192 Jul 20 14:01 UPLOAD
drwxr-xr-x 2 root system 256 Nov 6 2013 lost+found
drwxrwxr-x 2 ---- staff 256 Apr 11 2019 test_1104

Can you try some previous versions of rclone to see if this is a regression please?

Interesting.

Doing this will cause files which start or end with a space not to work properly, so ideally we need to fix this in the library: GitHub - jlaffaye/ftp: FTP client package for Go

Robert_Timis · July 20, 2022, 11:02pm

Sadly I can't tell what kind of FTP it is... I don't know.
I also tried with 1.49.1-1 and the issue is still present.
At the moment I'm using the rclone with the quick fix counting on the fact that 99% I won't have files with trailing or leading spaces. I will try having a look at the FTP package maybe I can help.
Thanks

Robert_Timis · July 20, 2022, 11:18pm

Ill leave here a pastebin, because funny enough when I paste here the output of rclone from my cli the extra whitespace seems to disappear.... total 200\n-rwxr----- 1 ---- staff 255 Apr 4 2019 .profile\ndr - Pastebin.com

ncw · July 21, 2022, 8:35am

That seems reasonable. We could apply the workaround on a flag or if the server doesn't support MLIST.

If you want to have a go a fixing the library you'd add some failing test lines here

github.com

jlaffaye/ftp/blob/11536801d1ffdf2a5b7cab8edcb1f1a62ff8bad2/parse_test.go#L38


      
          	line   string
          	name   string
          	target string
          }
          
          
type unsupportedLine struct {
          	line string
          	err  error
          }
          
          
var listTests = []line{
          	// UNIX ls -l style
          	{"drwxr-xr-x    3 110      1002            3 Dec 02  2009 pub", "pub", 0, EntryTypeFolder, newTime(2009, time.December, 2)},
          	{"drwxr-xr-x    3 110      1002            3 Dec 02  2009 p u b", "p u b", 0, EntryTypeFolder, newTime(2009, time.December, 2)},
          	{"-rw-r--r--   1 marketwired marketwired    12016 Mar 16  2016 2016031611G087802-001.newsml", "2016031611G087802-001.newsml", 12016, EntryTypeFile, newTime(2016, time.March, 16)},
          
          
	{"-rwxr-xr-x    3 110      1002            1234567 Dec 02  2009 fileName", "fileName", 1234567, EntryTypeFile, newTime(2009, time.December, 2)},
          	{"lrwxrwxrwx   1 root     other          7 Jan 25 00:17 bin -> usr/bin", "bin", 0, EntryTypeLink, newTime(thisYear, time.January, 25, 0, 17)},
          
          
	// Another ls style
          	{"drwxr-xr-x               folder        0 Aug 15 05:49 !!!-Tipp des Haus!", "!!!-Tipp des Haus!", 0, EntryTypeFolder, newTime(thisYear, time.August, 15, 5, 49)},

Then fix the parser without breaking the other ones!

system · August 20, 2022, 8:36am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.