Double folder names on target destination paths ftp

EHCanadian83 · August 30, 2020, 11:23pm

Error:
ERROR : /Disk01/FamilyPictures/FamilyPictures: error reading source directory: 501 /Disk01/FamilyPictures/FamilyPictures: Not a directory.

Command:
rclone sync "$varremote:/$var_tmp_b/" "$varroot/$var_tmp_b/" --progress --transfers 1 --bwlimit 10M -vv -n
[root@readynas424 /]# rclone sync archives:/Disk01/ /serversbackups/Disk01/ --progress --transfers 1 --bwlimit 10M -vv -n

Connection Type:
FTP

Folder Structure:
Using symbolic links so, /serversbackups/D01/Disk01 -> turns into /serversbackups/Disk01

Version:
rclone v1.52.3

os/arch: linux/amd64
go version: go1.14.7

Basically rclone is doubling up on the source/destination directory folder paths (FamilyPictures/FamilyPictures/). Resulting in a 501 error.
I would like it to just sync local: /serversbackups/Disk01/ with /Disk01 remote (FTP) however I've likely missed something in terms of a switch.

$varremote is varremote="archives"
$varroot is varroot="/serversbackups"

$var_tmp_a is var_tmp_a="var"$i"dir";
$var_tmp_b is var_tmp_b="${!var_tmp_a}";

I just use the .sh bits to loop though the Disk variables and not have to make x changes.

Anyone got any ideas?

ncw · August 31, 2020, 2:02pm

I suspect it is your $variables that is the problem here rather than rclone but I'm not 100% sure. Can you make an example showing the problem without the $variables?

Thanks

EHCanadian83 · September 1, 2020, 10:34am

Confirmed it wasn't the $variables. Not sure why rclone is repeating the directories twice.

Ran the command

rclone copy archives:/Disk01 /serversbackups/D01/Disk01 -vvvv --dry-run -L

[root@readynas424 ~]# rclone copy archives:/Disk01 /serversbackups/D01/Disk01 -vvvvv --dry-run -L
2020/09/01 06:30:53 DEBUG : rclone: Version "v1.52.3" starting with parameters ["rclone" "copy" "archives:/Disk01" "/serversbackups/D01/Disk01" "-vvvvv" "--dry-run" "-L"]
2020/09/01 06:30:53 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2020/09/01 06:30:53 DEBUG : ftp://127.0.0.1:21/Disk01: Connecting to FTP server
2020/09/01 06:30:54 DEBUG : ftp://127.0.0.1:21/Disk01: Connecting to FTP server
2020/09/01 06:30:54 DEBUG : ftp://127.0.0.1:21/Disk01: Connecting to FTP server
2020/09/01 06:30:54 DEBUG : ftp://127.0.0.1:21/Disk01: Connecting to FTP server
2020/09/01 06:30:54 DEBUG : ftp://127.0.0.1:21/Disk01: Connecting to FTP server
2020/09/01 06:30:54 DEBUG : ftp://127.0.0.1:21/Disk01: Connecting to FTP server
2020/09/01 06:30:54 DEBUG : ftp://127.0.0.1:21/Disk01: Connecting to FTP server
2020/09/01 06:30:54 DEBUG : ftp://127.0.0.1:21/Disk01: Connecting to FTP server
2020/09/01 06:30:54 ERROR : Family.Pictures/Family.Pictures: error reading source directory: 501 /Disk01/Family.Pictures/Family.Pictures: Not a directory.

ncw · September 1, 2020, 5:04pm

That is much easier for me to understand - thanks!

What happens if you do

rclone lsf archives:/Disk01

And also

rclone lsf archives:/Disk01/Family.Pictures

EHCanadian83 · September 2, 2020, 1:34pm

／Disk01/
[root@readynas424 ~]# rclone lsf archives:/Disk01 -vvvv

yum.update/
Blah.Blah/
／Disk01/
2020/09/02 09:29:44 DEBUG : 2 go routines active

[root@readynas424 ~]# rclone lsf archives:/Disk01/Family.Pictures -vvvv
2020/09/02 09:39:58 DEBUG : rclone: Version "v1.52.3" starting with parameters ["rclone" "lsf" "archives:/Disk01/Family.Pictures" "-vvvv"]
2020/09/02 09:39:58 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2020/09/02 09:39:58 DEBUG : ftp://127.0.0.1:21/Disk01/Family.Pictures: Connecting to FTP server
Family.Pictures/
2020/09/02 09:39:59 DEBUG : 2 go routines active

EHCanadian83 · September 2, 2020, 2:45pm

I did more digging.

I ran another modern ftp server daemon and it executed as it should.
I tried the problem ftp server daemon. Aged. And looked at its log between rclone.

[5] Wed 02Sep20 10:40:05 - (000029) Connected to 127.0.0.1 (Local address 10.0.0.2)
[6] Wed 02Sep20 10:40:05 - (000029) 220 ---------- Ready
[2] Wed 02Sep20 10:40:05 - (000029) FEAT
[6] Wed 02Sep20 10:40:05 - (000029) 211-Extension supported
[6] Wed 02Sep20 10:40:05 - (000029)  CLNT
[6] Wed 02Sep20 10:40:05 - (000029)  MDTM
[6] Wed 02Sep20 10:40:05 - (000029)  MDTM YYYYMMDDHHMMSS[+-TZ];filename
[6] Wed 02Sep20 10:40:05 - (000029)  SIZE
[6] Wed 02Sep20 10:40:05 - (000029)  SITE PSWD;EXEC;SET;INDEX;ZONE;CHMOD;MSG
[6] Wed 02Sep20 10:40:05 - (000029)  REST STREAM
[6] Wed 02Sep20 10:40:05 - (000029)  XCRC filename;start;end
[6] Wed 02Sep20 10:40:05 - (000029)  MODE Z
[6] Wed 02Sep20 10:40:05 - (000029)  MLST Type*;Size*;Create;Modify*;Win32.ea*;
[6] Wed 02Sep20 10:40:05 - (000029) 211 End
[2] Wed 02Sep20 10:40:05 - (000029) USER --
[6] Wed 02Sep20 10:40:05 - (000029) 331 User name okay, need password.
[2] Wed 02Sep20 10:40:05 - (000029) PASS --
[5] Wed 02Sep20 10:40:05 - (000029) User -- logged in
[6] Wed 02Sep20 10:40:05 - (000029) 230 User logged in, proceed.
[2] Wed 02Sep20 10:40:05 - (000029) TYPE I
[6] Wed 02Sep20 10:40:05 - (000029) 200 Type set to I.
[2] Wed 02Sep20 10:40:05 - (000029) EPSV
[6] Wed 02Sep20 10:40:05 - (000029) 500 'EPSV': command not understood.
[2] Wed 02Sep20 10:40:05 - (000029) PASV
[6] Wed 02Sep20 10:40:05 - (000029) 227 Entering Passive Mode (--,--,--)
[2] Wed 02Sep20 10:40:05 - (000029) MLSD /
[6] Wed 02Sep20 10:40:05 - (000029) 150 Opening BINARY mode data connection for MLSD.
[6] Wed 02Sep20 10:40:05 - (000029) 226 Transfer complete.
[2] Wed 02Sep20 10:40:06 - (000029) PASV
[6] Wed 02Sep20 10:40:06 - (000029) 227 Entering Passive Mode (--,--,--)
[2] Wed 02Sep20 10:40:06 - (000029) MLSD /Disk01
[6] Wed 02Sep20 10:40:06 - (000029) 150 Opening BINARY mode data connection for MLSD.
[6] Wed 02Sep20 10:40:06 - (000029) 226 Transfer complete.
[2] Wed 02Sep20 10:40:06 - (000029) PASV
[6] Wed 02Sep20 10:40:06 - (000029) 227 Entering Passive Mode (--,--,--)
[2] Wed 02Sep20 10:40:06 - (000029) MLSD /Disk01/Family.Pictures
[6] Wed 02Sep20 10:40:06 - (000029) 150 Opening BINARY mode data connection for MLSD.
[6] Wed 02Sep20 10:40:06 - (000029) 226 Transfer complete.
[2] Wed 02Sep20 10:40:06 - (000029) PASV
[6] Wed 02Sep20 10:40:06 - (000029) 227 Entering Passive Mode (--,--,--)
[2] Wed 02Sep20 10:40:06 - (000029) MLSD /Disk01/Family.Pictures/Family.Pictures
[6] Wed 02Sep20 10:40:06 - (000029) 501 /Disk01/Family.Pictures/Family.Pictures: Not a directory.
[

ncw · September 2, 2020, 5:11pm

v1.53 has just been released and that has a potential fix for this - can you give that a go?

EHCanadian83 · September 2, 2020, 6:16pm

Update done.
No change in behavior. Still doing it.

[root@readynas424 ~]# rclone copy archives:/Disk01/Family.Pictures/ /serversbackups/Disk01/Family.Pictures -vvvv --dry-run -L --transfers 1 --checkers 1
2020/09/02 14:10:40 DEBUG : rclone: Version "v1.53.0" starting with parameters ["rclone" "copy" "archives:/Disk01/Family.Pictures/" "/serversbackups/Disk01/Family.Pictures" "-vvvv" "--dry-run" "-L" "--transfers" "1" "--checkers" "1"]
2020/09/02 14:10:40 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2020/09/02 14:10:40 DEBUG : Creating backend with remote "archives:/Disk01/Family.Pictures/"
2020/09/02 14:10:40 DEBUG : ftp://127.0.0.1:21/Disk01/Family.Pictures: Connecting to FTP server
2020/09/02 14:10:40 DEBUG : Creating backend with remote "/serversbackups/Disk01/Family.Pictures"
2020/09/02 14:10:40 ERROR : Family.Pictures: error reading source directory: 501 /Disk01/Family.Pictures/Family.Pictures: Not a directory.
2020/09/02 14:10:40 DEBUG : Local file system at /serversbackups/Disk01/Family.Pictures: Waiting for checks to finish
2020/09/02 14:10:40 DEBUG : Local file system at /serversbackups/Disk01/Family.Pictures: Waiting for transfers to finish
2020/09/02 14:10:40 INFO  : There was nothing to transfer
2020/09/02 14:10:40 ERROR : Attempt 1/3 failed with 1 errors and: 501 /Disk01/Family.Pictures/Family.Pictures: Not a directory.
2020/09/02 14:10:40 ERROR : Family.Pictures: error reading source directory: 501 /Disk01/Family.Pictures/Family.Pictures: Not a directory.
2020/09/02 14:10:40 DEBUG : Local file system at /serversbackups/Disk01/Family.Pictures: Waiting for checks to finish
2020/09/02 14:10:40 DEBUG : Local file system at /serversbackups/Disk01/Family.Pictures: Waiting for transfers to finish
2020/09/02 14:10:40 INFO  : There was nothing to transfer
2020/09/02 14:10:40 ERROR : Attempt 2/3 failed with 1 errors and: 501 /Disk01/Family.Pictures/Family.Pictures: Not a directory.
2020/09/02 14:10:41 ERROR : Family.Pictures: error reading source directory: 501 /Disk01/Family.Pictures/Family.Pictures: Not a directory.
2020/09/02 14:10:41 DEBUG : Local file system at /serversbackups/Disk01/Family.Pictures: Waiting for checks to finish
2020/09/02 14:10:41 DEBUG : Local file system at /serversbackups/Disk01/Family.Pictures: Waiting for transfers to finish
2020/09/02 14:10:41 INFO  : There was nothing to transfer
2020/09/02 14:10:41 ERROR : Attempt 3/3 failed with 1 errors and: 501 /Disk01/Family.Pictures/Family.Pictures: Not a directory.
2020/09/02 14:10:41 INFO  :
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.7s

2020/09/02 14:10:41 DEBUG : 3 go routines active
2020/09/02 14:10:41 Failed to copy: 501 /Disk01/Family.Pictures/Family.Pictures: Not a directory.

ncw · September 2, 2020, 6:56pm

Can you try this with --retries 1 --dump bodies too?

EHCanadian83 · September 2, 2020, 7:06pm

[root@readynas424 ~]# rclone copy archives:/Disk01/Family.Pictures /serversbackups/Disk01/Family.Pictures --transfers 1 --checkers 1 --retries 1 --dump bodies -L -vvvv --dry-run
2020/09/02 15:07:37 DEBUG : rclone: Version "v1.53.0" starting with parameters ["rclone" "copy" "archives:/Disk01/Family.Pictures" "/serversbackups/Disk01/Family.Pictures" "--transfers" "1" "--checkers" "1" "--retries" "1" "--dump" "bodies" "-L" "-vvvv" "--dry-run"]
2020/09/02 15:07:37 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2020/09/02 15:07:37 DEBUG : Creating backend with remote "archives:/Disk01/Family.Pictures"
2020/09/02 15:07:37 DEBUG : ftp://127.0.0.2:21/Disk01/Family.Pictures: Connecting to FTP server
2020/09/02 15:07:37 DEBUG : FTP Rx: "220 -- Ready"
2020/09/02 15:07:37 DEBUG : FTP Tx: "FEAT"
2020/09/02 15:07:37 DEBUG : FTP Rx: "211-Extension supported"
2020/09/02 15:07:37 DEBUG : FTP Rx: " CLNT"
2020/09/02 15:07:37 DEBUG : FTP Rx: " MDTM"
2020/09/02 15:07:37 DEBUG : FTP Rx: " MDTM YYYYMMDDHHMMSS[+-TZ];filename"
2020/09/02 15:07:37 DEBUG : FTP Rx: " SIZE"
2020/09/02 15:07:37 DEBUG : FTP Rx: " SITE PSWD;EXEC;SET;INDEX;ZONE;CHMOD;MSG"
2020/09/02 15:07:37 DEBUG : FTP Rx: " REST STREAM"
2020/09/02 15:07:37 DEBUG : FTP Rx: " XCRC filename;start;end"
2020/09/02 15:07:37 DEBUG : FTP Rx: " MODE Z"
2020/09/02 15:07:37 DEBUG : FTP Rx: " MLST Type*;Size*;Create;Modify*;Win32.ea*;"
2020/09/02 15:07:37 DEBUG : FTP Rx: "211 End"
2020/09/02 15:07:37 DEBUG : FTP Tx: "USER --"
2020/09/02 15:07:37 DEBUG : FTP Rx: "331 User name okay, need password."
2020/09/02 15:07:37 DEBUG : FTP Tx: PASS --
2020/09/02 15:07:37 DEBUG : FTP Rx: "230 User logged in, proceed."
2020/09/02 15:07:37 DEBUG : FTP Tx: "TYPE I"
2020/09/02 15:07:37 DEBUG : FTP Rx: "200 Type set to I."
2020/09/02 15:07:37 DEBUG : FTP Tx: "EPSV"
2020/09/02 15:07:37 DEBUG : FTP Rx: "500 'EPSV': command not understood."
2020/09/02 15:07:37 DEBUG : FTP Tx: "PASV"
2020/09/02 15:07:37 DEBUG : FTP Rx: "227 Entering Passive Mode (--,--,--,--,198,209)"
2020/09/02 15:07:37 DEBUG : FTP Tx: "MLSD /Latest"
2020/09/02 15:07:37 DEBUG : FTP Rx: "150 Opening BINARY mode data connection for MLSD."
2020/09/02 15:07:37 DEBUG : FTP Rx: "226 Transfer complete."
2020/09/02 15:07:37 DEBUG : Creating backend with remote "/serversbackups/Disk01/Family.Pictures"
2020/09/02 15:07:37 DEBUG : FTP Tx: "PASV"
2020/09/02 15:07:37 DEBUG : FTP Rx: "227 Entering Passive Mode (--,--,--,--,198,210)"
2020/09/02 15:07:37 DEBUG : FTP Tx: "MLSD /Disk01/Family.Pictures"
2020/09/02 15:07:38 DEBUG : FTP Rx: "150 Opening BINARY mode data connection for MLSD."
2020/09/02 15:07:38 DEBUG : FTP Rx: "226 Transfer complete."
2020/09/02 15:07:38 DEBUG : FTP Tx: "PASV"
2020/09/02 15:07:38 DEBUG : FTP Rx: "227 Entering Passive Mode (--,--,--,--,198,211)"
2020/09/02 15:07:38 DEBUG : FTP Tx: "MLSD /Disk01/Family.Pictures/Family.Pictures"
2020/09/02 15:07:38 DEBUG : FTP Rx: "501 /Disk01/Family.Pictures/Family.Pictures: Not a directory."
2020/09/02 15:07:38 ERROR : Family.Pictures: error reading source directory: 501 /Disk01/Family.Pictures/Family.Pictures: Not a directory.
2020/09/02 15:07:38 DEBUG : Local file system at /serversbackups/Disk01/Family.Pictures: Waiting for checks to finish
2020/09/02 15:07:38 DEBUG : Local file system at /serversbackups/Disk01/Family.Pictures: Waiting for transfers to finish
2020/09/02 15:07:38 INFO  : There was nothing to transfer
2020/09/02 15:07:38 ERROR : Attempt 1/1 failed with 1 errors and: 501 /Disk01/Family.Pictures/Family.Pictures: Not a directory.
2020/09/02 15:07:38 INFO  :
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.4s

2020/09/02 15:07:38 DEBUG : 3 go routines active
2020/09/02 15:07:38 Failed to copy: 501 /Disk01/Family.Pictures/Family.Pictures: Not a directory.

ncw · September 2, 2020, 7:53pm

Hmm...

When you did this

There is a unicode ／ at the start of ／Disk01/ which kind of indicates that the server returned the name with a / on the front of it. This is probably what is confusing rclone as that is non standard.

What server is this?

Is there any chance I could get as access to it temporarily?

EHCanadian83 · September 3, 2020, 12:23am

Ya, Its a older server daemon. I've been slowing migrating over to a modern daemon.
Shoot me a private message and I'll set you up with access.

system · November 2, 2020, 8:23pm

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.