What is the problem you are having with rclone?
I am trying to copy files to a ProFTPd server with TLS, and I'm getting "* EOF" errors for all files. After the command is run, I end up with mostly empty directories (some contain a few files of zero size) on the server.
I cannot test without TLS, as the server is configured to reject plain connections.
(A side question, since with every FTP server change, I have huge problems making rclone work. Would it be possible to somehow use FileZilla or WinSCP as a proxy/frontend by rclone? They seem to always work fine.)
Run the command 'rclone version' and share the full output of the command.
rclone v1.62.2
- os/version: Microsoft Windows 10 Home 22H2 (64 bit)
- os/kernel: 10.0.19045.2965 Build 19045.2965.2965 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.20.2
- go/linking: static
- go/tags: cmount
I also tried the latest beta version with the same result:
rclone v1.63.0-beta.7054.51a468b2b
- os/version: Microsoft Windows 10 Home 22H2 (64 bit)
- os/kernel: 10.0.19045.2965 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.20.4
- go/linking: static
- go/tags: cmount
Which cloud storage system are you using? (eg Google Drive)
FTP (ProFTPd)
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
I'm starting it from a PowerShell script, so I can't give the exact parameter values, but you will be able to see them in the debug output below.
$cmd = "$rclone_exe copy ""$($sourceset[0])"" ""${target_conf}:$($subdirset[0])/current"" --backup-dir=""${target_conf}:$($subdirset[0])/old_$dt"" --config=$config_file --password-command=""$password_cmd"" --filter-from=$filter_file --ftp-concurrency=1 --transfers=1 --bwlimit=${thrup}:${thrdown} --retries-sleep=10s --retries=1 $debug_str 2>&1 | % ToString | Tee-Object -FilePath D:\BackupScript\rclone\Last-run.txt"
Invoke-Expression $cmd
The rclone config contents with secrets removed.
[enklawa]
type = ftp
host = (removed)
user = kazik1
port = (removed)
pass = (removed)
explicit_tls = true
disable_mfmt = false
disable_tls13 = true
encoding = Slash,Asterisk,Ctl,Dot
concurrency = 3
tls_cache_size = 64
[enklawa_crypt]
type = crypt
remote = enklawa:Lapasus
directory_name_encryption = false
password = (removed)
A log from the command with the -vv
flag
The full log is very log, so I will just snip the relevant parts
2023/06/03 10:33:13 INFO : Starting bandwidth limiter at 5Mi:50Mi Byte/s
2023/06/03 10:33:13 DEBUG : rclone: Version "v1.63.0-beta.7054.51a468b2b" starting with parameters ["C:\\pprog\\Dysk\\rclone_beta\\rclone.exe" "copy" "D:\\" "enklawa_crypt:/d_moje/current" "--backup-dir=enklawa_crypt:/d_moje/old_2023-06-03_10-33" "--config=D:\\BackupScript\\rclone\\rclone.modtime.config" "--password-command=powershell -NoProfile -File D:\\BackupScript\\rclone\\rclone_pass.ps1" "--filter-from=D:\\BackupScript\\rclone\\rclone_filter.txt" "--ftp-concurrency=1" "--transfers=1" "--bwlimit=5M:50M" "--retries-sleep=10s" "--retries=1" "-vv"]
2023/06/03 10:33:13 DEBUG : Creating backend with remote "D:\\"
2023/06/03 10:33:13 DEBUG : Using config file from "D:\\BackupScript\\rclone\\rclone.modtime.config"
2023/06/03 10:33:13 DEBUG : fs cache: renaming cache item "D:\\" to be canonical "//?/D:/"
2023/06/03 10:33:13 DEBUG : Creating backend with remote "enklawa_crypt:/d_moje/current"
2023/06/03 10:33:13 DEBUG : Creating backend with remote "enklawa:Lapasus/d_moje/0rqboq96f6166id3mtqusk8qdk"
2023/06/03 10:33:13 DEBUG : enklawa: detected overridden config - adding "{klVtY}" suffix to name
2023/06/03 10:33:13 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/0rqboq96f6166id3mtqusk8qdk: Connecting to FTP server
2023/06/03 10:33:13 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/0rqboq96f6166id3mtqusk8qdk: dial("tcp","(removed):(removed)")
2023/06/03 10:33:13 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/0rqboq96f6166id3mtqusk8qdk: > dial: conn=*fshttp.timeoutConn, err=<nil>
2023/06/03 10:33:14 DEBUG : fs cache: renaming cache item "enklawa:Lapasus/d_moje/0rqboq96f6166id3mtqusk8qdk" to be canonical "enklawa,concurrency='1':Lapasus/d_moje/0rqboq96f6166id3mtqusk8qdk"
2023/06/03 10:33:14 DEBUG : Creating backend with remote "enklawa:Lapasus/d_moje/current"
2023/06/03 10:33:14 DEBUG : enklawa: detected overridden config - adding "{klVtY}" suffix to name
2023/06/03 10:33:14 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/current: Connecting to FTP server
2023/06/03 10:33:14 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/current: dial("tcp","(removed):(removed)")
2023/06/03 10:33:14 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/current: > dial: conn=*fshttp.timeoutConn, err=<nil>
2023/06/03 10:33:15 DEBUG : fs cache: renaming cache item "enklawa:Lapasus/d_moje/current" to be canonical "enklawa,concurrency='1':Lapasus/d_moje/current"
2023/06/03 10:33:15 DEBUG : Creating backend with remote "enklawa_crypt:/d_moje/old_2023-06-03_10-33"
2023/06/03 10:33:15 DEBUG : Creating backend with remote "enklawa:Lapasus/d_moje/bdqb6ed28isgrv84cstjgj5m5qq7vvsnsa6icrq6rhs60tn685jg"
2023/06/03 10:33:15 DEBUG : enklawa: detected overridden config - adding "{klVtY}" suffix to name
2023/06/03 10:33:15 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/bdqb6ed28isgrv84cstjgj5m5qq7vvsnsa6icrq6rhs60tn685jg: Connecting to FTP server
2023/06/03 10:33:15 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/bdqb6ed28isgrv84cstjgj5m5qq7vvsnsa6icrq6rhs60tn685jg: dial("tcp","(removed):(removed)")
2023/06/03 10:33:15 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/bdqb6ed28isgrv84cstjgj5m5qq7vvsnsa6icrq6rhs60tn685jg: > dial: conn=*fshttp.timeoutConn, err=<nil>
2023/06/03 10:33:15 DEBUG : fs cache: renaming cache item "enklawa:Lapasus/d_moje/bdqb6ed28isgrv84cstjgj5m5qq7vvsnsa6icrq6rhs60tn685jg" to be canonical "enklawa,concurrency='1':Lapasus/d_moje/bdqb6ed28isgrv84cstjgj5m5qq7vvsnsa6icrq6rhs60tn685jg"
2023/06/03 10:33:15 DEBUG : Creating backend with remote "enklawa:Lapasus/d_moje/old_2023-06-03_10-33"
2023/06/03 10:33:15 DEBUG : enklawa: detected overridden config - adding "{klVtY}" suffix to name
2023/06/03 10:33:15 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/old_2023-06-03_10-33: Connecting to FTP server
2023/06/03 10:33:15 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/old_2023-06-03_10-33: dial("tcp","(removed):(removed)")
2023/06/03 10:33:15 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/old_2023-06-03_10-33: > dial: conn=*fshttp.timeoutConn, err=<nil>
2023/06/03 10:33:16 DEBUG : fs cache: renaming cache item "enklawa:Lapasus/d_moje/old_2023-06-03_10-33" to be canonical "enklawa,concurrency='1':Lapasus/d_moje/old_2023-06-03_10-33"
2023/06/03 10:33:16 DEBUG : $RECYCLE.BIN: Excluded
2023/06/03 10:33:16 DEBUG : System Volume Information: Excluded
2023/06/03 10:33:16 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/current: dial("tcp","(removed):6696")
2023/06/03 10:33:16 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/current: > dial: conn=*tls.Conn, err=<nil>
2023/06/03 10:33:16 DEBUG : test2.txt: Need to transfer - File not found at Destination
2023/06/03 10:33:16 DEBUG : za┼╝├│┼é─ç g─Ö┼øl─à ja┼║┼ä__.txt: Need to transfer - File not found at Destination
2023/06/03 10:33:16 DEBUG : Schematy/DESCRIPTION: Need to transfer - File not found at Destination
2023/06/03 10:33:16 DEBUG : Schematy/Thumbs.db: Need to transfer - File not found at Destination
2023/06/03 10:33:16 DEBUG : Projekty-Stare/Etykiety.svg: Need to transfer - File not found at Destination
2023/06/03 10:33:16 DEBUG : EA/CubeIDEWorkspaces: Excluded
2023/06/03 10:33:16 DEBUG : Stronka/Header-location musi mie─ç pe┼én─à ┼øcie┼╝k─Ö.txt: Need to transfer - File not found at Destination
(...)
2023/06/03 10:33:20 DEBUG : Projekty/TrueStudioWorkspaces/SDLogger/.metadata/.plugins/org.eclipse.ui.intro: Excluded
2023/06/03 10:33:20 DEBUG : Projekty/TrueStudioWorkspaces/SDLogger/.metadata/.plugins/org.eclipse.ui.workbench: Excluded
2023/06/03 10:33:20 DEBUG : Projekty/TrueStudioWorkspaces/SDLogger/.metadata/.plugins/org.eclipse.ui.workbench.texteditor: Excluded
2023/06/03 10:33:20 DEBUG : Projekty/MartinoEscapeRoom/Czujniki/soft/IRSen_1/Debug/IRSen_1.elf: Need to transfer - File not found at Destination
2023/06/03 10:33:21 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/current: Connecting to FTP server
2023/06/03 10:33:21 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/current: dial("tcp","(removed):(removed)")
2023/06/03 10:33:21 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/current: > dial: conn=*fshttp.timeoutConn, err=<nil>
2023/06/03 10:33:22 DEBUG : Schematy/2nkp28gptuj02v5tg96ascdhhk: Removed after failed upload: 2 errors occurred:
* EOF
* EOF
System.Management.Automation.RemoteException
2023/06/03 10:33:22 ERROR : Schematy/DESCRIPTION: Failed to copy: update stor: 2 errors occurred:
* EOF
* EOF
System.Management.Automation.RemoteException
2023/06/03 10:33:22 DEBUG : EA/Projekty/GS_GM_switch/FS203SP_2sfp_3e_poe/v1/FS203SP_1.PRJPCBVariants: Need to transfer - File not found at Destination
2023/06/03 10:33:22 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/current: dial("tcp","(removed):6698")
2023/06/03 10:33:22 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/current: > dial: conn=*tls.Conn, err=<nil>
2023/06/03 10:33:23 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/current: Connecting to FTP server
2023/06/03 10:33:23 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/current: dial("tcp","(removed):(removed)")
2023/06/03 10:33:23 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/current: > dial: conn=*fshttp.timeoutConn, err=<nil>
2023/06/03 10:33:23 DEBUG : Schematy/lnpio3aomg34422r02hhkjqa4g: Removed after failed upload: 2 errors occurred:
* EOF
* EOF
System.Management.Automation.RemoteException
2023/06/03 10:33:23 ERROR : Schematy/Thumbs.db: Failed to copy: update stor: 2 errors occurred:
* EOF
* EOF
System.Management.Automation.RemoteException
A log from the command with the -vv
and --dump bodies
flags
Only snipped the output around one of the errors. I can provide startup output too, or the full log, if necessary.
2023/06/03 10:55:12 DEBUG : Projekty-Stare/AVRBreakout/aukcje/LD1117AV33.txt: Need to transfer - File not found at Destination
2023/06/03 10:55:12 DEBUG : FTP Tx: "MLST Lapasus/d_moje/current/Instrukcje"
2023/06/03 10:55:12 DEBUG : FTP Rx: "550 'Lapasus/d_moje/current/Instrukcje' cannot be listed"
2023/06/03 10:55:12 DEBUG : FTP Tx: "NOOP"
2023/06/03 10:55:12 DEBUG : FTP Rx: "200 NOOP command successful"
2023/06/03 10:55:12 DEBUG : FTP Tx: "MLST Lapasus/d_moje/current"
2023/06/03 10:55:12 DEBUG : FTP Rx: "250-Start of list for Lapasus/d_moje/current"
2023/06/03 10:55:12 DEBUG : FTP Rx: " modify=20230603085508;perm=flcdmpe;type=dir;unique=35U9010000A3ADB94C;UNIX.group=100;UNIX.groupname=users;UNIX.mode=0755;UNIX.owner=1002;UNIX.ownername=kazik1; /Lapasus/d_moje/current"
2023/06/03 10:55:12 DEBUG : FTP Rx: "250 End of list"
2023/06/03 10:55:12 DEBUG : FTP Tx: "MKD Lapasus/d_moje/current/Instrukcje"
2023/06/03 10:55:12 DEBUG : FTP Rx: "257 \"/Lapasus/d_moje/current/Instrukcje\" - Directory successfully created"
2023/06/03 10:55:12 DEBUG : FTP Tx: "EPSV"
2023/06/03 10:55:12 DEBUG : FTP Rx: "229 Entering Extended Passive Mode (|||6695|)"
2023/06/03 10:55:12 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/current: dial("tcp","(removed):6695")
2023/06/03 10:55:12 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/current: > dial: conn=*tls.Conn, err=<nil>
2023/06/03 10:55:12 DEBUG : FTP Tx: "STOR Lapasus/d_moje/current/Instrukcje/ale553hqh0f24dbtirlra7913o4r0kl21qe9i6onqrag6tdg3d80"
2023/06/03 10:55:12 DEBUG : FTP Rx: "150 Opening BINARY mode data connection for Lapasus/d_moje/current/Instrukcje/ale553hqh0f24dbtirlra7913o4r0kl21qe9i6onqrag6tdg3d80"
2023/06/03 10:55:12 DEBUG : FTP Tx: "QUIT"
2023/06/03 10:55:13 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/current: Connecting to FTP server
2023/06/03 10:55:13 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/current: dial("tcp","(removed):(removed)")
2023/06/03 10:55:13 DEBUG : ftp://(removed):(removed)/Lapasus/d_moje/current: > dial: conn=*fshttp.timeoutConn, err=<nil>
2023/06/03 10:55:13 DEBUG : FTP Rx: "220 ProFTPD Server (ProFTPd) [(removed)]"
2023/06/03 10:55:13 DEBUG : FTP Tx: "AUTH TLS"
2023/06/03 10:55:13 DEBUG : FTP Rx: "234 AUTH TLS successful"
2023/06/03 10:55:13 DEBUG : FTP Tx: "USER kazik1"
2023/06/03 10:55:14 DEBUG : FTP Rx: "331 Password required for kazik1"
2023/06/03 10:55:14 DEBUG : FTP Tx: PASS *****
2023/06/03 10:55:14 DEBUG : FTP Rx: "230 User kazik1 logged in"
2023/06/03 10:55:14 DEBUG : FTP Tx: "FEAT"
2023/06/03 10:55:14 DEBUG : FTP Rx: "211-Features:"
2023/06/03 10:55:14 DEBUG : FTP Rx: " AUTH TLS"
2023/06/03 10:55:14 DEBUG : FTP Rx: " CCC"
2023/06/03 10:55:14 DEBUG : FTP Rx: " CLNT"
2023/06/03 10:55:14 DEBUG : FTP Rx: " EPRT"
2023/06/03 10:55:14 DEBUG : FTP Rx: " EPSV"
2023/06/03 10:55:14 DEBUG : FTP Rx: " HOST"
2023/06/03 10:55:14 DEBUG : FTP Rx: " LANG en-US*"
2023/06/03 10:55:14 DEBUG : FTP Rx: " MDTM"
2023/06/03 10:55:14 DEBUG : FTP Rx: " MFF modify;UNIX.group;UNIX.mode;"
2023/06/03 10:55:14 DEBUG : FTP Rx: " MFMT"
2023/06/03 10:55:14 DEBUG : FTP Rx: " MLST modify*;perm*;size*;type*;unique*;UNIX.group*;UNIX.groupname*;UNIX.mode*;UNIX.owner*;UNIX.ownername*;"
2023/06/03 10:55:14 DEBUG : FTP Rx: " PBSZ"
2023/06/03 10:55:14 DEBUG : FTP Rx: " PROT"
2023/06/03 10:55:14 DEBUG : FTP Rx: " RANG STREAM"
2023/06/03 10:55:14 DEBUG : FTP Rx: " REST STREAM"
2023/06/03 10:55:14 DEBUG : FTP Rx: " SIZE"
2023/06/03 10:55:14 DEBUG : FTP Rx: " SSCN"
2023/06/03 10:55:14 DEBUG : FTP Rx: " TVFS"
2023/06/03 10:55:14 DEBUG : FTP Rx: " UTF8"
2023/06/03 10:55:14 DEBUG : FTP Rx: "211 End"
2023/06/03 10:55:14 DEBUG : FTP Tx: "TYPE I"
2023/06/03 10:55:14 DEBUG : FTP Rx: "200 Type set to I"
2023/06/03 10:55:14 DEBUG : FTP Tx: "OPTS UTF8 ON"
2023/06/03 10:55:14 DEBUG : FTP Rx: "200 UTF8 set to on"
2023/06/03 10:55:14 DEBUG : FTP Tx: "PBSZ 0"
2023/06/03 10:55:14 DEBUG : FTP Rx: "200 PBSZ 0 successful"
2023/06/03 10:55:14 DEBUG : FTP Tx: "PROT P"
2023/06/03 10:55:14 DEBUG : FTP Rx: "200 Protection set to Private"
2023/06/03 10:55:14 DEBUG : FTP Tx: "MLST Lapasus/d_moje/current/Instrukcje/ale553hqh0f24dbtirlra7913o4r0kl21qe9i6onqrag6tdg3d80"
2023/06/03 10:55:14 DEBUG : FTP Rx: "250-Start of list for Lapasus/d_moje/current/Instrukcje/ale553hqh0f24dbtirlra7913o4r0kl21qe9i6onqrag6tdg3d80"
2023/06/03 10:55:14 DEBUG : FTP Rx: " modify=20230603085513;perm=adfrw;size=0;type=file;unique=35U9010000A45B3638;UNIX.group=100;UNIX.groupname=users;UNIX.mode=0644;UNIX.owner=1002;UNIX.ownername=kazik1; /Lapasus/d_moje/current/Instrukcje/ale553hqh0f24dbtirlra7913o4r0kl21qe9i6onqrag6tdg3d80"
2023/06/03 10:55:14 DEBUG : FTP Rx: "250 End of list"
2023/06/03 10:55:14 DEBUG : FTP Tx: "DELE Lapasus/d_moje/current/Instrukcje/ale553hqh0f24dbtirlra7913o4r0kl21qe9i6onqrag6tdg3d80"
2023/06/03 10:55:14 DEBUG : FTP Rx: "250 DELE command successful"
2023/06/03 10:55:14 DEBUG : Instrukcje/ale553hqh0f24dbtirlra7913o4r0kl21qe9i6onqrag6tdg3d80: Removed after failed upload: 2 errors occurred:
* EOF
* EOF
System.Management.Automation.RemoteException
2023/06/03 10:55:14 ERROR : Instrukcje/121GW-Manual-v12-proto.pdf: Failed to copy: update stor: 2 errors occurred:
* EOF
* EOF
System.Management.Automation.RemoteException