V1.55-failed to read-not deleting files as there were IO errors

Suspected Bug
1

What is the problem you are having with rclone?

SFTP tasks running with errors after upgrading to v1.55 (see log below)
Saw this... "not deleting files as there were IO errors" but.. there were no deletions necessary.
Rolled back to 1.54.1 and all is well.

What is your rclone version (output from rclone version)

1.55

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Win10x64

Which cloud storage system are you using? (eg Google Drive)

Synology NAS, DSM v6.2.3

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync --error-on-no-transfer D:\weekly mrbak:/mrbak

The rclone config contents with secrets removed.

[mrbak]
type = sftp
host = 192.168.xxx.xxx
user = username
port = 99999
pass = xxxxxxx
md5sum_command = none
sha1sum_command = none

A log from the command with the -vv flag

2021-04-15 09:50:04 ERROR : Failed to copy: Update ReadFrom failed: failed to send packet: EOF
2021-04-15 09:50:04 ERROR : not deleting files as there were IO errors
2021-04-15 09:50:04 ERROR : sftp: not deleting directories as there were IO errors
2021-04-15 09:50:04 ERROR : Attempt 1/3 failed with 1 errors and: Update ReadFrom failed: failed to send packet: EOF
2021-04-15 09:51:05 ERROR : Failed to copy: Update ReadFrom failed: Connection Lost
2021-04-15 09:51:05 ERROR : sftp: not deleting files as there were IO errors
2021-04-15 09:51:05 ERROR : sftp: not deleting directories as there were IO errors
2021-04-15 09:51:05 ERROR : Attempt 2/3 failed with 1 errors and: Update ReadFrom failed: Connection Lost
2

hi,

could you post the full debug log, including the top lines.

3

Added -vv... Where do I find debug file after I CTRL-C?

I'm using Win10.

4

add this to your command, for example
--log-level=DEBUG --log-file=c:\path\to\log\log.txt

5 
2021/04/15 14:52:09 DEBUG : Using config file from "C:\\Users\\UserName\\.config\\rclone\\rclone.conf"
2021/04/15 14:52:09 DEBUG : rclone: Version "v1.55.0" starting with parameters ["rclone" "sync" "--error-on-no-transfer" "--progress" "--log-level=DEBUG" "--log-file=f:\\TEST\\log.txt" "D:\\Reflect\\weeklies" "mrbak:/mrbak"]
2021/04/15 14:52:09 DEBUG : Creating backend with remote "D:\\Reflect\\weeklies"
2021/04/15 14:52:09 DEBUG : fs cache: renaming cache item "D:\\Reflect\\weeklies" to be canonical "//?/D:/Reflect/weeklies"
2021/04/15 14:52:09 DEBUG : Creating backend with remote "mrbak:/mrbak"
2021/04/15 14:52:10 DEBUG : sftp://secret@192.xxx.xxx.xxx:9999//mrbak: New connection 192.168.96.153:8976->192.xxx.xxx.xxx:9999 to "SSH-2.0-OpenSSH_7.4"
2021/04/15 14:52:10 DEBUG : QWERTYUIOP-00-02.mrimg: Size and modification time the same (differ by -228.4894ms, within tolerance 1s)
2021/04/15 14:52:10 DEBUG : QWERTYUIOP-00-00.mrimg: Size and modification time the same (differ by -548.5828ms, within tolerance 1s)
2021/04/15 14:52:10 DEBUG : QWERTYUIOP-00-00.mrimg: Unchanged skipping
2021/04/15 14:52:10 DEBUG : QWERTYUIOP-00-08.mrimg: Size and modification time the same (differ by -750.1592ms, within tolerance 1s)
2021/04/15 14:52:10 DEBUG : QWERTYUIOP-00-08.mrimg: Unchanged skipping
2021/04/15 14:52:10 DEBUG : QWERTYUIOP-01-09.mrimg: Sizes differ (src 4240292574 vs dst 174157824)
2021/04/15 14:52:10 DEBUG : QWERTYUIOP-00-01.mrimg: Size and modification time the same (differ by -404.489ms, within tolerance 1s)
2021/04/15 14:52:10 DEBUG : QWERTYUIOP-00-06.mrimg: Size and modification time the same (differ by -893.4733ms, within tolerance 1s)
2021/04/15 14:52:10 DEBUG : sftp://secret@192.xxx.xxx.xxx:9999//mrbak: Waiting for checks to finish
2021/04/15 14:52:10 DEBUG : QWERTYUIOP-00-07.mrimg: Size and modification time the same (differ by -133.0258ms, within tolerance 1s)
2021/04/15 14:52:10 DEBUG : QWERTYUIOP-00-07.mrimg: Unchanged skipping
2021/04/15 14:52:10 DEBUG : QWERTYUIOP-00-05.mrimg: Size and modification time the same (differ by -804.0487ms, within tolerance 1s)
2021/04/15 14:52:10 DEBUG : QWERTYUIOP-00-05.mrimg: Unchanged skipping
2021/04/15 14:52:10 DEBUG : QWERTYUIOP-00-03.mrimg: Size and modification time the same (differ by -74.1562ms, within tolerance 1s)
2021/04/15 14:52:10 DEBUG : QWERTYUIOP-00-03.mrimg: Unchanged skipping
2021/04/15 14:52:10 DEBUG : QWERTYUIOP-00-04.mrimg: Size and modification time the same (differ by -248.5824ms, within tolerance 1s)
2021/04/15 14:52:10 DEBUG : QWERTYUIOP-00-04.mrimg: Unchanged skipping
2021/04/15 14:52:10 DEBUG : QWERTYUIOP-00-02.mrimg: Unchanged skipping
2021/04/15 14:52:10 DEBUG : QWERTYUIOP-00-01.mrimg: Unchanged skipping
2021/04/15 14:52:10 DEBUG : QWERTYUIOP-00-06.mrimg: Unchanged skipping
2021/04/15 14:52:10 DEBUG : sftp://secret@192.xxx.xxx.xxx:9999//mrbak: Waiting for transfers to finish
2021/04/15 14:53:10 DEBUG : sftp://secret@192.xxx.xxx.xxx:9999//mrbak: closing 1 unused connections
2021/04/15 14:53:11 DEBUG : sftp://secret@192.xxx.xxx.xxx:9999//mrbak: New connection 192.168.96.153:8977->192.xxx.xxx.xxx:9999 to "SSH-2.0-OpenSSH_7.4"
2021/04/15 14:53:11 DEBUG : QWERTYUIOP-01-09.mrimg: Removed after failed upload: failed to send packet: EOF
2021/04/15 14:53:11 ERROR : QWERTYUIOP-01-09.mrimg: Failed to copy: Update ReadFrom failed: failed to send packet: EOF
2021/04/15 14:53:11 ERROR : sftp://secret@192.xxx.xxx.xxx:9999//mrbak: not deleting files as there were IO errors
2021/04/15 14:53:11 ERROR : sftp://secret@192.xxx.xxx.xxx:9999//mrbak: not deleting directories as there were IO errors
2021/04/15 14:53:11 ERROR : Attempt 1/3 failed with 1 errors and: Update ReadFrom failed: failed to send packet: EOF
2021/04/15 14:53:11 DEBUG : QWERTYUIOP-00-01.mrimg: Size and modification time the same (differ by -404.489ms, within tolerance 1s)
2021/04/15 14:53:11 DEBUG : QWERTYUIOP-00-01.mrimg: Unchanged skipping
2021/04/15 14:53:11 DEBUG : QWERTYUIOP-00-08.mrimg: Size and modification time the same (differ by -750.1592ms, within tolerance 1s)
2021/04/15 14:53:11 DEBUG : QWERTYUIOP-00-08.mrimg: Unchanged skipping
2021/04/15 14:53:11 DEBUG : QWERTYUIOP-00-04.mrimg: Size and modification time the same (differ by -248.5824ms, within tolerance 1s)
2021/04/15 14:53:11 DEBUG : QWERTYUIOP-00-04.mrimg: Unchanged skipping
2021/04/15 14:53:11 DEBUG : sftp://secret@192.xxx.xxx.xxx:9999//mrbak: Waiting for checks to finish
2021/04/15 14:53:11 DEBUG : QWERTYUIOP-00-02.mrimg: Size and modification time the same (differ by -228.4894ms, within tolerance 1s)
2021/04/15 14:53:11 DEBUG : QWERTYUIOP-00-02.mrimg: Unchanged skipping
2021/04/15 14:53:11 DEBUG : QWERTYUIOP-00-06.mrimg: Size and modification time the same (differ by -893.4733ms, within tolerance 1s)
2021/04/15 14:53:11 DEBUG : QWERTYUIOP-00-06.mrimg: Unchanged skipping
2021/04/15 14:53:11 DEBUG : QWERTYUIOP-00-00.mrimg: Size and modification time the same (differ by -548.5828ms, within tolerance 1s)
2021/04/15 14:53:11 DEBUG : QWERTYUIOP-00-00.mrimg: Unchanged skipping
2021/04/15 14:53:11 DEBUG : QWERTYUIOP-00-05.mrimg: Size and modification time the same (differ by -804.0487ms, within tolerance 1s)
2021/04/15 14:53:11 DEBUG : QWERTYUIOP-00-05.mrimg: Unchanged skipping
2021/04/15 14:53:11 DEBUG : QWERTYUIOP-00-03.mrimg: Size and modification time the same (differ by -74.1562ms, within tolerance 1s)
2021/04/15 14:53:11 DEBUG : QWERTYUIOP-00-03.mrimg: Unchanged skipping
2021/04/15 14:53:11 DEBUG : QWERTYUIOP-00-07.mrimg: Size and modification time the same (differ by -133.0258ms, within tolerance 1s)
2021/04/15 14:53:11 DEBUG : QWERTYUIOP-00-07.mrimg: Unchanged skipping
2021/04/15 14:53:11 DEBUG : sftp://secret@192.xxx.xxx.xxx:9999//mrbak: Waiting for transfers to finish
2021/04/15 14:54:11 DEBUG : sftp://secret@192.xxx.xxx.xxx:9999//mrbak: closing 1 unused connections
2021/04/15 14:54:12 DEBUG : sftp://secret@192.xxx.xxx.xxx:9999//mrbak: New connection 192.168.96.153:8978->192.xxx.xxx.xxx:9999 to "SSH-2.0-OpenSSH_7.4"
2021/04/15 14:54:13 DEBUG : QWERTYUIOP-01-09.mrimg: Removed after failed upload: failed to send packet: EOF
2021/04/15 14:54:13 ERROR : QWERTYUIOP-01-09.mrimg: Failed to copy: Update ReadFrom failed: failed to send packet: EOF
2021/04/15 14:54:13 ERROR : sftp://secret@192.xxx.xxx.xxx:9999//mrbak: not deleting files as there were IO errors
2021/04/15 14:54:13 ERROR : sftp://secret@192.xxx.xxx.xxx:9999//mrbak: not deleting directories as there were IO errors
2021/04/15 14:54:13 ERROR : Attempt 2/3 failed with 1 errors and: Update ReadFrom failed: failed to send packet: EOF
2021/04/15 14:54:13 DEBUG : sftp://secret@192.xxx.xxx.xxx:9999//mrbak: Waiting for checks to finish
2021/04/15 14:54:13 DEBUG : QWERTYUIOP-00-01.mrimg: Size and modification time the same (differ by -404.489ms, within tolerance 1s)
2021/04/15 14:54:13 DEBUG : QWERTYUIOP-00-01.mrimg: Unchanged skipping
2021/04/15 14:54:13 DEBUG : QWERTYUIOP-00-05.mrimg: Size and modification time the same (differ by -804.0487ms, within tolerance 1s)
2021/04/15 14:54:13 DEBUG : QWERTYUIOP-00-05.mrimg: Unchanged skipping
2021/04/15 14:54:13 DEBUG : QWERTYUIOP-00-00.mrimg: Size and modification time the same (differ by -548.5828ms, within tolerance 1s)
2021/04/15 14:54:13 DEBUG : QWERTYUIOP-00-02.mrimg: Size and modification time the same (differ by -228.4894ms, within tolerance 1s)
2021/04/15 14:54:13 DEBUG : QWERTYUIOP-00-02.mrimg: Unchanged skipping
2021/04/15 14:54:13 DEBUG : QWERTYUIOP-00-00.mrimg: Unchanged skipping
2021/04/15 14:54:13 DEBUG : QWERTYUIOP-00-03.mrimg: Size and modification time the same (differ by -74.1562ms, within tolerance 1s)
2021/04/15 14:54:13 DEBUG : QWERTYUIOP-00-03.mrimg: Unchanged skipping
2021/04/15 14:54:13 DEBUG : QWERTYUIOP-00-04.mrimg: Size and modification time the same (differ by -248.5824ms, within tolerance 1s)
2021/04/15 14:54:13 DEBUG : QWERTYUIOP-00-04.mrimg: Unchanged skipping
2021/04/15 14:54:13 DEBUG : QWERTYUIOP-00-06.mrimg: Size and modification time the same (differ by -893.4733ms, within tolerance 1s)
2021/04/15 14:54:13 DEBUG : QWERTYUIOP-00-06.mrimg: Unchanged skipping
2021/04/15 14:54:13 DEBUG : QWERTYUIOP-00-07.mrimg: Size and modification time the same (differ by -133.0258ms, within tolerance 1s)
2021/04/15 14:54:13 DEBUG : QWERTYUIOP-00-08.mrimg: Size and modification time the same (differ by -750.1592ms, within tolerance 1s)
2021/04/15 14:54:13 DEBUG : QWERTYUIOP-00-07.mrimg: Unchanged skipping
2021/04/15 14:54:13 DEBUG : QWERTYUIOP-00-08.mrimg: Unchanged skipping
2021/04/15 14:54:13 DEBUG : sftp://secret@192.xxx.xxx.xxx:9999//mrbak: Waiting for transfers to finish
6

now that we have a debug log, i am sure that one of our rclone experts will stop by soon.

rclone has its own sftp server rclone serve sftp.
i wonder if you have the same problem with rclone sync to that?

7

I have no problems with 1.54.1, just 1.55. FWIW, these are 10GB files, if that matters.

8

have you searched the forum?

https://forum.rclone.org/t/crypt-sftp-failed-to-send-packet-eof-slow/23241/9

https://forum.rclone.org/t/how-to-install-on-dreamhost-hosting-site-ssh-on-sandy-server-with-no-root-sudo-access/23093/7

1 Like
9

I search here and GitHub and didn't see any recent result. Nice find. Hopefully this is settled in 1.5.6.

10

I'm pretty sure this issue is fixed.

Please try the latest v1.55.1 beta - it should be fixed in there I think

https://beta.rclone.org/branch/v1.55-stable/

11

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.