Crypt + SFTP - failed to send packet: EOF & Slow!

Help and Support
1

What is the problem you are having with rclone?

First and more pressing issue: Failing to upload files to an SFTP server.
Error: Failed to copy: Update ReadFrom failed: failed to send packet: EOF

Two are in the region of 20GBytes the other 300MBytes.

Another backup to the same server using the same settings (but different source folder/destination folder) seems to work just fine but all the files are < 100MB.

If I upload the un-encrypted directly to the SFTP server it uploads seemingly fine. Updated the SFTP software on the destination to without success.

It also fails via the Rclone Web UI.

Second Issue
Each transfer thread uploads at approx 950KBytes/sec. CPU on the local UnRAID server is a Quad Core E3-1230 V3 (3.3-3.6ghz) and CPU load isn't 100% on any thread.

I can read and write to the local disk at full gigabit so its not a local disk IO issue.

Uploading the files mentioned above one at a time to the SFTP server via Cyberduck gets full line speed (9MBytes/sec) - but obviously unencrypted!

So:

A) Why are these files failing to upload.
B) Why is each transfer thread so slow?

What is your rclone version (output from rclone version)

Various, including latest beta: v1.56.0-beta.5356.9e2fbe0f1

Which OS you are using and how many bits (eg Windows 7, 64 bit)

UnRAID v6.9

Which cloud storage system are you using? (eg Google Drive)

SFTP to remote server

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync --config .rclone.conf "...path..." Disk3-Remote-Encrypted: --delete-before --transfers=4 --check-first --sftp-set-modtime=false --checkers 4  -P --stats=1m --stats-one-line --buffer-size=64M --log-level DEBUG

The rclone config contents with secrets removed.

[Disk3-Remote-Encrypted]
type = crypt
password = BLAH
remote = Remote:/BU/Disk3

[Remote]
type = sftp
user = BLAH
host = BLAH
key_file = BLAH-BLAH
key_file_pass = BLAH
key_use_agent = false
port = BLAH
use_insecure_cipher = false
md5sum_command = none
sha1sum_command = none

A log from the command with the -vv flag

2021-04-01 11:45:17 DEBUG : sftp://BLAH@BLAH:BLAH//BU/Disk3: closing 4 unused connections
2021-04-01 11:45:18 DEBUG : sftp://BLAH@BLAH:BLAH//BU/Disk3: New connection LOCAL-IP:LOCAL-PORT->REMOTE-IP:REMOTE-PORT to "SSH-2.0-OpenSSH_7.4p1 Debian-10+deb9u4"
2021-04-01 11:45:18 DEBUG : TL/2021/IT2 - 2021/IT2 - 2021 - DNG files.zip: Removed after failed upload: failed to send packet: EOF
2021-04-01 11:45:18 ERROR : TL/2021/IT2 - 2021/IT2 - 2021 - DNG files.zip: Failed to copy: Update ReadFrom failed: failed to send packet: EOF
2021-04-01 11:45:18 ERROR : Attempt 2/3 failed with 3 errors and: Update ReadFrom failed: failed to send packet: EOF
2

hello,

have you tried the latest stable v1.55.0?

so the issue is only when uploading to a rclone crypted remote when the file size is larger than 100MB?

as a test, could try to upload to rclone sftp server using rclone serve sftp

3

Its not my server but a colleagues, thus I don't have direct access to it (I've just got him to add a SFTP docker and pass disks through). Running an rclone server is more complicated...

I've not tried the stable, I'll wait until the current transfers are complete (33k <50 Mbytes) files before changing rclone versions.

4

Using latest stable too - fails as well.

400GB of small files (50MB or less) transferred absolutely fine...

Anything else that can be tried other than using rclone serve?

5

This sounds like the same as this issue

Can you see if 1.54.1 works OK?

6

That latest beta on Mac OS/Linux fixes the speed issue but not the EOF issue.

I'll try 1.54.

Thanks.

Edit: Went back to 1.53 and that appears to work OK (Up to 9GB of one file so far...)
Edit 2: Three 20GB files uploaded at full line speed (--transfers 1) OK with 1.53.

So something between 1.53 and >1.55 is broken!

7

I can have a go at bisecting the EOF issue. What do you think the best way of reproducing it is? Can you get it to go wrong every time?

The bisection will take a very long time if I have to test each candidate for too long!

8

Yep - every time! I haven't paid too much attention to if it happens exactly the same place or not.

I've downloaded that beta again and I'll test how far it gets along of the troublesome files once or twice.

Edit: Using rclone-v1.56.0-beta.5357.b89d085cb, looking at the server folder the file gets deleted so I don't know the exact size...

Run 1: Error before 500MB
Run 2: Error before 475MB (start: 22:16:03, end 22:17:04)
Run 3: Error just before 470MB (start: 22:17:10, end 22:18:11)
Run 4: Error just before 467MB (start: 22:18:45, end 22:19:46)

It seems it dumps out after exactly one minute rather than the size...

So Run 5 (with --bwlimit 1M - speeds above approx 8M) results in an error at... the 1 minute mark!

So it seems that in after rclone 1.53 with SFTP transfers, there is a timeout somewhere of 60 seconds.

9

I have managed to discover what is going on.

In sftp: Update ReadFrom failed: failed to send packet: EOF errors and very slow ¡ Issue #5197 ¡ rclone/rclone ¡ GitHub I have posted a new test binary which should fix the problem - testing appreciated!

It it works I'll put it in 1.55.1

10

@ncw Sorry to add to a slightly older post... but I am seeing exactly the same issue. Works perfectly on Ubuntu, but fails as originally described on Windows.

UPDATE: it looks like things are OK on the 32 bit version, it’s the 64 bit that has the problem. It closes the connection(s) and doesn’t have the ‘not closing unused connections’ message.

12

Did you try v1.55.1?

13

Yes this is using v1.55.1. The 32 bit failed for a separate reason, so actually that still looks OK. The 64 bit is definitely failing though, due to the connection being lost.

14

Can you open a new topic and fill in the support template with all the details please @Daniel_Kennedy ? I this must be a different issue...

15

Yes no problem.

1 Like
16

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.