Update ReadFrom failed: sftp: "seek 159744 not supported with cursor 126976." (SSH_FX_FAILURE)

Help and Support
1

What is the problem you are having with rclone?

Hello everyone,

I am having issues with moving files from S3 to SFTP. The error doesn't always occur and it tends to be happening on larger files (still a couple MBs only). Sometimes rclone is able to move them after many retries. Do you have an idea why this could be happening?

The move works reliably using --sftp-disable-concurrent-reads --sftp-disable-concurrent-writes parameters, but there is an error in the log even though the file arrives successfully:
2021/07/23 08:35:03 ERROR : sftp://...@...////: Ignoring disable_concurrent_reads after library reversion - see #5197

What is your rclone version (output from rclone version)

v1.56.0

Which OS you are using and how many bits (eg Windows 7, 64 bit)

ubuntu 18.04 (64 bit)

Which cloud storage system are you using? (eg Google Drive)

AWS S3

The command you were trying to run (eg rclone copy /tmp remote:tmp)

"rclone" "moveto" "--ignore-size" "--log-level" "DEBUG" "--log-file=rclone.log" "my-s3:my-s3-bucket/my-file.zip" "my-sftp:///my-file.zip"

The rclone config contents with secrets removed.

[my-sftp]
type = sftp
host = ...
user = ...
port = ...
pass = ...
key_file = /home/ubuntu/.ssh/id_rsa
disable_hashcheck = true
use_insecure_cipher = true
md5sum_command = none
set_modtime = false
sha1sum_command = none

[my-s3]
type = s3
provider = AWS
env_auth = true
region = eu-west-2
location_constraint = eu-west-2
acl = private
server_side_encryption = AES256

A log from the command with the -vv flag

2021/07/22 18:15:03 DEBUG : rclone: Version "v1.56.0" starting with parameters ["rclone" "moveto" "--ignore-size" "--log-level" "DEBUG" "--log-file=rclone.log" "my-s3:my-s3-bucket/my-file.zip" "my-sftp:///my-file.zip"]
2021/07/22 18:15:03 DEBUG : Creating backend with remote "my-s3:my-s3-bucket/my-file.zip"
2021/07/22 18:15:03 DEBUG : Using config file from "/home/ubuntu/.config/rclone/rclone.conf"
2021/07/22 18:15:03 DEBUG : fs cache: adding new entry for parent of "my-s3:my-s3-bucket/my-file.zip", "my-s3:my-s3-bucket"
2021/07/22 18:15:03 DEBUG : Creating backend with remote "my-sftp:///"
2021/07/22 18:15:03 DEBUG : sftp://...////: New connection ...->... to "SSH-2.0-sftp server"
2021/07/22 18:15:03 DEBUG : my-file.zip: Need to transfer - File not found at Destination
2021/07/22 18:15:04 DEBUG : my-file.zip: Removed after failed upload: sftp: "seek 618496 not supported with cursor 520192." (SSH_FX_FAILURE)
2021/07/22 18:15:04 ERROR : my-file.zip: Failed to copy: Update ReadFrom failed: sftp: "seek 618496 not supported with cursor 520192." (SSH_FX_FAILURE)
2021/07/22 18:15:04 ERROR : my-file.zip: Not deleting source as copy failed: Update ReadFrom failed: sftp: "seek 618496 not supported with cursor 520192." (SSH_FX_FAILURE)
2021/07/22 18:15:04 ERROR : Attempt 1/3 failed with 1 errors and: Update ReadFrom failed: sftp: "seek 618496 not supported with cursor 520192." (SSH_FX_FAILURE)
2021/07/22 18:15:04 DEBUG : my-file.zip: Need to transfer - File not found at Destination
2021/07/22 18:15:05 DEBUG : my-file.zip: Removed after failed upload: sftp: "seek 192512 not supported with cursor 126976." (SSH_FX_FAILURE)
2021/07/22 18:15:05 ERROR : my-file.zip: Failed to copy: Update ReadFrom failed: sftp: "seek 192512 not supported with cursor 126976." (SSH_FX_FAILURE)
2021/07/22 18:15:05 ERROR : my-file.zip: Not deleting source as copy failed: Update ReadFrom failed: sftp: "seek 192512 not supported with cursor 126976." (SSH_FX_FAILURE)
2021/07/22 18:15:05 ERROR : Attempt 2/3 failed with 1 errors and: Update ReadFrom failed: sftp: "seek 192512 not supported with cursor 126976." (SSH_FX_FAILURE)
2021/07/22 18:15:05 DEBUG : my-file.zip: Need to transfer - File not found at Destination
2021/07/22 18:15:05 DEBUG : my-file.zip: Removed after failed upload: sftp: "seek 159744 not supported with cursor 126976." (SSH_FX_FAILURE)
2021/07/22 18:15:05 ERROR : my-file.zip: Failed to copy: Update ReadFrom failed: sftp: "seek 159744 not supported with cursor 126976." (SSH_FX_FAILURE)
2021/07/22 18:15:05 ERROR : my-file.zip: Not deleting source as copy failed: Update ReadFrom failed: sftp: "seek 159744 not supported with cursor 126976." (SSH_FX_FAILURE)
2021/07/22 18:15:05 ERROR : Attempt 3/3 failed with 1 errors and: Update ReadFrom failed: sftp: "seek 159744 not supported with cursor 126976." (SSH_FX_FAILURE)
2021/07/22 18:15:05 INFO  :
2021/07/22 18:15:05 DEBUG : 15 go routines active
2021/07/22 18:15:05 Failed to moveto: Update ReadFrom failed: sftp: "seek 159744 not supported with cursor 126976." (SSH_FX_FAILURE)
2

Any idea what could be causing this? The issue is still present.

3

Hi akos, welcome to the forum

Good post!

I use SFTP daily and haven’t encountered that error, so my best guess that it has something to do with your SFTP server (in combination with rclone).

Somebody may be inclined to take a deeper look if you can reproduce the issue on a well-known publicly available SFTP server with many users. Preferable using a simple example reading from the local file system.

Otherwise, my best advice is to continue using --sftp-disable-concurrent-writes to get maximum stability. And then try to remove --sftp-disable-concurrent-reads to avoid the ERROR message triggered by #5197

Probably not the answer you were hoping to get, but that is the best I can offer.

4

This is useful for SFTP servers which can't cope with concurrent reads and writes which it looks like yours is.

What server is it?

It looks like this is an error message left over from a previous patch. You can ignore it - I'll remove it now.

5

...actually we removed it already in the latest beta. I've queued the patch for 1.56.1 too.

6

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.