SSH_FX_BAD_MESSAGE when syncing files with long filename to encrypted sftp storage

Suspected Bug
1

What is the problem you are having with rclone?

I want to sync files to a remote sftp storage (Hetzner Storagebox) with filename encryption. Sync fails with "SSH_FX_BAD_MESSAGE" if files with long filenames are being synced.

Run the command 'rclone version' and share the full output of the command.

rclone v1.66.0

  • os/version: linuxmint 21.3 (64 bit)
  • os/kernel: 5.15.0-107-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.22.1
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Hetzner Stroagebox

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync -vv --retries=1 ./rclone-test/ "SFTP Hetzner Enc:"

The rclone config contents with secrets removed.

[SFTP Hetzner]
type = sftp
host = uXXXXXX-sub2.your-storagebox.de
user = uXXXXXX-sub2
pass = ***REMOVED***
port = 23
shell_type = unix
md5sum_command = md5 -r
sha1sum_command = sha1 -r

[SFTP Hetzner Enc]
type = crypt
remote = SFTP Hetzner:rclone_enc
password = ***REMOVED***
password2 = ***REMOVED***

A log from the command with the -vv flag

024/05/30 14:48:08 DEBUG : rclone: Version "v1.66.0" starting with parameters ["rclone" "sync" "-vv" "--retries=1" "./rclone-test/" "SFTP Hetzner Enc:"]
2024/05/30 14:48:08 DEBUG : Creating backend with remote "./rclone-test/"
2024/05/30 14:48:08 DEBUG : Using config file from "/home/XXXXXX/.config/rclone/rclone.conf"
2024/05/30 14:48:08 DEBUG : fs cache: renaming cache item "./rclone-test/" to be canonical "/home/XXXXXX/tmp/rclone-test"
2024/05/30 14:48:08 DEBUG : Creating backend with remote "SFTP Hetzner Enc:"
2024/05/30 14:48:08 DEBUG : Creating backend with remote "SFTP Hetzner:rclone_enc"
2024/05/30 14:48:09 DEBUG : sftp://uXXXXXX-sub2@uXXXXXX-sub2.your-storagebox.de:23/rclone_enc: New connection [IPV6_REMOVED]:35918->[IPV6_REMOVED]:23 to "SSH-2.0-OpenSSH_9.3 FreeBSD-20230719"
2024/05/30 14:48:09 DEBUG : sftp://uXXXXXX-sub2@uXXXXXX-sub2.your-storagebox.de:23/rclone_enc: Shell type "unix" from config
2024/05/30 14:48:09 DEBUG : sftp://uXXXXXX-sub2@uXXXXXX-sub2.your-storagebox.de:23/rclone_enc: Relative path resolved to "/home/rclone_enc"
2024/05/30 14:48:09 DEBUG : sftp://uXXXXXX-sub2@uXXXXXX-sub2.your-storagebox.de:23/rclone_enc: Using root directory "/home/rclone_enc"
2024/05/30 14:48:09 DEBUG : a very long file name Lorem ipsum dolor sit amet consetetur sadipscing elitr sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat sed diam voluptua At vero eos et accusam et justo duo dolores et ea rebum.txt: Need to transfer - File not found at Destination
2024/05/30 14:48:09 DEBUG : Encrypted drive 'SFTP Hetzner Enc:': Waiting for checks to finish
2024/05/30 14:48:09 DEBUG : Encrypted drive 'SFTP Hetzner Enc:': Waiting for transfers to finish
2024/05/30 14:48:09 ERROR : a very long file name Lorem ipsum dolor sit amet consetetur sadipscing elitr sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat sed diam voluptua At vero eos et accusam et justo duo dolores et ea rebum.txt: Failed to copy: Update Create failed: sftp: "Bad message" (SSH_FX_BAD_MESSAGE)
2024/05/30 14:48:09 INFO  : a very long file name Lorem ipsum dolor sit amet consetetur sadipscing elitr sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat sed diam voluptua At vero eos et accusam et justo duo dolores et ea rebum.txt.sawokab4.partial: Failed to remove failed partial copy: stat failed: sftp: "Bad message" (SSH_FX_BAD_MESSAGE)
2024/05/30 14:48:09 ERROR : Encrypted drive 'SFTP Hetzner Enc:': not deleting files as there were IO errors
2024/05/30 14:48:09 ERROR : Encrypted drive 'SFTP Hetzner Enc:': not deleting directories as there were IO errors
2024/05/30 14:48:09 ERROR : Attempt 1/1 failed with 1 errors and: Update Create failed: sftp: "Bad message" (SSH_FX_BAD_MESSAGE)
2024/05/30 14:48:09 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.5s

2024/05/30 14:48:09 DEBUG : 15 go routines active
2024/05/30 14:48:09 DEBUG : sftp://uXXXXXX-sub2@uXXXXXX-sub2.your-storagebox.de:23/rclone_enc: Closing 1 unused connections
2024/05/30 14:48:09 Failed to sync: Update Create failed: sftp: "Bad message" (SSH_FX_BAD_MESSAGE)
2

welcome to the forum,

that is a known issue with crypt remotes.

might test using another encoding.
An experimental advanced option filename_encoding is now provided to address this problem to a certain degree

might test obfuscation, but based on forum posts, not guaranteed to work in all cases.

3

Thank you. Switching to filename obfuscation solves the issue for me.
It would be great if full filename encryption was working as well, but for now, obfuscation is good enough in my point of view, since the file content is encrypted anyway.

4

I wonder if it is time to remove this as experimental. It's been out for a while and I haven't heard of many issues

5

i have no idea about that. but there are a bunch of issues in the forum.
imho, not a good look for just keep so many experimental features and backends, for year after year.

yeah, but that also implies that there are issues not getting addressed for years.

6

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.