Mangled file endings when mounted via SFTP

What is the problem you are having with rclone?

I tried to use rclone instead of sshfs to mount a remote machine. I was pleasantly surprised by the speed, but I started having a weird problem when editing files, which doesn’t happen with sshfs. I am editing python files with Visual Studio Code, and after saving the files have some mangled endings, like these:

            cnt_0_total=cnt_0_running,
            cnt_1_total=cnt_1_running,
            cumulative_precision=cum_precision[rec.catboost_result_rounded],
            cumulative_recall=round(cnt_1_running / cnt_1_total * 100.0, 2),
            roc_auc_score=roc_auc_score
        )  # actual end of the file
)  # the mangled ending starts here
100.0, 2),
            roc_auc_score=roc_auc_score
        )
ion=cum_precision[rec.catboost_result_rounded],
            cumulative_recall=round(cnt_1_running / cnt_1_total * 100.0, 2),
            roc_auc_score=roc_auc_score
        )

I open the file, delete the ending, save it and re-open — the mangled ending is still there!

I’m confused as to what might the nature of this bug be. It might not be your bug (but VSCode’s for example), but it doesn’t happen when using sshfs. Maybe anyone got a clue...

Run the command 'rclone version' and share the full output of the command.

rclone v1.63.1

  • os/version: darwin 13.4.1 (64 bit)
  • os/kernel: 22.5.0 (arm64)
  • os/type: darwin
  • os/arch: arm64 (ARMv8 compatible)
  • go/version: go1.20.6
  • go/linking: dynamic
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

SFTP

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount remotename: foldername --daemon

The rclone config contents with secrets removed.

[remotename]
type = sftp
host = path.to.host
shell_type = unix
md5sum_command = md5sum
sha1sum_command = sha1sum

A log from the command with the -vv flag

2023/09/05 19:39:15 DEBUG : rclone: Version "v1.63.1" starting with parameters ["rclone" "mount" "remotename:" "foldername" "--daemon" "-vv"]
2023/09/05 19:39:15 DEBUG : Creating backend with remote "remotename:"
2023/09/05 19:39:15 DEBUG : Using config file from "/Users/pecheny/.config/rclone/rclone.conf"
2023/09/05 19:39:16 DEBUG : sftp://pecheny@host-path-redacted:22/: New connection [ip-redacted]:50145->[ip-redacted]:22 to "SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.1"
2023/09/05 19:39:16 DEBUG : sftp://pecheny@host-path-redacted:22/: Shell type "unix" from config
2023/09/05 19:39:16 DEBUG : sftp://pecheny@host-path-redacted:22/: Relative path resolved to "/home/pecheny"
2023/09/05 19:39:16 DEBUG : sftp://pecheny@host-path-redacted:22/: Using root directory "/home/pecheny"
2023/09/05 19:39:21 DEBUG : rclone: Version "v1.63.1" finishing with parameters ["/usr/local/bin/rclone" "mount" "remotename:" "foldername" "--daemon" "-vv"]

welcome to the forum,

might try --vfs-cache-mode=writes or --vfs-cache-mode=full

that log is just a snippet of the top lines, can you post the full debug log?

Great advice to use --vfs-cache-mode writes

I suspect you are going to see moans in the log about not being able to truncate files if you check the log when you try the edit.

--vfs-cache-mode=writes did it, thank you!

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.