What is the problem you are having with rclone?
When copying many files with sftp, I get occasional erroneous permission denied errors
Run the command 'rclone version' and share the full output of the command.
rclone v1.69.0
- os/version: darwin 15.1.1 (64 bit)
- os/kernel: 24.1.0 (arm64)
- os/type: darwin
- os/arch: arm64 (ARMv8 compatible)
- go/version: go1.23.4
- go/linking: dynamic
- go/tags: none
In addition, I have experienced the same error with v1.67.0 - v1.69.0 on Ubuntu.
Which cloud storage system are you using? (eg Google Drive)
sftp
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone copy \
--modify-window 2s \
--fast-list \
--update \
--sftp-connections 4 \
--multi-thread-streams 4 \
--transfers 4 \
-vv \
remote-sftp:. .
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
[remote-sftp]
type = sftp
host = XXX
user = XXX
port = 6671
pass = XXX
shell_type = unix
md5sum_command = none
sha1sum_command = none
connections = 5
macs = hmac-sha2-512-etm@openssh.com
key_exchange = ecdh-sha2-nistp384
ciphers = aes256-ctr
host_key_algorithms = rsa-sha2-512
use_fstat = true
A log from the command that you were trying to run with the -vv flag
2025/02/06 17:54:10 DEBUG : rclone: Version "v1.69.0" starting with parameters ["rclone" "copy" "--modify-window" "2s" "--fast-list" "--update" "--sftp-shell-type" "unix" "--sftp-md5sum-command" "none" "--sftp-sha1sum-command" "none" "--sftp-connections" "4" "--multi-thread-streams" "4" "--transfers" "4" "--sftp-macs" "hmac-sha2-512-etm@openssh.com" "--sftp-key-exchange" "ecdh-sha2-nistp384" "--sftp-ciphers" "aes256-ctr" "--sftp-host-key-algorithms" "rsa-sha2-512" "-vv" "remote-sftp:." "."]
2025/02/06 17:54:10 DEBUG : Creating backend with remote "remote-sftp:."
2025/02/06 17:54:10 DEBUG : Using config file from "/Users/russ/.config/rclone/rclone.conf"
2025/02/06 17:54:10 DEBUG : remote-sftp: detected overridden config - adding "{bi1gX}" suffix to name
2025/02/06 17:54:11 DEBUG : sftp://XXXX@XXXX:6671/.: New connection 192.168.4.127:62829->XXXX:6671 to "SSH-2.0-SSHD"
2025/02/06 17:54:11 DEBUG : sftp://XXXX@XXXX:6671/.: Shell type "unix" from config
2025/02/06 17:54:11 DEBUG : sftp://XXXX@XXXX:6671/.: Relative path resolved to "/"
2025/02/06 17:54:11 DEBUG : sftp://XXXX@XXXX:6671/.: Using root directory "/"
2025/02/06 17:54:11 DEBUG : fs cache: renaming cache item "remote-sftp:." to be canonical "remote-sftp{bi1gX}:."
2025/02/06 17:54:11 DEBUG : Creating backend with remote "."
2025/02/06 17:54:11 DEBUG : fs cache: renaming cache item "." to be canonical "/home/XXXX/log"
2025/02/06 17:54:12 DEBUG : Added delayed dir = "log", newDst=<nil>
2025/02/06 17:54:12 DEBUG : test_file: Need to transfer - File not found at Destination
2025/02/06 17:54:12 DEBUG : sftp://XXXX@XXXX:6671/.: New connection 192.168.4.127:62845->XXXX:6671 to "SSH-2.0-SSHD"
2025/02/06 17:54:12 DEBUG : log/t0_d0_a0.pdf: Need to transfer - File not found at Destination
*[SNIPPED - 525 lines of similar output, with diffferent filenames]*
2025/02/06 17:54:12 DEBUG : log/t0_d0_a99.pdf: Need to transfer - File not found at Destination
2025/02/06 17:54:12 DEBUG : log/ADS_PDF_DATA.pubtag: Need to transfer - File not found at Destination
2025/02/06 17:54:12 DEBUG : Local file system at /home/XXXX/log: Waiting for checks to finish
2025/02/06 17:54:12 DEBUG : Local file system at /home/XXXX/log: Waiting for transfers to finish
2025/02/06 17:54:13 DEBUG : test_file.8e81cfff.partial: renamed to: test_file
2025/02/06 17:54:13 INFO : test_file: Copied (new)
2025/02/06 17:54:13 DEBUG : sftp://XXXX@XXXX:6671/.: New connection 192.168.4.127:62864->XXXX:6671 to "SSH-2.0-SSHD"
2025/02/06 17:54:13 DEBUG : sftp://XXXX@XXXX:6671/.: New connection 192.168.4.127:62865->XXXX:6671 to "SSH-2.0-SSHD"
2025/02/06 17:54:13 DEBUG : log/t0_d0_a0.pdf.2f80f023.partial: renamed to: log/t0_d0_a0.pdf
2025/02/06 17:54:13 INFO : log/t0_d0_a0.pdf: Copied (new)
*[SNIPPED - 207 pairs of line of similar output, with diffferent filenames]*
2025/02/06 17:54:54 DEBUG : log/t0_d0_a285.pdf.cc7fc973.partial: renamed to: log/t0_d0_a285.pdf
2025/02/06 17:54:54 INFO : log/t0_d0_a285.pdf: Copied (new)
2025/02/06 17:54:54 DEBUG : sftp://XXXX@XXXX:6671/.: Connection OK after error: permission denied
2025/02/06 17:54:54 ERROR : log/t0_d0_a289.pdf: Failed to copy: failed to open source object: Open failed: permission denied
2025/02/06 17:54:55 DEBUG : sftp://XXXX@XXXX:6671/.: Connection OK after error: permission denied
2025/02/06 17:54:55 ERROR : log/t0_d0_a290.pdf: Failed to copy: failed to open source object: Open failed: permission denied
2025/02/06 17:54:55 DEBUG : log/t0_d0_a287.pdf.e1b4a4e.partial: renamed to: log/t0_d0_a287.pdf
2025/02/06 17:54:55 INFO : log/t0_d0_a287.pdf: Copied (new)
2025/02/06 17:54:55 DEBUG : log/t0_d0_a288.pdf.dc41fbfa.partial: renamed to: log/t0_d0_a288.pdf
2025/02/06 17:54:55 INFO : log/t0_d0_a288.pdf: Copied (new)
2025/02/06 17:54:55 DEBUG : sftp://XXXX@XXXX:6671/.: Connection OK after error: permission denied
2025/02/06 17:54:55 ERROR : log/t0_d0_a291.pdf: Failed to copy: failed to open source object: Open failed: permission denied
2025/02/06 17:54:55 DEBUG : sftp://XXXX@XXXX:6671/.: Connection OK after error: permission denied
2025/02/06 17:54:55 ERROR : log/t0_d0_a293.pdf: Failed to copy: failed to open source object: Open failed: permission denied
2025/02/06 17:54:55 DEBUG : log/t0_d0_a29.pdf.f380b30f.partial: renamed to: log/t0_d0_a29.pdf
2025/02/06 17:54:55 INFO : log/t0_d0_a29.pdf: Copied (new)
^C2025/02/06 17:55:03 INFO : Signal received: interrupt
2025/02/06 17:55:03 INFO : log/t0_d0_a342.pdf.23571269.partial: Removing failed copy
2025/02/06 17:55:03 INFO : log/t0_d0_a345.pdf.e17d6383.partial: Removing failed copy
2025/02/06 17:55:03 INFO : Exiting...