Rclone copy copies unchanged files sometimes

What is the problem you are having with rclone?

Using rclone to copy a directory tree from a SFTP remote (actually a Linux container on the same host) copies some files repeatedly. This only affects a small subset of files. Narrowing this down suggests that it is not the specific files that are the issue, but the containing directory.

Issue detected on latest stable and reproduced on latest beta.

Run the command 'rclone version' and share the full output of the command.

rclone v1.59.2
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 5.15.0-48-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.18.6
- go/linking: static
- go/tags: none

also tested on:

rclone v1.60.0-beta.6466.2aa264b33
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 5.15.0-48-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.19.1
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

SFTP to local filesystem

The command you were trying to run (eg rclone copy /tmp remote:tmp)

./rclone copy -vv orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/ /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/

The rclone config contents with secrets removed.

[orchestra-jammy]
type = sftp
host = 10.51.175.82
user = ubuntu
key_file = ~/.ssh/id_rsa
key_file_pass = <removed>
shell_type = unix
md5sum_command = /usr/bin/md5sum
sha1sum_command = /usr/bin/sha1sum

A log from the command with the -vv flag

[snip]
2022/10/01 12:21:40 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell path "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md"
2022/10/01 12:21:40 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Running remote command: /usr/bin/md5sum /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md
2022/10/01 12:21:40 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Remote command result: 34a3da2c76628872ceafaa9a6307df42  /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md
2022/10/01 12:21:40 DEBUG : template/disclaimer.md: Parsed hash: 34a3da2c76628872ceafaa9a6307df42
2022/10/01 12:21:40 DEBUG : template/disclaimer.md: md5 = 34a3da2c76628872ceafaa9a6307df42 OK
2022/10/01 12:21:40 INFO  : template/disclaimer.md: Copied (new)
2022/10/01 12:21:40 INFO  : 
Transferred:   	  788.268 KiB / 788.268 KiB, 100%, 0 B/s, ETA -
Checks:                 6 / 6, 100%
Transferred:            8 / 8, 100%
Elapsed time:         0.8s

2022/10/01 12:21:40 DEBUG : 57 go routines active
2022/10/01 12:21:40 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Closing 6 unused connections

Repeating the command results in the same output, with all 8 files in the template remote sub-directory reported as being copied each time, including the final file disclaimer.md shown in log.

The copy should not happen as it is present on both the remote and local file system:

david@alpe:~/Bin/rclone-v1.59.2-linux-amd64$ ./rclone lsl orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md
      994 2022-08-15 10:07:21.000000000 disclaimer.md
david@alpe:~/Bin/rclone-v1.59.2-linux-amd64$ ./rclone lsl /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md
      994 2022-08-15 10:07:21.000000000 disclaimer.md

However if just the template sub-directory itself is copied, or just the disclaimer.md file within it, then the existing version is detected as expected and no copy takes place. It only happens when copying the higher level directory.

sub-directory:

david@alpe:~/Bin/rclone-v1.59.2-linux-amd64$ ./rclone copy -vv orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/template/ /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/
2022/10/01 12:29:32 DEBUG : rclone: Version "v1.59.2" starting with parameters ["./rclone" "copy" "-vv" "orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/template/" "/home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/"]
2022/10/01 12:29:32 DEBUG : Creating backend with remote "orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/template/"
2022/10/01 12:29:32 DEBUG : Using config file from "/home/david/.config/rclone/rclone.conf"
2022/10/01 12:29:32 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/template/: New connection 10.51.175.1:48456->10.51.175.82:22 to "SSH-2.0-OpenSSH_8.9p1 Ubuntu-3"
2022/10/01 12:29:33 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/template/: Shell type "unix" from config
2022/10/01 12:29:33 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/template/: Relative path resolved to "/home/ubuntu/github/ltng-documentation/template"
2022/10/01 12:29:33 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/template/: Using root directory "/home/ubuntu/github/ltng-documentation/template"
2022/10/01 12:29:33 DEBUG : Creating backend with remote "/home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/"
2022/10/01 12:29:33 DEBUG : cover.pdf: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/10/01 12:29:33 DEBUG : finalpage_A4.pdf: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/10/01 12:29:33 DEBUG : cover.pdf: Unchanged skipping
2022/10/01 12:29:33 DEBUG : finalpage_A4.pdf: Unchanged skipping
2022/10/01 12:29:33 DEBUG : .gitignore: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/10/01 12:29:33 DEBUG : eisvogel.tex: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/10/01 12:29:33 DEBUG : .gitignore: Unchanged skipping
2022/10/01 12:29:33 DEBUG : eisvogel.tex: Unchanged skipping
2022/10/01 12:29:33 DEBUG : disclaimer.md: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/10/01 12:29:33 DEBUG : disclaimer.md: Unchanged skipping
2022/10/01 12:29:33 DEBUG : background.pdf: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/10/01 12:29:33 DEBUG : background.pdf: Unchanged skipping
2022/10/01 12:29:33 DEBUG : finalpage_A3.pdf: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/10/01 12:29:33 DEBUG : finalpage_A3.pdf: Unchanged skipping
2022/10/01 12:29:33 DEBUG : Local file system at /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/: Waiting for checks to finish
2022/10/01 12:29:33 DEBUG : templateParams.md: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/10/01 12:29:33 DEBUG : templateParams.md: Unchanged skipping
2022/10/01 12:29:33 DEBUG : Local file system at /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/: Waiting for transfers to finish
2022/10/01 12:29:33 INFO  : There was nothing to transfer
2022/10/01 12:29:33 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Checks:                 8 / 8, 100%
Elapsed time:         0.5s

2022/10/01 12:29:33 DEBUG : 12 go routines active
2022/10/01 12:29:33 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/template/: Closing 1 unused connections

file:

david@alpe:~/Bin/rclone-v1.59.2-linux-amd64$ ./rclone copy -vv orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/
2022/10/01 12:30:57 DEBUG : rclone: Version "v1.59.2" starting with parameters ["./rclone" "copy" "-vv" "orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md" "/home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/"]
2022/10/01 12:30:57 DEBUG : Creating backend with remote "orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md"
2022/10/01 12:30:57 DEBUG : Using config file from "/home/david/.config/rclone/rclone.conf"
2022/10/01 12:30:57 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md: New connection 10.51.175.1:42648->10.51.175.82:22 to "SSH-2.0-OpenSSH_8.9p1 Ubuntu-3"
2022/10/01 12:30:58 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md: Shell type "unix" from config
2022/10/01 12:30:58 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md: Relative path resolved to "/home/ubuntu/github/ltng-documentation/template/disclaimer.md"
2022/10/01 12:30:58 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md: Using root directory "/home/ubuntu/github/ltng-documentation/template"
2022/10/01 12:30:58 DEBUG : fs cache: adding new entry for parent of "orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md", "orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/template"
2022/10/01 12:30:58 DEBUG : Creating backend with remote "/home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/"
2022/10/01 12:30:58 DEBUG : disclaimer.md: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/10/01 12:30:58 DEBUG : disclaimer.md: Unchanged skipping
2022/10/01 12:30:58 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Checks:                 1 / 1, 100%
Elapsed time:         0.5s

2022/10/01 12:30:58 DEBUG : 11 go routines active
2022/10/01 12:30:58 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md: Closing 1 unused connections

Can you share the full log of the actual error and not just the snippet as I need to see the full command in the debug log and a few lines above the error?

and the output from these commands:

./rclone lsl orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md
./rclone lsl /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md
./rclone hashsum md5 orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md
./rclone hashsum md5 /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md

executed immediately before and after the failing sync documented in the full debug log.

Thinking a bit more about it, perhaps it is a concurrency issue on the server.

Are you able to reproduce if adding --checkers=1 --dry-run to your sync command?

Here is the full output from the command. This is a copy of the parent directory MFS that holds the sub-directory template. I ran it twice in succession so that you can see that it copies the files in the sub-directory repeatedly.

first run:

david@alpe:~/Bin/rclone-v1.59.2-linux-amd64$ ./rclone copy -vv orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/ /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/
2022/10/01 13:42:56 DEBUG : rclone: Version "v1.59.2" starting with parameters ["./rclone" "copy" "-vv" "orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/" "/home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/"]
2022/10/01 13:42:56 DEBUG : Creating backend with remote "orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/"
2022/10/01 13:42:56 DEBUG : Using config file from "/home/david/.config/rclone/rclone.conf"
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: New connection 10.51.175.1:54506->10.51.175.82:22 to "SSH-2.0-OpenSSH_8.9p1 Ubuntu-3"
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell type "unix" from config
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Relative path resolved to "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS"
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Using root directory "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS"
2022/10/01 13:42:56 DEBUG : Creating backend with remote "/home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/"
2022/10/01 13:42:56 NOTICE: template: Can't follow symlink without -L/--copy-links
2022/10/01 13:42:56 DEBUG : makefile: Size and modification time the same (differ by 994.645823ms, within tolerance 1s)
2022/10/01 13:42:56 DEBUG : .gitignore: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/10/01 13:42:56 DEBUG : makefile: Unchanged skipping
2022/10/01 13:42:56 DEBUG : .gitignore: Unchanged skipping
2022/10/01 13:42:56 DEBUG : img/warning_26a0-fe0f.png: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/10/01 13:42:56 DEBUG : img/warning_26a0-fe0f.png: Unchanged skipping
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: New connection 10.51.175.1:54512->10.51.175.82:22 to "SSH-2.0-OpenSSH_8.9p1 Ubuntu-3"
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell path "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/.gitignore"
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Running remote command: /usr/bin/md5sum /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/.gitignore
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell path "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/background.pdf"
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Running remote command: /usr/bin/md5sum /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/background.pdf
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Remote command result: 93549a0f8f18bd1022b839c0484fd7cf  /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/.gitignore
2022/10/01 13:42:56 DEBUG : template/.gitignore: Parsed hash: 93549a0f8f18bd1022b839c0484fd7cf
2022/10/01 13:42:56 DEBUG : template/.gitignore: md5 = 93549a0f8f18bd1022b839c0484fd7cf OK
2022/10/01 13:42:56 INFO  : template/.gitignore: Copied (new)
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Remote command result: 7f70c0cdc431830a5dd611b58dd93491  /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/background.pdf
2022/10/01 13:42:56 DEBUG : template/background.pdf: Parsed hash: 7f70c0cdc431830a5dd611b58dd93491
2022/10/01 13:42:56 DEBUG : template/background.pdf: md5 = 7f70c0cdc431830a5dd611b58dd93491 OK
2022/10/01 13:42:56 INFO  : template/background.pdf: Copied (new)
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell path "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/eisvogel.tex"
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Running remote command: /usr/bin/md5sum /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/eisvogel.tex
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell path "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/finalpage_A3.pdf"
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Running remote command: /usr/bin/md5sum /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/finalpage_A3.pdf
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Remote command result: 485753eebd570199fcc0c70fa2225054  /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/eisvogel.tex
2022/10/01 13:42:56 DEBUG : template/eisvogel.tex: Parsed hash: 485753eebd570199fcc0c70fa2225054
2022/10/01 13:42:56 DEBUG : template/eisvogel.tex: md5 = 485753eebd570199fcc0c70fa2225054 OK
2022/10/01 13:42:56 INFO  : template/eisvogel.tex: Copied (new)
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Remote command result: 1f7a18651bef9f699261b627ed03a030  /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/finalpage_A3.pdf
2022/10/01 13:42:56 DEBUG : template/finalpage_A3.pdf: Parsed hash: 1f7a18651bef9f699261b627ed03a030
2022/10/01 13:42:56 DEBUG : template/finalpage_A3.pdf: md5 = 1f7a18651bef9f699261b627ed03a030 OK
2022/10/01 13:42:56 INFO  : template/finalpage_A3.pdf: Copied (new)
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell path "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/finalpage_A4.pdf"
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Running remote command: /usr/bin/md5sum /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/finalpage_A4.pdf
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell path "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/templateParams.md"
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Running remote command: /usr/bin/md5sum /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/templateParams.md
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Remote command result: affded0ebdd0883d1a0305235ed0d4a1  /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/templateParams.md
2022/10/01 13:42:56 DEBUG : template/templateParams.md: Parsed hash: affded0ebdd0883d1a0305235ed0d4a1
2022/10/01 13:42:56 DEBUG : template/templateParams.md: md5 = affded0ebdd0883d1a0305235ed0d4a1 OK
2022/10/01 13:42:56 INFO  : template/templateParams.md: Copied (new)
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Remote command result: 9fc7d564358851497822d190c452439c  /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/finalpage_A4.pdf
2022/10/01 13:42:56 DEBUG : template/finalpage_A4.pdf: Parsed hash: 9fc7d564358851497822d190c452439c
2022/10/01 13:42:56 DEBUG : template/finalpage_A4.pdf: md5 = 9fc7d564358851497822d190c452439c OK
2022/10/01 13:42:56 INFO  : template/finalpage_A4.pdf: Copied (new)
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: New connection 10.51.175.1:54514->10.51.175.82:22 to "SSH-2.0-OpenSSH_8.9p1 Ubuntu-3"
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: New connection 10.51.175.1:54530->10.51.175.82:22 to "SSH-2.0-OpenSSH_8.9p1 Ubuntu-3"
2022/10/01 13:42:56 DEBUG : docs/1-Coverpage.md: Size and modification time the same (differ by 685.717159ms, within tolerance 1s)
2022/10/01 13:42:56 DEBUG : docs/1-Coverpage.md: Unchanged skipping
2022/10/01 13:42:56 DEBUG : docs/2-Version.md: Size and modification time the same (differ by 685.717159ms, within tolerance 1s)
2022/10/01 13:42:56 DEBUG : docs/PFDT.md: Size and modification time the same (differ by 994.645823ms, within tolerance 1s)
2022/10/01 13:42:56 DEBUG : Local file system at /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/: Waiting for checks to finish
2022/10/01 13:42:56 DEBUG : docs/PFDT.md: Unchanged skipping
2022/10/01 13:42:56 DEBUG : docs/2-Version.md: Unchanged skipping
2022/10/01 13:42:56 DEBUG : Local file system at /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/: Waiting for transfers to finish
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: New connection 10.51.175.1:54532->10.51.175.82:22 to "SSH-2.0-OpenSSH_8.9p1 Ubuntu-3"
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell path "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/cover.pdf"
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Running remote command: /usr/bin/md5sum /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/cover.pdf
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Remote command result: b5d4ec500bf0b90516b7d02920ca9716  /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/cover.pdf
2022/10/01 13:42:56 DEBUG : template/cover.pdf: Parsed hash: b5d4ec500bf0b90516b7d02920ca9716
2022/10/01 13:42:56 DEBUG : template/cover.pdf: md5 = b5d4ec500bf0b90516b7d02920ca9716 OK
2022/10/01 13:42:56 INFO  : template/cover.pdf: Copied (new)
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell path "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md"
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Running remote command: /usr/bin/md5sum /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Remote command result: 34a3da2c76628872ceafaa9a6307df42  /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md
2022/10/01 13:42:56 DEBUG : template/disclaimer.md: Parsed hash: 34a3da2c76628872ceafaa9a6307df42
2022/10/01 13:42:56 DEBUG : template/disclaimer.md: md5 = 34a3da2c76628872ceafaa9a6307df42 OK
2022/10/01 13:42:56 INFO  : template/disclaimer.md: Copied (new)
2022/10/01 13:42:56 INFO  : 
Transferred:   	  788.268 KiB / 788.268 KiB, 100%, 0 B/s, ETA -
Checks:                 6 / 6, 100%
Transferred:            8 / 8, 100%
Elapsed time:         0.7s

2022/10/01 13:42:56 DEBUG : 48 go routines active
2022/10/01 13:42:56 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Closing 5 unused connections

second run:

david@alpe:~/Bin/rclone-v1.59.2-linux-amd64$ ./rclone copy -vv orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/ /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/
2022/10/01 13:43:46 DEBUG : rclone: Version "v1.59.2" starting with parameters ["./rclone" "copy" "-vv" "orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/" "/home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/"]
2022/10/01 13:43:46 DEBUG : Creating backend with remote "orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/"
2022/10/01 13:43:46 DEBUG : Using config file from "/home/david/.config/rclone/rclone.conf"
2022/10/01 13:43:46 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: New connection 10.51.175.1:42844->10.51.175.82:22 to "SSH-2.0-OpenSSH_8.9p1 Ubuntu-3"
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell type "unix" from config
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Relative path resolved to "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS"
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Using root directory "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS"
2022/10/01 13:43:47 DEBUG : Creating backend with remote "/home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/"
2022/10/01 13:43:47 NOTICE: template: Can't follow symlink without -L/--copy-links
2022/10/01 13:43:47 DEBUG : .gitignore: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/10/01 13:43:47 DEBUG : .gitignore: Unchanged skipping
2022/10/01 13:43:47 DEBUG : makefile: Size and modification time the same (differ by 994.645823ms, within tolerance 1s)
2022/10/01 13:43:47 DEBUG : makefile: Unchanged skipping
2022/10/01 13:43:47 DEBUG : docs/1-Coverpage.md: Size and modification time the same (differ by 685.717159ms, within tolerance 1s)
2022/10/01 13:43:47 DEBUG : docs/1-Coverpage.md: Unchanged skipping
2022/10/01 13:43:47 DEBUG : docs/2-Version.md: Size and modification time the same (differ by 685.717159ms, within tolerance 1s)
2022/10/01 13:43:47 DEBUG : docs/2-Version.md: Unchanged skipping
2022/10/01 13:43:47 DEBUG : docs/PFDT.md: Size and modification time the same (differ by 994.645823ms, within tolerance 1s)
2022/10/01 13:43:47 DEBUG : docs/PFDT.md: Unchanged skipping
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: New connection 10.51.175.1:42856->10.51.175.82:22 to "SSH-2.0-OpenSSH_8.9p1 Ubuntu-3"
2022/10/01 13:43:47 DEBUG : img/warning_26a0-fe0f.png: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/10/01 13:43:47 DEBUG : img/warning_26a0-fe0f.png: Unchanged skipping
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: New connection 10.51.175.1:42860->10.51.175.82:22 to "SSH-2.0-OpenSSH_8.9p1 Ubuntu-3"
2022/10/01 13:43:47 DEBUG : Local file system at /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/: Waiting for checks to finish
2022/10/01 13:43:47 DEBUG : Local file system at /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/: Waiting for transfers to finish
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell path "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md"
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Running remote command: /usr/bin/md5sum /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell path "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/.gitignore"
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Running remote command: /usr/bin/md5sum /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/.gitignore
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell path "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/cover.pdf"
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Running remote command: /usr/bin/md5sum /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/cover.pdf
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Remote command result: 93549a0f8f18bd1022b839c0484fd7cf  /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/.gitignore
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Remote command result: 34a3da2c76628872ceafaa9a6307df42  /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md
2022/10/01 13:43:47 DEBUG : template/disclaimer.md: Parsed hash: 34a3da2c76628872ceafaa9a6307df42
2022/10/01 13:43:47 DEBUG : template/disclaimer.md: md5 = 34a3da2c76628872ceafaa9a6307df42 OK
2022/10/01 13:43:47 INFO  : template/disclaimer.md: Copied (new)
2022/10/01 13:43:47 DEBUG : template/.gitignore: Parsed hash: 93549a0f8f18bd1022b839c0484fd7cf
2022/10/01 13:43:47 DEBUG : template/.gitignore: md5 = 93549a0f8f18bd1022b839c0484fd7cf OK
2022/10/01 13:43:47 INFO  : template/.gitignore: Copied (new)
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Remote command result: b5d4ec500bf0b90516b7d02920ca9716  /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/cover.pdf
2022/10/01 13:43:47 DEBUG : template/cover.pdf: Parsed hash: b5d4ec500bf0b90516b7d02920ca9716
2022/10/01 13:43:47 DEBUG : template/cover.pdf: md5 = b5d4ec500bf0b90516b7d02920ca9716 OK
2022/10/01 13:43:47 INFO  : template/cover.pdf: Copied (new)
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell path "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/eisvogel.tex"
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Running remote command: /usr/bin/md5sum /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/eisvogel.tex
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell path "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/finalpage_A3.pdf"
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Running remote command: /usr/bin/md5sum /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/finalpage_A3.pdf
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell path "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/finalpage_A4.pdf"
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Running remote command: /usr/bin/md5sum /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/finalpage_A4.pdf
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Remote command result: 485753eebd570199fcc0c70fa2225054  /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/eisvogel.tex
2022/10/01 13:43:47 DEBUG : template/eisvogel.tex: Parsed hash: 485753eebd570199fcc0c70fa2225054
2022/10/01 13:43:47 DEBUG : template/eisvogel.tex: md5 = 485753eebd570199fcc0c70fa2225054 OK
2022/10/01 13:43:47 INFO  : template/eisvogel.tex: Copied (new)
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell path "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/templateParams.md"
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Running remote command: /usr/bin/md5sum /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/templateParams.md
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Remote command result: 1f7a18651bef9f699261b627ed03a030  /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/finalpage_A3.pdf
2022/10/01 13:43:47 DEBUG : template/finalpage_A3.pdf: Parsed hash: 1f7a18651bef9f699261b627ed03a030
2022/10/01 13:43:47 DEBUG : template/finalpage_A3.pdf: md5 = 1f7a18651bef9f699261b627ed03a030 OK
2022/10/01 13:43:47 INFO  : template/finalpage_A3.pdf: Copied (new)
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Remote command result: 9fc7d564358851497822d190c452439c  /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/finalpage_A4.pdf
2022/10/01 13:43:47 DEBUG : template/finalpage_A4.pdf: Parsed hash: 9fc7d564358851497822d190c452439c
2022/10/01 13:43:47 DEBUG : template/finalpage_A4.pdf: md5 = 9fc7d564358851497822d190c452439c OK
2022/10/01 13:43:47 INFO  : template/finalpage_A4.pdf: Copied (new)
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Remote command result: affded0ebdd0883d1a0305235ed0d4a1  /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/templateParams.md
2022/10/01 13:43:47 DEBUG : template/templateParams.md: Parsed hash: affded0ebdd0883d1a0305235ed0d4a1
2022/10/01 13:43:47 DEBUG : template/templateParams.md: md5 = affded0ebdd0883d1a0305235ed0d4a1 OK
2022/10/01 13:43:47 INFO  : template/templateParams.md: Copied (new)
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: New connection 10.51.175.1:42862->10.51.175.82:22 to "SSH-2.0-OpenSSH_8.9p1 Ubuntu-3"
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell path "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/background.pdf"
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Running remote command: /usr/bin/md5sum /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/background.pdf
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Remote command result: 7f70c0cdc431830a5dd611b58dd93491  /home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/background.pdf
2022/10/01 13:43:47 DEBUG : template/background.pdf: Parsed hash: 7f70c0cdc431830a5dd611b58dd93491
2022/10/01 13:43:47 DEBUG : template/background.pdf: md5 = 7f70c0cdc431830a5dd611b58dd93491 OK
2022/10/01 13:43:47 INFO  : template/background.pdf: Copied (new)
2022/10/01 13:43:47 INFO  : 
Transferred:   	  788.268 KiB / 788.268 KiB, 100%, 0 B/s, ETA -
Checks:                 6 / 6, 100%
Transferred:            8 / 8, 100%
Elapsed time:         0.7s

2022/10/01 13:43:47 DEBUG : 39 go routines active
2022/10/01 13:43:47 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Closing 4 unused connections

before:

david@alpe:~/Bin/rclone-v1.59.2-linux-amd64$ ./rclone lsl orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md
      994 2022-08-15 10:07:21.000000000 disclaimer.md
david@alpe:~/Bin/rclone-v1.59.2-linux-amd64$ ./rclone lsl /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md
      994 2022-08-15 10:07:21.000000000 disclaimer.md
david@alpe:~/Bin/rclone-v1.59.2-linux-amd64$ ./rclone hashsum md5 orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md
34a3da2c76628872ceafaa9a6307df42  disclaimer.md
david@alpe:~/Bin/rclone-v1.59.2-linux-amd64$ ./rclone hashsum md5 /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md
34a3da2c76628872ceafaa9a6307df42  disclaimer.md

after:

david@alpe:~/Bin/rclone-v1.59.2-linux-amd64$ ./rclone lsl orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md
      994 2022-08-15 10:07:21.000000000 disclaimer.md
david@alpe:~/Bin/rclone-v1.59.2-linux-amd64$ ./rclone lsl /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md
      994 2022-08-15 10:07:21.000000000 disclaimer.md
david@alpe:~/Bin/rclone-v1.59.2-linux-amd64$ ./rclone hashsum md5 orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md
34a3da2c76628872ceafaa9a6307df42  disclaimer.md
david@alpe:~/Bin/rclone-v1.59.2-linux-amd64$ ./rclone hashsum md5 /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/template/disclaimer.md
34a3da2c76628872ceafaa9a6307df42  disclaimer.md

Yes I suspected concurrency issues too, and had already tried both --sftp-concurrency=1 and --transfers=1 to no avail. Also checkers=1 seems to make no difference, the files in the parent directory are correctly identified as unchanged but the files in the sub-directory are detected as needing to be copied:

david@alpe:~/Bin/rclone-v1.59.2-linux-amd64$ ./rclone copy --checkers=1 --dry-run -vv orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/ /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/
2022/10/01 13:50:42 DEBUG : rclone: Version "v1.59.2" starting with parameters ["./rclone" "copy" "--checkers=1" "--dry-run" "-vv" "orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/" "/home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/"]
2022/10/01 13:50:42 DEBUG : Creating backend with remote "orchestra-jammy:github/ltng-documentation/FunctionalDescription/MFS/"
2022/10/01 13:50:42 DEBUG : Using config file from "/home/david/.config/rclone/rclone.conf"
2022/10/01 13:50:42 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: New connection 10.51.175.1:39846->10.51.175.82:22 to "SSH-2.0-OpenSSH_8.9p1 Ubuntu-3"
2022/10/01 13:50:42 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Shell type "unix" from config
2022/10/01 13:50:42 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Relative path resolved to "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS"
2022/10/01 13:50:42 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Using root directory "/home/ubuntu/github/ltng-documentation/FunctionalDescription/MFS"
2022/10/01 13:50:42 DEBUG : Creating backend with remote "/home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/"
2022/10/01 13:50:42 NOTICE: template: Can't follow symlink without -L/--copy-links
2022/10/01 13:50:42 DEBUG : .gitignore: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/10/01 13:50:42 DEBUG : .gitignore: Unchanged skipping
2022/10/01 13:50:42 DEBUG : makefile: Size and modification time the same (differ by 994.645823ms, within tolerance 1s)
2022/10/01 13:50:42 DEBUG : makefile: Unchanged skipping
2022/10/01 13:50:42 NOTICE: template/.gitignore: Skipped copy as --dry-run is set (size 6)
2022/10/01 13:50:42 NOTICE: template/disclaimer.md: Skipped copy as --dry-run is set (size 994)
2022/10/01 13:50:42 NOTICE: template/background.pdf: Skipped copy as --dry-run is set (size 72.930Ki)
2022/10/01 13:50:42 NOTICE: template/finalpage_A3.pdf: Skipped copy as --dry-run is set (size 446.823Ki)
2022/10/01 13:50:42 NOTICE: template/cover.pdf: Skipped copy as --dry-run is set (size 92.469Ki)
2022/10/01 13:50:42 NOTICE: template/templateParams.md: Skipped copy as --dry-run is set (size 458)
2022/10/01 13:50:42 NOTICE: template/finalpage_A4.pdf: Skipped copy as --dry-run is set (size 141.626Ki)
2022/10/01 13:50:42 NOTICE: template/eisvogel.tex: Skipped copy as --dry-run is set (size 32.996Ki)
2022/10/01 13:50:42 DEBUG : docs/1-Coverpage.md: Size and modification time the same (differ by 685.717159ms, within tolerance 1s)
2022/10/01 13:50:42 DEBUG : docs/1-Coverpage.md: Unchanged skipping
2022/10/01 13:50:42 DEBUG : docs/2-Version.md: Size and modification time the same (differ by 685.717159ms, within tolerance 1s)
2022/10/01 13:50:42 DEBUG : docs/2-Version.md: Unchanged skipping
2022/10/01 13:50:42 DEBUG : docs/PFDT.md: Size and modification time the same (differ by 994.645823ms, within tolerance 1s)
2022/10/01 13:50:42 DEBUG : docs/PFDT.md: Unchanged skipping
2022/10/01 13:50:42 DEBUG : img/warning_26a0-fe0f.png: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/10/01 13:50:42 DEBUG : Local file system at /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/: Waiting for checks to finish
2022/10/01 13:50:42 DEBUG : img/warning_26a0-fe0f.png: Unchanged skipping
2022/10/01 13:50:42 DEBUG : Local file system at /home/david/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS/: Waiting for transfers to finish
2022/10/01 13:50:42 NOTICE: 
Transferred:   	  788.268 KiB / 788.268 KiB, 100%, 0 B/s, ETA -
Checks:                 6 / 6, 100%
Transferred:            8 / 8, 100%
Elapsed time:         0.3s

2022/10/01 13:50:42 DEBUG : 13 go routines active
2022/10/01 13:50:42 DEBUG : sftp://ubuntu@10.51.175.82:22/github/ltng-documentation/FunctionalDescription/MFS/: Closing 1 unused connections

Is the template a link to somewhere else?

1 Like

** UPDATE **

I just spotted this in the log and I'm sure it is at the root of the behaviour NOTICE: template: Can't follow symlink without -L/--copy-links. The directory whose contents gets copied repeatedly is a symlink, which I didn't realise before:

from the remote:

ubuntu@orchestra-jammy:~/github/ltng-documentation/FunctionalDescription/MFS$ ls -l
total 8
drwxrwxr-x 2 ubuntu ubuntu   5 Aug 19 21:09 docs
drwxrwxr-x 2 ubuntu ubuntu   3 Aug 15 09:07 img
-rw-rw-r-- 1 ubuntu ubuntu 608 Aug 19 21:09 makefile
lrwxrwxrwx 1 ubuntu ubuntu  15 Aug 15 09:07 template -> ../../template/

The SFTP storage driver copies symlinks by default, and I have not specfied --sftp-skip-links. So it copies the link but then copies it again every time. Note that it is the link itself that is copied, not the referenced file (as with --copy-links), nor a pseudo-link (as with --links on some remote types).

After the copy the local directory has the symbolic link (which functions correctly):

david@alpe:~/Bin/rclone-v1.59.2-linux-amd64$ ls -l ~/Temp/lxc-backups/orchestra-jammy/ubuntu/github/ltng-documentation/FunctionalDescription/MFS
total 13
-rw-rw-r-- 1 david david 994 Aug 15 10:07 disclaimer.md
drwxrwxr-x 2 david david   5 Aug 19 22:09 docs
drwxrwxr-x 2 david david   3 Aug 15 10:07 img
-rw-rw-r-- 1 david david 608 Aug 19 22:09 makefile
lrwxrwxrwx 1 david david  15 Aug 15 10:07 template -> ../../template/

Yes template is a symbolic link. And sure enough using --copy-links solves the problem: the link is followed, the file found, and not copied.

The thing which surprises me is that even with --copy-links it is the link itself that is copied. I thought that "following" the link meant that the linked-to file target would be copied. So I have a complete solution to the issue I reported.

Unfortunately I'm not sure it helps, since my actual goal is to copy from the SFTP remote, direct to a Swift remote (Memstore), without creating a local copy, and I'm not sure that supports symbolic links, but that is another topic.

I generally don't consider rclone a backup tool as I wouldn't use as it as such but many people do.

That still seems a bit strange as I thought the file was in the linked area but from that output it is not.

I tried a quick test and I can't quite make the same thing happen or I'm not simulating what you have.

felix@gemini:~$ ls -al test
total 8
drwxrwxr-x 1 felix felix  26 Oct  1 11:03 .
drwxr-xr-x 1 felix felix 296 Oct  1 11:03 ..
-rw-r--r-- 1 felix felix 327 Oct  1 11:03 hosts
lrwxrwxrwx 1 felix felix  11 Oct  1 11:03 template -> ../template
felix@gemini:~$ ls -al test2
total 4
drwxrwxr-x 1 felix felix  10 Oct  1 11:03 .
drwxr-xr-x 1 felix felix 296 Oct  1 11:03 ..
-rw-rw-r-- 1 felix felix 327 Oct  1 11:03 hosts

and I just did

felix@gemini:~$ rclone copy SFTP:/home/felix/test SFTP:/home/felix/test2 -vv
2022/10/01 11:04:36 DEBUG : Setting --config "/opt/rclone/rclone.conf" from environment variable RCLONE_CONFIG="/opt/rclone/rclone.conf"
2022/10/01 11:04:36 DEBUG : rclone: Version "v1.59.2" starting with parameters ["rclone" "copy" "SFTP:/home/felix/test" "SFTP:/home/felix/test2" "-vv"]
2022/10/01 11:04:36 DEBUG : Creating backend with remote "SFTP:/home/felix/test"
2022/10/01 11:04:36 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test: New connection 127.0.0.1:40564->127.0.0.1:22 to "SSH-2.0-OpenSSH_8.9p1 Ubuntu-3"
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test: Running shell type detection remote command: echo ${ShellId}%ComSpec%
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test: Remote command result: %ComSpec%
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test: Shell type "unix" detected (set option shell_type to override)
2022/10/01 11:04:36 DEBUG : Saving config "shell_type" in section "SFTP" of the config file
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test: Using root directory "/home/felix/test"
2022/10/01 11:04:36 DEBUG : Creating backend with remote "SFTP:/home/felix/test2"
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test2: New connection 127.0.0.1:40576->127.0.0.1:22 to "SSH-2.0-OpenSSH_8.9p1 Ubuntu-3"
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test2: Shell type "unix" from config
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test2: Using root directory "/home/felix/test2"
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test: Checking default md5 hash commands
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test: Running remote command: md5sum
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test: Remote command result: d41d8cd98f00b204e9800998ecf8427e  -
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test: Hash command accepted
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test: Checking default sha1 hash commands
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test: Running remote command: sha1sum
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test: Remote command result: da39a3ee5e6b4b0d3255bfef95601890afd80709  -
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test: Hash command accepted
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test: Setting hash command for md5 to "md5sum" (set sha1sum_command to override)
2022/10/01 11:04:36 DEBUG : Saving config "md5sum_command" in section "SFTP" of the config file
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test: Setting hash command for sha1 to "sha1sum" (set md5sum_command to override)
2022/10/01 11:04:36 DEBUG : Saving config "sha1sum_command" in section "SFTP" of the config file
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test2: Checking default md5 hash commands
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test2: Running remote command: md5sum
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test2: Remote command result: d41d8cd98f00b204e9800998ecf8427e  -
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test2: Hash command accepted
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test2: Checking default sha1 hash commands
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test2: Running remote command: sha1sum
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test2: Remote command result: da39a3ee5e6b4b0d3255bfef95601890afd80709  -
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test2: Hash command accepted
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test2: Setting hash command for md5 to "md5sum" (set sha1sum_command to override)
2022/10/01 11:04:36 DEBUG : Saving config "md5sum_command" in section "SFTP" of the config file
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test2: Setting hash command for sha1 to "sha1sum" (set md5sum_command to override)
2022/10/01 11:04:36 DEBUG : Saving config "sha1sum_command" in section "SFTP" of the config file
2022/10/01 11:04:36 DEBUG : hosts: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/10/01 11:04:36 DEBUG : hosts: Unchanged skipping
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test2: Waiting for checks to finish
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test2: Waiting for transfers to finish
2022/10/01 11:04:36 INFO  : There was nothing to transfer
2022/10/01 11:04:36 INFO  :
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Checks:                 1 / 1, 100%
Elapsed time:         0.6s

2022/10/01 11:04:36 DEBUG : 21 go routines active
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test: Closing 1 unused connections
2022/10/01 11:04:36 DEBUG : sftp://felix@localhost:22//home/felix/test2: Closing 1 unused connections

It's not clear whether in your test the target of the symbolic link, i.e. ../template, exists or not, and if it exists whether it is a regular file or a directory, and if a directory whether it contains files. In my example the target of the link does exist, is a directory and does contain files.

I suspect there is a subtle issue here, but I'd have to do more tests to confirm it. What I suspect is happening is that the rclone SFTP protocol is copying links (I had not set --sftp-skip-links, so this is expected behaviour). This copies the link itself, it does not dereference it and copy the linked object. The next rclone copy operation should therefore just be testing to see if the link itself exists and is unchanged, but I suspect it is dereferencing the link. In my case the link was to a directory, and since copying a directory means copying its contents, the check is looking for the contents to be there. They are not, so it 'copies' them again, but in fact all that gets copied again is the link itself. The consequence is that however many times it is performed, rclone copy copies the link again.

The 'solution' of using --copy-links changes the behaviour. With this flag the link is deferenced and the target of the link is copied, in my case the contents of the linked directory, and subsequent checks find the files. But of course this is not the same as copying the link: I've got duplicates of the file in two different directories now.

I'm coming to the same conclusion as you about using rclone for backup. I've tried a test restore, and some files are missing, haven't been copied to the cloud storage. I'll raise a separate report for that. Then maybe I'll checkout restic, a backup tool (also written in go) that can use rclone as a driver for cloud storage. That seems a good separation of responsibilities: a backup tool to package up the blob and rclone to store it remotely.

I use rclone as a backup tool for the things I have in my home/user folder (including git repos) - and find it perfect for that because I copy to targets with built in versioning and snapshots. The advantage is easy no fuss access, browsing and restore.

I recently had a crash due to bug in a BIOS update and have restored from my backup without any issues.

Having say that, I would be very cautious with filesystems containing links and wouldn't use rclone for system files or system backups.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.