When I mount a sftp backend, then open any file for viewing (e.g. a document) and then perform a rclone sync between that backend and another backend, the file is considered as modified.
$ rclone version rclone v1.55.1 - os/type: linux - os/arch: amd64 - go/version: go1.16.3 - go/linking: static - go/tags: none
Linux Mint 20.1 64-bit
Hetzner Storage Box via sftp
rclone --dry-run -vv --progress sync sftp-backend: b2-backend:
The remote is mounted via systemd:
[Unit] Description=Hetzner Storage Box - encrypted union AssertPathIsDirectory=/mnt/rclone/sftp-crypt After=network-online.target Wants=network-online.target [Service] Type=simple ExecStart=/usr/bin/rclone mount \ --config=/root/.config/rclone/rclone.conf \ --allow-other \ --vfs-cache-mode full \ --checksum \ sftp-crypt:/ /mnt/rclone/sftp-crypt ExecStop=/root/scripts/rclone_unmount.sh /mnt/rclone/sftp-crypt Restart=always RestartSec=10 [Install] WantedBy=default.target
[sftp-backend-01] type = sftp host = <sftp-host> user = <sftp-user> key_file = <sftp-key> port = <sftp-port> md5sum_command = md5 -r sha1sum_command = sha1 -r [sftp-backend-02] type = sftp host = <sftp-host2> user = <sftp-user2> port = <sftp-port2> key_file = <sftp-key2> md5sum_command = md5 -r sha1sum_command = sha1 -r [sftp-union] type = union upstreams = sftp-backend-01:data sftp-backend-02:data action_policy = all create_policy = all search_policy = all [sftp-crypt] type = crypt remote = sftp-union:crypt password = <pwd> password2 = <salt>
2021-06-03 11:41:06 DEBUG : aaaaaaaaaa.csv: Size and modification time the same (differ by 0s, within tolerance 1s) 2021-06-03 11:41:06 DEBUG : aaaaaaaaaa.csv: Unchanged skipping 2021-06-03 11:41:06 DEBUG : bbbbbbbbbb.pdf: Size and modification time the same (differ by 2ms, within tolerance 1s) 2021-06-03 11:41:06 DEBUG : bbbbbbbbbb.pdf: Unchanged skipping 2021-06-03 11:41:06 DEBUG : test.txt: Modification times differ by -669h20m32.351s: 2021-06-03 11:39:10 +0200 CEST, 2021-05-06 12:18:37.649 +0000 UTC 2021-06-03 11:41:06 DEBUG : cccccccccc.pdf: Size and modification time the same (differ by 377ms, within tolerance 1s) 2021-06-03 11:41:06 DEBUG : cccccccccc.pdf: Unchanged skipping 2021-06-03 11:41:06 DEBUG : dddddddddd.pdf: Size and modification time the same (differ by 0s, within tolerance 1s) 2021-06-03 11:41:06 NOTICE: test.txt: Skipped copy as --dry-run is set (size 38) 2021-06-03 11:41:06 DEBUG : dddddddddd.pdf: Unchanged skipping
The file test.txt had a timestamp of 2021-05-06 12:18:37.649 +0000 UTC, but after I opened it in an editor and then closed it again, without saving the file, the timestamp changed to the "current timestamp" (i.e. the time when I opened the file).
I also attached a screenshot of the filemanagers where you can clearly see that the files in both locations had the exact same timestamp right before I opened it.
This only occurs with sftp and AFAIK only with the Hetzner Storage boxes. When using a Backblaze (B2) backend or a Wasabi (S3) backend then the timestamps don't change when opening a file for reading.
I also tested with 2 local linux servers as an encrypted sftp union backend and until now I couldn't reproduce this same behaviour.
So to summarize: an rclone sync works fine, incl correct timestamps, but as soon as I start opening files on the Hetzner storage boxes backend the files will have a different timestamp and therefore will be synced (while they shouldn't because they didn't actually change at all)