Rclone mount caching wrong or old contents for large files?

What is the problem you are having with rclone?

Under some circumstances rclone mount appears to be not showing the right file contents. I want to know if it might be some cache setting I am using, or an edge case bug. "Some circumstances" might be large database files and an rclone mount persisting beyond the laptop being suspended.

I am using rclone mount from a Linux laptop ("local") to an SFTP backend on a Linux container ("remote"). My use case is a shared document repository using Recoll to index the files, as it has the capability to use external indexes like this. I run recollindex on the remote, and it creates a large XapianDB in its ~/.recoll. When I point Recoll on my local machine at the rclone-mounted index, I get an error that the file is corrupt.

To debug, I ran a sha1sum on the same files both remotely (direct) and locally (via rclone mount). The digest for some of the files is different (some are the same):

[tony@local] ➜  ~ sha1sum /home/tony/mount/remote/.recoll/xapiandb/*
f3006de9b32150618e956736bce80752d85c245d  /home/tony/mount/remote/.recoll/xapiandb/docdata.glass
da39a3ee5e6b4b0d3255bfef95601890afd80709  /home/tony/mount/remote/.recoll/xapiandb/flintlock
e7fb27fec48e0a04b9203e77d4956975328e1ee4  /home/tony/mount/remote/.recoll/xapiandb/iamglass
5d6afbc3416c483aa9bfd019d9e88234a0bf4a55  /home/tony/mount/remote/.recoll/xapiandb/position.glass
6ea67148123b9d6594fb859c5c572def16547ef6  /home/tony/mount/remote/.recoll/xapiandb/postlist.glass
16fea0b41c551e98ad33c40080a35440dca3dc4c  /home/tony/mount/remote/.recoll/xapiandb/synonym.glass
8d1fda69e14f1d96ab04ddd9dcddfcc2d4b59275  /home/tony/mount/remote/.recoll/xapiandb/termlist.glass


[tony@remote] ➜  ~ sha1sum ~/.recoll/xapiandb/*
6cda653125b9d6afeaefe65f42992f5cf37291fb  /home/tony/.recoll/xapiandb/docdata.glass
da39a3ee5e6b4b0d3255bfef95601890afd80709  /home/tony/.recoll/xapiandb/flintlock
e7fb27fec48e0a04b9203e77d4956975328e1ee4  /home/tony/.recoll/xapiandb/iamglass
42ea538deb3d27343bc326c78d90a097ade90ee8  /home/tony/.recoll/xapiandb/position.glass
c4a4450b2539e82cfc54065cd07c876a3ddcdf61  /home/tony/.recoll/xapiandb/postlist.glass
16fea0b41c551e98ad33c40080a35440dca3dc4c  /home/tony/.recoll/xapiandb/synonym.glass
580a375507b7c4dbda472d95df2a574c257b4d9e  /home/tony/.recoll/xapiandb/termlist.glass

The files are more than an hour old, and more than an hour since being modified, and so it shouldn't be a short term cache / sync issue. The files are large, up to 35GB for the largest one and 57GB in total in the index directory. It's the large files that have mismatching digests.

A point to note: the local device gets suspended regularly, but that doesn't seem to bother the persistent rclone mount; I can us it to create a new file on the remote just fine right now.

I also just ran a second mount command without terminating the first one, to see what it would report for the sha1sum. It "fixed" the issue in that it reported matching checksums. And then when I repeated the sha1sum command on the original mount, it is now reporting the correct checksums.

That makes this a very intermittent problem that, with a little jolt, seems to fix itself. It also makes it hard to replicate.

Run the command 'rclone version' and share the full output of the command.

rclone v1.75.2 
- os/version: ubuntu 26.04 (64 bit) 
- os/kernel: 7.0.0-42-generic (x86_64) 
- os/type: linux 
- os/arch: amd64 
- go/version: go1.26.9 
- go/linking: static 
- go/tags: none```

Which cloud storage system are you using? (eg Google Drive)

SFTP on a container on a Linux server on the LAN. Using a specific --sftp-ssh command so I can use a ProxyJump command

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone --sftp-ssh "ssh remote.local" mount --vfs-cache-mode full remote:. ~/mount/remote

The rclone config contents with secrets removed.

[remote]
type = sftp
host = XXX
key_file = ~/.ssh/id_ed25519
use_insecure_cipher = true
shell_type = unix
md5sum_command = md5sum
sha1sum_command = sha1sum

A log from the command with the -vv flag

No logs at this point, because this mount command was run a few days ago.