I'm using rclone to mount a Cloudflare R2 bucket as the mail directory for my dovecot emailserver. However, I am running into errors from dovecot regarding "duplicate transaction logs".
I have read through the dovecot docs and it seems this issue is related to using virtual filesystems like NFS and fuse and dovecot's feature of "locking" files to prevent multiple sources from writing to a file at the same time.
I am able to sync email to the rclone mounted mail directory but when I access the mail via IMAP I get the following error logs. I am thinking this has something to do with my rclone mount config. Can you please review my config and let me know if there is something I can adjust? To be clear, there are no errors from rclone.
The rclone fuse mount doesn't support locking (as in flock or fcntl). Rclone could possibly implement it but it would be unreliable as cloud storage systems don't support locking themselves so rclone would have to emulate it.
Thanks, Nick. That's what I suspected too. So Dovecot has a third option for locking called dotlock and with that enabled, I was able to create the Dovecot mail folders on the rclone-mounted bucket. With either flock or fcntl I wasn't able to create the mail folders.
Do you have any insight into why Dovecot would be seeing "duplicate transaction logs" with the fuse mount? Is there anything with the VFS cache settings I can try to adjust to resolve this?
Also, to help me understand better, do you know, to the linux kernel, how similar are NFS mounts versus fuse mounts? There's a lot of documentation about mail storage on NFS but not much on fuse mounts. Dovecot says they also support mail storage on GlusterFS, so I'm going to explore that more.
Could I try to implement in a fork of rclone the emulation of locking? Do you have any recommendations on how I could start with this?
dotlock just creates files doesn't it which will be fine with rclone mount.
No - does it do that with dotlock? I would take a squint in the source code to see if that give you any ideas.
If there was a scale from 10 (behaves like a perfectly normal file system) to 1 (does weird stuff) then NFS is probably about 8, rclone mount with --vfs-cache-mode writes/full is about 7 and rclone mount without vfs cache mode is about 5.
I think GluserFS will act more like a proper file system, more at the NFS levels of proper so may be a better choice for this.
The first thing to do would be to see how to implement locking in a FUSE file system (I don't know the answer to that).
Then we'd have to work out how to implement that on the cloud storage backend. Creating files is all we can do so it would have to work like that.
I think you'd end up implementing something rather like dotlock in the end!