Crypt remote / delayed saving of SQLite database

Help and Support
1

What is the problem you are having with rclone?

Hi everyone,
I'm using a mounted (local) crypt remote which works well, but I've noticed that an SQLite database it only updated on the mounted/decrypted folder, while the "source" encrypted file only changes when the database is closed. This represents a significant risk of data loss e.g. if an unexpected reboot occurs before that.
Is there a way to force rclone to update the encrypted file every time the mounted/decrypted one is changed?
Thanks!
Alex

Run the command 'rclone version' and share the full output of the command.

rclone v1.69.1

  • os/version: debian 12.9 (64 bit)
  • os/kernel: 6.6.74+rpt-rpi-v8 (aarch64)
  • os/type: linux
  • os/arch: arm64 (ARMv8 compatible)
  • go/version: go1.24.0
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

local storage (with crypt)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount --allow-other --vfs-cache-mode writes crypt:paperless-ngx /home/dietpi/paperless-ngx

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[crypt]
type = crypt
remote = rpi:/home/dietpi/_crypt
password = XXX

[rpi]
type = local

A log from the command that you were trying to run with the -vv flag

2025/02/16 12:53:29 DEBUG : rclone: Version "v1.69.1" starting with parameters ["rclone" "mount" "--allow-other" "--vfs-cache-mode" "writes" "crypt:paperless-ngx" "/home/dietpi/paperless-ngx" "-vv"]
2025/02/16 12:53:29 DEBUG : Creating backend with remote "crypt:paperless-ngx"
2025/02/16 12:53:29 DEBUG : Using config file from "/home/dietpi/.config/rclone/rclone.conf"
2025/02/16 12:53:30 DEBUG : Creating backend with remote "rpi:/home/dietpi/_crypt/ir15meq97vs8r8qcnmaa7130ao"
2025/02/16 12:53:30 INFO  : Encrypted drive 'crypt:paperless-ngx': poll-interval is not supported by this remote
2025/02/16 12:53:30 DEBUG : Encrypted drive 'crypt:paperless-ngx': vfs cache: root is "/home/dietpi/.cache/rclone"
2025/02/16 12:53:30 DEBUG : Encrypted drive 'crypt:paperless-ngx': vfs cache: data root is "/home/dietpi/.cache/rclone/vfs/crypt/paperless-ngx"
2025/02/16 12:53:30 DEBUG : Encrypted drive 'crypt:paperless-ngx': vfs cache: metadata root is "/home/dietpi/.cache/rclone/vfsMeta/crypt/paperless-ngx"
2025/02/16 12:53:30 DEBUG : Creating backend with remote ":local,encoding='Slash,Dot',links=false:/home/dietpi/.cache/rclone/vfs/crypt/paperless-ngx"
2025/02/16 12:53:30 DEBUG : :local: detected overridden config - adding "{8un-i}" suffix to name
2025/02/16 12:53:30 DEBUG : fs cache: renaming cache item ":local,encoding='Slash,Dot',links=false:/home/dietpi/.cache/rclone/vfs/crypt/paperless-ngx" to be canonical ":local{8un-i}:/home/dietpi/.cache/rclone/vfs/crypt/paperless-ngx"
2025/02/16 12:53:30 DEBUG : Creating backend with remote ":local,encoding='Slash,Dot',links=false:/home/dietpi/.cache/rclone/vfsMeta/crypt/paperless-ngx"
2025/02/16 12:53:30 DEBUG : :local: detected overridden config - adding "{8un-i}" suffix to name
2025/02/16 12:53:30 DEBUG : fs cache: renaming cache item ":local,encoding='Slash,Dot',links=false:/home/dietpi/.cache/rclone/vfsMeta/crypt/paperless-ngx" to be canonical ":local{8un-i}:/home/dietpi/.cache/rclone/vfsMeta/crypt/paperless-ngx"
2025/02/16 12:53:30 DEBUG : Encrypted drive 'crypt:paperless-ngx': vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item data/classification_model.pickle not removed, freed 0 bytes
2025/02/16 12:53:30 DEBUG : Encrypted drive 'crypt:paperless-ngx': vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item data/db.sqlite3 not removed, freed 0 bytes
2025/02/16 12:53:30 DEBUG : Encrypted drive 'crypt:paperless-ngx': vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item data/log/celery.log not removed, freed 0 bytes
2025/02/16 12:53:30 DEBUG : Encrypted drive 'crypt:paperless-ngx': vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item data/log/paperless.log not removed, freed 0 bytes
2025/02/16 12:53:30 DEBUG : Encrypted drive 'crypt:paperless-ngx': vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item data/migration_lock not removed, freed 0 bytes
2025/02/16 12:53:30 DEBUG : Encrypted drive 'crypt:paperless-ngx': vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item data/celerybeat-schedule.db not removed, freed 0 bytes
2025/02/16 12:53:30 INFO  : Encrypted drive 'crypt:paperless-ngx': vfs cache: cleaned: objects 6 (was 6) in use 0, to upload 0, uploading 0, total size 146.131Mi (was 146.131Mi)
2025/02/16 12:53:30 DEBUG : Encrypted drive 'crypt:paperless-ngx': Mounting on "/home/dietpi/paperless-ngx"
2025/02/16 12:53:30 DEBUG : : Root: 
2025/02/16 12:53:30 DEBUG : : >Root: node=/, err=<nil>
2025/02/16 12:53:40 DEBUG : /: Attr: 
2025/02/16 12:53:40 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2025/02/16 12:53:40 DEBUG : : Statfs: 
2025/02/16 12:53:40 DEBUG : : >Statfs: stat={Blocks:15347267 Bfree:11194722 Bavail:11194722 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2

This is expected behaviour. Opened file can not be uploaded.

More importantly this is not very good idea to use rclone mount to host database like this. It is not designed to handle it. You risk corruption and losing all your database data.

And why to even attempt to do it on local remote?! Use native disk encryption. It is very straightforward on Linux to use LUKS or truecrypt (veracrypt). You can encrypt all or dedicated filesystem or even create one hosted in local container (file). For case like yours you can also look at eCryptfs - Linux native, file system level, can encrypt only specific directory.

rclone crypt is NOT replacement for local disk encryption. It is VERY BAD idea to use it as such unless for very limited scope like dump storage for some backup files.

Use the right tool for the job you have.

3

Thanks for the reply!

Well that was a test before using a remote storage, in which case LUKS or eCryptfs would not work.

As I understand, having a database on a (FUSE-)mounted file system is not optimal. What I am ultimately trying to do is to have a database in an encrypted form on a remote storage, while still being able to work with it without syncing it every time. Is there any way to achieve this with Rclone without a major corruption risk?

4

Nope. And on true remote it is even worse as rclone does have to upload all file even if only one byte changes.

rent some VPS and run your database there in encrypted container. You can interact with database remotely over VPN etc.

Let's see maybe others have different/better suggestions.

5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.