What is the problem you are having with rclone?
Background
On my Android device (using termux), I am trying to bisync a local directory with a local crypt remote, which (the crypt location) in turn is bisynced with a WebDAV remote. The reason I take this extra step is to be able to keep a potentially large amount of files locally, and to delete the decrypted files any time i want to. This setup is thought to be an alternative to file encryption software like Cryptomator or VeraCrypt, none of which does provide filesystem integration.
Expected behaviour
The expected behaviour would be that rclone does not care whether the local directory it tries to bisync to a backend is the location of a crypt or something else entirely. I expect rclone to treat the crypt as normal files.
Issue
Bisyncing a "normal" folder to the same backend works like a charm. Using a crypt instead and updating it internally results in broken behaviour of the following bisync call to the WebDAV remote. It seems to be changing the modtime of updated files in the crypt to their creation time / previous time (have not tested which is true), aka ignores the update, even when the WebDAV remote remained unchanged and detected no differences.
In the debug scenario, i made a folder for each remote and then just ran the commands below.
rclone version:
rclone v1.68.2-termux
- os/version: unknown
- os/kernel: 5.10.209-android13-4-00011-g17e85e80b432-ab12324538 (aarch64)
- os/type: android
- os/arch: arm64 (ARMv8 compatible)
- go/version: go1.23.2
- go/linking: dynamic
- go/tags: noselfupdate
Which cloud storage system are you using?
WebDAV
rclone config:
[local]
type = local
copy_links = true
[local_Test-encrypted]
type = crypt
remote = local:/storage/emulated/0/documents/Test-encrypted
password = Deleted
password2 = Deleted
[WEBDAV-REMOTE]
type = webdav
url = Deleted
vendor = nextcloud
user = Deleted
pass = Deleted
Command sequence, tracking modtimes along (bisync logs are further below):
~ $ rclone lsl local:/storage/emulated/0/documents/Test
~ $ rclone lsl local:/storage/emulated/0/documents/Test-encrypted/
~ $ rclone lsl WEBDAV-REMOTE:Test-encrypted/
~ $ touch storage/shared/Documents/Test/file1.md
~ $ touch storage/shared/Documents/Test/file2.md
~ $ touch storage/shared/Documents/Test/file3.md
~ $ rclone lsl local:/storage/emulated/0/documents/Test
0 2024-12-01 21:38:41.396155956 file1.md
0 2024-12-01 21:38:44.644155958 file2.md
0 2024-12-01 21:38:48.028155959 file3.md
~ $ stat /storage/emulated/0/documents/Test/file1.md
File: /storage/emulated/0/documents/Test/file1.md
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: 0,194 Inode: 439150 Links: 1
Access: (0660/-rw-rw----) Uid: (10234/ u0_a234) Gid: ( 1023/media_rw)
Context: u:object_r:fuse:s0
Access: 2024-12-01 22:38:41.396155956 +0100
Modify: 2024-12-01 22:38:41.396155956 +0100
Change: 2024-12-01 22:38:41.396155956 +0100
Birth: -
~ $
~ $ rclone bisync local:/storage/emulated/0/documents/Test local_Test-encrypted: --ask-password=false --log-file="01_local-resync.log" --log-level INFO --resync
~ $ stat /storage/emulated/0/documents/Test/file1.md
File: /storage/emulated/0/documents/Test/file1.md
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: 0,194 Inode: 439150 Links: 1
Access: (0660/-rw-rw----) Uid: (10234/ u0_a234) Gid: ( 1023/media_rw)
Context: u:object_r:fuse:s0
Access: 2024-12-01 22:38:41.396155956 +0100
Modify: 2024-12-01 22:38:41.396155956 +0100
Change: 2024-12-01 22:38:41.396155956 +0100
Birth: -
~ $ stat /storage/emulated/0/documents/Test-encrypted/qlfj5q8esklhka5t6nddghnatg
File: /storage/emulated/0/documents/Test-encrypted/qlfj5q8esklhka5t6nddghnatg
Size: 32 Blocks: 8 IO Block: 4096 regular file
Device: 0,194 Inode: 394493 Links: 1
Access: (0660/-rw-rw----) Uid: (10234/ u0_a234) Gid: ( 1023/media_rw)
Context: u:object_r:fuse:s0
Access: 2024-12-01 22:38:41.396155956 +0100
Modify: 2024-12-01 22:38:41.396155956 +0100
Change: 2024-12-01 22:41:51.004156049 +0100
Birth: -
~ $
~ $ rclone bisync local:/storage/emulated/0/documents/Test-encrypted/ WEBDAV-REMOTE:Test-encrypted --ask-password=false --log-file="02_remote-resync.log" --log-level INFO --resync
~ $ stat /storage/emulated/0/documents/Test/file1.md
File: /storage/emulated/0/documents/Test/file1.md
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: 0,194 Inode: 439150 Links: 1
Access: (0660/-rw-rw----) Uid: (10234/ u0_a234) Gid: ( 1023/media_rw)
Context: u:object_r:fuse:s0
Access: 2024-12-01 22:38:41.396155956 +0100
Modify: 2024-12-01 22:38:41.396155956 +0100
Change: 2024-12-01 22:38:41.396155956 +0100
Birth: -
~ $ stat /storage/emulated/0/documents/Test-encrypted/qlfj5q8esklhka5t6nddghnatg
File: /storage/emulated/0/documents/Test-encrypted/qlfj5q8esklhka5t6nddghnatg
Size: 32 Blocks: 8 IO Block: 4096 regular file
Device: 0,194 Inode: 394493 Links: 1
Access: (0660/-rw-rw----) Uid: (10234/ u0_a234) Gid: ( 1023/media_rw)
Context: u:object_r:fuse:s0
Access: 2024-12-01 22:38:41.396155956 +0100
Modify: 2024-12-01 22:38:41.396155956 +0100
Change: 2024-12-01 22:41:51.008156049 +0100
Birth: -
~ $ rclone lsl WEBDAV-REMOTE:Test-encrypted/qlfj5q8esklhka5t6nddghnatg
32 2024-12-01 21:38:41.000000000 qlfj5q8esklhka5t6nddghnatg
~ $
~ $ nano storage/shared/Documents/Test/file1.md
~ $ stat /storage/emulated/0/documents/Test/file1.md
File: /storage/emulated/0/documents/Test/file1.md
Size: 5 Blocks: 8 IO Block: 4096 regular file
Device: 0,194 Inode: 439150 Links: 1
Access: (0660/-rw-rw----) Uid: (10234/ u0_a234) Gid: ( 1023/media_rw)
Context: u:object_r:fuse:s0
Access: 2024-12-01 22:38:41.396155956 +0100
Modify: 2024-12-01 22:47:52.884156225 +0100
Change: 2024-12-01 22:47:52.884156225 +0100
Birth: -
~ $
~ $ rclone bisync local:/storage/emulated/0/documents/Test/ local_Test-encrypted: --ask-password=false --log-file="03_local-bisync.log" --log-level INFO
~ $ stat /storage/emulated/0/documents/Test/file1.md
File: /storage/emulated/0/documents/Test/file1.md
Size: 5 Blocks: 8 IO Block: 4096 regular file
Device: 0,194 Inode: 439150 Links: 1
Access: (0660/-rw-rw----) Uid: (10234/ u0_a234) Gid: ( 1023/media_rw)
Context: u:object_r:fuse:s0
Access: 2024-12-01 22:38:41.396155956 +0100
Modify: 2024-12-01 22:47:52.884156225 +0100
Change: 2024-12-01 22:47:52.884156225 +0100
Birth: -
~ $ stat /storage/emulated/0/documents/Test-encrypted/qlfj5q8esklhka5t6nddghnatg
File: /storage/emulated/0/documents/Test-encrypted/qlfj5q8esklhka5t6nddghnatg
Size: 53 Blocks: 8 IO Block: 4096 regular file
Device: 0,194 Inode: 325453 Links: 1
Access: (0660/-rw-rw----) Uid: (10234/ u0_a234) Gid: ( 1023/media_rw)
Context: u:object_r:fuse:s0
Access: 2024-12-01 22:47:52.884156225 +0100
Modify: 2024-12-01 22:47:52.884156225 +0100
Change: 2024-12-01 22:49:11.412156264 +0100
Birth: -
~ $
~ $ # FAILS BECAUSE OF MOD TIME CHANGE (/storage/emulated/0/documents/Test-encrypted/qlfj5q8esklhka5t6nddghnatg):
~ $ rclone bisync local:/storage/emulated/0/documents/Test-encrypted WEBDAV-REMOTE:Test-encrypted --ask-password=false --log-file="04_remote-bisync.log" --log-level INFO
~ $ stat /storage/emulated/0/documents/Test/file1.md
File: /storage/emulated/0/documents/Test/file1.md
Size: 5 Blocks: 8 IO Block: 4096 regular file
Device: 0,194 Inode: 439150 Links: 1
Access: (0660/-rw-rw----) Uid: (10234/ u0_a234) Gid: ( 1023/media_rw)
Context: u:object_r:fuse:s0
Access: 2024-12-01 22:38:41.396155956 +0100
Modify: 2024-12-01 22:47:52.884156225 +0100
Change: 2024-12-01 22:47:52.884156225 +0100
Birth: -
~ $ stat /storage/emulated/0/documents/Test-encrypted/qlfj5q8esklhka5t6nddghnatg
File: /storage/emulated/0/documents/Test-encrypted/qlfj5q8esklhka5t6nddghnatg
Size: 53 Blocks: 8 IO Block: 4096 regular file
Device: 0,194 Inode: 325453 Links: 1
Access: (0660/-rw-rw----) Uid: (10234/ u0_a234) Gid: ( 1023/media_rw)
Context: u:object_r:fuse:s0
Access: 2024-12-01 22:47:52.884156225 +0100
Modify: 2024-12-01 22:38:41.396155956 +0100
Change: 2024-12-01 22:49:11.420156264 +0100
Birth: -
~ $ rclone lsl WEBDAV-REMOTE:Test-encrypted/qlfj5q8esklhka5t6nddghnatg
2024/12/01 21:52:34 ERROR : : error listing: directory not found
2024/12/01 21:52:34 NOTICE: Failed to lsl with 2 errors: last error was: directory not found
Logs of bisync commands
Inital resync of local and crypt (01_local-resync.log):
2024/12/01 21:41:50 NOTICE: bisync is IN BETA. Don't use in production!
2024/12/01 21:41:50 INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
2024/12/01 21:41:50 INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"HashType1": 0,
"HashType2": 0,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"SlowHashDetected": true,
"DownloadHash": false
}
2024/12/01 21:41:50 INFO : Synching Path1 "/storage/emulated/0/documents/Test/" with Path2 "local_Test-encrypted:/"
2024/12/01 21:41:50 INFO : Copying Path2 files to Path1
2024/12/01 21:41:50 INFO : - Path2 Resync is copying files to - Path1
2024/12/01 21:41:50 INFO : There was nothing to transfer
2024/12/01 21:41:50 INFO : - Path1 Resync is copying files to - Path2
2024/12/01 21:41:51 INFO : file1.md: Copied (new)
2024/12/01 21:41:51 INFO : file2.md: Copied (new)
2024/12/01 21:41:51 INFO : file3.md: Copied (new)
2024/12/01 21:41:51 INFO : Resync updating listings
2024/12/01 21:41:51 INFO : Validating listings for Path1 "/storage/emulated/0/documents/Test/" vs Path2 "local_Test-encrypted:/"
2024/12/01 21:41:51 INFO : Bisync successful
2024/12/01 21:41:51 INFO :
Transferred: 96 B / 96 B, 100%, 0 B/s, ETA -
Transferred: 3 / 3, 100%
Elapsed time: 0.1s
Inital resync of crypt location and WebDAV (02_remote-resync.log):
2024/12/01 21:44:10 NOTICE: bisync is IN BETA. Don't use in production!
2024/12/01 21:44:10 INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
2024/12/01 21:44:10 INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"HashType1": 0,
"HashType2": 0,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"SlowHashDetected": true,
"DownloadHash": false
}
2024/12/01 21:44:10 INFO : Synching Path1 "/storage/emulated/0/documents/Test-encrypted/" with Path2 "WEBDAV-REMOTE:Test-encrypted/"
2024/12/01 21:44:10 INFO : Copying Path2 files to Path1
2024/12/01 21:44:10 INFO : - Path2 Resync is copying files to - Path1
2024/12/01 21:44:10 INFO : There was nothing to transfer
2024/12/01 21:44:10 INFO : - Path1 Resync is copying files to - Path2
2024/12/01 21:44:13 INFO : qlfj5q8esklhka5t6nddghnatg: Copied (new)
2024/12/01 21:44:13 INFO : 3n093r1vebf0p5cbllan3orj30: Copied (new)
2024/12/01 21:44:13 INFO : 34jdtfj20v9so1pia70s51qp18: Copied (new)
2024/12/01 21:44:13 INFO : Resync updating listings
2024/12/01 21:44:13 INFO : Validating listings for Path1 "/storage/emulated/0/documents/Test-encrypted/" vs Path2 "WEBDAV-REMOTE:Test-encrypted/"
2024/12/01 21:44:13 INFO : Bisync successful
2024/12/01 21:44:13 INFO :
Transferred: 96 B / 96 B, 100%, 48 B/s, ETA 0s
Transferred: 3 / 3, 100%
Elapsed time: 4.0s
Bisync of local and crypt after local file modification (03_local-bisync.log):
2024/12/01 21:49:11 NOTICE: bisync is IN BETA. Don't use in production!
2024/12/01 21:49:11 INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
2024/12/01 21:49:11 INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"HashType1": 0,
"HashType2": 0,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"SlowHashDetected": true,
"DownloadHash": false
}
2024/12/01 21:49:11 INFO : Synching Path1 "/storage/emulated/0/documents/Test/" with Path2 "local_Test-encrypted:/"
2024/12/01 21:49:11 INFO : Building Path1 and Path2 listings
2024/12/01 21:49:11 INFO : Path1 checking for diffs
2024/12/01 21:49:11 INFO : - Path1 File changed: size (larger), time (newer) - file1.md
2024/12/01 21:49:11 INFO : Path1: 1 changes: 0 new, 1 modified, 0 deleted
2024/12/01 21:49:11 INFO : (Modified: 1 newer, 0 older, 1 larger, 0 smaller)
2024/12/01 21:49:11 INFO : Path2 checking for diffs
2024/12/01 21:49:11 INFO : Applying changes
2024/12/01 21:49:11 INFO : - Path1 Queue copy to Path2 - local_Test-encrypted:/file1.md
2024/12/01 21:49:11 INFO : - Path1 Do queued copies to - Path2
2024/12/01 21:49:11 INFO : file1.md: Copied (replaced existing)
2024/12/01 21:49:11 INFO : Updating listings
2024/12/01 21:49:11 INFO : Validating listings for Path1 "/storage/emulated/0/documents/Test/" vs Path2 "local_Test-encrypted:/"
2024/12/01 21:49:11 INFO : Bisync successful
2024/12/01 21:49:11 INFO :
Transferred: 53 B / 53 B, 100%, 0 B/s, ETA -
Checks: 7 / 7, 100%
Transferred: 1 / 1, 100%
Elapsed time: 0.1s
Bisync of crypt location and WebDAV after local file modification (04_remote-bisync.log):
2024/12/01 21:50:47 NOTICE: bisync is IN BETA. Don't use in production!
2024/12/01 21:50:47 INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
2024/12/01 21:50:47 INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"HashType1": 0,
"HashType2": 0,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"SlowHashDetected": true,
"DownloadHash": false
}
2024/12/01 21:50:47 INFO : Synching Path1 "/storage/emulated/0/documents/Test-encrypted/" with Path2 "WEBDAV-REMOTE:Test-encrypted/"
2024/12/01 21:50:47 INFO : Building Path1 and Path2 listings
2024/12/01 21:50:47 INFO : Path1 checking for diffs
2024/12/01 21:50:47 INFO : - Path1 File changed: size (larger), time (newer) - qlfj5q8esklhka5t6nddghnatg
2024/12/01 21:50:47 INFO : Path1: 1 changes: 0 new, 1 modified, 0 deleted
2024/12/01 21:50:47 INFO : (Modified: 1 newer, 0 older, 1 larger, 0 smaller)
2024/12/01 21:50:47 INFO : Path2 checking for diffs
2024/12/01 21:50:47 INFO : Applying changes
2024/12/01 21:50:47 INFO : - Path1 Queue copy to Path2 - WEBDAV-REMOTE:Test-encrypted/qlfj5q8esklhka5t6nddghnatg
2024/12/01 21:50:47 INFO : - Path1 Do queued copies to - Path2
2024/12/01 21:50:50 ERROR : qlfj5q8esklhka5t6nddghnatg: Failed to copy: Put "https://box.jamail.one/cloud/remote.php/dav/files/jb%40jamail.one/Test-encrypted/qlfj5q8esklhka5t6nddghnatg": can't copy - source file is being updated (mod time changed from 2024-12-01 21:47:52.884156225 +0000 UTC to 2024-12-01 21:38:41.396155956 +0000 UTC)
2024/12/01 21:50:50 ERROR : webdav root 'Test-encrypted': not deleting files as there were IO errors
2024/12/01 21:50:50 ERROR : webdav root 'Test-encrypted': not deleting directories as there were IO errors
2024/12/01 21:50:50 ERROR : Bisync critical error: Put "https://box.jamail.one/cloud/remote.php/dav/files/jb%40jamail.one/Test-encrypted/qlfj5q8esklhka5t6nddghnatg": can't copy - source file is being updated (mod time changed from 2024-12-01 21:47:52.884156225 +0000 UTC to 2024-12-01 21:38:41.396155956 +0000 UTC)
2024/12/01 21:50:50 ERROR : Bisync aborted. Must run --resync to recover.