Rclone bisync in systemd

Help and Support
1

What is the problem you are having with rclone?

This is a continuation of the post I made earlier called Beginner Question, that question was resolved and I am now on to step 2 - using my rclone mount with bisync and setting up a systemd timer to do it. However, this issue happened at the very beginning of the process, trying to run the rclone bisync command, after changing one file in the remote with another device that has access to it. I have not yet tried to make the service and timer.

Run the command 'rclone version' and share the full output of the command.

$ rclone version
rclone v1.67.0
- os/version: ubuntu 24.04 (64 bit)
- os/kernel: 6.8.0-40-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.22.4
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

/usr/bin/rclone bisync wopa: ~/wopa  \   
-MvP     
--create-empty-src-dirs     
--compare size,modtime,checksum          
--conflict-resolve newer     
--conflict-loser delete     
--conflict-suffix sync-conflict-{DateOnly}-     
--suffix-keep-extension     
--resilient     
--recover     
--no-slow-hash     
--drive-skip-gdocs     
--fix-case

This is a modification of the command given in tutorials/Rclone bisync Google Drive.md at main · notnout/tutorials · GitHub. I didn't need filters, as I am cloning only a single directory on Google Drive.

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

$ rclone config redacted
[wopa]
type = drive
scope = drive
token = XXX
team_drive = 
root_folder_id = XXX
client_id = XXX
client_secret = XXX
### Double check the config for sensitive info before posting publicly

A log from the command that you were trying to run with the -vv flag

2024/08/09 12:17:54 NOTICE: bisync is IN BETA. Don't use in production!
2024/08/09 12:17:54 INFO  : Not checking for common hash as at least one slow hash detected.
2024/08/09 12:17:54 NOTICE: Local file system at /home/scohen/wopa: --checksum is in use but Path1 and Path2 have no hashes in common; falling back to --compare modtime,size for sync. (Use --compare size or --size-only to ignore modtime)
2024/08/09 12:17:54 INFO  : Path1 hashes: [md5, sha1, sha256]
2024/08/09 12:17:54 INFO  : Path2 hashes: [md5, sha1, whirlpool, crc32, sha256, dropbox, hidrive, mailru, quickxor]
2024/08/09 12:17:54 NOTICE: Google drive root '': will use md5 for same-side diffs on Path1 only
2024/08/09 12:17:54 INFO  : Slow hash detected on Path2. Will ignore checksum due to slow-hash settings
2024/08/09 12:17:54 NOTICE: WARNING: Ignoring checksums globally as hashes are ignored or unavailable on both sides.
2024/08/09 12:17:54 INFO  : Bisyncing with Comparison Settings: 
{
	"Modtime": true,
	"Size": true,
	"Checksum": false,
	"HashType1": 0,
	"HashType2": 0,
	"NoSlowHash": true,
	"SlowHashSyncOnly": false,
	"SlowHashDetected": true,
	"DownloadHash": false
}
2024/08/09 12:17:54 INFO  : Synching Path1 "wopa{YRXYK}:/" with Path2 "/home/scohen/wopa/"
2024/08/09 12:17:54 INFO  : Building Path1 and Path2 listings
2024/08/09 12:17:55 INFO  : Path1 checking for diffs
2024/08/09 12:17:55 INFO  : - Path1    File changed: size (smaller), time (newer) - 871204.kdbx
2024/08/09 12:17:55 INFO  : Path1:    1 changes:    0 new,    1 modified,    0 deleted
2024/08/09 12:17:55 INFO  : (Modified:    1 newer,    0 older,    0 larger,    1 smaller)
2024/08/09 12:17:55 INFO  : Path2 checking for diffs
2024/08/09 12:17:55 INFO  : - Path2    File changed: size (smaller), time (newer) - 871204.kdbx
2024/08/09 12:17:55 INFO  : Path2:    1 changes:    0 new,    1 modified,    0 deleted
2024/08/09 12:17:55 INFO  : (Modified:    1 newer,    0 older,    0 larger,    1 smaller)
2024/08/09 12:17:55 ERROR : Safety abort: all files were changed on Path1 "wopa{YRXYK}:/". Run with --force if desired.
2024/08/09 12:17:55 ERROR : Safety abort: all files were changed on Path2 "/home/scohen/wopa/". Run with --force if desired.
2024/08/09 12:17:55 NOTICE: Bisync aborted. Please try again.
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Checks:                 2 / 2, 100%
Elapsed time:         0.4s
2024/08/09 12:17:55 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Checks:                 2 / 2, 100%
Elapsed time:         0.4s

2024/08/09 12:17:55 Failed to bisync: all files were changed

I had changed one file from another computer with access to the Google Drive folder the remote was set up with and the above was the result.

2

If you want file(s) synced with cloud remote you have two options:

  1. mount remote - any changes you make locally in the mount will be uploaded to the cloud, any changes in the cloud will be downloaded locally

  2. bisync local directory with your remote - sync will happen only when you run rclone bisync

Neither method provides any locking nor conflict resolution if you will have multiple clients accessing the same remote....

3

Thank you. If I understand you correctly, I don't need bisync at all in my simple situation. And as I look at my local copy of the file I edited on another device this morning, I see, indeed that it is updated.

And so, I am done! Whoopee. Thank you @kapitainsky and @asdffdsa. You guys are terrific!

4

Actually though,

Neither method provides any locking nor conflict resolution if you will have multiple clients accessing the same remote....

please define "accessing". If one device has the file open, but is not actively editing it, and another device opens and saves the file will those changes propapagate to the first device that had the file open? Does this imply that it would be advisable for each device to only hold the file open as long as it is needed, especially if I expect the other devices to need to access it in the same time frame? Or is keeping the file open on two devices okay as long as simultaneous edits do not occur?

5

Whoever saves the last it will become cloud version.

If file is open locally than I think rclone is not able to download in the meantime updated remote version.

All together I think if you want to allow multiple users to edit remotely stored file you have to manage it "manually".

Not sure how your previous Dropbox solution managed it.

6

I suspect it only came up rarely. I think all the applications that use the file save automatically upon clicking "OK". Still, actually closing the file on completing the task seems like the safest thing to do. Thanks.

7

FYI: I think this is happening because you have a total of 1 file, and it was changed (therefore 100% of the files were changed.)

It is a safety feature which can be disabled with --force.

8

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.