What is the problem you are having with rclone?
A logfile of the rclone bisync task is included in the directory which is being "bisynced". However, every second run (always the same command), the logfile on Path1 (source, raspberry pi) and the logfile on Path2 (destination, google drive) is seen as a conflicting file. Path1 is the "winner" as it is newer and overwrites the file on Path2, which is what i want. However, if a file conflicts, additional renamed copies of the "loser" are kept, which is desired and from the command. However, since this is a scheduled task, it is unnecessary to have many logfile copies of the "loser", as nothing actually has been modified since the previous run on Path2, as the logfile is only modified on Path1 (logs of the rclone bisync run).
Not quite confident to suspect a bug, but might be.
MY QUESTION:
- Why does rclone see the logfile on Path2 as also modified, triggering the conflict? Or differently asked: what is the difference between a user modifying a python file on Path1 (e.g. mycode.py) and on the next run rclone not seeing the copy on Path2 as also modified but just does a classic "copy-to-replace" from Path1 to Path2, as mycode.py has only been modified on Path1 and our logfile on Path1 having additional logs since the last run? (which seems to be the same, if i understand it correctly)
- Why is this only the case every second run?
- (optional :)), there are more important questions above Why does it queue the renamed copy of the logfile on Path1 at
Not renaming Path1 copy, as it was determined the winner, but then not copy it to Path2 after- Path2 Do queued copies to - Path1? And then on the following run, it sees this file (that has not been created or copied (i checked on Path1 and Path2, not there)) as "was deleted"? Is this like a "rclone memory from a previous run?" It doesn't do anything wrong, it just seems unnecessary creating and deleting a file "rclone internally" and then realising that happened on the next run... anyway :), not the main question here, just thought about it.
Run the command 'rclone version' and share the full output of the command.
rclone v1.68.2
- os/version: raspbian 12.8 (64 bit)
- os/kernel: 6.6.62+rpt-rpi-v8 (aarch64)
- os/type: linux
- os/arch: arm64 (ARMv8 compatible)
- go/version: go1.23.3
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
Google Drive
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone bisync /home/user/AutoEinm googledrive:raspberrypi_home_user_AutoEinm_gdrive -Mv --compare size,modtime,checksum --resilient --recover --max-lock 2m --disable ListR --check-access --track-renames --conflict-resolve newer --conflict-loser pathname --conflict-suffix "{20060102_150405}_raspberrypi","{20060102_150405}_googledrive" --exclude-if-present .nosync --log-file /home/user/AutoEinm/LOGFILE_bisync_AutoEinm.txt
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
[googledrive]
type = drive
scope = drive
token = XXX
team_drive =
### Double check the config for sensitive info before posting publicly
A log from the command that you were trying to run with the -vv flag
If more detailed logs or a specific part of it is wished for, please let me know. The logs tend to be very long, so I rather send detailed logs of parts of interest if requested.
INFORMATION ABOUT LOG: The log shows 4 consecutive runs right after eachother with no other files being manually changed. It shows the behaviour of the logfile copying and change conflicts.
NOTE: The logs are "identical" in sets of 2.
### RUN 1:
2024/12/12 11:07:42 NOTICE: bisync is IN BETA. Don't use in production!
2024/12/12 11:07:42 INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": true,
"HashType1": 1,
"HashType2": 1,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"SlowHashDetected": true,
"DownloadHash": false
}
2024/12/12 11:07:42 INFO : lock file renewed for 2m0s. New expiration: 2024-12-12 11:09:42.653412594 +0100 CET m=+120.588369592
2024/12/12 11:07:42 INFO : Synching Path1 "/home/user/AutoEinm/" with Path2 "googledrive:raspberrypi_home_user_AutoEinm_gdrive/"
2024/12/12 11:07:42 INFO : Building Path1 and Path2 listings
2024/12/12 11:07:44 INFO : Path1 checking for diffs
2024/12/12 11:07:44 INFO : - Path1 File changed: size (larger), time (newer), hash - LOGFILE_bisync_AutoEinm.txt
2024/12/12 11:07:44 INFO : Path1: 1 changes: 0 new, 1 modified, 0 deleted
2024/12/12 11:07:44 INFO : (Modified: 1 newer, 0 older, 1 larger, 0 smaller, 1 hash differs)
2024/12/12 11:07:44 INFO : Path2 checking for diffs
2024/12/12 11:07:44 INFO : - Path2 File changed: size (larger), time (newer), hash - LOGFILE_bisync_AutoEinm.txt
2024/12/12 11:07:44 INFO : Path2: 1 changes: 0 new, 1 modified, 0 deleted
2024/12/12 11:07:44 INFO : (Modified: 1 newer, 0 older, 1 larger, 0 smaller, 1 hash differs)
2024/12/12 11:07:44 INFO : Checking access health
2024/12/12 11:07:44 INFO : Found 1 matching "RCLONE_TEST" files on both paths
2024/12/12 11:07:44 INFO : Applying changes
2024/12/12 11:07:44 NOTICE: - WARNING New or changed in both paths - LOGFILE_bisync_AutoEinm.txt
2024/12/12 11:07:44 INFO : LOGFILE_bisync_AutoEinm.txt: Path1 is newer. Path1: 2024-12-12 11:07:42.651994904 +0100 CET, Path2: 2024-12-12 11:07:39.792 +0100 CET, Difference: 2.859994904s
2024/12/12 11:07:44 INFO : LOGFILE_bisync_AutoEinm.txt: The winner is: Path1
2024/12/12 11:07:44 NOTICE: - Path1 Not renaming Path1 copy, as it was determined the winner - /home/user/AutoEinm/LOGFILE_bisync_AutoEinm.txt.20241212_110742_raspberrypi
2024/12/12 11:07:44 NOTICE: - Path1 Queue copy to Path2 - googledrive:raspberrypi_home_user_AutoEinm_gdrive/LOGFILE_bisync_AutoEinm.txt.20241212_110742_raspberrypi
2024/12/12 11:07:44 NOTICE: - Path2 Renaming Path2 copy - googledrive:raspberrypi_home_user_AutoEinm_gdrive/LOGFILE_bisync_AutoEinm.txt.20241212_110742_googledrive
2024/12/12 11:07:45 INFO : LOGFILE_bisync_AutoEinm.txt: Moved (server-side) to: LOGFILE_bisync_AutoEinm.txt.20241212_110742_googledrive
2024/12/12 11:07:45 NOTICE: - Path2 Queue copy to Path1 - /home/user/AutoEinm/LOGFILE_bisync_AutoEinm.txt.20241212_110742_googledrive
2024/12/12 11:07:45 INFO : - Path2 Do queued copies to - Path1
2024/12/12 11:07:46 INFO : Local file system at /home/user/AutoEinm: Making map for --track-renames
2024/12/12 11:07:46 INFO : Local file system at /home/user/AutoEinm: Finished making map for --track-renames
2024/12/12 11:07:46 INFO : LOGFILE_bisync_AutoEinm.txt.20241212_110742_googledrive: Copied (new)
2024/12/12 11:07:46 INFO : - Path1 Do queued copies to - Path2
2024/12/12 11:07:47 INFO : Google drive root 'raspberrypi_home_user_AutoEinm_gdrive': Making map for --track-renames
2024/12/12 11:07:47 INFO : Google drive root 'raspberrypi_home_user_AutoEinm_gdrive': Finished making map for --track-renames
2024/12/12 11:07:47 INFO : Updating listings
2024/12/12 11:07:47 INFO : Validating listings for Path1 "/home/user/AutoEinm/" vs Path2 "googledrive:raspberrypi_home_user_AutoEinm_gdrive/"
2024/12/12 11:07:47 INFO : Bisync successful
2024/12/12 11:07:47 INFO :
Transferred: 180.744 KiB / 180.744 KiB, 100%, 45.186 KiB/s, ETA 0s
Checks: 58 / 58, 100%
Renamed: 1
Transferred: 2 / 2, 100%
Server Side Moves: 1 @ 90.372 KiB
Elapsed time: 5.4s
### RUN 2:
2024/12/12 11:07:49 NOTICE: bisync is IN BETA. Don't use in production!
2024/12/12 11:07:49 INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": true,
"HashType1": 1,
"HashType2": 1,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"SlowHashDetected": true,
"DownloadHash": false
}
2024/12/12 11:07:49 INFO : lock file renewed for 2m0s. New expiration: 2024-12-12 11:09:49.55516834 +0100 CET m=+120.596420237
2024/12/12 11:07:49 INFO : Synching Path1 "/home/user/AutoEinm/" with Path2 "googledrive:raspberrypi_home_user_AutoEinm_gdrive/"
2024/12/12 11:07:49 INFO : Building Path1 and Path2 listings
2024/12/12 11:07:51 INFO : Path1 checking for diffs
2024/12/12 11:07:51 INFO : - Path1 File was deleted - LOGFILE_bisync_AutoEinm.txt.20241212_110742_raspberrypi
2024/12/12 11:07:51 INFO : - Path1 File is new - LOGFILE_bisync_AutoEinm.txt
2024/12/12 11:07:51 INFO : Path1: 2 changes: 1 new, 0 modified, 1 deleted
2024/12/12 11:07:51 INFO : Path2 checking for diffs
2024/12/12 11:07:51 INFO : - Path2 File was deleted - LOGFILE_bisync_AutoEinm.txt.20241212_110742_raspberrypi
2024/12/12 11:07:51 INFO : Path2: 1 changes: 0 new, 0 modified, 1 deleted
2024/12/12 11:07:51 INFO : Checking access health
2024/12/12 11:07:51 INFO : Found 1 matching "RCLONE_TEST" files on both paths
2024/12/12 11:07:51 INFO : Applying changes
2024/12/12 11:07:51 INFO : - Path1 Queue copy to Path2 - googledrive:raspberrypi_home_user_AutoEinm_gdrive/LOGFILE_bisync_AutoEinm.txt
2024/12/12 11:07:51 INFO : - Path1 Do queued copies to - Path2
2024/12/12 11:07:51 INFO : Google drive root 'raspberrypi_home_user_AutoEinm_gdrive': Making map for --track-renames
2024/12/12 11:07:51 INFO : Google drive root 'raspberrypi_home_user_AutoEinm_gdrive': Finished making map for --track-renames
2024/12/12 11:07:53 INFO : LOGFILE_bisync_AutoEinm.txt: Copied (new)
2024/12/12 11:07:53 INFO : Updating listings
2024/12/12 11:07:53 INFO : Validating listings for Path1 "/home/user/AutoEinm/" vs Path2 "googledrive:raspberrypi_home_user_AutoEinm_gdrive/"
2024/12/12 11:07:53 INFO : Bisync successful
2024/12/12 11:07:53 INFO :
Transferred: 96.968 KiB / 96.968 KiB, 100%, 96.968 KiB/s, ETA 0s
Checks: 59 / 59, 100%
Transferred: 1 / 1, 100%
Elapsed time: 3.9s
### RUN 3:
2024/12/12 11:07:54 NOTICE: bisync is IN BETA. Don't use in production!
2024/12/12 11:07:54 INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": true,
"HashType1": 1,
"HashType2": 1,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"SlowHashDetected": true,
"DownloadHash": false
}
2024/12/12 11:07:54 INFO : lock file renewed for 2m0s. New expiration: 2024-12-12 11:09:54.875922046 +0100 CET m=+120.589343009
2024/12/12 11:07:54 INFO : Synching Path1 "/home/user/AutoEinm/" with Path2 "googledrive:raspberrypi_home_user_AutoEinm_gdrive/"
2024/12/12 11:07:54 INFO : Building Path1 and Path2 listings
2024/12/12 11:07:56 INFO : Path1 checking for diffs
2024/12/12 11:07:56 INFO : - Path1 File changed: size (larger), time (newer), hash - LOGFILE_bisync_AutoEinm.txt
2024/12/12 11:07:56 INFO : Path1: 1 changes: 0 new, 1 modified, 0 deleted
2024/12/12 11:07:56 INFO : (Modified: 1 newer, 0 older, 1 larger, 0 smaller, 1 hash differs)
2024/12/12 11:07:56 INFO : Path2 checking for diffs
2024/12/12 11:07:56 INFO : - Path2 File changed: size (larger), time (newer), hash - LOGFILE_bisync_AutoEinm.txt
2024/12/12 11:07:56 INFO : Path2: 1 changes: 0 new, 1 modified, 0 deleted
2024/12/12 11:07:56 INFO : (Modified: 1 newer, 0 older, 1 larger, 0 smaller, 1 hash differs)
2024/12/12 11:07:56 INFO : Checking access health
2024/12/12 11:07:56 INFO : Found 1 matching "RCLONE_TEST" files on both paths
2024/12/12 11:07:56 INFO : Applying changes
2024/12/12 11:07:56 NOTICE: - WARNING New or changed in both paths - LOGFILE_bisync_AutoEinm.txt
2024/12/12 11:07:56 INFO : LOGFILE_bisync_AutoEinm.txt: Path1 is newer. Path1: 2024-12-12 11:07:54.87587638 +0100 CET, Path2: 2024-12-12 11:07:51.903 +0100 CET, Difference: 2.97287638s
2024/12/12 11:07:56 INFO : LOGFILE_bisync_AutoEinm.txt: The winner is: Path1
2024/12/12 11:07:56 NOTICE: - Path1 Not renaming Path1 copy, as it was determined the winner - /home/user/AutoEinm/LOGFILE_bisync_AutoEinm.txt.20241212_110754_raspberrypi
2024/12/12 11:07:56 NOTICE: - Path1 Queue copy to Path2 - googledrive:raspberrypi_home_user_AutoEinm_gdrive/LOGFILE_bisync_AutoEinm.txt.20241212_110754_raspberrypi
2024/12/12 11:07:56 NOTICE: - Path2 Renaming Path2 copy - googledrive:raspberrypi_home_user_AutoEinm_gdrive/LOGFILE_bisync_AutoEinm.txt.20241212_110754_googledrive
2024/12/12 11:07:57 INFO : LOGFILE_bisync_AutoEinm.txt: Moved (server-side) to: LOGFILE_bisync_AutoEinm.txt.20241212_110754_googledrive
2024/12/12 11:07:57 NOTICE: - Path2 Queue copy to Path1 - /home/user/AutoEinm/LOGFILE_bisync_AutoEinm.txt.20241212_110754_googledrive
2024/12/12 11:07:57 INFO : - Path2 Do queued copies to - Path1
2024/12/12 11:07:57 INFO : Local file system at /home/user/AutoEinm: Making map for --track-renames
2024/12/12 11:07:57 INFO : Local file system at /home/user/AutoEinm: Finished making map for --track-renames
2024/12/12 11:07:58 INFO : LOGFILE_bisync_AutoEinm.txt.20241212_110754_googledrive: Copied (new)
2024/12/12 11:07:58 INFO : - Path1 Do queued copies to - Path2
2024/12/12 11:07:59 INFO : Google drive root 'raspberrypi_home_user_AutoEinm_gdrive': Making map for --track-renames
2024/12/12 11:07:59 INFO : Google drive root 'raspberrypi_home_user_AutoEinm_gdrive': Finished making map for --track-renames
2024/12/12 11:07:59 INFO : Updating listings
2024/12/12 11:07:59 INFO : Validating listings for Path1 "/home/user/AutoEinm/" vs Path2 "googledrive:raspberrypi_home_user_AutoEinm_gdrive/"
2024/12/12 11:07:59 INFO : Bisync successful
2024/12/12 11:07:59 INFO :
Transferred: 193.936 KiB / 193.936 KiB, 100%, 32.322 KiB/s, ETA 0s
Checks: 60 / 60, 100%
Renamed: 1
Transferred: 2 / 2, 100%
Server Side Moves: 1 @ 96.968 KiB
Elapsed time: 5.2s
### RUN 4:
2024/12/12 11:08:52 NOTICE: bisync is IN BETA. Don't use in production!
2024/12/12 11:08:52 INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": true,
"HashType1": 1,
"HashType2": 1,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"SlowHashDetected": true,
"DownloadHash": false
}
2024/12/12 11:08:52 INFO : lock file renewed for 2m0s. New expiration: 2024-12-12 11:10:52.982314051 +0100 CET m=+120.548760199
2024/12/12 11:08:52 INFO : Synching Path1 "/home/user/AutoEinm/" with Path2 "googledrive:raspberrypi_home_user_AutoEinm_gdrive/"
2024/12/12 11:08:52 INFO : Building Path1 and Path2 listings
2024/12/12 11:08:54 INFO : Path1 checking for diffs
2024/12/12 11:08:54 INFO : - Path1 File was deleted - LOGFILE_bisync_AutoEinm.txt.20241212_110754_raspberrypi
2024/12/12 11:08:54 INFO : - Path1 File is new - LOGFILE_bisync_AutoEinm.txt
2024/12/12 11:08:54 INFO : Path1: 2 changes: 1 new, 0 modified, 1 deleted
2024/12/12 11:08:54 INFO : Path2 checking for diffs
2024/12/12 11:08:54 INFO : - Path2 File was deleted - LOGFILE_bisync_AutoEinm.txt.20241212_110754_raspberrypi
2024/12/12 11:08:54 INFO : Path2: 1 changes: 0 new, 0 modified, 1 deleted
2024/12/12 11:08:54 INFO : Checking access health
2024/12/12 11:08:54 INFO : Found 1 matching "RCLONE_TEST" files on both paths
2024/12/12 11:08:54 INFO : Applying changes
2024/12/12 11:08:54 INFO : - Path1 Queue copy to Path2 - googledrive:raspberrypi_home_user_AutoEinm_gdrive/LOGFILE_bisync_AutoEinm.txt
2024/12/12 11:08:54 INFO : - Path1 Do queued copies to - Path2
2024/12/12 11:08:55 INFO : Google drive root 'raspberrypi_home_user_AutoEinm_gdrive': Making map for --track-renames
2024/12/12 11:08:55 INFO : Google drive root 'raspberrypi_home_user_AutoEinm_gdrive': Finished making map for --track-renames
2024/12/12 11:08:56 INFO : LOGFILE_bisync_AutoEinm.txt: Copied (new)
2024/12/12 11:08:56 INFO : Updating listings
2024/12/12 11:08:56 INFO : Validating listings for Path1 "/home/user/AutoEinm/" vs Path2 "googledrive:raspberrypi_home_user_AutoEinm_gdrive/"
2024/12/12 11:08:56 INFO : Bisync successful
2024/12/12 11:08:56 INFO :
Transferred: 103.562 KiB / 103.562 KiB, 100%, 103.562 KiB/s, ETA 0s
Checks: 61 / 61, 100%
Transferred: 1 / 1, 100%
Elapsed time: 3.9s