Logfile missing data when using --partial-suffix option

Help and Support
1

What is the problem you are having with rclone?

When executing:

rclone bisync PATH1 PATH2 --partial-suffix -verbose --progress --log-file LOGFILE

the only output in the logfile is the "... do not use in production ..." warning.

Omitting the "--partial-suffix" option produces the expected logfile behavior.

Run the command 'rclone version' and share the full output of the command.

rclone v1.65.2
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 6.5.0-21-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.21.6
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone bisync /etc/scripts/ GoogleDrive:/ZZ_Sync/Scripts/ \
	--check-access --check-filename .zz_rclone_access_test_do_not_delete \
	--metadata --partial-suffix --verbose --progress \
	--log-file $HOME/.cache/rclone/rclone-test.log

Output on screen:

Transferred:            287 B / 287 B, 100%, 143 B/s, ETA 0s
Checks:                 1 / 1, 100%
Transferred:            1 / 1, 100%
Elapsed time:         3.9s

Output in logfile:

2024/03/03 09:07:38 NOTICE: bisync is EXPERIMENTAL. Don't use in production!

Removing the --partial-suffix option:

rclone bisync /etc/scripts/ GoogleDrive:/ZZ_Sync/Scripts/ \
	--check-access --check-filename .zz_rclone_access_test_do_not_delete \
	--metadata --verbose --progress \
	--log-file $HOME/.cache/rclone/rclone-test.log

Output on screen:

Transferred:            279 B / 279 B, 100%, 278 B/s, ETA 0s
Checks:                 1 / 1, 100%
Transferred:            1 / 1, 100%
Elapsed time:         3.6s

Output in log file is as expected:

2024/03/03 09:09:23 NOTICE: bisync is EXPERIMENTAL. Don't use in production!
2024/03/03 09:09:23 INFO  : Synching Path1 "/etc/scripts/" with Path2 "GoogleDrive:ZZ_Sync/Scripts/"
2024/03/03 09:09:23 INFO  : Path1 checking for diffs
2024/03/03 09:09:23 INFO  : - Path1    File is newer                       - zz_test.sh
2024/03/03 09:09:23 INFO  : Path1:    1 changes:    0 new,    1 newer,    0 older,    0 deleted
2024/03/03 09:09:23 INFO  : Path2 checking for diffs
2024/03/03 09:09:23 INFO  : Checking access health
2024/03/03 09:09:23 INFO  : Found 1 matching ".zz_rclone_access_test_do_not_delete" files on both paths
2024/03/03 09:09:23 INFO  : Applying changes
2024/03/03 09:09:23 INFO  : - Path1    Queue copy to Path2                 - GoogleDrive:ZZ_Sync/Scripts/zz_test.sh
2024/03/03 09:09:23 INFO  : - Path1    Do queued copies to                 - Path2
2024/03/03 09:09:25 INFO  : zz_test.sh: Copied (replaced existing)
2024/03/03 09:09:25 INFO  : Updating listings
2024/03/03 09:09:25 INFO  : Validating listings for Path1 "/etc/scripts/" vs Path2 "GoogleDrive:/Scripts/"
2024/03/03 09:09:25 INFO  : Bisync successful
2024/03/03 09:09:25 INFO  : 
Transferred:   	        279 B / 279 B, 100%, 278 B/s, ETA 0s
Checks:                 1 / 1, 100%
Transferred:            1 / 1, 100%
Elapsed time:         3.6s

The rclone config contents with secrets removed.

[GoogleDrive]
type = drive
client_id = 
client_secret = 
token = {"access_token":"","token_type":"Bearer","refresh_token":"","expiry":"2024-03-03T09:56:42.224410092-08:00"}
team_drive =

A log from the command with the -vv flag

Here is the debug WITH "--partial-suffix"


2024/03/03 09:26:54 DEBUG : rclone: Version "v1.65.2" starting with parameters ["rclone" "bisync" "/etc/scripts/" "GoogleDrive:/ZZ_Sync/Scripts/" "--check-access" "--check-filename" ".zz_rclone_access_test_do_not_delete" "--metadata" "--partial-suffix" "--verbose" "--progress" "-vv" "--log-file" "$HOME/.cache/rclone/rclone-test.log"]
2024/03/03 09:26:54 DEBUG : Creating backend with remote "/etc/scripts/"
2024/03/03 09:26:54 DEBUG : Using config file from "$HOME/.config/rclone/rclone.conf"
2024/03/03 09:26:54 DEBUG : fs cache: renaming cache item "/etc/scripts/" to be canonical "/etc/scripts"
2024/03/03 09:26:54 DEBUG : Creating backend with remote "GoogleDrive:/ZZ_Sync/Scripts/"
2024/03/03 09:26:54 DEBUG : Google drive root 'ZZ_Sync/Scripts': 'root_folder_id = OMITTED' - save this in the config to speed up startup
2024/03/03 09:26:55 DEBUG : fs cache: renaming cache item "GoogleDrive:/ZZ_Sync/Scripts/" to be canonical "GoogleDrive:ZZ_Sync/Scripts"
2024/03/03 09:26:55 NOTICE: bisync is EXPERIMENTAL. Don't use in production!
2024/03/03 09:26:55 DEBUG : Lock file created: $HOME/.cache/rclone/bisync/etc_scripts..GoogleDrive_ZZ_Sync_Scripts.lck
2024/03/03 09:26:55 INFO  : Synching Path1 "/etc/scripts/" with Path2 "GoogleDrive:ZZ_Sync/Scripts/"
2024/03/03 09:26:55 INFO  : Path1 checking for diffs
2024/03/03 09:26:55 INFO  : - Path1    File is newer                       - zz_test.sh
2024/03/03 09:26:55 INFO  : Path1:    1 changes:    0 new,    1 newer,    0 older,    0 deleted
2024/03/03 09:26:55 INFO  : Path2 checking for diffs
2024/03/03 09:26:55 INFO  : Checking access health
2024/03/03 09:26:55 INFO  : Found 1 matching ".zz_rclone_access_test_do_not_delete" files on both paths
2024/03/03 09:26:55 INFO  : Applying changes
2024/03/03 09:26:55 INFO  : - Path1    Queue copy to Path2                 - GoogleDrive:ZZ_Sync/Scripts/zz_test.sh
2024/03/03 09:26:55 INFO  : - Path1    Do queued copies to                 - Path2
2024/03/03 09:26:55 DEBUG : file1: Excluded
2024/03/03 09:26:55 DEBUG : file2: Excluded
2024/03/03 09:26:55 DEBUG : file3: Excluded
2024/03/03 09:26:55 DEBUG : file4: Excluded
2024/03/03 09:26:55 DEBUG : file5: Excluded
2024/03/03 09:26:55 DEBUG : file6: Excluded
2024/03/03 09:26:55 DEBUG : file7: Excluded
2024/03/03 09:26:55 DEBUG : .zz_rclone_access_test_do_not_delete: Excluded
2024/03/03 09:26:55 DEBUG : file8: Excluded
2024/03/03 09:26:55 DEBUG : file9: Excluded
2024/03/03 09:26:55 DEBUG : Google drive root 'ZZ_Sync/Scripts': Waiting for checks to finish
2024/03/03 09:26:55 DEBUG : zz_test.sh: Sizes differ (src 300 vs dst 283)
2024/03/03 09:26:55 DEBUG : Google drive root 'ZZ_Sync/Scripts': Waiting for transfers to finish
2024/03/03 09:26:55 DEBUG : Google drive root 'ZZ_Sync/Scripts': Skipping btime metadata as can't update it on an existing file: 2024-03-03T09:26:45.686458072-08:00
2024/03/03 09:26:57 DEBUG : zz_test.sh: md5 = OMITTED OK
2024/03/03 09:26:57 INFO  : zz_test.sh: Copied (replaced existing)
2024/03/03 09:26:57 INFO  : Updating listings
2024/03/03 09:26:58 INFO  : Validating listings for Path1 "/etc/scripts/" vs Path2 "GoogleDrive:ZZ_Sync/Scripts/"
2024/03/03 09:26:58 DEBUG : Lock file removed: $HOME/.cache/rclone/bisync/etc_scripts..GoogleDrive_ZZ_Sync_Scripts.lck
2024/03/03 09:26:58 INFO  : Bisync successful
2024/03/03 09:26:58 INFO  : 
Transferred:   	        300 B / 300 B, 100%, 149 B/s, ETA 0s
Checks:                 1 / 1, 100%
Transferred:            1 / 1, 100%
Elapsed time:         3.7s

2024/03/03 09:26:58 DEBUG : 8 go routines active

Here is the debug output WITHOUT "--partial-suffix"

2024/03/03 09:16:55 DEBUG : rclone: Version "v1.65.2" starting with parameters ["rclone" "bisync" "/etc/scripts/" "GoogleDrive:/ZZ_Sync/Scripts/" "--check-access" "--check-filename" ".zz_rclone_access_test_do_not_delete" "--metadata" "--verbose" "--progress" "-vv" "--log-file" "$HOME/.cache/rclone/rclone-test.log"]
2024/03/03 09:16:55 DEBUG : Creating backend with remote "/etc/scripts/"
2024/03/03 09:16:55 DEBUG : Using config file from "$HOME/.config/rclone/rclone.conf"
2024/03/03 09:16:55 DEBUG : fs cache: renaming cache item "/etc/scripts/" to be canonical "/etc/scripts"
2024/03/03 09:16:55 DEBUG : Creating backend with remote "GoogleDrive:/ZZ_Sync/Scripts/"
2024/03/03 09:16:56 DEBUG : Google drive root 'ZZ_Sync/Scripts': 'root_folder_id = OMITTED' - save this in the config to speed up startup
2024/03/03 09:16:56 DEBUG : fs cache: renaming cache item "GoogleDrive:/ZZ_Sync/Scripts/" to be canonical "GoogleDrive:ZZ_Sync/Scripts"
2024/03/03 09:16:56 NOTICE: bisync is EXPERIMENTAL. Don't use in production!
2024/03/03 09:16:56 DEBUG : Lock file created: $HOME/.cache/rclone/bisync/etc_scripts..GoogleDrive_ZZ_Sync_Scripts.lck
2024/03/03 09:16:56 INFO  : Synching Path1 "/etc/scripts/" with Path2 "GoogleDrive:ZZ_Sync/Scripts/"
2024/03/03 09:16:56 INFO  : Path1 checking for diffs
2024/03/03 09:16:56 INFO  : - Path1    File is newer                       - zz_test.sh
2024/03/03 09:16:56 INFO  : Path1:    1 changes:    0 new,    1 newer,    0 older,    0 deleted
2024/03/03 09:16:56 INFO  : Path2 checking for diffs
2024/03/03 09:16:57 INFO  : Checking access health
2024/03/03 09:16:57 INFO  : Found 1 matching ".zz_rclone_access_test_do_not_delete" files on both paths
2024/03/03 09:16:57 INFO  : Applying changes
2024/03/03 09:16:57 INFO  : - Path1    Queue copy to Path2                 - GoogleDrive:ZZ_Sync/Scripts/zz_test.sh
2024/03/03 09:16:57 INFO  : - Path1    Do queued copies to                 - Path2
2024/03/03 09:16:57 DEBUG : file1: Excluded
2024/03/03 09:16:57 DEBUG : file2: Excluded
2024/03/03 09:16:57 DEBUG : file3: Excluded
2024/03/03 09:16:57 DEBUG : file4: Excluded
2024/03/03 09:16:57 DEBUG : file5: Excluded
2024/03/03 09:16:57 DEBUG : file6: Excluded
2024/03/03 09:16:57 DEBUG : file7: Excluded
2024/03/03 09:16:57 DEBUG : .zz_rclone_access_test_do_not_delete: Excluded
2024/03/03 09:16:57 DEBUG : file8: Excluded
2024/03/03 09:16:57 DEBUG : file9: Excluded
2024/03/03 09:16:57 DEBUG : Google drive root 'ZZ_Sync/Scripts': Waiting for checks to finish
2024/03/03 09:16:57 DEBUG : zz_test.sh: Sizes differ (src 283 vs dst 279)
2024/03/03 09:16:57 DEBUG : Google drive root 'ZZ_Sync/Scripts': Waiting for transfers to finish
2024/03/03 09:16:57 DEBUG : Google drive root 'ZZ_Sync/Scripts': Skipping btime metadata as can't update it on an existing file: 2024-03-03T09:16:46.885432447-08:00
2024/03/03 09:16:59 DEBUG : zz_test.sh: md5 = OMITTED OK
2024/03/03 09:16:59 INFO  : zz_test.sh: Copied (replaced existing)
2024/03/03 09:16:59 INFO  : Updating listings
2024/03/03 09:17:00 INFO  : Validating listings for Path1 "/etc/scripts/" vs Path2 "GoogleDrive:ZZ_Sync/Scripts/"
2024/03/03 09:17:00 DEBUG : Lock file removed: $HOME/.cache/rclone/bisync/etc_scripts..GoogleDrive_ZZ_Sync_Scripts.lck
2024/03/03 09:17:00 INFO  : Bisync successful
2024/03/03 09:17:00 INFO  : 
Transferred:   	        283 B / 283 B, 100%, 141 B/s, ETA 0s
Checks:                 1 / 1, 100%
Transferred:            1 / 1, 100%
Elapsed time:         4.1s

2024/03/03 09:17:00 DEBUG : 8 go routines active
2

bisync is changing. Big time. For better. With tones of fixes and improvements.

Wait for rclone v1.66 or try beta now and see bisync beta docs.

3

Thank you.

Is there a mailing list that will alert me when the latest versions are released?

4

Use rclone github - and set custom Watch to be notified about releases

5

The --partial-suffix flag requires an argument, so I'm guessing what is happening here is that it is interpreting -verbose as that argument, and therefore ignoring it as a flag.

Does the following work?

rclone bisync PATH1 PATH2 --partial-suffix ".partial" --verbose --progress --log-file LOGFILE

[EDIT: --verbose should have two dashes, or one with the -v shorthand.]

Also true :slightly_smiling_face:

You can "watch" the GitHub repo to be notified of new releases. They are also announced in this forum.

For bisync specifically, this project is used to track progress.

1 Like
6

the correct flag is --verbose not -verbose, see docs:

-v, --verbose count Print lots more stuff (repeat for more)

1 Like
7

Thank you, the single dash is a typo which is not included in the command which was used to reproduce the behavior.

8

Yes, including ".partial" fixed the problem. I could be wrong, but, I thought that I read in the man page that ".partial" was the default.

9

It is. So if you want ".partial" you can just omit the --partial-suffix flag entirely. --partial-suffix is mainly useful for specifying some other custom suffix like --partial-suffix .tmp or something.

(I used ".partial" in my example only to debug and see if the flag is working.)

1 Like
10

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.