Best flags for a specific use of rclone bisync

Help and Support
1

What is the problem you are having with rclone?

It's not a problem, I need advice. I'm lost with the so many flags for rclone (and rclone bisync which is the one that interests me now).

I want to run an infinite loop of "rclone bisync" to keep my local files in sync with the cloud. Here there are many very badly explained exemples (eg what does the -RCs flag do???).

But I want to use a command that 1- I understand fully and 2- that's adapted to my needs.

More about my needs: five folders less than 10GB big I'd like to keep in constant sync with Proton Drive.

So if someone knows how best to do that... I do know that both Proton Drive and rclone bisync are newcomers in the rclone ecosystem, but they would really fit my needs I think. Oh and the synchro is between the cloud and several devices, that's why it needs to be biderectional.

The only commands I've tried so far were:

rclone bisync proton:Pictures /home/tome/Pictures --resync --create-empty-src-dirs -MPc -v

...that took 58 minutes for a 2,6 GiB, 1 633 files, 56 sub-folderdirectory containing pictures, and after that:

rclone bisync proton:Pictures /home/tome/Pictures --resilient --create-empty-src-dirs -MPc -v

...which took 32min, not even half the time....

I really want a much, much faster solution, if possible.

rclone v1.64.2

  • os/version: opensuse-tumbleweed (64 bit)
  • os/kernel: 6.5.8-1-default (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.21.3
  • go/linking: dynamic
  • go/tags: none

Are you on the latest version of rclone? You can validate by checking the version listed here: Rclone downloads
--> Yes (rclone v1.64.2)

Which cloud storage system are you using? (eg Google Drive)

Proton Drive

[proton]
type = protondrive
username = myusername
password = XXX
mailbox_password = XXX
client_uid = XXX
client_access_token = XXX
client_refresh_token = XXX
client_salted_key_pass = XXX
replace_existing_draft = true
### Double check the config for sensitive info before posting publicly
2

@nielash would you mind to provide your comment here? Any obvious way to speed up bisync?

1 Like
3

--ignore-listing-checksum can potentially make it a lot faster. Since one of your remotes is local, bisync is otherwise having to calculate hashes for every local file (much slower than retrieving a stored hash), even though it will not even end up using these hashes for anything (because bisync currently only supports modtime.)

Another one I'd recommend is --check-sync=false, which will make the run slightly faster, because it will skip the step of re-loading both listings into memory at the end and comparing them. (For an integrity check that is more robust than --check-sync, see the suggestion here.)

Additionally, I recently wrote at length here about bisync's current speed issues and my plan to address them. I think it's worth a read if the topic interests you, but the TL;DR is: I agree that bisync is currently much slower than it should be, and it will be dramatically improved in the near future. (The new version is now actually already developed locally and working great -- I just need to find the time to clean up the commit history and write new docs before opening a PR.)

2 Likes
4

Thank you very much! I'm trying these flags right now. Could you also explaine what the -MPc flag does? I found no explaination for it in the official doc.

5

-MPc is shortened way for -M -P -c which is shortened for --metadata --progress --checksum. Maybe it is good idea to avoid shortened flags in documentation for sake of clarity.

  -M, --metadata  If set, preserve metadata when copying objects
  -P, --progress  Show progress during transfer
  -c, --checksum  Check for changes with size & checksum (if available, or fallback to size only).

All explained in docs.

To quickly find some short flag reference use:

$ rclone help flags | grep "\-P,"
  -P, --progress                            Show progress during transfer

Then you can find --progress in docs for more detailed explanation.

1 Like
6

Im having trouble moving mega: to protondrive: I need help

7

open a new thread - provide all details. Somebody will help you.

8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.