Use Rclone instead of Backup & Sync?

Help and Support
1

What is the problem you are having with rclone?

Due to limitations with the Google Backup & Sync Windows app, I was thinking about using Rclone to do the syncing for me. I already use it for a few things. I would probably schedule it to run a couple times a day. Is there anything wrong with this plan? I don't know how much data I sync in a typical day, but I am guessing on average we are talking < 100 MB.

What is your rclone version (output from rclone version)

1.5.1

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Window 10 Pro

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Paste command here

The rclone config contents with secrets removed.

Paste config here

A log from the command with the -vv flag

Paste  log here
2

sure, that is a good plan.

"limitations with the Google Backup & Sync Windows app".
what does that google app lack, that you are looking to rclone for?

--backup-dir is a good flag to use for backups.

you can run rclone in a loop

:looper
rclone sync local remote:
timeout /t 3600 /nobreak
goto looper

or use task scheduler.

if you need something more complex, let me know

3

The main limitation (for me) are you cannot exclude directories (except manually) or files (except by extension).

An example:

  1. I uncheck 2 folders in the app to not sync
  2. I add a new folder to GDrive

The result is the new folder doesn't sync automatically and I have to go in and check it to sync since I am not syncing everything.

4

yeah, that is an accident waiting to happen.
also, when testing you can use --dry-run

5

Okay. Thanks. I was mainly worried about efficiency and if Google had any limitations I might hit (which I thought unlikely given my limited usage).

Edit: Another solution might be to keep using their app but run a scheduled Rclone to pick any folders it misses.

6

gdrive does have some limitations.
https://rclone.org/drive/#limitations

the most important thing is to make sure you do this
https://rclone.org/drive/#making-your-own-client-id

7

Wow - I used to think it was just me, that explains a lot.

If your sync is from Drive to Local, and it is a while since I tried to use Backup & sync, the biggest issue I had was that for native Google docs - writely, sheets etc - it only created a hyperlink in the local backup which was no backup at all. It also used to choke intermittently on files with long paths/filenames, and which were being actively changed during backup, and never seemed to fix itself elegantly*.

rclone is a happier experience if you want t a backup or local copy you know you can rely on.

  • Far too often had to disable sync for a Drive file, make a copy of it's content, delete both drive and local copy; recreate and hope.
8

Is the info in that link on making your own ID dated? It didn't work for me. It seems like the process might have changed. I got an API Key but no Client ID.

9

Not sure what you mean. You get a Client ID and a Client Secret if you follow the steps.

What is dated on the steps? Can you be more specific?

10

It was user error. I didn't see Create Client under the Credentials drop down. Thanks

11

Do you mean this? It should say create oauth client id.

image

12

Yes. I wasn't paying attention.

13

Is there a way to differentiate between a deleted file and a new file (that hasn't been synced yet)?

For example, if I delete a file off GD and leave it on my PC it just gets re-added to GD during the copy.

Obviously, if I add a file to one I want it synced to the other.

The obvious solution is to delete in both places if I have to.

I believe the Backup and Sync app handles this properly (file deleted one place gets deleted both places).

14

imho,
backup and sync are two very different things.
rclone is not a backup solution, but can sort of act like a backup if you use the correct flags in the correct way
rclone is a one way sync/copy/move solution

"file deleted one place gets deleted both places"
this is a bi-directional sync, not a backup.

15

rclone currently does not do this. It synchronises the files in one direction, or the other.

Google's Backup & Sync does attempt to do a bidirectional sync. It depends upon some heroic assumptions and faith in timestamps to decide what to do if both the Drive and local file are changed. Indeed, changes are lost even in simple cases if that happens because typically the Backup & Sync is run only intermittently and is not triggered by a file change. There is no lock or merge capability. In the case of native files there isn't a problem because Backup & Sync only keeps a hyperlink - there isn't really any synchronisation or backup in any case.

If you need bi-directional sync, typically to create edge or offline repositories of your files where changes will be made, then you should use Backup & Sync, not rclone.

If you really want a local copy of all your Drive files, and can work out a workflow that needs sync in only one direction, rclone is your thing.

No simple file sync mechanism is a backup because if a file is erroneously deleted or modified on the master, that is synced through and propagated to the other file storage. Sync mechanisms can be used to marshal files to go into a backup process, or rclone can be sort of set up to do non destructive syncs, but that is it.

I do have a problem with the name 'Backup & Sync' which Google used because it isn't what it does, or wasn't when I last used it.

1 Like
16

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.