Sync without versioning

Help and Support
1

What is the problem you are having with rclone?

When conducting rclone sync, a new file version is created if the local file is different than the existing file on the cloud. I would like to sync without creating a new file version; that is, I would like to completely remove the old existing file on the cloud.

There may be two possible approaches, but neither works well:

  1. Delete old versions of the files after syncing. However, rclone cleanup doesn't accomplish this for GoogleDrive - it only empties the trash. Manually removing all the old versions is inconvenient.

  2. Delete the targeted files on the cloud before syncing. I don't know a good (and automatic) way to do so, other than parsing the output from rclone sync --dry-run.

Any suggestions and thoughts are welcomed :slight_smile:

What is your rclone version (output from rclone version)

v1.57.0-beta.5684.8b8a943dd

Which cloud storage system are you using? (eg Google Drive)

GoogleDrive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync E:/Test GoogleDrive:/Test
rclone cleanup GoogleDrive:/Test

The rclone config contents with secrets removed.

Not applicable. I only have credentials in the config file.

A log from the command with the -vv flag

Not applicable.

2

What's the actual issue with having more versions of a file?

You could do a dry run, delete everything and run a new sync which requires some scripting.

I don't think the API provides a way to delete all versions while copying it so not sure this would be an easy item to do.

Deleting files before sounds awful if something bad happens too.

3

Thanks. Because multiple versions of a file count toward the storage. Due to my account setting (part of an education account), the versions are kept permanently unless I manually delete them. I am not the organization admin and cannot change the setting, hence the ask.

I was just wondering if there is any other approach beyond what I had written in the main post. What you said is what I had known, which is bad news :hushed:

4

Other than scripting, I'm not aware of any other way to do it.

5

OneDrive and Jottacloud backends have option no_versions for this. Could perhaps be implemented for Google Drive as well..

6

I don't see it in the API:

Am I missing something?

7

No, I'm not saying that. I was mostly just pointing out that we had such options in rclone for some backends...

Actually the Jottacloud implementation of it does perform a delete of existing file before uploading the new file.

And Google Drive API does have revisions list and revisions delete endpoints.

8

Seems like a sensible idea. I don't see an obvious way to implement it though other than reading the revisions list and deleting the old revisions after a successful upload which would take quite a few transactions.

BTW this should be possible with --delete-before

 --delete-before     When synchronizing, delete files on destination before transferring
9

Thanks Nick for the tip on --delete-before. But it doesn't seem to solve my user case - maybe I'm doing something wrong. I find it only deleting the files that no longer exist on the source directory. If a file exists on both source and destination and if the two don't match, --delete-before won't delete the file from destination; it still uploads on top of the file and makes a new version. What I was hoping is to delete (before syncing) not only the files that no longer exist on the source side, but also those files that need update. In other words, I was hoping to delete every file that doesn't match the source and then do a clean re-upload.

10

I thought delete before is for deleting files not in the destination and it does that before the sync starts as opposed to after so I don't think it fits for your use case.

11

Yes you are right, didn't think that through...

You could use rclone check --differ to find files which are present on both sides and then feed that into rclone delete to remove them before you sync, something like this

rclone check --differ modified.txt E:/Test GoogleDrive:/Test
rclone delete --files-from modified.txt GoogleDrive:/Test
rclone sync E:/Test GoogleDrive:/Test

That scans the directories twice, but it should do what you want. Test carefully before using for real!

12

Thank Nick! Exactly what I wanted :laughing: Much appreciated! :slight_smile:

1 Like
13

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.