Full sync but excluding certain files from deletion on the remote

Help and Support
1

What is the problem you are having with rclone?

Good morning all!

I would really appreciate some guidance on what's the best way to accomplish a task I've been having difficulty with.

I need to sync a local folder to a remote folder on Google Drive. I already have the Remote fully synced in the past, however a lot of things have changed on the Source folder since. Some large video files have been deleted from Source that I would like to ensure are preserved on the Destination after re-syncing, yet for all other files I would like Rclone to sync the Destination to look exactly like the source.

So essentially, I need a way to do full rclone sync operation that replicates the entire contents of Source to the Remote, including copying any new video files it finds in Source, but that does not delete any video files already present in the Destination even if they were removed from the Source. All other files should be deleted on the Destination if they no longer exist in Source as in a regular syncing operation. Video files are understood to be files ending with the extensions ".mp4", ".mkv" or ".webm".

What would be the best way to accomplish this task? Thank you very much for the assistance!

What is your rclone version (output from rclone version)

v1.57.0

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Ideally something like rclone sync Source Remote: --exclude-from-deletion *.mp4

The rclone config contents with secrets removed.

[Google-Drive]
type = drive
acknowledge_abuse = true
keep_revision_forever = true
client_id = 
client_secret = 
token = 
scope = drive
2

So this doesn't match this

You contradict yourself here so I'm not sure what would win.

You'd want to list those files you want to keep and exclude them specifically from the sync.

3

I need existing video files to not be deleted from the destination (i.e. they exist only in the destination), but if they only exist in the source I need them to still be copied to the destination. All other file types that are not video files should still sync as normal.

4

Yeah, that doesn't quite work.

Sync makes destination identical to the source. You can filter the source to match only video files, but in your scenario you asking to not delete the destination if it doesn't match the source.

Do you want to copy source to destination and filter on video files?

For the second, you can just sync as normal and exclude the video file patterns as that sounds like it operates as normal. --dry-run and testing with rclone first would be very important.

5

I suppose the idea of doing it in two sets of operations, first a sync excluding video files, then a copy to just move the remaining video files might work. How would I go about excluding the video files from the sync operation? What kind of pattern should I use? The documentation for filter seems a little confusing to me. Thank you for the input so far!

6

You'd have to figure out what your logic is to determine what's included or not.

If you can explain that, we can translate it to a filter assuming there is some pattern there.

7

The goal is to exclude all .mkv, .mp4 and .webm files from the Source folder and all subfolders so that they dont get synced then I guess. How would that look like as a filter?

8

did not see the output of rclone version posted, so not sure which operating system,

on linux
rclone ls Source --exclude='*.{mkv,mp4,wbem}'

on windows
rclone ls Source --exclude="*.{mkv,mp4,wbem}"

1 Like
9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.