Sync: Keep local directories and files, only download if files do not exists

[mikoim]

Hello.
I am in a difficult situation because I have executed a complex operation. Is there any way to solve this problem? I'm planning to patch rclone to disable delete operation against local directories and files.

Situation:

• S3 bucket and local directory had been synced.
• External service created new files and directories on S3 after last sync (>1,000,000)
• I created files and directories on local to merge other bucket (>100,000)
• I modified files on local. (>100,000)

What I want to accomplish:

• Download created files on S3 if the file does not exist on local
• Keep modified and created files on local

Trials:

rclone --checkers 32 --transfers 32 --progress sync wasabi:foobar /foobar

• New files and directories on local will be deleted
• Modified files on local will be overwritten

rclone --checkers 32 --transfers 32 --ignore-existing -progress sync wasabi:foobar /foobar

• New files and directories on local will be deleted

rclone --checkers 32 --transfers 32 --ignore-existing --exclude-from keep -progress sync wasabi:foobar /foobar

• It seems to work but it is too slow to sync. (ignore list has over 100K lines)

---

[ncw (Nick Craig-Wood)]

@mikoim

rclone --checkers 32 --transfers 32 --ignore-existing -progress sync wasabi:foobar /foobar

• New files and directories on local will be deleted

This is almost right, but use copy instead of sync

[asdffdsa (jojothehumanmonkey)]

hello and welcome to the forum,

might try using --fast-list.

@mikoim

patch rclone to disable delete operation against local directories and files.
Keep modified and created files on local
only download if files do not exists

might try --immutable
might try --ignore-existing

[mikoim]

Finally, copy and --ignore-existing accomplished this task. Thank you for your advice! @ncw @asdffdsa

rclone --checkers 32 --transfers 32 --fast-list --ignore-existing -progress copy wasabi:foobar /foobar