What is the problem you are having with rclone?
I'm periodically syncing with some items on Internet Archive. Occasionally, an item will be removed from Internet Archive for a variety of reasons (uploader deletes it, specious (or maybe even valid) DMCA claim, etc.). At that point, the collection becomes empty (no files are listed via rclone ls or similar operations) and the frontend view of the collection on Internet Archive is replaced with a "This item is no longer available" page.
Because the collection is now empty, a sync results in a deletion of all local content.
I'm currently just avoiding deletion of local content in these situations by doing a dry run of every sync before doing the "real" sync. This isn't perfect, though. (It requires some manual labor, and what if a collection manages to get taken down between the dry run and the real sync?)
Is there any reliable, trivial way to detect that the collection has been taken down and, consequently, not sync? I've been thinking of just grabbing the collection page in my sync script and checking for text like "no longer available", but I'm not sure how reliable this might be.
Has anyone come up with a good solution for this in your own efforts, and wouldn't mind sharing?
Thanks!