Support for "plan files"

I use rclone on a cron job to back up certain directories to S3. Since I am paranoid about bugs in my own scripts, etc., the cron job does a --dry-run to generate all the NOTICEs about what it will do and e-mails me those for manual approval. If everything looks good, I reply to the e-mail and rclone re-does the same sync to apply those changes, without the --dry-run.

The problem, of course, is that theoretically something may have changed (either locally or remotely) in between the times that the --dry-run was executed and when I approved and applied those changes. I would have no way of even detecting this happened except to manually diff the sync output against the dry run (and even then it would be to late to prevent it from happening, it would just notify me).

What I’m proposing is an additional flag to --dry-run called, maybe, --plan-file, that outputs a machine-readable version of all the NOTICEs to a file which can then be passed back to the sync call (without dry-run) to only apply the changes listed in the plan file. That way, if I approve a plan file, I can be confident I know exactly what changes will be applied.

Users of tools like Terraform will be familiar with this concept; it is how every single plan/apply cycle works.

I’d be happy to put together a PR with such functionality (I’ll probably be coding it for my own use anyway), I just wanted to get some feedback, and to find out if such a feature exists somewhere and I just haven’t found it.

Thanks! :slight_smile:

Sounds like an interesting idea. The --files-from flag does part of what you might need, limiting the actions to a given set of files. You could probably process the logs of a --dry-run to produce it…

Yeah, after reading a bit more, I found the --files-from flag, which is close but not quite there. Firstly, as far as I can tell, it only applies to copy operations to the remote, not back. And secondly, while it limits the scope of potential damage, there are lots of bugs (probably most) that would manifest on a single group of files.

The plan file should be quite strict - it should even encode a checksum/modification time of the files it intends to modify so that if they have changed underneath it, it will detect that too.

But the --files-from feature encourages me that the feature I am looking for would be accepted if written :slight_smile:

rclone syncs are all uni-directional…