Copy to S3 incrementally

That is a good optimization.

If the files are never updated you can use --size-only which might be better than those two flags.

That will speed up the listing times greatly at the cost of memory.

Once everything is in sync then using rclone copy --max-age --no-traverse --size-only) will probably be the most efficient way of doing a top-up sync.

If files never get deleted then copy will work fine.

I'd be generous with the --max-age so if you crontab runs every hour, I might use --max-age 24h.

You can always do a topup sync without --max-age - with rclone copy --size-only --fast-list

I wouldn't bother with --ignore-existing - rclone will skip them equally well with --size-only