Writing Jobs That Can Rerun

A scheduler that retries a failed job makes one promise: the job will run again. It makes no promise about what the failed attempt left behind. That attempt may have written half a file, or appended rows that the next attempt appends again. The Airflow documentation states the rule that follows: because Airflow can retry a task, "the tasks should produce the same outcome on every re-run."

A job that meets this rule can run again at any time, with no one checking first what the last run did (a rerunnable job)