Orchestration with Airflow

The meter pipeline from ETL and ELT has three steps: extract a day of readings, load them into raw_readings, and build daily_usage. Here are those steps written for Apache Airflow 3, a Python tool that runs such steps on a schedule and in order:

from datetime import timedelta

import pendulum
from airflow.sdk import Asset, dag, task

# ...
DAILY_USAGE = Asset("daily_usage")


@dag(
    schedule="@daily",
    start_date=START,
    catchup=False,
Copy