Monitor Apache Airflow DAGs

Airflow shows failed runs in its UI, but only to someone who looks. A DAG that’s paused, a scheduler that’s down, or a run stuck in queued doesn’t fail at all; it just doesn’t happen. A heartbeat check catches all of these: the DAG reports to Pingstack in a final task, and Pingstack alerts you when that report doesn’t arrive on time.

1. Create a heartbeat check that matches the schedule

Find the DAG’s schedule:

@dag(schedule="0 2 * * *", start_date=pendulum.datetime(2026, 1, 1, tz="UTC"), catchup=False)

Create a heartbeat check and set its schedule to Cron with the same expression. Airflow interprets the cron in the timezone of the DAG’s start_date, or the instance’s default timezone (UTC unless changed). Choose the same timezone in Pingstack. See schedules and timezones.

Set a grace period that covers the whole DAG’s runtime, time waiting for worker slots, and task retries. Attach a notification group.

2. Store the ping URL as an Airflow Variable

Copy the location’s ping URL from the check in Pingstack and save it as a Variable named pingstack_nightly_report_url, under Admin › Variables, or in your secrets backend if you use one. Don’t put it in the DAG file.

3. Report from a final task

Add a task that runs after everything else:

import pendulum
import requests
from airflow.decorators import dag, task
from airflow.models import Variable


@dag(schedule="0 2 * * *", start_date=pendulum.datetime(2026, 1, 1, tz="UTC"), catchup=False)
def nightly_report():
    @task
    def build_report():
        ...

    @task
    def send_report():
        ...

    @task
    def report_to_pingstack():
        url = Variable.get("pingstack_nightly_report_url")
        requests.get(url, timeout=10).raise_for_status()

    build_report() >> send_report() >> report_to_pingstack()


nightly_report()

Tasks run only when every upstream task succeeded (the default all_success trigger rule), so a failed run never reaches report_to_pingstack and Pingstack alerts you when the deadline passes.

On Airflow 3, import dag, task and Variable from airflow.sdk instead.

4. Confirm the first report

Make sure the DAG is unpaused: new DAGs start paused by default. Then trigger a run from the UI (Trigger DAG) or the CLI:

airflow dags trigger nightly_report

When it finishes, open the check in Pingstack and confirm the location received a heartbeat and shows a next deadline.

Optional: report a number

Pass a count from an earlier task through XCom and send it as a numeric value:

@task
def report_to_pingstack(rows: int):
    url = Variable.get("pingstack_nightly_report_url")
    requests.get(url, params={"value": rows}, timeout=10).raise_for_status()

Set warning and critical thresholds on the check, so a run that succeeds but processes nothing still raises an alert.

Why a healthy DAG might still alert

• Paused DAG. A paused DAG never runs. Pingstack alerts, which is usually what you want.
• When runs start. Airflow starts a run when its data interval ends. For a daily 0 2 * * * schedule, that’s still 02:00, but custom timetables can differ. Match the check to when runs actually start.
• Retries and queues. Retries and busy workers push the final task later. Lengthen the grace period.
• Timezone mismatch. The check and the DAG’s timezone must agree.

If alerts still don’t match what you expect, see troubleshooting monitoring.