Skip to content

Automation Conditions

Teamster uses four custom AutomationCondition builders in teamster.core.automation_conditions that replace Dagster's default AutomationCondition.eager() for dbt assets. All four share a common skeleton (_build_dbt_condition) and differ only in what triggers a materialization request beyond newly_missing.

VIEW vs TABLE

The core distinction is that views are computed on read — materializing a view when upstream data changes just re-executes the same SQL against already-updated data, so re-triggering on upstream changes is redundant.

dbt_view_automation_condition()

For materialized: view dbt models. Triggers when:

  • The asset is newly missing (e.g. first run, or dropped in BigQuery)
  • The compiled SQL has changed (code_version_changed) — a dbt code deploy
  • The previous run failed

any_deps_updated is intentionally omitted. When an upstream table materializes, view assets correctly do not re-run.

dbt_table_automation_condition()

For materialized: table models. Triggers on everything above, plus:

  • Upstream data changes — including changes propagated through intermediate view chains (up to 10 levels deep via recursive dependency traversal)

The recursive traversal means a table asset will re-run when any upstream table materializes, even if the direct dependency is a chain of views.

union_relations views

Views that use the union_relations macro are a special case. Unlike regular views, their compiled SQL resolves column lists at run time — the macro expands upstream table columns into an explicit SELECT with UNION ALL. If an upstream regional table adds or removes a column and the view is not re-run, its compiled SQL becomes stale.

Importantly, code_version_changed does not catch this. Dagster derives code_version from a SHA1 of the model's raw_code (the Jinja source), not the compiled SQL. When the union_relations macro produces different output because upstream schemas changed, the raw source file hasn't changed, so the code version stays the same.

dbt_union_relations_automation_condition()

A third condition that sits between view and table. Triggers on everything in the view condition, plus one extra trigger:

  • A parent's code change lands — the view re-runs on the tick a parent's post-code-change materialization lands. "Code change" covers the parent's own raw SQL or any of its ancestors'. A view parent counts when it rebuilds itself, and the check also looks through it to the table behind it (up to 10 levels).

It does not re-run on the deploy tick itself. At that point the parent still has its old schema, and re-running then compiles the stale column list (#4290). The trigger stays armed until the view is requested, so a run that started before the parent landed cannot consume it.

Unlike the table condition, this does not trigger on upstream data changes (any_deps_updated). Re-materializing views on every upstream data refresh would waste Dagster credits and Kubernetes resources.

Known limits:

  • A parent that reloads and finishes rebuilding within one sensor tick is missed.
  • If a parent table does a data rebuild before its changed ancestor rebuilds, the view re-runs early against the old schema.
  • A second parent that lands on the tick right after the view was requested is dropped. This usually does no harm, because the requested run compiles after the landing. The exception is a table behind a view parent.

In each case, rematerialize the view from the Dagster UI.

The CustomDagsterDbtTranslator auto-detects these views by checking for "union_relations" in the model's raw_code.

Cron-cadence tables

dbt_cron_automation_condition(cron_schedule, cron_timezone)

For expensive materialized: table models whose consumers refresh on a schedule (nightly extracts, Cube pre-aggregation refresh keys). Under the eager table condition, a wide table rebuilds on every upstream data refresh — freshness nothing consumes. This condition swaps the ancestor-updated trigger for cron_tick_passed: the table rebuilds on each cron tick instead.

Everything else in the shared skeleton is retained — newly_missing, code_version_changed (deploys still rebuild immediately), and the deps-missing/in-progress guards. The tick trigger is wrapped in .since(newly_requested | newly_updated), so a tick that fires while a dep is missing or in progress is latched, not lost — the request goes out once the guard clears.

It is deliberately not Dagster's stock AutomationCondition.on_cron(): that condition's all_deps_updated_since_cron gate waits for every dep to update after each tick, so one slower-cadence upstream (e.g. a weekly model) silently starves the schedule. cron_tick_passed is unconditional — a rebuild on a tick where nothing changed is accepted waste (one scan) in exchange for a deadlock-free cadence.

Opt a model in via meta.dagster.automation_condition.cron_schedule in its properties YAML:

models:
  - name: my_expensive_model
    config:
      materialized: table
      meta:
        dagster:
          automation_condition:
            cron_schedule: 0 0 * * *
            # cron_timezone: America/New_York  # optional

Contract and constraints:

  • Tables only. The translator ignores the key on view / ephemeral models — a view must keep its view or union_relations condition (losing ancestor code-version detection makes a stale union_relations view non-self-healing).
  • cron_timezone defaults to the code location's LOCAL_TIMEZONE (passed to CustomDagsterDbtTranslator at construction); set the meta key only to deviate from it.
  • The cron expression is not validated at dbt parse — a typo surfaces when Dagster loads definitions. Multi-tick expressions (0 0,10,15 * * *) are fine; align ticks ahead of the downstream consumers' refresh schedules.
  • Downstream eager tables cascade normally: assets requested on the same evaluation batch into one run, and dbt orders them by DAG position.

Non-dbt partitioned assets

Non-dbt assets (e.g., BigQuery→SFTP extracts) use AutomationCondition.eager() directly when they need reactive triggering. Two important behaviors when a partitioned asset has mixed dep types:

  • Partitioned dep (same PartitionsDefinition): updating partition X triggers only partition X of the downstream asset.
  • Non-partitioned dep: any update fans out to all partitions of the downstream asset.

This means wiring a non-partitioned lookup table as a dep is intentional but has cost — every refresh of that table triggers a full re-extract across all partitions. Design accordingly.

The "Unsynced" indicator

When an upstream table materializes, directly-dependent view assets show as Unsynced in the Dagster UI. This is driven by Dagster's internal data versioning system, not the automation condition.

The automation condition correctly suppresses any run trigger for view assets, but there is no Dagster API to suppress the UI indicator on a per-asset basis.

This is expected behavior, not a misconfiguration. Unsynced views will clear on their next scheduled or code-version-triggered materialization.