Python Models: dbt and Interlace, Side by Side

We have written before that Python and SQL models are interchangeable nodes in the same graph. That is true of Interlace, and it is worth being careful about what it implies about dbt — because the most common version of this comparison, including one we drafted ourselves, gets it wrong.

dbt Python models participate fully in dbt's DAG. A SQL model can ref() a Python model, and a Python model can dbt.ref() a SQL one. The dbt documentation is explicit about it, with examples in both directions. Anyone claiming dbt Python models are "not real DAG nodes" has not read the page.

The differences are real, but they are elsewhere. Every claim below about dbt is from dbt's current Python models documentation; every claim about Interlace was checked against the source.

Where you can run one

This is the largest difference and the one that decides most cases.

dbt Python models require a data platform with a fully featured Python runtime. In practice that means Snowflake (Snowpark), BigQuery (BigFrames), or Databricks (PySpark). The code runs in the warehouse, on the warehouse's Python runtime.

Interlace Python models run in the Interlace process, so they work on any engine — including DuckDB and Postgres, where dbt has no Python model story at all.

The practical consequence: on a DuckDB or Postgres project, a dbt Python model is not slower or more limited, it is unavailable. If your stack is one of the three supported warehouses, dbt's approach has a real advantage in return — the computation happens next to the data, and never crosses the network.

What a model actually is

dbt's Python model is a function with a required signature, called by dbt with two arguments it supplies:

def model(dbt, session):
    dbt.config(materialized="table")
    orders = dbt.ref("stg_orders")
    return orders.groupBy("customer_id").agg(...)

Interlace's is an ordinary function whose parameters name its upstreams:

from interlace import model

@model(strategy="merge", key="user_id")
def user_ltv(by_user):
    for batch in by_user.reader():
        yield score(batch)

The difference that matters is testability. @model registers the model and returns the function unchanged, so it is callable in a unit test with no warehouse, no session and no dbt object:

>>> from models.user_ltv import user_ltv
>>> callable(user_ltv)
True
>>> user_ltv(fake_batches)     # a plain call, no infrastructure

dbt's model(dbt, session) requires both a dbt object and a live warehouse session, so testing it means testing through dbt.

The data type differs too. dbt hands you a warehouse DataFrame — Snowpark, BigFrames or PySpark depending on the platform, each with a different API, which is a portability cost if you ever change warehouse. Interlace hands you Arrow, streamed as RecordBatches, so memory stays bounded on inputs far larger than RAM.

What you can materialise

Both tools restrict Python models, in different places.

dbt supports table (default) and incremental. It does not support view or ephemeral, and Python models cannot reference ephemeral models at all.

Interlace supports only virtual — its owned, snapshot-backed default. view and ephemeral are rejected because both require SQL the engine can evaluate; table and file are rejected because terminal delivery is SQL-only for now. The errors say so directly:

Python models cannot be ephemeral; ephemeral requires SQL (it is inlined as a CTE)
Python models cannot materialise as 'table' yet; write a SQL model over this model's output

Interlace Python models can read an ephemeral SQL model, which dbt's cannot. The upstream is inlined as a CTE and never materialises:

 Model        Output    Strategy   Depends on   Rows   Time
 raw          virtual   replace    —              +3   0.06s
 pyconsumer   virtual   replace    mid            +3   0.06s

mid is the ephemeral model. It does not appear, because it was compiled into its consumer.

Incremental work

Here dbt is straightforwardly ahead. dbt Python models support incremental with the same incremental strategies as SQL models, subject to adapter support.

Interlace Python models can use incremental when they declare a key — the Arrow output is staged and the window's rows are upserted. Without a key it is refused, because a Python function has already computed everything before the window could narrow it; the window would bound what is written, not what is done. cursor is the tool for bounding the fetch.

So the gap is narrower than it looks, but real: dbt pushes the incremental predicate into the query it generates, and for a Python model we cannot.

Side by side

Choosing

Neither column is a win. The honest decision rule:

dbt if your warehouse is Snowflake, BigQuery or Databricks and you want computation to stay next to the data, or if you need incremental Python models. Both are real advantages and we do not have an answer to either.

Interlace if your engine is DuckDB or Postgres, where dbt Python models do not exist; if you want to unit-test transformation logic as plain functions; or if the memory profile of streaming Arrow matters more than warehouse-side execution.

The thing worth avoiding is picking on a claim neither tool actually makes. dbt Python models are full DAG participants. Ours cannot do incremental. Both statements are inconvenient for the usual marketing, and both are true.

Start with the Python models guide, or install it:

pip install interlaced