Datagrunt 4.6.3: DuckDB 2.0 Compatibility Fix

Datagrunt 4.6.3 is now available, and it is a recommended upgrade.

Datagrunt depends on duckdb>=1.5.3, so a fresh install now resolves DuckDB 2.0. DuckDB 2.0 removed several methods from its relation objects, and on Datagrunt 4.6.2 and earlier every DuckDB-engine CSV read and write then failed with:

AttributeError: Relations no longer have execute(): a relation keeps no result between calls, so every call runs it again. Use rel.materialize() to run it once and keep its rows, or a method such as fetchall() instead.

Datagrunt 4.6.3 works with both DuckDB 1.x and 2.0. The full test suite passes on both backends against DuckDB 1.5.3 (the minimum supported version) and DuckDB 2.0.0.

Deep Dive 1: What Changed in DuckDB 2.0

DuckDB 2.0 removed four methods from DuckDBPyRelation: execute(), fetchone(), fetchmany(), and fetch_df_chunk(). A relation no longer keeps a result between calls; each consuming call (.pl(), .arrow(), .fetchall(), …) runs it.

Internally, Datagrunt’s DuckDB engine imported the CSV into a table and returned that table as an executed relation. 4.6.3 returns the relation itself instead:

# 4.6.2 — DuckDBQueries.create_table()
return self.connection.sql(self.select_from_duckdb_table()).execute()

# 4.6.3
return self.connection.sql(self.select_from_duckdb_table())

Every internal caller either consumes that relation exactly once (.pl() / .arrow()) or only needs the table to exist. A side benefit on DuckDB 1.x: write and export paths (write_csv, write_parquet, …) no longer run a full SELECT * they never used.

Action Needed: query_data() Results Under DuckDB 2.0

With the DuckDB engine, CSVReader.query_data() returns a DuckDB relation unchanged. If your own code calls one of the removed methods on that result, it will fail under DuckDB 2.0 whichever Datagrunt version you run. Use the methods that remain:

from datagrunt import CSVReader

reader = CSVReader("sales_data.csv", engine="duckdb")

relation = reader.query_data(f"SELECT region, SUM(amount) AS total FROM {reader.db_table} GROUP BY 1")

df = relation.pl()           # Polars DataFrame
rows = relation.fetchall()   # list of tuples — works on DuckDB 1.x and 2.0

Under the Hood: Keeping Our Own CI Green

Two changes in this release affect only Datagrunt’s own development tooling. Nothing changes in your environment or in your own type checking.

• A regression test that does not need DuckDB 2.0 installed. The test suite wraps a DuckDB connection in a proxy whose relations refuse the four removed methods, so any future reliance on them fails on every DuckDB version, not only when CI happens to install 2.0.
• mypy and NumPy 2.5 stubs. NumPy 2.5’s type stubs use Python 3.12 type statements. Datagrunt’s own mypy gate checks against a Python 3.10 target and reached those stubs through Polars, so it stopped with a syntax error before checking any Datagrunt code. Datagrunt never imports NumPy, so the project’s mypy configuration now skips NumPy’s stubs.

Summary of Changes

Upgrading

uv pip install --upgrade datagrunt