Datagrunt 5.4.1: Exposing Per-Model Audit Outputs & Flat By-Model Exports

November 20, 2026by Martin Graham

Datagrunt 5.4.1 is now live! This release introduces structured per-model audit visibility and flat per-model audit exports across the rationalization pipeline (PR #500).


Key Highlights in Datagrunt 5.4.1

  1. Structured Per-Model Audit Data (ModelOutput) — GroundedRecord instances now expose supporting_models and typed model_outputs (ModelOutput dataclass), capturing every model’s exact votes, grounding quotes, cited element IDs, and model-specific grounding reasons.
  2. Grounded-Union Provenance & Model Outputs — DataFrame, Arrow, dict, and JSONL exports with include_grounded_union=True (or explicit include_model_outputs=True) automatically append _supporting_models and nested _model_outputs columns.
  3. Flat Per-Model Audit Exports (by_model=True) — Pass by_model=True to to_dataframe(), to_dicts(), to_arrow_table(), or to_jsonl() to explode records into individual per-model audit rows (_model_id, _model_family, _vote_index, _status, _doc_id, _is_grounded, _grounding_quotes, _element_ids).
  4. Disputed Item Audit Inspection — Setting include_disputed=True alongside by_model=True preserves individual model votes even on ungrounded or disputed items where consensus produced record=None.

1. Accessing Per-Model Audit State (ModelOutput)

When extracting entities across multiple LLM/VLM providers using rationalize() or rationalize_many(), Datagrunt computes cross-family consensus and element grounding. In Datagrunt 5.4.1, every folded GroundedRecord carries complete audit metadata per provider model:

from datagrunt import PDFReader, rationalize
from pydantic import BaseModel

class PartNumber(BaseModel):
    part_id: str
    quantity: int

doc = PDFReader("schematic.pdf").document
result = rationalize(
    doc,
    schema=PartNumber,
    prompt="Extract all part numbers and quantities.",
    models=["openai:gpt-5.4", "anthropic:claude-sonnet-4-6", "gemini:gemini-2.5-flash"],
    min_families=2,
)

# Inspect per-model audit outputs on folded grounded records
for record in result.grounded_records:
    print(f"Record Key: {record.key_value}")
    print(f"Supporting Models: {record.supporting_models}")
    
    for mo in record.model_outputs:
        print(f"  Model: {mo.model_id} ({mo.model_family})")
        print(f"    Supported: {mo.supported}, Grounded: {mo.is_grounded}")
        print(f"    Votes: {mo.votes}")
        print(f"    Quotes: {mo.grounding_quotes}")
        print(f"    Element IDs: {mo.element_ids}")
        if mo.grounding_reasons:
            print(f"    Grounding Failures: {[r.reason for r in mo.grounding_reasons]}")

Technical Design:

  • Privacy & Formatting Protection: Text-bearing fields (votes, grounding_quotes, element_ids, grounding_reasons) are omitted from ModelOutput.__repr__ to prevent terminal log pollution.
  • Defensive Deep Copy (to_dict()): Calling mo.to_dict() returns a pure, stdlib-JSON-serializable deep copy of the model’s extraction and audit state.

2. Exporting Grounded Union Records with Model Outputs

When analyzing extraction results in Polars or Pandas DataFrames, include_grounded_union=True includes strict consensus items alongside element-grounded single-family catches ("missing_in_some_models" with zero grounding issues).

In Datagrunt 5.4.1, grounded-union exports include _supporting_models and _model_outputs by default:

# Export grounded union to Polars DataFrame with typed _model_outputs
union_df = result.to_dataframe(include_grounded_union=True)
print(union_df.schema)
# Struct columns include _status, _doc_id, _supporting_models, _model_outputs

# Atomic UTF-8 JSONL export with full provenance and model outputs
result.to_jsonl("out/extracted_parts.jsonl", include_grounded_union=True)

3. Flat Per-Model Audit Exports (by_model=True)

For down-stream auditing, LLM benchmarking, or model disagreement troubleshooting, you can unroll records into flat per-model rows by setting by_model=True:

# Explode items into flat per-model audit rows
by_model_df = result.to_dataframe(include_grounded_union=True, by_model=True)

# Filter to inspect votes from a specific provider
anthropic_votes = by_model_df.filter(by_model_df["_model_family"] == "anthropic")
for row in anthropic_votes.iter_rows(named=True):
    print(f"[{row['_model_id']}] Doc {row['_doc_id']} Part {row['part_id']}: grounded={row['_is_grounded']}")

Unrolled Audit Schema (by_model=True):

When by_model=True is passed to to_dataframe(), to_dicts(), to_arrow_table(), or to_jsonl(), each output row represents one (item, model_id, vote) tuple with metadata columns:

  • _model_id: Canonical model string (e.g. "openai:gpt-5.4").
  • _model_family: Model provider family (e.g. "openai", "anthropic", "google").
  • _vote_index: Zero-based vote index within the model’s contribution to the unit.
  • _status: Record status ("consensus", "single_model", "field_disagreement", etc.).
  • _doc_id: Source document identifier.
  • _is_grounded: Boolean flag indicating zero grounding failures for this model.
  • _grounding_quotes: Grounding quotes cited by this specific model.
  • _element_ids: Source element IDs cited by this specific model.

Installation

Upgrade to Datagrunt 5.4.1 today using uv:

uv pip install --upgrade "datagrunt[pdf,rationalize]"

For complete usage documentation, check out the PDF Parsing & Rationalization Guide.