Datagrunt 5.4.0: Multi-Pass rationalize_many, Truncation Bisection & High-Concurrency Engine

November 17, 2026by Martin Graham

Datagrunt 5.4.0 is now officially available! This major feature release delivers the Cloud-Agnostic PDF Rationalization Roadmap (PR #498), introducing multi-schema extraction (rationalize_many()), automatic unit bisection on LLM output truncation, first-class provenance JSONL exports, 256 in-flight concurrency scaling, live telemetry streaming, and sub-millisecond hot-path speedups across consensus and grounding engines.


Highlights in Datagrunt 5.4.0

  1. Multi-Schema rationalize_many() & PassSpec — Extract multiple Pydantic schemas in a single pass over a loaded PDF corpus using shared provider HTTP pools, unified rate limiters, and isolated per-pass caching without redundant bundle verification.
  2. Automatic Truncation Bisection (split_on_truncation=True) — Automatically bisects multi-element extraction units, multi-row tables, and bullet lists at their midpoint (up to 16 rounds) whenever an LLM response hits token limit truncation (finish="truncated").
  3. First-Class Provenance-by-Default JSONL Export (to_jsonl()) — Export atomic UTF-8 JSONL with full attribution fields (_doc_id, _source_name, _status, _grounding_quotes, _disputed_fields, _sources) and pure JSON-serializable telemetry dicts (result.telemetry.to_dict()).
  4. 256 In-Flight Concurrency & Live Telemetry (event_delivery="live") — Scale concurrency up to CONCURRENCY_LIMIT = 256 with 64-socket TLS keep-alive connection pools per provider and live event streaming without head-of-line blocking.
  5. Textless Scan Grounding via Cross-Family Agreement — Pure textless scan items backed by $\ge \max(2, min_families)$ distinct model families with substantive field agreement waive peer-quote string overlap while maintaining strict key verification.
  6. Sub-Millisecond Hot-Path Speedups — Up to 71.5x faster unkeyed table consensus, 7.2x faster table grounding, 2.9x faster reply JSON parsing, and 2.6x–10x faster corpus loading and preflight reuse.

1. Multi-Schema Extraction: rationalize_many() & PassSpec

When extracting complex PDF documents (such as engineering packages, invoices, or medical records), you often need to extract multiple distinct schemas (e.g. document classification headers alongside granular bill-of-materials line items). Previously, running separate rationalize() calls incurred repeated disk reads, redundant bundle verification, and uncoordinated HTTP connection pools.

Datagrunt 5.4.0 introduces rationalize_many() and PassSpec:

from pydantic import BaseModel
from typing import Annotated
from datagrunt import Key, PassSpec, load_corpus, rationalize_many

# Schema 1: High-level document classification
class DocProfile(BaseModel):
    drawing_number: Annotated[str, Key("drawing")]
    title: str
    revision: str

# Schema 2: Granular bill-of-materials parts list
class PartMention(BaseModel):
    part_id: Annotated[str, Key("part", normalizer="alnum_upper")]
    qty: int
    description: str

# Load corpus documents once
corpus = load_corpus("engineering_drawings_corpus/")

# Execute multiple schemas across the corpus with shared runner infrastructure
results = rationalize_many(
    corpus,
    passes=[
        PassSpec(
            schema=DocProfile,
            prompt="Extract drawing title and revision header.",
            name="profile",
        ),
        PassSpec(
            schema=PartMention,
            prompt="Extract all BOM part line items.",
            name="parts",
            max_output_tokens=16_384,
        ),
    ],
    models=["openai:gpt-5.4", "anthropic:claude-sonnet-4-6"],
    min_families=2,
    max_concurrency=64,
    event_delivery="live",   # Stream telemetry events in arrival order
    cache_dir=".cache/rationalize",
)

# Access individual pass RationalizedResult objects
results["profile"].to_jsonl("output/doc_profiles.jsonl")
results["parts"].to_jsonl("output/part_mentions.jsonl")

Technical Benefits of Multi-Pass Execution:

  • 1x Corpus Loading (load_corpus_documents): Manifest bundles and PDF documents are loaded and verified once across all extraction passes using lstat-stamp validation.
  • Shared Connection Pools & Rate Limiters: One provider HTTP connection pool and unified per-model token/RPM rate limiters are shared across all passes.
  • Isolated Per-Pass Caching: Each PassSpec maintains an isolated CacheMeta namespace so cache keys never collide between passes.

2. Automatic Truncation Bisection (split_on_truncation=True)

When extracting large tables or long lists from PDF documents, provider output token limits (e.g., 4,096 or 8,192 tokens) can cause LLM responses to be cut off mid-output (finish="truncated"), losing trailing table rows or list items.

In Datagrunt 5.4.0, split_on_truncation=True is enabled by default in both rationalize() and rationalize_many(). When a provider response encounters output truncation, Datagrunt’s unit scheduler (bisect_unit()) automatically bisects multi-element extraction units and multi-row Table or BulletList elements at their exact midpoint:

from datagrunt import PDFReader, rationalize

doc = PDFReader("massive_parts_catalog.pdf").document

# If an extraction unit hits provider token limits, it bisects automatically up to 16 rounds
result = rationalize(
    doc,
    schema=PartMention,
    prompt="Extract every single part number in this catalog.",
    models=["gemini:gemini-2.5-flash", "anthropic:claude-3-5-sonnet-20241022"],
    split_on_truncation=True,  # Enabled by default
)

print(f"Extracted {len(result.records)} records cleanly across {result.telemetry.total_units} units.")

How Truncation Bisection Works:

  1. Midpoint Splitting: Splittable elements (Table, BulletList) and multi-element units are divided into two equal halves.
  2. Deterministic Part IDs: Bisected units generate collision-free sub-unit IDs with (part_index, part_count) suffixes (e.g. unit_001_p1_of_2, unit_001_p2_of_2).
  3. Iterative Resolution: Bisection executes up to MAX_TRUNCATION_ROUNDS = 16 rounds before consensus, quote grounding, and entity folding, preserving parent token counts and cache telemetry.

3. First-Class Provenance-by-Default Exports & Telemetry

Datagrunt 5.4.0 makes result provenance first-class across JSONL exports and runtime telemetry.

Atomic JSONL Export (to_jsonl())

The new RationalizedResult.to_jsonl() method exports atomic UTF-8 JSONL containing strict records, grounded union records, disputed items, and complete attribution metadata:

result = rationalize(doc, schema=PartMention, prompt="Extract parts.")

# Write full provenance JSONL (includes _doc_id, _source_name, _status, _sources, _grounding_quotes)
count = result.to_jsonl(
    "out/parts_provenance.jsonl",
    include_provenance=True,
    include_disputed=True,
)
print(f"Wrote {count} audit records to JSONL.")

Each JSONL record includes first-class metadata attributes:

  • _doc_id & _source_name: Document identifier and original file name.
  • _status: Consensus status ("consensus", "single_model", "missing_in_some_models", or "disputed").
  • _is_grounded & _is_grounded_union: Grounding state flags.
  • _supporting_models & _grounding_quotes: Model citations and verified text quotes.
  • _sources: Page numbers, element IDs, and exact bounding boxes (bbox) for auditability.

Structured Telemetry (result.telemetry.to_dict())

Export complete run stats, unit counters, and per-model latency/token usage as pure Python dictionaries ready for stdlib json.dumps():

import json

telemetry_data = result.telemetry.to_dict()
print(json.dumps(telemetry_data, indent=2))
# Output includes:
# - total_units, cache_hits, cache_misses, skipped_units
# - model_usage: prompt_tokens, completion_tokens, latency_ms per model

4. High-Concurrency Engine Scaling & Live Telemetry

To support high-volume document extraction pipelines, Datagrunt 5.4.0 significantly boosts concurrent execution throughput and streaming delivery:

  • CONCURRENCY_LIMIT = 256: In-flight request concurrency caps have been increased from 64 to 256 across RunLimits and provider wrappers.
  • Pooled 64-Socket TLS Keep-Alive Transports: Expanded HTTP connection pools from 8 to 64 sockets per provider with 60-second TLS keep-alive (KEEPALIVE_EXPIRY_S = 60.0), eliminating connection teardowns during extended model thinking pauses.
  • Live Telemetry Events (event_delivery="live"): Passing event_delivery="live" streams completion callbacks instantly as units finish, preventing fast units from waiting on slow stragglers.
  • Vertex AI Quota Isolation: Added per-(project, location, publisher, model) rate keys and lock-free bearer token caching (_CachedBearer), ensuring high-volume Claude workloads on Vertex AI never starve Gemini models.

5. Hot-Path Concurrency Multipliers & Latency Speedups

Datagrunt 5.4.0 delivers massive performance improvements across hot-path execution routines while preserving 100% bit-identical extraction invariants:

Execution Engine / Layer Optimization Strategy Measured Speedup
Unkeyed Consensus FIFO score == 1.0 pre-pass before fuzzy candidate postings 227.55 ms -> 3.18 ms (71.5x faster) on 200-row tables
BOM Table Grounding Pre-computed lazy per-element word_set & joined_plain indices 254.67 ms -> 35.13 ms (7.2x faster) on 2,000-cell tables
Reply JSON Parsing Single-walk parsed_response with UTF-8 byte length & ASCII fast-paths 199.99 ms -> 69.04 ms (2.9x faster) on 200-row structured replies
CAD Quality & Gate Classifier x0-sorted _GlyphBand split, _gate_selection bisection, ASCII fast-path 2.5x / 2.4x faster on dense CAD sheets
Preflight & Corpus Loading 16-thread doc packing, cross-pass cache prewarm, manifest-first reuse 2.6x–10x faster preflight & incremental corpus loading

6. Textless Scan Grounding via Cross-Family Agreement

On textless scanned document pages (image_only scans with ocr="never"), visible elements contain no extracted text strings for verbatim quote matching. In previous releases, this required vision models to rely solely on peer quote superset matching.

Datagrunt 5.4.0 introduces Cross-Family Field Agreement for textless scans. When pure textless_anchor consensus items are backed by $\ge \max(2, min_families)$ distinct known model families with substantive structured field agreement, Datagrunt waives peer-quote string overlap while strictly enforcing key verification (key_not_quoted) and low-entropy safeguards to prevent false positives.


Upgrading to Datagrunt 5.4.0

To upgrade your environment, run:

uv pip install --upgrade "datagrunt[pdf,rationalize]"

Or with standard pip:

pip install --upgrade "datagrunt[pdf,rationalize]"

For full details on the new APIs, check out the updated PDF Parsing & Rationalization Guide and API Reference.