Datagrunt 5.3.8: Rationalization Hot-Path Performance and Include Schema Cache Collision Fix

Datagrunt 5.3.8 is now available! This release focuses on deep performance optimizations across the rationalization structured extraction subsystem (src/datagrunt/rationalization/), introduces a new fault-isolated batch envelope builder (build_envelopes_isolated), and resolves a subtle cache collision bug involving include_schema=False (#490).

Key Highlights in Datagrunt 5.3.8

1. include_schema Cache Collision Fix (envelope.py) — Resolves a cache key material generation issue in Envelope.key_material() where include_schema=False and include_schema=True produced identical JSON key material when include_schema was omitted when True.
2. Fault-Isolated Batch Envelope Building (build_envelopes_isolated) — Introduces build_envelopes_isolated() to process extraction units while isolating per-unit non-fatal PDF rendering errors into SkippedUnit(reason="image_only") objects without aborting batch envelope assembly.
3. Iterative Zero-Parse Deep JSON Cloning (_clone_json in wire.py) — Replaces costly json.dumps() / json.loads() canonical serialization cycles with an iterative, non-recursive dict/list deep copy routine bounded at 1,000,000 nodes.
4. Pre-Validated Pipeline Fast Paths (_trusted_* constructors) — Bypasses redundant UTF-8 encodability and string validation loops during internal batch extraction pipelines via _trusted_model_request, _trusted_provider_response, and _trusted_envelope.
5. Schema & Reference Hot-Path Acceleration (_refs.py, _canonical.py) — Adds $O(1)$ ASCII fast-paths in canonical_json, fast string pointer formatting in json_pointer, fast token checks in def_name, and stack-allocated subschema traversal (_push_subschemas).

1. Resolving the include_schema Cache Key Collision Bug (#490)

Understanding the Cache Collision Risk

In Datagrunt’s schema-driven PDF extraction pipeline, the Envelope object represents the formatted system and user prompt pair supplied to LLM providers. To avoid redundant LLM requests, Envelope.key_material() generates canonical JSON representing the cache key inputs.

Prior to version 5.3.8, Envelope.key_material() omitted the include_schema flag when include_schema=True and only set material["include_schema"] = False when disabled. Under specific prompt configurations, this could lead to identical cache key material across requests where include_schema differed, causing potential cache collisions across provider execution modes.

Hardened Component Reference Tracking & Caching

Datagrunt 5.3.8 fixes this collision and optimizes key material generation:

• Explicit False Mapping: include_schema is explicitly specified when False.
• Exact Reference State (_key_state): Internal memoization captures exact object identity tuples across all 13 envelope components (system_rules, schema_json, context_json, payload, images, etc.).
• Instantaneous Key Material Reuse: If component references remain untouched, _key_material_json() returns pre-rendered canonical JSON without re-traversing the material dictionary.

from datagrunt.rationalization.envelope import Envelope

# Create envelopes with identical payloads but different schema inclusion settings
env_with_schema = Envelope(
    unit_id="doc1:p0001",
    prompt="Extract invoice items",
    context_json="{}",
    payload="<el id=\"p0001_e001\">...</el>",
    schema_json='{"type": "object"}',
    header_element_ids=(),
    include_schema=True,
)

env_no_schema = Envelope(
    unit_id="doc1:p0001",
    prompt="Extract invoice items",
    context_json="{}",
    payload="<el id=\"p0001_e001\">...</el>",
    schema_json='{"type": "object"}',
    header_element_ids=(),
    include_schema=False,
)

# Guaranteed unique cache key materials
assert env_with_schema.key_material() != env_no_schema.key_material()
assert "include_schema" in env_no_schema.key_material()

2. Fault-Isolated Batch Envelope Building (build_envelopes_isolated) (#490)

Graceful Error Handling for Large-Scale PDF Batches

When processing thousands of PDF pages in multi-tenant ingestion pipelines, individual page image renders can occasionally fail due to corrupted stream data or isolated renderer faults. Previously, an exception during page rendering within build_envelopes() would interrupt the entire document envelope creation process.

Isolating Soft Render Faults

Datagrunt 5.3.8 introduces build_envelopes_isolated() in src/datagrunt/rationalization/envelope.py:

from datagrunt import PDFReader
from datagrunt.rationalization.envelope import build_envelopes_isolated
from datagrunt.rationalization.units import build_units

document = PDFReader("complex_multipage_report.pdf").document
units = build_units(document, multimodal=True)

# Build envelopes with per-unit fault isolation
kept_units, envelopes, skipped_units = build_envelopes_isolated(
    units,
    document,
    prompt="Extract financial tables",
    context=None,
    schema_json='{"type": "object"}',
    multimodal=True,
)

print(f"Successfully generated {len(envelopes)} envelopes across {len(kept_units)} units.")
if skipped_units:
    print(f"Skipped {len(skipped_units)} problematic page units:")
    for skipped in skipped_units:
        print(f"  Unit: {skipped.unit_id}, Reason: {skipped.reason}")

Fatal vs. Soft Exception Isolation

build_envelopes_isolated() catches non-fatal DatagruntPDFError instances during page rendering and converts the unit into a SkippedUnit(reason="image_only"). Fatal errors (PDFEncryptedError, PDFCorruptError, PDFSourceChangedError, BundleCorruptError) continue to be re-raised immediately to prevent downstream processing of invalid or tampered documents.

3. High-Throughput Wire & Framing Hot-Path Performance (#490)

Iterative Zero-Parse Deep JSON Cloning (_clone_json)

In batch pipeline operations, transferring schemas and model requests between provider interfaces required creating deep copies of JSON structures. Standard copy.deepcopy() or json.loads(json.dumps()) roundtrips incur heavy CPU and memory allocation overhead.

Datagrunt 5.3.8 implements _clone_json() in src/datagrunt/rationalization/wire.py:

• Iterative Stack Processing: Copies dict and list structures using explicit queues, avoiding Python recursion limit issues on deep schemas.
• ASCII & Scalar Fast Paths: Bypasses UTF-8 re-encoding for ASCII string keys and primitive scalars (int, bool, finite float).
• Node Cap Safety (_CLONE_MAX_NODES = 1_000_000): Enforces a maximum node limit during cloning to prevent resource exhaustion attacks on untrusted schemas.

Pre-Computed Header & Opener Escaping

Within envelope.py, rendering unit section openers and header context blocks previously re-escaped header texts repeatedly for every unit in a document.

• Single-Pass Opener Escaping (_opener_lines): Precomputes and caches single-line escaped strings (_one_line) once per unique visible opener heading across all document group scopes.
• Pre-Padded Document Line (doc_line): Pre-escapes the document ID header string (<document>...</document>) on _Frame creation, removing string formatting overhead inside hot loops.

Summary of Changes in 5.3.8

Getting Started

To upgrade to Datagrunt 5.3.8 using uv:

uv pip install --upgrade datagrunt

For schema-driven PDF extraction support:

uv pip install --upgrade "datagrunt[pdf,rationalize]"