Datagrunt 5.3.5: Overhang-Verified CAD Edge-Letter Drift and Sparse/MRC Scan Deduplication

Datagrunt 5.3.5 is officially available! This maintenance and precision performance update delivers overhang-verified vector stroke recovery for CAD technical drawing text callouts affected by edge-letter SHX font drift, alongside advanced sparse tile grid propagation and multi-layer Mixed Raster Content (MRC) scan image deduplication (#487).

Key Highlights in Datagrunt 5.3.5

1. Overhang-Verified CAD Edge-Letter Drift Recovery (quality.py) — Resolves edge-character truncation in CAD engineering schematics where SHX vector font stroke paths drift into adjacent inter-word whitespace beyond strict bounding boxes.
2. Multi-Layer MRC & Segmentation Mask Suppression (assembly.py) — Automatically identifies and suppresses background layer masks, 1-bit CCITT Fax / JBIG2 monochrome segmentation patches, and grayscale text zones in multi-layer scanned PDFs (_mrc_and_bitonal_positions).
3. Full-Span & Ink-Trimmed Scan Band Axis Assembly — Detects multi-band full-span scan partitions (_full_span_band_axis) and cross-overlapping ink-trimmed scan bands (_ink_trimmed_band_axis) across white gaps.
4. Sparse Tile Propagation (_propagate_sparse_tiles) — Extends confirmed scan tile grid matches in $O(T)$ time to absorb isolated or edge-remainder tiles that omit blank white areas.

1. Overhang-Verified CAD Edge-Letter Drift Recovery (#487)

The Challenge with CAD SHX Edge Characters

In engineering drawings produced by CAD applications (AutoCAD, MicroStation, Revit), text callouts and dimension specs are rendered using invisible PDF text streams (Text Render Mode 3 / Tr 3) overlaid on visible vector stroke paths (glyph_stroke_boxes) generated from SHX shape fonts.

While Datagrunt 5.3.4 introduced word-level vector stroke backing, initial or terminal letters at word boundaries (such as “A” in “ASSEMBLY” or “D” in “DIM-100”) occasionally exhibit minor physical drift relative to underlying vector paths. When an edge letter’s invisible bounding box shifted outside the exact vector stroke boundary, traditional strict spatial checks failed to mark the edge character as backed, causing truncated words.

Overhang Verification in Inter-Word Whitespace

Datagrunt 5.3.5 introduces targeted overhang verification in src/datagrunt/pdf/quality.py:

• Adjacent Line Bound Calculation (_adjacent_word_bound): Locates nearest non-space Tr 3 character edges on the same text line across all spans using $O(\log N)$ binary searches over horizontal spatial bands (_GlyphBand).
• Whitespace Overhang Check (_word_has_drift_overhang): Verifies if an unbacked edge character (alnum[0] or alnum[-1]) has underlying vector stroke geometry extending into unclaimed inter-word whitespace (gap_lo to gap_hi).
• Edge Letter Recovery (_mark_word_backing): When interior word characters are backed by vector strokes ($\ge 2$ alpha hits), overhang verification restores drifting lead and trail letters (lead_ok and trail_ok).

from datagrunt import PDFReader

# Process an engineering schematic with SHX font callouts and dimension tags
reader = PDFReader("schematic_drawing_sheet.pdf")

# CAD callouts with edge-letter SHX drift (e.g., "ASSEMBLY", "DIM-100")
# are now fully recovered without letter truncation:
for element in reader.elements:
    if element.kind in ("heading", "paragraph"):
        print(f"[{element.kind}]: {element.text}")

# Retains clean markdown output for downstream AI extraction or search indexing
markdown_doc = reader.to_markdown()
print(markdown_doc[:400])

2. Sparse Tile Propagation & Multi-Layer MRC Scan Deduplication (#487)

Eliminating Multi-Layer Scan Noise & Micro-Tile Artifacts

Scanned PDFs often use Mixed Raster Content (MRC) compression or tile-based scan drivers, breaking pages into multiple overlapping image layers: full-page background images, 1-bit CCITT Fax or JBIG2 monochrome text masks, and small micro-tiles.

Without intelligent deduplication, PDF readers extract dozens of redundant background fragments, mask layers, or narrow scanline strips as separate Figure objects.

Advanced MRC & Sparse Grid Deduplication

Datagrunt 5.3.5 expands src/datagrunt/pdf/assembly.py with multi-tier scan deduplication:

• MRC Background & Bitonal Mask Suppression (_mrc_and_bitonal_positions): Identifies full-page MRC background images covering $\ge 94%$ of page dimensions at $\ge 300\text{ px}$ scan resolution (_is_mrc_background), 1-bit CCITT/JBIG2 monochrome masks (_is_bitonal_mask), and grayscale text zone bands (_is_gray_text_zone).
• Full-Span & Ink-Trimmed Band Detection (_full_span_band_axis, _ink_trimmed_band_axis): Unifies abutting scan bands partitioning $\ge 55%$ of page area and ink-trimmed bands across white gaps.
• Sparse Tile Grid Propagation (_propagate_sparse_tiles): Given a confirmed connected tile grid, propagates suppression in $O(T)$ time to isolated or edge-remainder tiles matching grid cell dimensions along at least one axis.

from datagrunt import PDFReader

# Load a complex 5-page scanned blueprint containing MRC background masks and sparse tile grids
reader = PDFReader("scanned_mrc_blueprint.pdf", min_image_dimension=30)

# Multi-layer background masks and sparse edge-remainder tiles are automatically suppressed:
print(f"Extracted genuine figures count: {len(reader.figures)}")

# Export deduplicated figures cleanly to disk:
reader.extract_images("output_figures")

3. Performance & Benchmark Suite (#487)

Datagrunt 5.3.5 updates performance micro-benchmarks in tests/pdf_tests/test_cad_and_tiled_speed.py, verifying linear $O(S)$ span classification and $O(T \log T)$ tile grid assembly. Wall-clock performance tolerances have also been calibrated across Python 3.11 and Python 3.12 sysmon coverage environments.

Summary of Changes in 5.3.5

Getting Started

To update to Datagrunt 5.3.5 using uv:

uv pip install --upgrade datagrunt

For PDF parsing support:

uv pip install --upgrade "datagrunt[pdf]"