Datagrunt 5.3.7: Corroborated CAD 3 Tr SHX Drift and Hardened Numeric Guards

Datagrunt 5.3.7 is officially available! This release delivers advanced CAD technical drawing text extraction enhancements and hardened sign, numeric, and negation guards in src/datagrunt/pdf/quality.py (#489).

Building on the CAD vector stroke recovery introduced in versions 5.3.4 and 5.3.5, Datagrunt 5.3.7 resolves complex kerning drift scenarios in CAD drawings rendered with invisible 3 Tr text streams and underlying SHX font stroke geometry, while introducing strict guards against false-positive text restoration on numbers, signed values, or arithmetic/negation tokens.

Key Highlights in Datagrunt 5.3.7

1. Multi-Word Span Directional Drift Corroboration (quality.py) — Verifies horizontal drift orientation across multi-word spans (_with_span_corroboration, _multi_word_span_qualifies, _word_drifts_toward) to confirm that peer words shift in the same direction before applying edge letter restoration.
2. 2-Letter Preposition & Word-Edge Drift Bridging (_bridge_two_char_word, _bridge_two_char_edge_drift) — Restores allowlisted 2-character prepositions (“TO”, “IN”, “AT”, “OF”, “ON”, “BY”, “AS”, “IS”, “BE”, “IT”, “AN”, “LA”, “LE”, “DU”, “DE”, “AU”, “EN”, “ET”) and 2-glyph edge drifts when corroborated by span direction and overhang geometry.
3. EU Directive Standard Format Recognition (_is_eu_directive_lead_digit) — Recognizes standard EU directive patterns (e.g., 2002/95/CE RoHS compliance tags or YYYY/NN/XX specifications) with 4-digit year leads.
4. Narrow-Glyph Dropout Bridging with Footnotes (_gap_is_bridgeable, _bridge_short_drift_gaps) — Bridges interior narrow-glyph dropouts ($\le 2$ characters) within $\ge 60%$-backed digit-free words, including trailing footnote asterisks (*) (e.g., DANS LE TABLEAU*).
5. Hardened Sign, Numeric, and Negation Guards (_SIGN_CHARS, _UNSAFE_BRIDGE_WORDS, _has_sign_prefix) — Hardens boundaries to strictly block gap bridging or letter restoration on signed quantities (-100, +100), numeric values (1000, 100k), arithmetic operators (+, -, ±, ~), non-ASCII homoglyphs, or negation words (NO, NOT, NOR, NONE, NEVER, CANT, IGNORE).

1. Multi-Word CAD SHX Kerning Drift Corroboration (#489)

Resolving Complex CAD SHX Vector Font Overhangs

In engineering schematics (AutoCAD, MicroStation, Revit), callouts and technical specifications use invisible 3 Tr PDF text overlaying visible vector path segments generated from SHX shape fonts. In complex multi-word callouts (such as "COMPONENT PART LIST" or "CONFORMS TO 2002/95/CE"), character kerning drift can cause bounding boxes for edge letters or short prepositions to shift into adjacent whitespace.

Multi-Word Directional Corroboration

Datagrunt 5.3.7 introduces multi-word span directional corroboration in src/datagrunt/pdf/quality.py:

• Qualifying Multi-Word Spans (_multi_word_span_qualifies): Evaluates multi-word 3 Tr spans containing $\ge 8$ total alphanumeric characters, $\ge 2$ hit words, and $\ge 70%$ direct stroke backing across the span.
• Directional Consensus (_with_span_corroboration): Inspects individual word drift vectors (_word_drifts_toward) to ensure that peer words on the span shift in a unified direction (right_corroborated or left_corroborated).
• Allowlisted Preposition & 2-Glyph Edge Bridging (_bridge_two_char_word, _bridge_two_char_edge_drift): When span-level corroboration is confirmed, allowlisted 2-letter prepositions and 2-glyph edge drifts are safely restored without risking false-positive letter generation.

from datagrunt import PDFReader

# Load an engineering drawing containing CAD SHX callouts and specifications
reader = PDFReader("cad_drawing_sheet.pdf")

# Extracted Markdown cleanly recovers corroborated multi-word CAD text
# (e.g. "CONFORMS TO 2002/95/CE", "DANS LE TABLEAU*", "COMPONENT PART LIST")
markdown_doc = reader.to_markdown()
print(markdown_doc[:500])

2. Hardened Sign, Numeric, and Negation Guards (#489)

Preventing False-Positive Text Recovery

While kerning-drift recovery is vital for technical drawings, restoring characters near numbers, signed quantities, or negation terms could corrupt critical values if numeric characters or arithmetic symbols were improperly bridged.

Strict Numeric & Negation Boundaries

Datagrunt 5.3.7 introduces comprehensive guards across quality.py:

• Sign and Dash Hardening (_is_sign_or_dash, _has_sign_prefix): Detects ASCII and Unicode dashes, minus symbols, plus operators (+, -, ±, ~, \u2212, \u2010–\u2015), and rejects any gap bridging on signed numbers (-100, +100).
• Unsafe Negation & Keyword Blocklist (_UNSAFE_BRIDGE_WORDS): Explicitly blocks bridging on critical negation words and keywords (NO, NOT, NOR, NONE, NEVER, NEITHER, CANNOT, CANT, DONT, WONT, SHANT, ISNT, ARENT, WASNT, WERENT, HASNT, HADNT, DOESNT, DIDNT, WOULDNT, COULDNT, SHOULDNT, MUSTNT, NOK, IGNORE).
• Numeric Token Exclusions: Ensures words containing digits (1000, 100k) or non-ASCII homoglyphs bypass drift bridging entirely (_is_ascii_alpha).

from datagrunt import PDFReader

reader = PDFReader("schematic_with_specs.pdf")

# Text elements with signed quantities (-100, +100) or negation terms (NEVER, CANT, IGNORE)
# maintain exact numeric fidelity without false-positive character bridging:
for p in reader.paragraphs:
    if "UNAUTHORIZED" in p.text or "CONFORMS" in p.text:
        print(f"Verified Paragraph: {p.text}")

3. Engineering Canon & Scaling Micro-Benchmarks (#489)

Datagrunt 5.3.7 adds a comprehensive verification suite in tests/pdf_tests/test_cad_tr3_residual_polish.py:

• AST Nesting Limit ($\le 3$): Guarantees that all control flow structures across quality.py remain cleanly structured with maximum nesting depth $\le 3$.
• Helper Line Budget ($\le 50$ lines): Enforces modular design with all 22 drift and backing helper functions bounded under 50 lines.
• Defensive Boundary Safety: Validates fail-closed behavior on empty strings, out-of-bounds indices, and degenerate inputs.
• Sysmon Tracing Micro-Benchmarks: Confirms linear $O(N)$ single-token gap bridging and sub-quadratic span classification scaling under Python line tracing (sys.settrace).

Summary of Changes in 5.3.7

Getting Started

To upgrade to Datagrunt 5.3.7 using uv:

uv pip install --upgrade datagrunt

For PDF parsing support:

uv pip install --upgrade "datagrunt[pdf]"