Migrating to 5.0 (PDF)

Datagrunt 5.0 replaces the PDF extraction stack behind PDFReader and PDFWriter. This guide maps every 4.x PDF option and output to its 5.0 form. CSV, Excel, and Parquet code needs no changes.

What changes

• One engine. PyMuPDF, pdfplumber, and pytesseract are gone. Every PDF is read by PDFium (via pypdfium2), and scanned pages by the system tesseract program when it is installed.
• A typed document. The output is a typed document model — headings, paragraphs, key-value pairs, tables, lists, and figures — in reading order, with section scope and hidden-text flags. The 4.x unified and native schemas are gone, so the shape of to_dicts(), the DataFrame columns, and every written file change.
• Explicit outputs. Every write needs its path, every file is written atomically, and PDFWriter gains Parquet, CSV, and checksummed bundles.
• Python 3.11 or newer. 5.0 drops Python 3.10 for the whole library, so check your interpreter before you upgrade.

See the PDF Parsing guide for the 5.0 API in full.

# 4.x
from datagrunt import PDFReader, PDFWriter

reader = PDFReader("report.pdf", engine="pdfium")
document = reader.to_dicts(drop_layout_tables=True)    # {"document": {"pages": [...]}}
pages = reader.total_pages

writer = PDFWriter("report.pdf")
writer.write_json()                                     # output.json
writer.write_markdown("report.md", image_output_dir="images")
writer.render_pages_as_images(dpi=300, image_format="jpg")   # page_images/report_page_01.jpg

# 5.0
from datagrunt import PDFReader, PDFWriter

reader = PDFReader("report.pdf")
rows = reader.to_dicts()                                # one dict per element
document = reader.document                              # the whole typed document
pages = reader.metadata.page_count

writer = PDFWriter("report.pdf")
writer.write_json("output.json")                        # the path is required
writer.write_bundle("report_bundle")                    # Markdown that links its figures, checked
writer.render_pages_as_images("page_images", dpi=300)   # page_images/page_0001.png

Constructor (PDFReader / PDFWriter)

Reading

The document’s shape

Writing

Output names and return values

OCR

• pytesseract is removed. Install the tesseract program, with eng and osd data. Datagrunt runs it by an absolute path found in PATH’s absolute directories.
• Pages route digital, hybrid, or vision. A vision page (a scan, a page of mostly unreadable characters, or one with too little text beside images or drawings) is read by OCR at vision_dpi, with orientation detection and 900 px bands.
• Its own digital text is kept, flagged superseded.
• Without Tesseract, a vision page’s status is ocr_unavailable, and its digital text is kept.
• Each vision page’s render and OCR share a time budget (600 s). A page that runs out is ocr_unavailable: its digital text is kept, the words OCR read before the budget ran out are not used, and a warning says how many bands were read.

Hidden text (new)

Text a reader can’t see is never dropped: it is flagged with hidden_reasons. The reasons are:

• off_page, invisible, tiny_font, transparent;
• matches_background, zero_scale, duplicate;
• superseded, unchecked_background.

Markdown leaves hidden elements out by default (include_hidden=True keeps each behind a <!-- hidden: reasons --> comment); JSON and the DataFrame keep them. Filter on hidden_reasons before passing those rows to an LLM; see Hidden text.

Errors and pages

• A page that can’t be extracted is an error page: status="error", with its reason in page.error. The rest of the document extracts. With strict=True, it raises instead.
• PDFEncryptedError is a DatagruntPDFError and a ValueError, so except ValueError still works. PDFCorruptError is a DatagruntPDFError and a RuntimeError, as 4.x’s PdfiumError was, so except RuntimeError still works.
• Encrypted and unreadable PDFs raise on first use, not at construction: construction checks only the path and the options.
• Every page reports its status (ok, ocr_fallback, ocr_unavailable, error), its route and quality, and whether its elements were verified (integrity_verified).

Concurrency

Install

Python 3.11 or newer. 4.x supported Python 3.10; 5.0 needs 3.11 or newer, for every file type, not only PDF.

uv pip install "datagrunt[pdf]", which pulls in pypdfium2 and Pillow. Install the tesseract program separately for OCR. See Optional PDF Support.