Datagrunt 5.2.1: Robust Gemini Thinking Level Normalization & CodeQL Security Hardening

Datagrunt 5.2.1 is now officially available! This patch release introduces robust string normalization for Gemini reasoning controls (thinking_level) across all GeminiProvider constructors and updates test suite pattern matching to eliminate CodeQL static analysis alerts.

Key Highlights in Datagrunt 5.2.1

1. Flexible Gemini thinking_level Normalization — GeminiProvider, GeminiProvider.express(), GeminiProvider.cloud(), and model URI strings now automatically strip whitespace and normalize casing for the thinking_level parameter.
2. Type Hint Adjustments — Updated type annotations on Gemini provider constructors (ThinkingLevel | str | None) to accommodate standard string inputs without type-checker warnings.
3. Secret-Free Error Diagnostics — Invalid thinking_level values continue to trigger explicit RationalizationError exceptions without echoing raw parameter values or exposing secret credentials in error messages or tracebacks.
4. CodeQL Regex Security Hardening — Refactored R8 envelope escaping checks in the test suite (test_envelope.py) from regular expression range matches to explicit code-point tuple ranges and frozen character sets, resolving CodeQL static analysis alerts.

1. Case-Insensitive & Whitespace-Tolerant Gemini Thinking Level

With the release of reasoning-capable models in the Gemini 2.0 and 3 model series, fine-tuning model thinking budgets via thinking_level is a core workflow requirement.

Prior to 5.2.1, passing string variations like "low" or " High " raised a RationalizationError, requiring strict uppercase strings ("LOW", "MEDIUM", "HIGH"). Datagrunt 5.2.1 normalizes all string inputs by automatically stripping surrounding whitespace and converting to uppercase before validation.

Python Usage Example

from datagrunt.rationalization.providers import GeminiProvider
from datagrunt import rationalize

# 1. Flexible string parameters in provider instantiation
provider = GeminiProvider(
    api_key="GEMINI_API_KEY",
    thinking_level="low",  # Automatically normalized to "LOW"
)

# 2. Whitespace and mixed-case handling via helper methods
cloud_provider = GeminiProvider.cloud(
    auth=auth_credentials,
    thinking_level="  MEDIUM\n",  # Automatically normalized to "MEDIUM"
)

express_provider = GeminiProvider.express(
    api_key="GEMINI_API_KEY",
    thinking_level="High",  # Automatically normalized to "HIGH"
)

# 3. Model connection string queries
result = rationalize(
    document,
    schema=InvoiceSchema,
    models=["gemini:gemini-2.5-flash?thinking_level=low"],
)

Safety & Error Isolation

If an unsupported string value is supplied (e.g. "MINIMAL" or "NONE"), RationalizationError is raised immediately. In accordance with Datagrunt’s security posture, invalid values and sensitive credentials are never echoed in exception messages:

try:
    GeminiProvider(thinking_level="INVALID_LEVEL")
except RationalizationError as err:
    print(err)
    # Output: thinking_level must be None, "LOW", "MEDIUM" or "HIGH"

2. CodeQL Test Suite Security Hardening

Datagrunt enforces strict verification against unwanted or malicious Unicode escape characters in document payload envelopes (test_envelope.py).

In version 5.2.1, the internal pattern-matching logic was refactored to eliminate CodeQL alerts regarding regular expression character range evaluation (\x7f-\x9f, surrogate pairs \ud800-\udfff, and non-characters). The regex range string was replaced with discrete code-point boundary tuples and a frozenset lookup (ESCAPABLE_CHARS), providing identical runtime escape verification while maintaining clean static analysis metrics.

ESCAPABLE_RANGES = (
    (0x0000, 0x0008),
    (0x000B, 0x001F),
    (0x007F, 0x009F),
    (0x202A, 0x202E),
    (0x2066, 0x2069),
    (0xD800, 0xDFFF),
    (0xE0000, 0xE007F),
)
ESCAPABLE_CHARS = frozenset(chr(cp) for lo, hi in ESCAPABLE_RANGES for cp in range(lo, hi + 1))

Upgrading to Datagrunt 5.2.1

Upgrade to 5.2.1 today via uv or pip:

uv pip install --upgrade datagrunt

Or using standard pip:

pip install --upgrade datagrunt

For full details on earlier 5.2 features (including two-stage batch PDF extraction and greedy section coalescing), see the Datagrunt 5.2.0 Release Notes.