The Pydantic Trap: Why Strict Structured Outputs Cause Silent Semantic Truncation in Production

· Structured Outputs · By Hassan Nazir

Forcing LLMs to conform to deeply nested Pydantic schemas guarantees valid JSON syntax, but often causes silent semantic truncation and hallucinated null values on edge cases. Here is how Two-Phase Speculative Schema Decoding with AST repair solves it.

Over the past eighteen months, the AI ecosystem fell deeply in love with structured outputs. Libraries like Instructor, Outlines, and OpenAI's native responseformat: { type: "jsonschema" } solved one of the early headaches of working with LLMs: broken JSON brackets and malformed strings.

We wrapped our prompts in rigorous Pydantic models with nested lists, strict regex validators, and decimal constraints. If the model responded, the payload was guaranteed to parse cleanly into our backend database.

Then our production data started looking strangely hollow.

When extracting multi-page commercial loan agreements, the model's outputs were 100% syntactically valid Pydantic instances. But when we manually compared the extracted JSON against the source PDF contracts, we discovered that 35% of nuanced indemnity clauses and cross-default conditions had vanished completely.

The model was passing our schema validation by quietly throwing away critical business reality.

!Pydantic Schema Trap and Two-Phase AST Solution

The Solution: Two-Phase Speculative Decoding with AST Repair

Decouple semantic extraction from syntactic formatting:

from decimal import Decimal
from typing import List, Optional
from pydantic import BaseModel
import re

class IndemnityClause(BaseModel):
    clause_type: str
    liability_cap: Optional[Decimal] = None
    is_unlimited: bool = False
    governing_notes: str

class ContractSchema(BaseModel):
    jurisdiction: str
    indemnity_covenants: List[IndemnityClause]
    
    @classmethod
    def from_raw_ast(cls, raw_dict: dict) -> "ContractSchema":
        repaired_covenants = []
        for item in raw_dict.get("indemnity_covenants", []):
            raw_cap = str(item.get("liability_cap", ""))
            cleaned_cap = re.sub(r"[^d.]", "", raw_cap)
            cap_val = Decimal(cleaned_cap) if cleaned_cap else None
            
            repaired_covenants.append(IndemnityClause(
                clause_type=item.get("clause_type", "General"),
                liability_cap=cap_val,
                is_unlimited=item.get("is_unlimited", False) or (cap_val is None and "unlimited" in str(item).lower()),
                governing_notes=item.get("governing_notes", "")
            ))
        return cls(jurisdiction=raw_dict.get("jurisdiction", "Unknown"), indemnity_covenants=repaired_covenants)

Related field notes