Even with detailed output formatting rules, models occasionally output trailing text, invalid JSON keys, or markdown code block wrapping. Output-Constrained Decoding solves this by forcing the model's token generation to conform to a schema at the inference engine level.
Instead of checking output validation after the fact, libraries like Instructor and Outlines build a context-free grammar from Pydantic schemas. At each token selection step, any token that would violate the schema is masked (set to 0 probability). It is mathematically impossible for the model to generate malformed outputs or invalid JSON keys.
02 Weak vs. Strong
EX 01Extracting Recruiter Emails (Python)
from pydantic import BaseModel, Field
import instructor
from openai import OpenAI
class JobExtract(BaseModel):
name: str = Field(description="Candidate name")
salary_min: int = Field(description="Minimum salary mid-point")
salary_max: int = Field(description="Maximum salary mid-point")
stack: list[str] = Field(description="List of programming languages")
client = instructor.from_openai(OpenAI())
job = client.chat.completions.create(
model="gpt-4o",
response_model=JobExtract,
messages=[{"role": "user", "content": email_text}]
)
→ Why it works
Pydantic class enforces data types (e.g. integer bounds) and field names. The output is parsed directly into a Python object without regex or JSON.parse checks.
Uses native JSON Schema mode to constrain output at the token sampler layer, preventing spelling issues (e.g. 'Billing' instead of 'billing') and markdown backticks.
03 Key Points
01Eliminate parsing errors: mathematical guarantee of valid JSON structure.
02Instructor & Pydantic: define your schema as Python classes; let the library handle model calls.
03Token-level masking: the LLM's sampler only evaluates tokens that fit the active JSON regex/schema.
04Performance boost: avoids wasting output tokens on conversational preamble or formatting explanations.
05Production-grade pipelines: crucial for high-throughput pipelines where a single invalid key halts execution.
04 Model-Specific Notes
Claude does not support native json_schema constraints at the engine level yet; use Instructor with tool/function calling parameters to enforce Pydantic structures.
05 For Your Role
Think of it like a form on a website that won't let you submit unless your email has an '@' symbol. The AI is forced to type only valid fields.