Prompt Compilers & DSPy

Expert

Stop hand-tuning. Start compiling.

01 The Concept

In 2026, writing prompts by hand is increasingly viewed as writing low-level assembly. A Prompt Compiler (like DSPy) compiles a declarative system architecture into optimized prompts automatically.

Instead of tweaking words, you define a program structure (e.g. `Question -> Answer`), choose a model, define a programmatic metric (like accuracy or semantic similarity), and provide training examples. The compiler then optimizes the instructions and few-shot examples automatically, searching the space of options and even generating synthetic data.

02 Weak vs. Strong

EX 01Math Word Problem Solver

import dspy class MathQA(dspy.Signature): """Solve math word problems with step-by-step reasoning.""" question = dspy.InputField() solution = dspy.OutputField(desc="Step-by-step solution and final number") # Define the program using modules math_bot = dspy.ChainOfThought(MathQA) # Optimize using BootstrapFewShot optimizer teleprompter = dspy.teleprompt.BootstrapFewShot(metric=math_metric) compiled_bot = teleprompter.compile(math_bot, trainset=train_data)

→ Why it works

Replaces manual hand-tuning with structured python objects and programmatic optimization. When the underlying model changes, you simply recompile rather than rewrite all prompts.

EX 02Dynamic RAG Query Router

import dspy class RouteQuery(dspy.Signature): """Route user queries to the correct retrieval source.""" query = dspy.InputField() reasoning = dspy.OutputField(desc="Why this source is selected") source = dspy.OutputField(desc="One of: 'database', 'web_search', or 'direct_reply'") class QueryRouter(dspy.Module): def __init__(self): super().__init__() self.router = dspy.Predict(RouteQuery) def forward(self, query): return self.router(query=query) # Compile router using dynamic validation dataset teleprompter = dspy.teleprompt.BootstrapFewShot(metric=router_accuracy_metric) compiled_router = teleprompter.compile(QueryRouter(), trainset=router_train_data)

→ Why it works

The compiler programmatically selects the best few-shot query examples from historical data, resulting in higher routing accuracy than a hand-crafted rule set.

03 Key Points

01Programmatic optimization: separate application logic from specific model prompts.

02DSPy modules (like `dspy.Predict` or `dspy.ChainOfThought`) replace manual template strings.

03Bootstrapped few-shot selection: automatically generates and selects high-quality exemplars for your dataset.

04Evals act as loss functions: define a clear validation metric to guide the compilation process.

05Model independence: compile the same program for Claude, GPT, or Llama without manual rewrites.

04 Model-Specific Notes

Claude's long context allows DSPy to bootstrap larger few-shot contexts with extensive reasoning paths. Ensure your DSPy signatures translate well to Claude's XML preference.

05 For Your Role

Think of it like compiling code. You write high-level rules, and the compiler turns it into the exact machine instructions (prompts) the model needs.

Next: Constrained Decoding & Structured Outputs →