Dynamic Few-Shot Selection

Expert

Choose the best examples dynamically at runtime.

01 The Concept

In production systems, hardcoding a fixed set of few-shot examples can waste tokens and fail to represent the specific context of the user's input.

Dynamic Few-Shot Selection solves this by matching the user's input against an indexed vector database of examples at runtime.

By running a similarity search (e.g. cosine similarity over embeddings) on the query, the application retrieves the top-k most similar pairs of inputs and outputs, and injects them dynamically into the prompt before calling the LLM.

02 Weak vs. Strong

EX 01Dynamic Support Routing (Python)

import chromadb from openai import OpenAI

1. Search vector DB for similar queries

client = chromadb.Client() collection = client.get_collection("support_examples") results = collection.query(query_texts=[user_ticket], n_results=3)

2. Build prompt dynamically with retrieved examples

prompt = "Classify the incoming support ticket.\n\n" for doc, metadata in zip(results['documents'][0], results['metadatas'][0]): prompt += f"Ticket: {doc}\nCategory: {metadata['category']}\n---\n"

prompt += f"Ticket: {user_ticket}\nCategory:"

→ Why it works

Retrieves and formats only the examples most similar to the user's current query, saving up to 90% of prompt token space while ensuring hyper-relevant guidance.

03 Key Points

01Context Efficiency: Only load examples relevant to the current user query, preserving token limits.

02Dynamic Learning: Provide relevant demonstrations for rare edge cases that general prompts miss.

03Vector Search Hook: Embed historical inputs and retrieve the corresponding outputs at query time.

04Continuous Feedback: Human-corrected outputs can be added to the vector DB to automatically improve the prompt system over time.

05Reduced Regression: Prevents new few-shot examples from interfering with unrelated queries by keeping them scoped.

04 Model-Specific Notes

Claude's long context handles up to 20+ dynamic examples, making it suitable for complex pattern-matching.

05 For Your Role

It's like asking a librarian to find the three books most relevant to your exact question, instead of reading the whole library index.

Next: Context Windows & Caching Strategies →