Few-Shot Semantic Selection

Expert

Dynamically select relevant few-shot examples using vector embedding searches.

01 The Concept

Instead of hardcoding static few-shot examples, dynamic retrieval uses vector embeddings to calculate similarity between the incoming user query and a large database of labeled examples.

The prompt is then populated with the most semantically relevant examples, maximizing in-context learning relevance and saving context window space.

02 Weak vs. Strong

EX 01Semantic Retrieval Pipeline (Python)

1. Compute query vector

query_vector = get_embedding(user_query)

2. Query vector database for top 3 similar examples

matches = vector_db.search(query_vector, limit=3) # returns matching few-shots

3. Compile prompt dynamically

prompt = "Examples:\n" + compile_examples(matches) + f"\nQuery: {user_query}"

→ Why it works

Maintains prompt relevance. If the user asks about billing, it retrieves billing examples; if they ask about refund policies, it retrieves refund examples.

03 Key Points

01Dynamic few-shots: Selecting training examples on-the-fly based on active queries.

02Vector Embeddings: Translating text queries and examples into dense numerical arrays.

03Cosine Similarity: Calculating the semantic distance between the active query and example databases.

04Context Conservation: Injecting only the top 3-5 most relevant examples to save token space.

05Scale Efficiency: Utilizing databases with thousands of labeled examples without bloating context.

04 Model-Specific Notes

Claude benefits from highly relevant XML-tagged examples that align exactly with the input domain.

05 For Your Role

It's like a teacher selecting matching homework examples from a textbook that cover the exact math problem you are stuck on, rather than showing you random chapters.

Next: Zero-Shot vs Few-Shot →