Self-Consistency & Ensembling

Expert

Improve reasoning accuracy by sampling and voting over multiple generation paths.

01 The Concept

Self-Consistency samples multiple diverse reasoning paths (using a temperature > 0) from the model for the same prompt, then selects the most common final answer (majority voting), which significantly increases reasoning accuracy on math and logic tasks.

Rather than taking the first completion path, the system generates 3-10 completions, extracts the calculated result from each, and outputs the consensus answer, filtering out random logical errors.

02 Weak vs. Strong

EX 01Self-Consistency Vote Runner (Python)

# Sample 5 reasoning paths completions = [call_llm(temp=0.7, prompt="Solve: X + 3 = 10...") for _ in range(5)] # Extract final numbers answers = [extract_answer(c) for c in completions] # e.g. [7, 7, 5, 7, 9] # Majority vote final_consensus = max(set(answers), key=answers.count) print("Consensus Answer:", final_consensus)

→ Why it works

Calculation slip errors are usually random. By sampling multiple times and voting, the correct logic pathway (which is statistically more stable) wins.

03 Key Points

01Diverse Paths: Sample multiple chains of thought by setting temperature to 0.5-0.7.

02Extraction: Use structured output parsing (e.g. JSON or regex) to isolate the final answer.

03Consensus Voting: Run a majority vote over all outputs to identify the dominant answer.

04Slips Filtering: Self-Consistency cancels out random calculation slips or logic jumps.

05Ensembling: Combining completions from different models (e.g. Claude + GPT) for high-stakes decisions.

04 Model-Specific Notes

Claude is highly consistent, but ensembling Claude with GPT on safety/code classification tasks produces robust voting results.

05 For Your Role

It's like asking 5 different friends to do a math problem, and going with the answer that 3 or 4 of them agree on.

Next: Multi-Lingual Calibration →