Tree-of-Thought & Advanced Reasoning

Advanced

When one path isn't enough.

01 The Concept

Chain-of-Thought (CoT) tells the model to think step by step — one linear path. But some problems have multiple valid approaches, and the best answer requires exploring several, evaluating each, and choosing the strongest.

Tree of Thought (ToT) makes the model explore multiple reasoning branches in parallel, evaluate each intermediate step, backtrack from dead ends, and converge on the best path.

The pattern: decompose → explore multiple paths → evaluate each → prune losers → continue best path → synthesize.

This is overkill for simple tasks. It's transformative for strategy, architecture decisions, root cause analysis, and any problem where the first plausible answer isn't necessarily the best one.

02 Weak vs. Strong

EX 01Architecture decision with real trade-offs

I need to add real-time notifications to a B2B project management tool (Go backend, PostgreSQL, 2,000 concurrent users, 2-person eng team, hosted on Railway).

Use a Tree-of-Thought approach:

Step 1 — Generate 3 distinct approaches: Propose 3 architecturally different solutions for real-time notifications. Each must be genuinely different in technology and trade-offs, not variations of the same idea.

Step 2 — Evaluate each on these criteria (score 1–5):

Implementation complexity for a 2-person team
Operational cost at 2,000 concurrent users
Latency (time from event to user notification)
Scaling path to 50,000 users
Railway hosting compatibility

Step 3 — Eliminate: If any approach has a score of 1 on any criterion, eliminate it and explain the fatal flaw.

Step 4 — Develop the winner: For the winning approach, provide: architecture diagram (text), implementation sequence (what to build first), and the 3 biggest risks with mitigations.

Format the evaluation as a comparison table.

→ Why it works

Three genuinely different approaches prevent anchoring on the first idea. Scored criteria make trade-offs explicit. Fatal-flaw elimination prevents developing a doomed approach. The winner gets full development.

EX 02Root cause analysis with multiple hypotheses

Our Go API's P95 latency jumped from 120ms to 1,400ms yesterday at 3 PM. No code was deployed. 2,000 RPM. PostgreSQL on Railway.

Use a Tree-of-Thought diagnostic approach:

Step 1 — Generate 4 hypotheses for what could cause a 10× P95 spike with no code deployment. Each must be a different category of failure.

Step 2 — For each hypothesis, list:

The specific evidence I should check (exact query, command, or metric)
What result would CONFIRM this hypothesis
What result would ELIMINATE it
Expected investigation time

Step 3 — Rank by:

Prior probability (how common is this cause in production Go/PG systems?)
Investigation speed (fastest to confirm/eliminate first)

Step 4 — Give me a diagnostic runbook: Ordered steps to systematically confirm or eliminate each hypothesis, starting with the highest probability × fastest to check.

Context: I have access to Railway logs, PG slow query log (500ms threshold), Go pprof, and Grafana dashboards.

→ Why it works

Multiple hypotheses prevent anchoring on the first guess. Evidence-based confirmation/elimination prevents wild goose chases. Ranking by probability × speed gives the optimal investigation order.

03 Key Points

01Use ToT when the problem has multiple valid approaches and you need the BEST one, not just A valid one

02Structure: 'Generate 3 distinct approaches → evaluate each on [criteria] → select the strongest → develop it fully'

03Evaluation criteria must be explicit: cost, risk, complexity, time-to-value — not 'which is best'

04Backtracking instruction: 'If an approach hits a fatal flaw, abandon it and explain why before moving on'

05ToT is expensive (3–5× tokens vs. linear CoT) — reserve it for high-stakes decisions, not routine tasks

04 Model-Specific Notes

Claude excels at structured multi-path reasoning. Use numbered approaches and ask it to 'evaluate each against these criteria before choosing.' Claude will genuinely explore alternatives rather than rushing to one answer.

05 For Your Role

When you're stuck between options, ask the AI to explore 3 approaches, score them on your criteria, and develop the winner. You'll get a better answer than 'which should I pick?'

Next: Self-Consistency Prompting →