Standard sequential generation is slow because LLMs decode tokens one by one. Skeleton-of-Thought (SoT) addresses this by decomposing generation into two phases:
This approach reduces user-facing latency by up to 80% while often yielding deeper, more structured content since each section has its own full context window.
import asyncio from openai import AsyncOpenAI
client = AsyncOpenAI()
skeleton_res = await client.chat.completions.create( model="gpt-4o", messages=[{"role": "user", "content": "Outline a 5-point technical feature spec for a new Stripe subscription module. Output only 5 bullet points with clear, short titles."}] ) points = parse_bullets(skeleton_res.choices[0].message.content)
async def expand_point(p): res = await client.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": f"Write a detailed technical specification for this section of a Stripe module: {p}"}] ) return res.choices[0].message.content
full_spec = await asyncio.gather(*(expand_point(p) for p in points))
Claude handles highly structured outlines perfectly. Ensure the skeleton prompt requires clean XML tags for easy parsing.