Sonnet 5 Benchmarked: How AI Models Stack Up for GTM Tasks

The Gist

• Anthropic's Sonnet 5 outperforms Sonnet 4.6 in PRD quality and agentic tasks
• Lenny built a repeatable AI eval harness using Claude Code in under 45 minutes
• Combined human vibe scoring (70%) with LLM-as-judge (30%) for balanced results

• Sonnet 5 was benchmarked against four other frontier models (Sonnet 4.6, Opus 4.8, GPT-5.5, and Gemini 3 Pro) across PRD quality, prototype generation, agentic task completion, and agent personality, with surprising results.
• The How I AI Bench was built in under 45 minutes using Claude Code, enabling repeatable and comparable AI model evaluations.
• A combined scoring system of human vibe scoring (70%) and LLM as judge scoring (30%) was used to evaluate AI outputs, avoiding reliance on either method alone.
• Different models excel in different tasks: one is recommended for PRDs, another for complex prototypes, and another for daily agent interactions.

• Implement a repeatable evaluation harness like the How I AI Bench to compare AI models systematically over time.
• Use a combined scoring system (human + LLM) to evaluate AI outputs for more reliable results.
• Choose AI models based on specific tasks (e.g., PRDs, prototypes, agent interactions) rather than relying on a single model for all needs.

• 64 (Number of blind prototype generations, PRDs, and agent voice tests conducted across five frontier models.)
• 45 minutes (Time taken to build the How I AI Bench using Claude Code.)
• 70% human vibe scoring, 30% LLM as judge scoring (Weighting used in the combined scoring system for evaluating AI outputs.)