Braintrust CEO reveals how AI agents handle technical debt and database optimization

The Gist

• Ankur Goyal uses Codex to run week-long benchmark experiments across database indexes
• Agents can now handle deeply technical architecture work no single human could tackle
• Evals are the new PRD: encode 'what good looks like' so models figure out the 'how'

• AI agents can handle deeply technical architecture and infrastructure work that no single human engineer could tackle before.
• Evals are the modern version of a PRD (Product Requirements Document) and help encode 'what good looks like' so a model can figure out the 'how'.
• There’s no excuse to skip rigorous benchmarking now that AI agents can run them tirelessly.
• The 'agent line' framework helps decide which decisions, directions, and interactions can be handed off to an agent.
• Fixing your CI (Continuous Integration) is the highest-leverage way to speed up engineering velocity.
• Human attention decays on tedious work, making AI agents ideal for repetitive or monotonous tasks.

• Implement AI agents to handle repetitive or complex technical tasks like database optimization and benchmarking.
• Use evals to define and scale quality standards in AI-driven projects, replacing traditional PRDs.
• Invest in CI/CD infrastructure to maximize the efficiency of AI-accelerated engineering teams.
• Adopt the 'agent line' framework to systematically delegate tasks to AI agents based on their capabilities.

• 4 to 6 concurrent agents (Ankur Goyal's workflow involves running 4 to 6 concurrent AI agents for various tasks.)