Gorgias Sets New Standard with Transparent AI CX Benchmarks

The Gist

• Gorgias published a public benchmark of its AI agent against 18 competitors
• The evaluation included 8,356 live conversations across 212+ storefronts
• Gorgias ranked #1 in support but openly shared areas where competitors outperformed
• The full test harness is open-sourced on GitHub for transparency

• Gorgias has set a new standard by publishing transparent, detailed competitive evaluations of AI CX agents in ecommerce, including areas where competitors outperform them.
• Traditional benchmarks (like analyst quadrants or feature checklists) are insufficient for AI agents, which require continuous testing of real conversations due to their variable performance.
• Buyers trust vendor evaluations more when they include competitors' strengths, as it lends credibility to the vendor's own claims.
• AI agents are increasingly involved in vendor shortlisting, making structured, checkable evaluations more valuable for buyers.
• Gorgias's approach of open-sourcing benchmarks and updating them weekly creates internal accountability and transparency.

• Publish detailed, transparent competitive evaluations of your AI product, including areas where competitors outperform you.
• Continuously test AI agents with real conversations and update benchmarks regularly, as static feature checklists are inadequate.
• Open-source evaluation methodologies and scoring rubrics to build trust and enable buyers (and AI agents) to verify claims.
• Use benchmarks as internal accountability tools by making competitor gaps visible and regularly updated.

• 8,356 live conversations (Number of conversations analyzed in Gorgias's benchmark across 18 vendors.)
• $100M ARR (Gorgias's annual recurring revenue, with 80% coming from AI support for ecommerce brands.)
• 7.9 seconds (Envive's response time, a competitor metric publicly compared in Gorgias's benchmark.)