About

Go under the hood of what it takes to build an AI agent that can actually do real-world data analysis. Hex Head of Product Strategy Charles Schaefer shares lessons learned from developing an analytics agent that works across technical and non-technical interfaces.

You’ll learn why common “data science agent” benchmarks often overstate real performance, and why analytics is harder than coding: questions are ambiguous, follow-ups branch endlessly, and results require judgment. We’ll break down the core challenges that cause inaccuracies and the practical system design choices that can make agents reliable in production.

The session closes with a candid view on evaluation and governance for analytics agents, including why there’s no unit test for analytical truth—and why human-in-the-loop feedback and compounding organizational context are essential for trustworthy AI in data teams.

Takeaways:

• Key challenges AI faces when being asked to work on data – why common evaluation benchmarks fail to accurately capture the success of AI data agents
• How teams are changing to support AI data agents
• Hex’s lessons from building the world’s best AI agent for data – where we succeeded and where we struggled
• How you can take action if you’re interested in adding AI to your analytics work