258: Confidently Wrong: Why AI Needs Tools (and So Do We)

The Data Stack Show

This week on The Data Stack Show, John and Matt dive into the latest trends in AI, discussing the evolution of GPT models, the role of tools in reducing hallucinations, and the ongoing debate between data warehouses and agent-based approaches. They also explore the complexities of risk-taking in data teams, drawing lessons from Nate Silver’s book on risk and sharing real-world analogies from cybersecurity, football, and political campaigns. Key takeaways include the importance of balancing innovation with practical risk management, the need for clear recommendations from data professionals, the value of reading fiction to understand human behavior in data, and so much more.

Highlights from this week’s conversation include:

• Initial Impressions of GPT-5 (1:41)
• AI Hallucinations and the Open-Source GPT Model (4:06)
• Tools and Determinism in AI Agents (6:00)
• Risks of Tool Reliance in AI (8:05)
• The Next Big Data Fight: Warehouses vs. Agents (10:21)
• Real-Time Data Processing Limitations (12:56)
• Risk in Data and AI: Book Recommendation (17:08)
• Measurable vs. Perceived Risk in Business (20:10)
• Security Trade-Offs and Organizational Impact (22:31)
• The Quest for Certainty and Wicked Learning Environments (27:37)
• Poker, Process, and Data Team Longevity (29:11)
• Support Roles and Limits of Data Teams (32:56)
• Final Thoughts and Takeaways (34:20)
  

The Data Stack Show is a weekly podcast powered by RudderStack, customer data infrastructure that enables you to deliver real-time customer event data everywhere it’s needed to power smarter decisions and better customer experiences. Each week, we’ll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.

RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.