))
Greg Kamradt: Benchmarking Intelligence | ARC Prize
Manage episode 490585147 series 3241972
Content provided by Demetrios. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Demetrios or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://ppacc.player.fm/legal.
What makes a good AI benchmark? Greg Kamradt joins Demetrios to break it down—from human-easy, AI-hard puzzles to wild new games that test how fast models can truly learn. They talk hidden datasets, compute tradeoffs, and why benchmarks might be our best bet for tracking progress toward AGI. It’s nerdy, strategic, and surprisingly philosophical.
// Bio
Greg has mentored thousands of developers and founders, empowering them to build AI-centric applications.By crafting tutorial-based content, Greg aims to guide everyone from seasoned builders to ambitious indie hackers.Greg partners with companies during their product launches, feature enhancements, and funding rounds. His objective is to cultivate not just awareness, but also a practical understanding of how to optimally utilize a company's tools.He previously led Growth @ Salesforce for Sales & Service Clouds in addition to being early on at Digits, a FinTech Series-C company.
// Related Links
Website: https://gregkamradt.com/
YouTube channel: https://www.youtube.com/@DataIndependent
~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~
Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore
MLOps Swag/Merch: [https://shop.mlops.community/]
Connect with Demetrios on LinkedIn: /dpbrinkm
Connect with Greg on LinkedIn: /gregkamradt/
Timestamps:
[00:00] Human-Easy, AI-Hard
[05:25] When the Model Shocks Everyone
[06:39] “Let’s Circle Back on That Benchmark…”
[09:50] Want Better AI? Pay the Compute Bill
[14:10] Can We Define Intelligence by How Fast You Learn?
[16:42] Still Waiting on That Algorithmic Breakthrough
[20:00] LangChain Was Just the Beginning
[24:23] Start With Humans, End With AGI
[29:01] What If Reality’s Just... What It Seems?
[32:21] AI Needs Fewer Vibes, More Predictions
[36:02] Defining Intelligence (No Pressure)
[36:41] AI Building AI? Yep, We're Going There
[40:13] Open Source vs. Prize Money Drama
[43:05] Architecting the ARC Challenge
[46:38] Agent 57 and the Atari Gauntlet
446 episodes
Share
