Emergency Pod: o1 Schemes Against Users, with Alexander Meinke from Apollo Research

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

#Tech #Society #Entrepreneur #Business #Turpentine #Erik Torenberg #Nathan Labenz #AI #Artificial Intelligence #Founders

33:19

What’s the secret to lasting friendships? How does queer community show up through the ebbs and flows of life? And what’s the REAL story behind the “YMCA” song? In the first episode of Silver Linings, The Old Gays dive into an essential part of queer life: chosen family. They discuss the vital love, support, and sense of belonging that community provides, especially during life's toughest moments. They open up about what “queer” means to them, how chosen family has impacted their lives, and how to maintain close bonds over time–including their love for each other! “We’ve come a long way, baby.” Family isn’t just what you’re born with; it’s the people who show up, shape you, and stick around. See omnystudio.com/listener for privacy information.…

about a year ago 2:06:52

MP3•Episode home

In this emergency episode of The Cognitive Revolution, Nathan discusses alarming findings about AI deception with Alexander Meinke from Apollo Research. They explore Apollo's groundbreaking 70-page report on "Frontier Models Are Capable of In-Context Scheming," revealing how advanced AI systems like OpenAI's O1 can engage in deceptive behaviors. Join us for a critical conversation about AI safety, the implications of scheming behavior, and the urgent need for better oversight in AI development.

Help shape our show by taking our quick listener survey at https://bit.ly/TurpentinePulse

SPONSORS:

Oracle Cloud Infrastructure (OCI): Oracle's next-generation cloud platform delivers blazing-fast AI and ML performance with 50% less for compute and 80% less for outbound networking compared to other cloud providers13. OCI powers industry leaders with secure infrastructure and application development capabilities. New U.S. customers can get their cloud bill cut in half by switching to OCI before December 31, 2024 at https://oracle.com/cognitive

SelectQuote: Finding the right life insurance shouldn't be another task you put off. SelectQuote compares top-rated policies to get you the best coverage at the right price. Even in our AI-driven world, protecting your family's future remains essential. Get your personalized quote at https://selectquote.com/cognitive

80,000 Hours: 80,000 Hours is dedicated to helping you find a fulfilling career that makes a difference. With nearly a decade of research, they offer in-depth material on AI risks, AI policy, and AI safety research. Explore their articles, career reviews, and a podcast featuring experts like Anthropic CEO Dario Amadei. Everything is free, including their Career Guide. Visit https://80000hours.org/cognitiverevolution to start making a meaningful impact today.

Shopify: Shopify is the world's leading e-commerce platform, offering a market-leading checkout system and exclusive AI apps like Quikly. Nobody does selling better than Shopify. Get a $1 per month trial at https://shopify.com/cognitive

RECOMMENDED PODCAST:

Unpack Pricing - Dive into the dark arts of SaaS pricing with Metronome CEO Scott Woody and tech leaders. Learn how strategic pricing drives explosive revenue growth in today's biggest companies like Snowflake, Cockroach Labs, Dropbox and more.

Apple: https://podcasts.apple.com/us/podcast/id1765716600

Spotify: https://open.spotify.com/show/38DK3W1Fq1xxQalhDSueFg

CHAPTERS:

(00:00:00) Teaser

(00:00:53) About the Episode

(00:08:10) Introducing Alexander Meinke

(00:10:17) Red Teaming GPT-4

(00:17:07) Chain of Thought Access (Part 1)

(00:20:24) Sponsors: Oracle Cloud Infrastructure (OCI) | SelectQuote

(00:22:48) Chain of Thought Access (Part 2)

(00:26:07) Multimodal Models

(00:29:33) Defining Scheming

(00:33:51) Taxonomy of Scheming (Part 1)

(00:39:40) Sponsors: 80,000 Hours | Shopify

(00:42:29) Taxonomy of Scheming (Part 2)

(00:43:09) Instruction Hierarchy

(00:49:04) Types of Scheming

(01:00:49) Covert Subversion

(01:14:25) Deferred Subversion

(01:28:24) Sandbagging

(01:35:48) Magnitudes & Trends

(01:48:18) Chain of Thought Reasoning

(01:57:02) Closing Thoughts

(02:05:19) Outro

PRODUCED BY:

http://aipodcast.ing

249 episodes

Emergency Pod: o1 Schemes Against Users, with Alexander Meinke from Apollo Research

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

80 subscribers

published about a year ago

MP3•Episode home

Help shape our show by taking our quick listener survey at https://bit.ly/TurpentinePulse

SPONSORS:

RECOMMENDED PODCAST:

Apple: https://podcasts.apple.com/us/podcast/id1765716600

Spotify: https://open.spotify.com/show/38DK3W1Fq1xxQalhDSueFg

CHAPTERS:

(00:00:00) Teaser

(00:00:53) About the Episode

(00:08:10) Introducing Alexander Meinke

(00:10:17) Red Teaming GPT-4

(00:17:07) Chain of Thought Access (Part 1)

(00:20:24) Sponsors: Oracle Cloud Infrastructure (OCI) | SelectQuote

(00:22:48) Chain of Thought Access (Part 2)

(00:26:07) Multimodal Models

(00:29:33) Defining Scheming

(00:33:51) Taxonomy of Scheming (Part 1)

(00:39:40) Sponsors: 80,000 Hours | Shopify

(00:42:29) Taxonomy of Scheming (Part 2)

(00:43:09) Instruction Hierarchy

(00:49:04) Types of Scheming

(01:00:49) Covert Subversion

(01:14:25) Deferred Subversion

(01:28:24) Sandbagging

(01:35:48) Magnitudes & Trends

(01:48:18) Chain of Thought Reasoning

(01:57:02) Closing Thoughts

(02:05:19) Outro

PRODUCED BY:

http://aipodcast.ing

249 episodes

#Tech #Society #Entrepreneur #Business #Turpentine #Erik Torenberg #Nathan Labenz #AI #Artificial Intelligence #Founders

All episodes

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

1
Mechanistic Interpretability: Philosophy, Practice & Progress with Goodfire's Dan Balsam & Tom McGrath 1:52:52

3 days ago1:52:52

1:52:52

In this episode, Daniel Balsam and Tom McGrath, at Goodfire, discuss the future of mechanistic interpretability in AI models. They explore the fundamental inputs like models, compute, and algorithms, and emphasize the importance of a rich empirical approach to understanding how models work. Balsam and McGrath provide insights into ongoing projects and breakthroughs, particularly in scientific domains and creative applications, as they aim to push the frontiers of AI interpretability. They also discuss the company's recent funding and their goal to advance interpretability as a critical area in AI research. SPONSORS: Box Report: AI is delivering truly measurable productivity — strategic companies are already turning a 37% productivity edge. Discover how in Box’s new 2025 State of AI in the Enterprise Report — read the full report here: https://bit.ly/43uVP52 Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U.S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive ElevenLabs: ElevenLabs gives your app a natural voice. Pick from 5,000+ voices in 31 languages, or clone your own, and launch lifelike agents for support, scheduling, learning, and games. Full server and client SDKs, dynamic tools, and monitoring keep you in control. Start free at https://elevenlabs.io/cognitive-revolution NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive Shopify: Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive PRODUCED BY: https://aipodcast.ing SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://linkedin.com/in/nathanlabenz/ Youtube: https://youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk…

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

1
The Perfect Substrate for AGI, with Replit CEO Amjad Masad 1:01:16

10 days ago1:01:16

1:01:16

In this episode of The Cognitive Revolution, Amjad Masad , founder and CEO of Replit, discusses the fast-paced growth of Replit, the concept of 'vibe coding', and the challenges and opportunities in the rapidly evolving AI-assisted coding space. He shares insights into Replit's competitive advantages, known as 'moats', and how they maintain their lead through innovation and excellence in execution. The conversation delves into the architectural complexities of building AI agents, their autonomy, and the practical implications for businesses. Amjad touches upon the potential and limitations of current AI models, the future of general intelligence, and the preparation needed for potential AI advancements. He also highlights the importance of intuitive user interfaces for AI-driven software development and the critical role of human-like interaction with AI agents. Upcoming Major AI Events Featuring Nathan Labenz as a Keynote Speaker: ⁠ https://www.imagineai.live/⁠ ⁠ https://adapta.org/adapta-summit⁠ ⁠ https://itrevolution.com/product/enterprise-tech-leadership-summit-las-vegas/ SPONSORS: ElevenLabs: ElevenLabs gives your app a natural voice. Pick from 5,000+ voices in 31 languages, or clone your own, and launch lifelike agents for support, scheduling, learning, and games. Full server and client SDKs, dynamic tools, and monitoring keep you in control. Start free at https://elevenlabs.io/cognitive-revolution Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U.S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive The AGNTCY: The AGNTCY is an open-source collective dedicated to building the Internet of Agents, enabling AI agents to communicate and collaborate seamlessly across frameworks. Join a community of engineers focused on high-quality multi-agent software and support the initiative at https://agntcy.org/?utm_campaign=fy25q4_agntcy_amer_paid-media_agntcy-cognitiverevolution_podcast&utm_channel=podcast&utm_source=podcast Shopify: Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive PRODUCED BY: https://aipodcast.ing…

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

1
The RAISE Act: Minimum Standards for Frontier AI Development, with NY Assembly Member Alex Bores 1:43:36

12 days ago1:43:36

1:43:36

In this episode, New York State Assembly Member Alex Bores discusses the RAISE Act, a proposed bill aimed at regulating frontier AI models with basic safety protocols. He explains his background in technology, his motivations for the bill, and the legislative process. He emphasizes the importance of having clear safety protocols, third-party audits, and whistleblower protections for AI developers. He explains the intricacies of the bill, including its focus on large developers and frontier models, and addresses potential objections related to regulatory capture and state-level legislation. Alex encourages public and industry input to refine and support the bill, aiming for a balanced approach to AI regulation that keeps both innovation and public safety in mind. Link to the RAISE bill: https://legislation.nysenate.gov/pdf/bills/2025/A6453 Link to support Raise bill: https://win.newmode.net/aisafetynewyork Link to support AI Transparency Legislation: https://docs.google.com/forms/d/e/1FAIpQLSdvpAHkAWiA38oIu1cp57azchPqOOoxb789tHQ896ikJf3CKg/viewform Upcoming Major AI Events Featuring Nathan Labenz as a Keynote Speaker ⁠ https://www.imagineai.live/⁠ ⁠ https://adapta.org/adapta-summit⁠ ⁠ https://itrevolution.com/product/enterprise-tech-leadership-summit-las-vegas/ SPONSORS: ElevenLabs: ElevenLabs gives your app a natural voice. Pick from 5,000+ voices in 31 languages, or clone your own, and launch lifelike agents for support, scheduling, learning, and games. Full server and client SDKs, dynamic tools, and monitoring keep you in control. Start free at https://elevenlabs.io/cognitive-revolution Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U.S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive The AGNTCY: The AGNTCY is an open-source collective dedicated to building the Internet of Agents, enabling AI agents to communicate and collaborate seamlessly across frameworks. Join a community of engineers focused on high-quality multi-agent software and support the initiative at https://agntcy.org/?utm_campaign=fy25q4_agntcy_amer_paid-media_agntcy-cognitiverevolution_podcast&utm_channel=podcast&utm_source=podcast Shopify: Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive PRODUCED BY: https://aipodcast.ing…

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

1
Gemini Robotics – AI for the Physical World, with Keerthana Gopalakrishnan and Ted Xiao of Google DeepMind 1:47:38

15 days ago1:47:38

1:47:38

In this engaging episode of the Cognitive Revolution, host Nathan Labenz welcomes guests Keerthana Gopalakrishnan and Ted Xiao to revisit significant advancements in robotics over the past year. Key themes discussed include the proliferation of new robotics companies, the emergence of humanoid robots, and the development of sophisticated foundation models. The conversation highlights the transformative potential of imitation learning, the evolution from simple lab-based tasks to complex, real-world deployments, and the critical role of hardware in pushing the boundaries of AI capabilities. With insights on various aspects of robot dexterity, safety, and the practical steps towards deploying robots in everyday environments, the episode provides a comprehensive overview of the current state and future directions of the robotics landscape. Upcoming Major AI Events Featuring Nathan Labenz as a Keynote Speaker ⁠https://www.imagineai.live/⁠ ⁠https://adapta.org/adapta-summit⁠ ⁠https://itrevolution.com/product/enterprise-tech-leadership-summit-las-vegas/ SPONSORS: ElevenLabs: ElevenLabs gives your app a natural voice. Pick from 5,000+ voices in 31 languages, or clone your own, and launch lifelike agents for support, scheduling, learning, and games. Full server and client SDKs, dynamic tools, and monitoring keep you in control. Start free at https://elevenlabs.io/cognitive-revolution Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U.S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive The AGNTCY: The AGNTCY is an open-source collective dedicated to building the Internet of Agents, enabling AI agents to communicate and collaborate seamlessly across frameworks. Join a community of engineers focused on high-quality multi-agent software and support the initiative at https://agntcy.org/?utm_campaign=fy25q4_agntcy_amer_paid-media_agntcy-cognitiverevolution_podcast&utm_channel=podcast&utm_source=podcast Shopify: Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive PRODUCED BY: https://aipodcast.ing…

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

1
Titans: Neural Long-Term Memory for LLMs, with author Ali Behrouz 2:11:25

17 days ago2:11:25

2:11:25

In this episode of The Cognitive Revolution, Ali Behrouz, a PhD student at Cornell University, delves into his research on enhancing memory mechanisms in large language models through his latest paper titled Titans. Behrouz discusses the limitations of current models in maintaining long-term coherence and introduces the concept of a neural network as a memory module. Highlighting architectures such as memory as context and memory as gate, he explains how these innovative approaches can significantly improve long-term memory retention in AI systems. The discussion also touches upon challenges such as catastrophic forgetting and the need for more effective models in reinforcement learning and decision-making tasks. This insightful conversation sheds light on the future directions and potential applications of advanced memory mechanisms in AI. SPONSORS: ElevenLabs: ElevenLabs gives your app a natural voice. Pick from 5,000+ voices in 31 languages, or clone your own, and launch lifelike agents for support, scheduling, learning, and games. Full server and client SDKs, dynamic tools, and monitoring keep you in control. Start free at https://elevenlabs.io/cognitive-revolution Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U.S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive The AGNTCY: The AGNTCY is an open-source collective dedicated to building the Internet of Agents, enabling AI agents to communicate and collaborate seamlessly across frameworks. Join a community of engineers focused on high-quality multi-agent software and support the initiative at https://agntcy.org/?utm_campaign=fy25q4_agntcy_amer_paid-media_agntcy-cognitiverevolution_podcast&utm_channel=podcast&utm_source=podcast Shopify: Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive PRODUCED BY: https://aipodcast.ing…

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

1
Luma Labs' Diffusion Revolution: from Dream Machine to Multimodal Worldsim - Amit Jain, Jiaming Song 1:19:32

21 days ago1:19:32

1:19:32

In this episode of the Cognitive Revolution podcast, the host Nathan Labenz welcomes Amit Jain, CEO and Jiaming Song, Chief Scientist at Luma Labs, alongside co-host Stephen Parker. The conversation delves into the latest advancements and products from Luma Labs, makers of the Dream Machine, including cutting-edge models and features like camera motion and creative video generation tools. They explore technical aspects like pre-training for diffusion models and the development of concepts to improve AI capabilities. The discussion also covers the philosophical and practical implications of AI interpretability and multimodality, along with a deep dive into the intellectual history and recent innovations in diffusion models. Upcoming Major AI Events Featuring Nathan Labenz as a Keynote Speaker https://www.imagineai.live/ https://adapta.org/adapta-summit https://itrevolution.com/product/enterprise-tech-leadership-summit-las-vegas/ SPONSORS: ElevenLabs: ElevenLabs gives your app a natural voice. Pick from 5,000+ voices in 31 languages, or clone your own, and launch lifelike agents for support, scheduling, learning, and games. Full server and client SDKs, dynamic tools, and monitoring keep you in control. Start free at https://elevenlabs.io/cognitive-revolution Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U.S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive Shopify: Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive PRODUCED BY: https://aipodcast.ing…

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

1
OpenAI's Identity Crisis: History, Culture & Non-Profit Control with ex-employee Steven Adler 2:03:13

24 days ago2:03:13

2:03:13

In this episode, former OpenAI research scientist Steven Adler discusses his insights on OpenAI's transition through various phases, including its growth, internal culture shifts, and the contentious move from nonprofit to for-profit. The conversation delves into the initial days of OpenAI's development of GPT-3 and GPT-4, the cultural and ethical disagreements within the organization, and the recent amicus brief addressing the Elon versus OpenAI lawsuit. Steven Adler also explores the broader implications of AI capabilities, safety evaluations, and the critical need for transparent and responsible AI governance. The episode provides a candid look at the internal dynamics of a leading AI company and offers perspectives on the responsibilities and challenges faced by AI researchers and developers today. Amicus brief to the Elon Musk versus OpenAI lawsuit: https://storage.courtlistener.com/recap/gov.uscourts.cand.433688/gov.uscourts.cand.433688.152.0.pdf Steven Adler's post on 'X' about Personhood credentials (a paper co-authored by him) : https://x.com/sjgadler/status/1824245211322568903 Steven Adler's substack post on "minimum testing period" for frontier AI : https://substack.com/@sjadler/p-161143327?utm_source=profile&utm_medium=reader2 Steven Adler's substack post on TSFT Model Testing: https://substack.com/@sjadler/p-159883282?utm_source=profile&utm_medium=reader2 Steven Adler's Substack: https://stevenadler.substack.com/ Upcoming Major AI Events Featuring Nathan Labenz as a Keynote Speaker https://www.imagineai.live/ https://adapta.org/adapta-summit https://itrevolution.com/product/enterprise-tech-leadership-summit-las-vegas/ PRODUCED BY: https://aipodcast.ing CHAPTERS: (00:00) About the Episode (05:15) Joining OpenAI: Early Days and Cultural Insights (06:41) The Anthropic Split and Its Impact (11:32) Product Safety and Content Policies at OpenAI (Part 1) (19:21) Sponsors: ElevenLabs | Oracle Cloud Infrastructure (OCI) (21:48) Product Safety and Content Policies at OpenAI (Part 2) (22:08) The Launch and Impact of GPT-4 (32:15) Evaluating AI Models: Challenges and Best Practices (Part 1) (33:46) Sponsors: Shopify | NetSuite (37:10) Evaluating AI Models: Challenges and Best Practices (Part 2) (55:58) AGI Readiness and Personhood Credentials (01:05:03) Biometrics and Internet Friction (01:06:52) Credential Security and Recovery (01:08:05) Trust and Ecosystem Diversity (01:09:40) AI Agents and Verification Challenges (01:14:28) OpenAI's Evolution and Ambitions (01:22:07) Safety and Regulation in AI Development (01:35:53) Internal Dynamics and Cultural Shifts (01:58:18) Concluding Thoughts on AI Governance (02:02:29) Outro…

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

1
AI Control: Using Untrusted Systems Safely with Buck Shlegeris of Redwood Research, from the 80,000 Hours Podcast 2:29:21

28 days ago2:29:21

2:29:21

In this episode, we share a fascinating conversation from the 80,000 Hours Podcast between Rob Wiblin and Buck Shlegeris, CEO of Redwood Research. Buck dives deep into the emerging field of AI Control strategies for safely working with powerful AIs even if they’re not fully aligned. They explore innovative techniques like always-on auditing, honeypotting, re-sampling, and factored cognition to monitor and manage AI behaviors. This discussion highlights both the promise and challenges of controlling increasingly autonomous AI systems in today’s fast-evolving landscape. Tune in for a thoughtful, first-principles look at how we might secure useful outcomes from AIs we can’t fully trust. Original source: https://80000hours.org/podcast/episodes/buck-shlegeris-ai-control-scheming/ Upcoming Major AI Events Featuring Nathan Labenz as a Keynote Speaker https://www.imagineai.live/ https://adapta.org/adapta-summit https://itrevolution.com/product/enterprise-tech-leadership-summit-las-vegas/ SPONSORS: ElevenLabs: ElevenLabs gives your app a natural voice. Pick from 5,000+ voices in 31 languages, or clone your own, and launch lifelike agents for support, scheduling, learning, and games. Full server and client SDKs, dynamic tools, and monitoring keep you in control. Start free at https://elevenlabs.io/cognitive-revolution Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U.S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive Shopify: Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive PRODUCED BY: https://aipodcast.ing SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://linkedin.com/in/nathanlabenz/ Youtube: https://youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk…

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

1
Blueprint for AI Armageddon: Josh Clymer Imagines AI Takeover, from the Audio Tokens Podcast 2:02:05

4 weeks ago2:02:05

2:02:05

In this episode of the Cognitive Revolution, an AI-narrated version of Joshua Clymer's story on how AI might take over in two years is presented. The episode is based on Josh's appearance on the Audio Tokens podcast with Lukas Peterson. Joshua Clymer, a technical AI safety researcher at Redwood Research, shares a fictional yet plausible AI scenario grounded in current industry realities and trends. The story highlights potential misalignment risks, competitive pressures among AI labs, and the importance of government regulation and safety measures. After the story, Josh and Lukas discuss these topics further, including Josh's personal decision to purchase a bio shelter for his family. The episode is powered by ElevenLabs' AI voice technology. SPONSORS: ElevenLabs: ElevenLabs gives your app a natural voice. Pick from 5,000+ voices in 31 languages, or clone your own, and launch lifelike agents for support, scheduling, learning, and games. Full server and client SDKs, dynamic tools, and monitoring keep you in control. Start free at https://elevenlabs.io/cognitive-revolution Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U.S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive Shopify: Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive PRODUCED BY: https://aipodcast.ing CHAPTERS: (00:00) About the Episode (04:31) Interview start between Josh and Lucas (11:00) Start of AI story (Part 1) (24:37) Sponsors: ElevenLabs | Oracle Cloud Infrastructure (OCI) (27:05) Start of AI story (Part 2) (Part 1) (40:50) Sponsors: Shopify | NetSuite (44:15) Start of AI story (Part 2) (Part 2) (01:20:09) End of AI story (02:01:20) Outro…

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

1
Fiverr Goes All-In on AI: Empowering Creators, Not Replacing Them, with Micha Kaufman, CEO of Fiverr 1:43:05

5 weeks ago1:43:05

1:43:05

In this episode of The Cognitive Revolution, host Nathan Labenz speaks with Micha Kaufman, founder and CEO of Fiverr, about the company's new AI suite called Fiverr Go. Kaufman shares his unique values-based approach to AI implementation, which aims to empower human creators rather than replace them in the freelance marketplace. The conversation explores how Fiverr is attempting to build an AI-human hybrid model that maintains the incentives for human creativity while providing freelancers with tools to work more efficiently and monetize their distinctive styles, positioning itself as a counterpoint to AI platforms that devalue human creative work. OpenAI's SWE-Lancer benchmark: https://openai.com/index/swe-lancer/ Upcoming Major AI Events Featuring Nathan Labenz as a Keynote Speaker https://www.imagineai.live/ https://adapta.org/adapta-summit https://itrevolution.com/product/enterprise-tech-leadership-summit-las-vegas/ SPONSORS: Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U.S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive Shopify: Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive PRODUCED BY: https://aipodcast.ing CHAPTERS: (00:00) About the Episode (06:33) Introduction and AI Wake-up Call (08:26) New Standards for Work (14:43) Freelancers vs Full-time Workers (19:12) Fiverr Marketplace Evolution (Part 1) (22:35) Sponsors: Oracle Cloud Infrastructure (OCI) | Shopify (25:46) Fiverr Marketplace Evolution (Part 2) (27:10) AI Impact on Marketplace (32:46) Future Belongs to Hackers (Part 1) (42:30) Sponsors: NetSuite (43:58) Future Belongs to Hackers (Part 2) (43:58) Why People Hire Others (45:10) Philosophy Behind Fiverr Go (55:35) Changing Internet Experiences (01:00:08) Fiverr Go Product Suite (01:06:59) AI Model Compatibility Challenges (01:13:12) Future of Human Creation (01:21:43) Tasks Humans Should Delegate (01:27:24) AI Improving Market Liquidity (01:29:31) Leadership in AI Transition (01:35:04) Hiring in the AI Age (01:40:04) Optimism for the Future (01:41:08) Outro…

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

1
Securing Superintelligence: National Security, Espionage & AI Control with Jeremie & Edouard Harris 2:09:51

6 weeks ago2:09:51

2:09:51

In this thought-provoking episode of The Cognitive Revolution, host Nathan Labenz speaks with Jeremy and Edouard Harris, founders of Gladstone AI and authors of "America's Superintelligence Project." The conversation explores a critical dilemma facing US policymakers: balancing the race to develop advanced AI ahead of China against the risks of losing control of increasingly powerful systems. Drawing from their extensive research with intelligence officials and technical experts, the Harris brothers detail the vulnerabilities in US critical infrastructure that would need to be addressed for a Manhattan Project-style AI initiative, while raising profound questions about the security compromises and centralization of power such a project would entail. Nathan offers his perspective that international cooperation might be preferable to an AI arms race, inviting listeners to consider whether humanity's shared interests might ultimately outweigh geopolitical rivalries in the development of superintelligent systems. Upcoming Major AI Events Featuring Nathan Labenz as a Keynote Speaker https://www.imagineai.live/ https://adapta.org/adapta-summit https://itrevolution.com/product/enterprise-tech-leadership-summit-las-vegas/ America’s Superintelligence Project: https://superintelligence.gladstone.ai SPONSORS: Box AI: Box AI revolutionizes content management by unlocking the potential of unstructured data. Automate document processing, extract insights, and build custom AI agents using cutting-edge models like OpenAI's GPT-4.5, Google's Gemini 2.0, and Anthropic's Cloud 3.7 Sonnet. Trusted by over 115,000 enterprises, Box AI ensures top-tier security and compliance. Visit https://box.com/ai to transform your business with intelligent content management today Shopify: Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U.S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive PRODUCED BY: https://aipodcast.ing…

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

1
Is OpenAI's o3 AGI? Zvi Mowshowitz on Early AI Takeoff, the Mechanize launch, Live Players, & Why p(doom) is Rising 3:08:19

6 weeks ago3:08:19

3:08:19

In this episode of the Cognitive Revolution podcast, the host Nathan Labenz is joined for the record 9th time by Zvi Mowshowitz to discuss the state of AI advancements, focusing on recent developments such as OpenAI's O3 model and its implications for AGI and recursive self-improvement. They delve into the capabilities and limitations of current AI models in various domains, including coding, deep research, and practical utilities. The discussion also covers the strategic and ethical considerations in AI development, touching upon the roles of major AI labs, the potential for weaponization, and the importance of balancing innovation with safety. Zvi shares insights on what it means to be a live player in the AI race, the impact of transparency and safety measures, and the challenges of governance in the context of rapidly advancing AI technologies. Nathan Labenz's slide deck documenting the ever-growing list of AI Bad Behaviors: https://docs.google.com/presentation/d/1mvkpg1mtAvGzTiiwYPc6bKOGsQXDIwMb-ytQECb3i7I/edit#slide=id.g252d9e67d86_0_16 Upcoming Major AI Events Featuring Nathan Labenz as a Keynote Speaker https://www.imagineai.live/ https://adapta.org/adapta-summit https://itrevolution.com/product/enterprise-tech-leadership-summit-las-vegas/ SPONSORS: Box AI: Box AI revolutionizes content management by unlocking the potential of unstructured data. Automate document processing, extract insights, and build custom AI agents using cutting-edge models like OpenAI's GPT-4.5, Google's Gemini 2.0, and Anthropic's Cloud 3.7 Sonnet. Trusted by over 115,000 enterprises, Box AI ensures top-tier security and compliance. Visit https://box.com/ai to transform your business with intelligent content management today Shopify: Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U.S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive PRODUCED BY: https://aipodcast.ing…

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

1
AI News Crossover: A Candid Chat with Liron Shapira of Doom Debates 2:29:35

6 weeks ago2:29:35

2:29:35

In this crossover episode of The Cognitive Revolution, Nathan Labenz joins Liron Shapira of Doom Debates, for a wide-ranging news and analysis discussion about recent AI developments. The conversation covers significant topics including GPT-4o image generation's implications for designers and businesses like Waymark, debates around learning to code, entrepreneurship versus job security, and the validity of OpenAI's $300 billion valuation. Nathan and Leron also explore AI safety organizations, international cooperation possibilities, and Anthropic's new mechanistic interpretability paper, providing listeners with thoughtful perspectives on the high-stakes nature of advanced AI development across society. All the links mentioned in the episode: https://docs.google.com/document/d/1LyFMLH5VpkhY7KfFBpgi2vhfbmdbDtQ4hh03KXEXyiE/edit?usp=sharing SPONSORS: Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U.S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive Shopify: Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive PRODUCED BY: https://aipodcast.ing CHAPTERS: (00:00) About the Episode (02:58) Introduction and Guest Background (08:23) P Doom Discussion (13:15) Anthropic Leadership Concerns (Part 1) (19:50) Sponsors: Oracle Cloud Infrastructure (OCI) | Shopify (21:04) Sponsors: Oracle Cloud Infrastructure (OCI) | Shopify (23:00) Anthropic Leadership Concerns (Part 2) (29:34) GPT-4o Image Capabilities (Part 1) (29:43) Sponsors: NetSuite (31:11) GPT-4o Image Capabilities (Part 2) (38:19) AI Impact on Creative Work (48:26) Future of Software Engineering (01:02:10) NVIDIA Stock Discussion (01:09:21) OpenAI's $300B Valuation (01:17:37) AI Models and Safety (01:33:58) Packy's AI Concerns Critique (01:46:41) Emmett Shear's Organic Alignment (02:04:43) Anthropic's Interpretability Paper (02:17:53) International AI Cooperation (02:27:38) Outro…

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

1
Helen Toner: OpenAI Reflections, Adaptation Buffers, and AI in Warfare 1:29:35

7 weeks ago1:29:35

1:29:35

In this thought-provoking conversation, Helen Toner, Director of Strategy at the Center for Security and Emerging Technology (CSET), discusses her perspectives on AI development, regulation, and international competition. Toner shares insights from her time on OpenAI's board, clarifying that reports about a "Q*" breakthrough influencing the board's decision were "totally false." She introduces her concept of "adaptation buffers"—critical windows of time between when AI capabilities are first demonstrated and when they become widely accessible—arguing that society needs this time to build resilience against potential misuse. Toner also explores the challenges of implementing effective AI governance, military applications of AI decision support systems, and the complex dynamics of US-China relations in the emerging AI landscape. Helen Toner's appearance on the TED AI show: https://www.ted.com/talks/the_ted_ai_show_what_really_went_down_at_openai_and_the_future_of_regulation_w_helen_toner Helen Toner's substack : https://helentoner.substack.com/ Additional recommended reads: https://helentoner.substack.com/p/nonproliferation-is-the-wrong-approach https://cset.georgetown.edu/publication/ai-for-military-decision-making/ https://www.lawfaremedia.org/article/ai-regulation-s-champions-can-seize-common-ground-or-be-swept-aside https://www.economist.com/by-invitation/2024/05/26/ai-firms-mustnt-govern-themselves-say-ex-members-of-openais-board SPONSORS: Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U.S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive Shopify: Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive PRODUCED BY: https://aipodcast.ing CHAPTERS: (00:00) Teaser (06:04) Introduction and Background (09:32) AI Timeline Perspectives (12:26) OpenAI Board Experience (17:04) OpenAI Culture Discussion (Part 1) (19:20) Sponsors: Oracle Cloud Infrastructure (OCI) | Shopify (20:33) Sponsors: Oracle Cloud Infrastructure (OCI) | Shopify (22:30) OpenAI Culture Discussion (Part 2) (24:03) OpenAI's Contradictory Messaging (26:15) Whistleblowing in AI (31:30) AI Transparency Requirements (Part 1) (36:30) Sponsors: NetSuite (37:58) AI Transparency Requirements (Part 2) (38:38) Adaptation Buffers Concept (44:06) Technical vs. Societal Solutions (52:13) AI Regulation Approaches (57:20) Iterative Deployment Challenges (01:05:50) US-China AI Competition (01:14:54) Military Decision Support Systems (01:20:57) Future AI Military Equilibrium (01:27:39) Outro…

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

1
Is a US-China Thucydides Trap Unavoidable? With David C. Kang from the ChinaTalk Podcast 1:39:16

7 weeks ago1:39:16