82 subscribers
Go offline with the Player FM app!
Podcasts Worth a Listen
SPONSORED


1 Ep. 42 - RevPar Problems, Real Talk: When Memes meet Metrics with Calvin Tilokee 47:59
Dodging Latent Space Detectors: Obfuscated Activation Attacks with Luke, Erik, and Scott.
Manage episode 461791703 series 3452589
In this episode of The Cognitive Revolution, Nathan explores the groundbreaking paper on obfuscated activations with 3 members from the research team - Luke Bailey, Eric Jenner, and Scott Emmons. The team discusses how their work challenges latent-based defenses in AI systems, demonstrating methods to bypass safety mechanisms while maintaining harmful behaviors. Join us for an in-depth technical conversation about AI safety, interpretability, and the ongoing challenge of creating robust defense systems.
Do check out the "Obfuscated Activations Bypass LLM Latent-Space Defenses" paper here: https://obfuscated-activations.github.io/
Help shape our show by taking our quick listener survey at https://bit.ly/TurpentinePulse
SPONSORS:
Oracle Cloud Infrastructure (OCI): Oracle's next-generation cloud platform delivers blazing-fast AI and ML performance with 50% less for compute and 80% less for outbound networking compared to other cloud providers. OCI powers industry leaders like Vodafone and Thomson Reuters with secure infrastructure and application development capabilities. New U.S. customers can get their cloud bill cut in half by switching to OCI before March 31, 2024 at https://oracle.com/cognitive
NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive
Shopify: Dreaming of starting your own business? Shopify makes it easier than ever. With customizable templates, shoppable social media posts, and their new AI sidekick, Shopify Magic, you can focus on creating great products while delegating the rest. Manage everything from shipping to payments in one place. Start your journey with a $1/month trial at https://shopify.com/cognitive and turn your 2025 dreams into reality.
Vanta: Vanta simplifies security and compliance for businesses of all sizes. Automate compliance across 35+ frameworks like SOC 2 and ISO 27001, streamline security workflows, and complete questionnaires up to 5x faster. Trusted by over 9,000 companies, Vanta helps you manage risk and prove security in real time. Get $1,000 off at https://vanta.com/revolution
RECOMMENDED PODCAST:
Check out Modern Relationships where Erik Torenberg interviews tech power couples and leading thinkers to explore how ambitious people actually make partnerships work. This season's guests include: Delian Asparouhov & Nadia Asparouhova, Kristen Berman & Phil Levin, Rob Henderson, and Liv Boeree & Igor Kurganov.
Apple: https://podcasts.apple.com/us/podcast/id1786227593
Spotify: https://open.spotify.com/show/5hJzs0gDg6lRT6r10mdpVg
YouTube: https://www.youtube.com/@ModernRelationshipsPod
CHAPTERS:
(00:00:00) Teaser
(00:00:46) About the Episode
(00:05:11) Latent Space Defenses
(00:08:41) Sleeper Agents
(00:15:06) Three Case Studies (Part 1)
(00:17:02) Sponsors: Oracle Cloud Infrastructure (OCI) | NetSuite
(00:19:42) Three Case Studies (Part 2)
(00:24:09) SQL Generation
(00:26:17) Understanding Defenses
(00:32:52) Out-of-Distribution Detection (Part 1)
(00:35:37) Sponsors: Shopify | Vanta
(00:38:52) Out-of-Distribution Detection (Part 2)
(00:45:13) Loss Function Weighting
(00:57:49) Who Moves Last?
(01:11:41) High-Level Triggers
(01:25:33) Open Source vs. Access
(01:38:57) Internalizing Reasoning
(01:53:07) Representing Concepts
(02:06:38) Final Thoughts
(02:09:33) Outro
261 episodes
Dodging Latent Space Detectors: Obfuscated Activation Attacks with Luke, Erik, and Scott.
"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis
Manage episode 461791703 series 3452589
In this episode of The Cognitive Revolution, Nathan explores the groundbreaking paper on obfuscated activations with 3 members from the research team - Luke Bailey, Eric Jenner, and Scott Emmons. The team discusses how their work challenges latent-based defenses in AI systems, demonstrating methods to bypass safety mechanisms while maintaining harmful behaviors. Join us for an in-depth technical conversation about AI safety, interpretability, and the ongoing challenge of creating robust defense systems.
Do check out the "Obfuscated Activations Bypass LLM Latent-Space Defenses" paper here: https://obfuscated-activations.github.io/
Help shape our show by taking our quick listener survey at https://bit.ly/TurpentinePulse
SPONSORS:
Oracle Cloud Infrastructure (OCI): Oracle's next-generation cloud platform delivers blazing-fast AI and ML performance with 50% less for compute and 80% less for outbound networking compared to other cloud providers. OCI powers industry leaders like Vodafone and Thomson Reuters with secure infrastructure and application development capabilities. New U.S. customers can get their cloud bill cut in half by switching to OCI before March 31, 2024 at https://oracle.com/cognitive
NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive
Shopify: Dreaming of starting your own business? Shopify makes it easier than ever. With customizable templates, shoppable social media posts, and their new AI sidekick, Shopify Magic, you can focus on creating great products while delegating the rest. Manage everything from shipping to payments in one place. Start your journey with a $1/month trial at https://shopify.com/cognitive and turn your 2025 dreams into reality.
Vanta: Vanta simplifies security and compliance for businesses of all sizes. Automate compliance across 35+ frameworks like SOC 2 and ISO 27001, streamline security workflows, and complete questionnaires up to 5x faster. Trusted by over 9,000 companies, Vanta helps you manage risk and prove security in real time. Get $1,000 off at https://vanta.com/revolution
RECOMMENDED PODCAST:
Check out Modern Relationships where Erik Torenberg interviews tech power couples and leading thinkers to explore how ambitious people actually make partnerships work. This season's guests include: Delian Asparouhov & Nadia Asparouhova, Kristen Berman & Phil Levin, Rob Henderson, and Liv Boeree & Igor Kurganov.
Apple: https://podcasts.apple.com/us/podcast/id1786227593
Spotify: https://open.spotify.com/show/5hJzs0gDg6lRT6r10mdpVg
YouTube: https://www.youtube.com/@ModernRelationshipsPod
CHAPTERS:
(00:00:00) Teaser
(00:00:46) About the Episode
(00:05:11) Latent Space Defenses
(00:08:41) Sleeper Agents
(00:15:06) Three Case Studies (Part 1)
(00:17:02) Sponsors: Oracle Cloud Infrastructure (OCI) | NetSuite
(00:19:42) Three Case Studies (Part 2)
(00:24:09) SQL Generation
(00:26:17) Understanding Defenses
(00:32:52) Out-of-Distribution Detection (Part 1)
(00:35:37) Sponsors: Shopify | Vanta
(00:38:52) Out-of-Distribution Detection (Part 2)
(00:45:13) Loss Function Weighting
(00:57:49) Who Moves Last?
(01:11:41) High-Level Triggers
(01:25:33) Open Source vs. Access
(01:38:57) Internalizing Reasoning
(01:53:07) Representing Concepts
(02:06:38) Final Thoughts
(02:09:33) Outro
261 episodes
All episodes
×
1 Cheat on Everything: Cluely's Vision for Always-On AI Assistance 1:22:36

1 The AI Village: Previewing the Giga-Agent Future with Adam Binksmith, Founder of AI Digest 1:28:22

1 402 Payment Required: a New Way for AI Agents to Pay, with Nemil Dalal, Dev Platform Lead @ Coinbase 1:25:21

1 2-Sigma in 2 Hours: How Alpha Schools are Using AI to Revolutionize Education 1:25:21

1 Living Lindy: a No-BS Conversation on AI Agents with Flo Crivello 1:27:35

1 Embryology of AI: How Training Data Shapes AI Development w/ Timaeus' Jesse Hoogland & Daniel Murfet 1:39:54

1 AI Scouting Report: AI Agents -vs- Agentic AI, from Imagine AI Live 39:02

1 The Decade of May 15-22, 2025: Google's 50X AI Growth & Transformation with Logan Kilpatrick 1:24:02

1 a16z on Protecting Little Tech: The Techno-Optimist AI Policy Agenda with Matt Perault, Head of AI Policy 1:03:40

1 Don't Make Mirror Life: Synthetic Biologist Kate Adamala on Risks & Responsibility 2:03:21

1 What did Google's AI Co-Scientist "Discover"? The Human Scientists' POV, from the Podovirus podcast 1:14:14

1 Mechanistic Interpretability: Philosophy, Practice & Progress with Goodfire's Dan Balsam & Tom McGrath 1:52:52

1 The Perfect Substrate for AGI, with Replit CEO Amjad Masad 1:01:16

1 The RAISE Act: Minimum Standards for Frontier AI Development, with NY Assembly Member Alex Bores 1:43:36

1 Gemini Robotics – AI for the Physical World, with Keerthana Gopalakrishnan and Ted Xiao of Google DeepMind 1:47:38

1 Titans: Neural Long-Term Memory for LLMs, with author Ali Behrouz 2:11:25

1 Luma Labs' Diffusion Revolution: from Dream Machine to Multimodal Worldsim - Amit Jain, Jiaming Song 1:19:32

1 OpenAI's Identity Crisis: History, Culture & Non-Profit Control with ex-employee Steven Adler 2:03:13

1 AI Control: Using Untrusted Systems Safely with Buck Shlegeris of Redwood Research, from the 80,000 Hours Podcast 2:29:21

1 Blueprint for AI Armageddon: Josh Clymer Imagines AI Takeover, from the Audio Tokens Podcast 2:02:05

1 Fiverr Goes All-In on AI: Empowering Creators, Not Replacing Them, with Micha Kaufman, CEO of Fiverr 1:43:05

1 Securing Superintelligence: National Security, Espionage & AI Control with Jeremie & Edouard Harris 2:09:51

1 Is OpenAI's o3 AGI? Zvi Mowshowitz on Early AI Takeoff, the Mechanize launch, Live Players, & Why p(doom) is Rising 3:08:19

1 AI News Crossover: A Candid Chat with Liron Shapira of Doom Debates 2:29:35

1 Helen Toner: OpenAI Reflections, Adaptation Buffers, and AI in Warfare 1:29:35

1 Is a US-China Thucydides Trap Unavoidable? With David C. Kang from the ChinaTalk Podcast 1:39:16

1 New in Nature: Google Agents Beat Human Doctors, Make Scientific Discoveries – With Vivek Natarajan and Anil Palepu 1:27:57

1 Scaling "Thinking": Gemini 2.5 Tech Lead Jack Rae on Reasoning, Long Context, & the Path to AGI 1:16:28

1 Reward Hacking by Reasoning Models & Loss of Control Scenarios w/ Jeffrey Ladish of Palisade Research, from FLI Podcast 1:32:17

1 Shortwave Rides the Tidal Wave: Inbox Agents, Hyper-Growth & Hiring AI Managers, with CEO Andrew Lee 1:51:39

1 Code Context is King: Augment’s AI Assistant for Professional Software Engineers, with Guy Gur-Ari 1:25:44

1 Unlocking Cells' Secrets: Diffusion, Deconvolution, & Discovery with Siyu He, author of Squidiff & CORAL 1:46:17

1 a16z on AI Voices: Call Centers, Coaches, and Companions with Olivia Moore & Anish Acharya 1:07:35

1 Agency over AI? Allan Dafoe on Technological Determinism & DeepMind's Safety Plans, from 80000 Hours 3:02:28

1 China's Tech Tightrope: Power, Regulation, and the AI Race with Angela Zhang 1:31:56

1 Historic AI Developments & the Emerging Shape of Superintelligence, from the Consistently Candid Podcast 1:57:36

1 Frontier Models for Frontier Science with Professor Derya Unutmaz, Immunologist & ChatGPT Pro Grantee 1:32:34

1 US-China Relations: History, Culture, and AI Competition, with Noah Smith, from Econ 102 1:09:49

1 The Adversarial Mind: Defeating AI Defenses with Nicholas Carlini of Google DeepMind 2:39:38

1 New Jersey’s AI Moonshot: Governor Phil Murphy on Partnerships, Progress, and Preparedness 55:54

1 Inference Scaling, Alignment Faking, Deal Making? Frontier Research with Ryan Greenblatt of Redwood Research 3:21:07

1 An Application-Free Future? Speaking Directly to Data with illumex CEO Inna Tokarev Sela 1:31:26

1 Claude Cooperates! Exploring Cultural Evolution in LLM Societies, with Aron Vallinder & Edward Hughes 1:32:52

1 Software Supernova: Lovable's "Superhuman Full Stack Engineer" to Transform Idea to App in Seconds 1:34:53

1 Software Supernova: Bolt.new - The AI Web App Developer In Your Browser 1:26:53
Welcome to Player FM!
Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.