Artwork

Content provided by Brian Bell. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Brian Bell or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://ppacc.player.fm/legal.
Player FM - Podcast App
Go offline with the Player FM app!

Ignite AI: Minha Hwang on Scaling AI Experiments and Building Smarter Models with Less Data | Ep167

36:54
 
Share
 

Manage episode 487747051 series 3515266
Content provided by Brian Bell. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Brian Bell or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://ppacc.player.fm/legal.

Minha Hwang is a Principal Applied Scientist at Microsoft, where he leads efforts in large-scale AI experimentation, causal inference, and model evaluation. With a rare blend of technical depth and business acumen, Minha’s background spans two PhDs—one in materials science from MIT and another in marketing science—along with leadership roles at McKinsey and in academia. At Microsoft, he’s helped pioneer innovative approaches to A/B testing, proxy metrics, and AI evaluation at scale, particularly in the age of large language models.

In Today’s Episode We Discuss:

00:00 Intro

00:40 Minha’s Engineering Roots and PhD at MIT

01:55 Jumping from Engineering to Consulting at McKinsey

03:15 Why He Went Back for a Second PhD

04:35 Transition from Academia to Applied Data Science

06:00 Building McKinsey’s Data Science Arm

07:30 Moving to Microsoft to Explore Unstructured Data

08:40 Making A/B Testing More Sensitive with ML

10:00 Why False Positives Are a Massive Problem

11:05 How to Validate Experiments Through “Solidification”

12:10 The Importance of Proxy and Debugging Metrics

13:35 Model Compression and Quantization Explained

15:00 Balancing Statistical Rigor with Product Speed

16:30 Why Data, Not Model Training, Is the Bottleneck

18:00 Causal Inference vs. Machine Learning

20:00 Measuring What You Can’t Observe

21:15 The Missing Role of Causality in AI Education

22:15 Reinforcement Learning and the Data Scarcity Problem

23:40 The Rise of Open-Weight Models Like DeepSeek

25:00 Can Open Source Overtake Closed Labs?

26:15 IP Grey Areas in Foundation Model Training

27:35 Multimodal Models and the Future of Robotics

29:20 Simulated Environments and Physical AI

30:25 AGI, Overfitting, and the Benchmark Illusion

32:00 Practical Usefulness over Philosophical Debates

33:25 Most Underrated Metrics in A/B Testing

34:35 Favorite AI Papers and Experimentation Tools

36:30 Measuring Preferences with Discrete Choice Models

36:55 Outro

Subscribe on Spotify:

https://open.spotify.com/show/6Ga6v0YUsHotLhjap67uu5

Subscribe on Apple Podcasts:

https://podcasts.apple.com/us/podcast/ignite-conversations-on-startups-venture-capital-tech/id1709248824

Follow Brian Bell on X:

https://x.com/brianrbell?lang=en

Follow Minha Hwang on LinkedIn:

https://www.linkedin.com/in/minha-hwang-7440771/

Follow Minha Hwang on Twitter:

https://x.com/minhahwang

Visit Our Website:

https://www.teamignite.ventures/

👂🎧 Watch, listen, and follow on your favorite platform: https://tr.ee/S2ayrbx_fL

🙏 Join the conversation on your favorite social network: https://linktr.ee/theignitepodcast

  continue reading

166 episodes

Artwork
iconShare
 
Manage episode 487747051 series 3515266
Content provided by Brian Bell. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Brian Bell or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://ppacc.player.fm/legal.

Minha Hwang is a Principal Applied Scientist at Microsoft, where he leads efforts in large-scale AI experimentation, causal inference, and model evaluation. With a rare blend of technical depth and business acumen, Minha’s background spans two PhDs—one in materials science from MIT and another in marketing science—along with leadership roles at McKinsey and in academia. At Microsoft, he’s helped pioneer innovative approaches to A/B testing, proxy metrics, and AI evaluation at scale, particularly in the age of large language models.

In Today’s Episode We Discuss:

00:00 Intro

00:40 Minha’s Engineering Roots and PhD at MIT

01:55 Jumping from Engineering to Consulting at McKinsey

03:15 Why He Went Back for a Second PhD

04:35 Transition from Academia to Applied Data Science

06:00 Building McKinsey’s Data Science Arm

07:30 Moving to Microsoft to Explore Unstructured Data

08:40 Making A/B Testing More Sensitive with ML

10:00 Why False Positives Are a Massive Problem

11:05 How to Validate Experiments Through “Solidification”

12:10 The Importance of Proxy and Debugging Metrics

13:35 Model Compression and Quantization Explained

15:00 Balancing Statistical Rigor with Product Speed

16:30 Why Data, Not Model Training, Is the Bottleneck

18:00 Causal Inference vs. Machine Learning

20:00 Measuring What You Can’t Observe

21:15 The Missing Role of Causality in AI Education

22:15 Reinforcement Learning and the Data Scarcity Problem

23:40 The Rise of Open-Weight Models Like DeepSeek

25:00 Can Open Source Overtake Closed Labs?

26:15 IP Grey Areas in Foundation Model Training

27:35 Multimodal Models and the Future of Robotics

29:20 Simulated Environments and Physical AI

30:25 AGI, Overfitting, and the Benchmark Illusion

32:00 Practical Usefulness over Philosophical Debates

33:25 Most Underrated Metrics in A/B Testing

34:35 Favorite AI Papers and Experimentation Tools

36:30 Measuring Preferences with Discrete Choice Models

36:55 Outro

Subscribe on Spotify:

https://open.spotify.com/show/6Ga6v0YUsHotLhjap67uu5

Subscribe on Apple Podcasts:

https://podcasts.apple.com/us/podcast/ignite-conversations-on-startups-venture-capital-tech/id1709248824

Follow Brian Bell on X:

https://x.com/brianrbell?lang=en

Follow Minha Hwang on LinkedIn:

https://www.linkedin.com/in/minha-hwang-7440771/

Follow Minha Hwang on Twitter:

https://x.com/minhahwang

Visit Our Website:

https://www.teamignite.ventures/

👂🎧 Watch, listen, and follow on your favorite platform: https://tr.ee/S2ayrbx_fL

🙏 Join the conversation on your favorite social network: https://linktr.ee/theignitepodcast

  continue reading

166 episodes

All episodes

×
 
Loading …

Welcome to Player FM!

Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.

 

Quick Reference Guide

Copyright 2025 | Privacy Policy | Terms of Service | | Copyright
Listen to this show while you explore
Play