Go offline with the Player FM app!
Ignite AI: Minha Hwang on Scaling AI Experiments and Building Smarter Models with Less Data | Ep167
Manage episode 487747051 series 3515266
Minha Hwang is a Principal Applied Scientist at Microsoft, where he leads efforts in large-scale AI experimentation, causal inference, and model evaluation. With a rare blend of technical depth and business acumen, Minha’s background spans two PhDs—one in materials science from MIT and another in marketing science—along with leadership roles at McKinsey and in academia. At Microsoft, he’s helped pioneer innovative approaches to A/B testing, proxy metrics, and AI evaluation at scale, particularly in the age of large language models.
In Today’s Episode We Discuss:
00:00 Intro
00:40 Minha’s Engineering Roots and PhD at MIT
01:55 Jumping from Engineering to Consulting at McKinsey
03:15 Why He Went Back for a Second PhD
04:35 Transition from Academia to Applied Data Science
06:00 Building McKinsey’s Data Science Arm
07:30 Moving to Microsoft to Explore Unstructured Data
08:40 Making A/B Testing More Sensitive with ML
10:00 Why False Positives Are a Massive Problem
11:05 How to Validate Experiments Through “Solidification”
12:10 The Importance of Proxy and Debugging Metrics
13:35 Model Compression and Quantization Explained
15:00 Balancing Statistical Rigor with Product Speed
16:30 Why Data, Not Model Training, Is the Bottleneck
18:00 Causal Inference vs. Machine Learning
20:00 Measuring What You Can’t Observe
21:15 The Missing Role of Causality in AI Education
22:15 Reinforcement Learning and the Data Scarcity Problem
23:40 The Rise of Open-Weight Models Like DeepSeek
25:00 Can Open Source Overtake Closed Labs?
26:15 IP Grey Areas in Foundation Model Training
27:35 Multimodal Models and the Future of Robotics
29:20 Simulated Environments and Physical AI
30:25 AGI, Overfitting, and the Benchmark Illusion
32:00 Practical Usefulness over Philosophical Debates
33:25 Most Underrated Metrics in A/B Testing
34:35 Favorite AI Papers and Experimentation Tools
36:30 Measuring Preferences with Discrete Choice Models
36:55 Outro
Subscribe on Spotify:
https://open.spotify.com/show/6Ga6v0YUsHotLhjap67uu5
Subscribe on Apple Podcasts:
Follow Brian Bell on X:
https://x.com/brianrbell?lang=en
Follow Minha Hwang on LinkedIn:
https://www.linkedin.com/in/minha-hwang-7440771/
Follow Minha Hwang on Twitter:
Visit Our Website:
https://www.teamignite.ventures/
👂🎧 Watch, listen, and follow on your favorite platform: https://tr.ee/S2ayrbx_fL
🙏 Join the conversation on your favorite social network: https://linktr.ee/theignitepodcast
166 episodes
Ignite AI: Minha Hwang on Scaling AI Experiments and Building Smarter Models with Less Data | Ep167
Ignite: Conversations on Startups, Venture Capital, Tech, Future, and Society
Manage episode 487747051 series 3515266
Minha Hwang is a Principal Applied Scientist at Microsoft, where he leads efforts in large-scale AI experimentation, causal inference, and model evaluation. With a rare blend of technical depth and business acumen, Minha’s background spans two PhDs—one in materials science from MIT and another in marketing science—along with leadership roles at McKinsey and in academia. At Microsoft, he’s helped pioneer innovative approaches to A/B testing, proxy metrics, and AI evaluation at scale, particularly in the age of large language models.
In Today’s Episode We Discuss:
00:00 Intro
00:40 Minha’s Engineering Roots and PhD at MIT
01:55 Jumping from Engineering to Consulting at McKinsey
03:15 Why He Went Back for a Second PhD
04:35 Transition from Academia to Applied Data Science
06:00 Building McKinsey’s Data Science Arm
07:30 Moving to Microsoft to Explore Unstructured Data
08:40 Making A/B Testing More Sensitive with ML
10:00 Why False Positives Are a Massive Problem
11:05 How to Validate Experiments Through “Solidification”
12:10 The Importance of Proxy and Debugging Metrics
13:35 Model Compression and Quantization Explained
15:00 Balancing Statistical Rigor with Product Speed
16:30 Why Data, Not Model Training, Is the Bottleneck
18:00 Causal Inference vs. Machine Learning
20:00 Measuring What You Can’t Observe
21:15 The Missing Role of Causality in AI Education
22:15 Reinforcement Learning and the Data Scarcity Problem
23:40 The Rise of Open-Weight Models Like DeepSeek
25:00 Can Open Source Overtake Closed Labs?
26:15 IP Grey Areas in Foundation Model Training
27:35 Multimodal Models and the Future of Robotics
29:20 Simulated Environments and Physical AI
30:25 AGI, Overfitting, and the Benchmark Illusion
32:00 Practical Usefulness over Philosophical Debates
33:25 Most Underrated Metrics in A/B Testing
34:35 Favorite AI Papers and Experimentation Tools
36:30 Measuring Preferences with Discrete Choice Models
36:55 Outro
Subscribe on Spotify:
https://open.spotify.com/show/6Ga6v0YUsHotLhjap67uu5
Subscribe on Apple Podcasts:
Follow Brian Bell on X:
https://x.com/brianrbell?lang=en
Follow Minha Hwang on LinkedIn:
https://www.linkedin.com/in/minha-hwang-7440771/
Follow Minha Hwang on Twitter:
Visit Our Website:
https://www.teamignite.ventures/
👂🎧 Watch, listen, and follow on your favorite platform: https://tr.ee/S2ayrbx_fL
🙏 Join the conversation on your favorite social network: https://linktr.ee/theignitepodcast
166 episodes
All episodes
×Welcome to Player FM!
Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.