Ignite AI: Minha Hwang on Scaling AI Experiments and Building Smarter Models with Less Data | Ep167

Ignite: Conversations on Startups, Venture Capital, Tech, Future, and Society

Minha Hwang is a Principal Applied Scientist at Microsoft, where he leads efforts in large-scale AI experimentation, causal inference, and model evaluation. With a rare blend of technical depth and business acumen, Minha’s background spans two PhDs—one in materials science from MIT and another in marketing science—along with leadership roles at McKinsey and in academia. At Microsoft, he’s helped pioneer innovative approaches to A/B testing, proxy metrics, and AI evaluation at scale, particularly in the age of large language models.

In Today’s Episode We Discuss:

00:00 Intro

00:40 Minha’s Engineering Roots and PhD at MIT

01:55 Jumping from Engineering to Consulting at McKinsey

03:15 Why He Went Back for a Second PhD

04:35 Transition from Academia to Applied Data Science

06:00 Building McKinsey’s Data Science Arm

07:30 Moving to Microsoft to Explore Unstructured Data

08:40 Making A/B Testing More Sensitive with ML

10:00 Why False Positives Are a Massive Problem

11:05 How to Validate Experiments Through “Solidification”

12:10 The Importance of Proxy and Debugging Metrics

13:35 Model Compression and Quantization Explained

15:00 Balancing Statistical Rigor with Product Speed

16:30 Why Data, Not Model Training, Is the Bottleneck

18:00 Causal Inference vs. Machine Learning

20:00 Measuring What You Can’t Observe

21:15 The Missing Role of Causality in AI Education

22:15 Reinforcement Learning and the Data Scarcity Problem

23:40 The Rise of Open-Weight Models Like DeepSeek

25:00 Can Open Source Overtake Closed Labs?

26:15 IP Grey Areas in Foundation Model Training

27:35 Multimodal Models and the Future of Robotics

29:20 Simulated Environments and Physical AI

30:25 AGI, Overfitting, and the Benchmark Illusion

32:00 Practical Usefulness over Philosophical Debates

33:25 Most Underrated Metrics in A/B Testing

34:35 Favorite AI Papers and Experimentation Tools

36:30 Measuring Preferences with Discrete Choice Models

36:55 Outro

Subscribe on Spotify:

https://open.spotify.com/show/6Ga6v0YUsHotLhjap67uu5

Subscribe on Apple Podcasts:

https://podcasts.apple.com/us/podcast/ignite-conversations-on-startups-venture-capital-tech/id1709248824

Follow Brian Bell on X:

https://x.com/brianrbell?lang=en

Follow Minha Hwang on LinkedIn:

https://www.linkedin.com/in/minha-hwang-7440771/

Follow Minha Hwang on Twitter:

https://x.com/minhahwang

Visit Our Website:

https://www.teamignite.ventures/

👂🎧 Watch, listen, and follow on your favorite platform: https://tr.ee/S2ayrbx_fL

🙏 Join the conversation on your favorite social network: https://linktr.ee/theignitepodcast