Go offline with the Player FM app!
Podcasts Worth a Listen
SPONSORED


Why Knowledge Distillation Works in Generative Models: A Minimal Working Explanation
Manage episode 483815430 series 3524393
This paper explains knowledge distillation's impact on generative models, revealing a precision-recall trade-off that enhances sample quality while managing distributional coverage, validated through simulations and large-scale language modeling.
https://arxiv.org/abs//2505.13111
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
2295 episodes
Manage episode 483815430 series 3524393
This paper explains knowledge distillation's impact on generative models, revealing a precision-recall trade-off that enhances sample quality while managing distributional coverage, validated through simulations and large-scale language modeling.
https://arxiv.org/abs//2505.13111
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
2295 episodes
All episodes
×
1 [QA] Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning 8:08

1 Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning 23:02

1 [QA] ALPHAONE: Reasoning Models Thinking Slow and Fast at Test Time 7:21

1 ALPHAONE: Reasoning Models Thinking Slow and Fast at Test Time 17:12

1 [QA] ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models 7:40

1 ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models 23:32

1 [QA] Are Reasoning Models More Prone to Hallucination? 7:52

1 Are Reasoning Models More Prone to Hallucination? 20:24

1 [QA] How does Transformer Learn Implicit Reasoning? 8:56

1 How does Transformer Learn Implicit Reasoning? 23:21

1 [QA] Let Me Think! A Long Chain-of-Thought Can Be Worth Exponentially Many Short Ones 7:26

1 Let Me Think! A Long Chain-of-Thought Can Be Worth Exponentially Many Short Ones 24:00

1 [QA] Maximizing Confidence Alone Improves Reasoning 7:08

1 Maximizing Confidence Alone Improves Reasoning 13:21

1 [QA] Hardware-Efficient Attention for Fast Decoding 7:57

1 Hardware-Efficient Attention for Fast Decoding 30:59

1 [QA] Reinforcing General Reasoning without Verifiers 7:08

1 Reinforcing General Reasoning without Verifiers 17:11

1 [QA] ENIGMATA: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzles 8:16

1 ENIGMATA: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzles 23:54

1 [QA] Temporal Sampling for Forgotten Reasoning in LLMs 7:04

1 Temporal Sampling for Forgotten Reasoning in LLMs 10:43

1 [QA] Are Large Language Models Reliable AI Scientists? Assessing Reverse-Engineering of Black-Box Systems 10:15

1 Are Large Language Models Reliable AI Scientists? Assessing Reverse-Engineering of Black-Box Systems 17:21



1 [QA] General-Reasoner: Advancing LLM Reasoning Across All Domains 7:40

1 General-Reasoner: Advancing LLM Reasoning Across All Domains 17:40
Welcome to Player FM!
Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.