Deepseek R1 is the biggest AI breakthrough in years - HS#36 solo

HockeyStick Show

🚀 TL;DR

Some Chinese nerd just beat OpenAI using old hardware and less than 0.1% of their yearly training budget, then proceeded to rub it in their face by releasing it free & open source (MIT), along with the paper explaining how they did it.Imagine the atmosphere at OpenAI right now, especially after they announced that $0.5 trillion funding round

📜 Long(er) version

Last week, an AI lab you’ve never heard of, called Deepseek, released a reasoning model that’s equal or better at various benchmarks when compared to the latest models from OpenAI, Anthropic, Meta, or anybody else.“So what” right? “Aren’t there new models coming out every other week”?Yes, but this one is special for 6 reasons:1/ It’s open source, and comes with a paper that explains how it works, in English. You can download it and use it under MIT license, so we know it’s 100% legit.2/ It was trained on a shoestring budget compared to what OpenAI is splurging on their models. 3/ It’s much smaller than the competitors, so it can be run more cheaply. It comes in a variety of sizes and can run on a phone locally!4/ Comes out of China, despite the US preventing them from using the latest chips. They basically trained this on previous gen hardware.5/ It’s using Reinforcement Learning (RL) & a technique called distillation, when they use a bigger model to train the smaller model.6/ They already have an app, which within a week has officially topped App Store ranking, dethorning ChatGPT.Is this the end of closed AI companies like OpenAI?Will billions in valuation suddenly vanish?We’ll find out soon!What we know for sure is that it’s unlocked a new era.And being open source, is so far the biggest gift to the world in the domain of AI.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.hockeystick.show