Episode 8: The New Parser Core.py podcast

1:42:23

Python 3.14? That's old news. Let's talk about the first big feature of Python 3.15 -- a built-in sampling profiler for Linux, macOS, and Windows. We also cover improvements in perf support, discuss memory.python.org, and as usual, recent changes in the codebase. ## Timestamps (00:00:00) INTRO (00:02:43) PART 1: THE SAMPLING PROFILER (00:05:07) Built-in profile is bad, long live cProfile (00:10:54) Out-of-process profiling (00:12:18) Shortcuts Compromise Accuracy, Leading Eventually to Numerous Errors (00:16:07) Selfish Łukasz vs benevolent Pablo (00:23:11) How does a sampling profiler even work? (00:30:42) One meeellion huuurtzzz (00:32:40) Free threading makes it extra spicy (00:41:26) AsyncIO makes it even spicier (00:49:49) You made this? I made this (00:54:06) What if the profiled process changes during sampling? (00:57:33) Coming in October 2026 (01:04:30) PART 2: PR OF THE WEEEEEEK (01:14:14) memory.python.org launched (01:23:15) PART 3: WHAT'S GOING ON IN CPYTHON (01:26:45) Performance updates (01:30:24) Features & Curiosities (01:41:45) OUTRO…

core.py

1
PyCon US 2025 Recap 1:36:11

6 weeks ago1:36:11

1:36:11

We’ve been gone a while. Here’s our excuse for being silent for a month: PyCon, PyCon, something something security. Come listen to how the conference looked like from our perspective! And whatever you do, DO NOT upgrade to Python 3.13.4. ## Timestamps (00:00:00) INTRO (00:01:06) PART 1: LANGUAGE SUMMIT (00:04:47) A bit about the Summit talks (00:06:19) Is free-threading happening? (00:09:20) Łukasz and his favorite discussion item at the Summit (00:13:38) Find actual competent coverage of the Summit on the PSF blog (00:14:17) PART 2: PYCON TALK HIGHLIGHTS (00:14:33) Cory Doctorow's opening keynote (00:18:17) Brandt Bucher's talk on JIT challenges (00:28:28) Lysandros and Nathan talk about community adoption of free-threading (00:36:23) Lynn Root's keynote (00:37:46) PyXL: Python-oriented chip (00:39:47) Łukasz and his tutorial on WebGL with PyScript (00:47:58) A new bet appears! (00:48:14) Zoom, Enhance the Banana (00:54:11) Watch out, Łukasz is talking about audio again (01:02:42) Ivona and Pablo talk about remote code execution as a service (01:05:44) Core Python sprints after the conference (01:12:48) PART 3: PR OF THE WEEK (01:13:00) tarfile security fixes (01:19:13) Pablo's PR: strncmp considered harmful (01:25:36) PART 4: WHAT'S GOING ON IN CPYTHON (01:26:06) compression.zstd lands (01:28:01) concurrent.futures → asyncio.Future transfer 4X faster (01:29:14) Bugfix: PyCFuncPtr_call no longer uses locks (01:30:13) Some curiosities (01:34:14) OUTRO…

core.py

1
Beta Frenzy 1:19:11

11 weeks ago1:19:11

1:19:11

Python 3.14 Beta 1 is coming! And that means we reach feature freeze. BUT QUICK, there’s still time to squeeze in one last thing! ## Timestamps (00:00:00) INTRO (00:01:58) PART 1: Template strings (00:07:10) PART 2: Asyncio Introspection (00:29:07) PART 3: Syntax highlighting (00:43:00) PART 4: Color themes (00:50:56) PART 5: Debugging a remote process with pdb (01:01:35) PART 6: Python Installation Manager for Windows (01:05:29) PART 7: Worship (01:08:53) PART 8: What else is happening? (01:16:03) OUTRO…

core.py

1
Episode 21: A Garbage Episode 1:57:34

14 weeks ago1:57:34

1:57:34

We talked about this episode for months now, and it's finally here. Garbage collection in its full glory. Classic and free-threaded. Generational and single-pass. With eager and delayed untracking. We cover it all! Explicitly. ## Timestamps (00:00:00) THE FUCKING INTRO (00:02:03) PART 0: SPORTS NEWS (00:03:19) PART 1: GARBAGE COLLECTION (00:03:57) The big problem with refcounting (00:08:35) Solving reference cycles through PyGC_Head (00:11:45) 64 bits ought to be enough for anybody (00:17:30) Why a doubly-linked list? (00:21:15) How reference counting makes finding cycles easier (00:26:25) Roots bloody roots (00:30:17) How are objects in the cycle destroyed? (00:31:58) Object resurrection (00:43:21) Why do you need "generations"? (00:52:26) Delayed untracking (00:54:46) Weak references, strong problems (00:59:19) GC in free threading (01:03:27) Reference counting in free-threading builds (01:10:08) Incremental GC talk is DEFERRED (01:11:00) PART 2: PR OF THE WEEK (01:17:15) Type checking the standard library itself? (01:29:51) PART 3: WHAT'S GOING ON IN CPYTHON? (01:30:15) Free-threading changes (01:32:54) Performance updates (01:36:11) http.server supports HTTPS! (01:37:01) PEP 768 and 758 landed (01:37:34) HACL* (01:38:24) fnmatch.filterfalse() (01:38:54) Bugfixes (01:42:46) Curiosities (01:54:49) OUTRO…

core.py

1
Episode 20: Remote Code Execution By Design 1:44:20

17 weeks ago1:44:20

1:44:20

In this episode, Pablo's avoiding the topic of garbage collection by talking about his latest PEP, which allows unprecedented interaction with a running Python process. We also resolve the bet about reference counting semantics, mention some notable changes in Python since the last episode, and discuss syntax highlighting in PyREPL and why it's bad, actually. ## Timestamps (00:00:00) INTRO (00:02:16) PART 1: PABLO'S LATEST PEP (00:04:34) gdb is IMPOSSIBLE (00:12:49) Make the process run code for you (00:14:14) This already works on PyPy (00:15:13) How does it work? (00:25:38) Why a file? (00:31:15) What if you don't trust Pablo? (00:32:57) sys.remote_exec() (00:36:09) Less obvious use cases (00:46:56) PART 2: BETS (00:55:44) PART 3: PR OF THE WEEK (00:55:50) Łukasz: syntax highlighting in PyREPL (01:10:14) Pablo's PR: allow the parser to activate future imports on the fly (01:20:11) PART 4: WHAT'S GOING ON IN CPYTHON (01:20:22) Free threading (01:23:30) Performance (01:34:41) PEP 765 implemented (01:36:08) concurrent.futures.Executor.map(buffersize=) (01:36:57) io.Reader and io.Writer (01:38:40) Pabluco's linecache fetching interactive source code (01:41:25) ast.unparse() roundtrip with semicolons (01:41:59) OUTRO…

core.py

1
Episode 19: Async hacks, unicorns and velociraptors 2:07:21

20 weeks ago2:07:21

2:07:21

In this asynchronous episode we're interviewing a fellow core developer Yury Selivanov to talk about asyncio's past and future, composable design, immutability, and databases you'd actually like using. We also broke the 2-hour episode barrier! ## Timestamps (00:00:00) INTRO (00:01:33) PART 1: INTERVIEW (00:02:27) What drives you? (00:04:47) How do you choose what to work on? (00:08:10) Hyperfocus (00:09:28) Things from Rust that Python could use (00:14:50) Nothing is sacred when you depend on glibc (00:18:47) TypeScript typing is god-tier (00:22:04) Adding async and await to Python (00:34:11) Adding new keywords to the language (00:41:17) Jumping into a new codebase (00:49:22) Any design regrets? (00:58:46) Contextvars (01:10:40) Is the frozenmap PEP happening? (01:19:21) uvloop (01:23:25) What makes Gel lovable? (01:39:57) PART 2: PR OF THE WEEK (01:47:08) Saturday talks at PyCon should be fun (01:50:35) PART 3: WHAT'S GOING ON IN CPYTHON (01:50:47) Ken Jin's tail-call interpreter (01:55:05) Barney Gale's glob.glob() optimization (01:55:43) Brandt's boolean guards to narrow types to values in the JIT (01:56:33) Mark Shannon's stack limits implemented with addresses, not counters (01:58:34) Brandt's removal of _DYNAMIC_EXIT (01:58:53) Mark Shannon's async for branches instrumented (01:59:36) Free-threading changes (01:59:58) Sam Gross' regression tests can now run in --parallel-threads (02:00:34) Tomasz Pytel's thread safety crusade (02:01:01) Xuanteng Huang's __annotations__ race fix (02:01:11) Kumar's per-thread linked lists for tasks (02:02:54) Serhiy's crashes related to PySys_GetObject() fixed (02:03:22) Sam's usage of stack pointers in thread stack traversal (02:03:38) Dino Viehland's lock avoidance during object cleanup (02:04:23) OUTRO…

core.py

1
Episode 18: Reference Counting 1:39:07

26 weeks ago1:39:07

1:39:07

After we talked about memory allocation in Python back in Episode 16, we're ready to complain, uh, explain reference counting. Or at least throw a bunch of reference counting facts at you. Plus a big assortment of recent Python changes. You ready? ## Timestamps (00:00:00) INTRO (00:04:17) PART 0: SPORTS NEWS (00:06:53) PART 1: REFERENCE COUNTING (00:08:28) New segment of 2025 (00:13:54) C++ is asymptotic Python (00:15:37) Is Rust game yet? (00:18:01) Names (00:20:25) Breaking the law (00:23:08) sys.getrefcount() (00:25:21) Pedantic Pablo (00:26:06) sys.gettotalrefcount() (00:31:24) TraceRefs (00:33:28) Advantages of refcounting (00:36:16) Disadvantages of refcounting (00:38:40) Reference cycles (00:40:39) Multithreading (00:41:25) When refcounting goes wrong (00:44:05) Freeing memory in Python doesn't return it to the OS (00:45:42) Leaks and cycles redux (00:50:29) Double free (00:53:05) Avoiding reference counting (00:54:59) Immortal objects (01:00:40) PART 2: WHAT'S GOING ON IN CPYTHON (01:02:43) New features (01:04:11) Assorted interesting changes (01:15:23) Performance (01:18:09) Free-threading changes galore (01:27:02) AsyncIO (01:34:25) Windows changes (01:36:45) Security (01:37:20) OUTRO…

core.py

1
Episode 17: Argparse, JIT, and balloons with Savannah Ostrowski 1:45:06

35 weeks ago1:45:06

1:45:06

Meet our newest member of the core developer team, Savannah! Currently at Snowflake, she also worked with development tools at Docker and Microsoft, but also flew drones over forests. In terms of CPython, Savannah works on argparse and the JIT, but that's not her last word. # Timestamps (00:00:00) INTRO (00:01:26) PART 1: INTERVIEW WITH SAVANNAH OSTROWSKI (00:02:12) Beginnings as a Python user (00:04:14) Carol Willing's nudge (00:06:55) First PR (00:08:56) Psychological damage from asyncio (00:11:51) Savannah at ***** Maps (00:14:04) Chipotle Claim to Fame (00:16:14) The most funky CPython discoveries (00:19:06) What if you could break backwards compatibility in argparse? (00:23:51) How do the JIT internals look to new eyes? (00:27:33) Is Savannah team typing? (00:33:55) Somebody's jealous (00:37:29) Favorite PEP and least favorite PEP (00:42:10) Big Fish (00:52:58) Hard conversations (01:02:31) Polska (01:06:37) Do it scared (01:08:14) PART 2: PR OF THE WEEK (01:08:22) Łukasz (01:12:01) Pabluco (01:14:46) Savannah (01:18:27) PART 3: WHAT'S GOING ON IN CPYTHON (01:21:21) Features (01:29:59) Bug fixes (01:33:42) Performance (01:40:12) Security (01:43:23) OUTRO…

core.py

1
Episode 16: Memory Allocation 1:45:52

38 weeks ago1:45:52

1:45:52

How does Python handle memory? Why does it need to perform custom forms of memory allocation? We talk about all that in this episode. We don't talk about Easter eggs, and we never mention Brandt by name, as promised last time! ## Timestamps (00:00:00) INTRO (00:00:22) PART 0: SPORTS NEWS (00:01:57) PART 1: MEMORY ALLOCATION (00:03:46) If you write C correctly, it manages memory for you (00:05:38) malloc and the heap (00:09:31) High-level allocators (00:10:48) pymalloc (00:11:15) 512 is a good number (00:12:43) Memory domains in pymalloc (00:15:09) Why is pymalloc faster than just using malloc? (00:20:16) To free or not to free (00:26:35) User-facing features of pymalloc (00:30:40) Łukasz replaced by an LLM Sim... almost (00:32:58) Debugging help with marked memory (00:39:24) 8-byte alignment (00:42:27) cymem (00:45:08) tracemalloc (00:49:12) memray (00:50:47) Coding like an ultravillain (00:55:22) Complaining about Apple (01:01:45) PART 2: PR OF THE WEEK (01:13:46) Pablo's buildbot release status page (01:18:26) PART 3: WHAT'S GOING ON IN CPYTHON (01:18:52) PEP 762 (01:22:30) PEP 758 (01:26:10) Performance updates (01:29:14) Argparse suggests correct spelling (01:30:13) Fraction.from_number() and Decimal.from_number() (01:30:39) switched in contextvars (01:31:42) Better type annotations in pydoc signatures (01:33:33) Fixes in TaskGroup and _PyFuture ref cycles (01:34:03) Data race in ZoneInfo (01:35:53) Testing certificates set to expire far in the future (01:36:32) InternalDocs (01:40:22) There are no Easter eggs (01:40:42) OUTRO…

core.py

1
Episode 15: Core sprint at Meta 1:56:03

42 weeks ago1:56:03

1:56:03

Over 40 core developers spent a week in Bellevue WA putting finishing touches on Python 3.13, planning, prototyping, and implementing features for Python 3.14. We talked to half of them. We laughed, we cried. We were happy watching graphs go up, and sad watching them go down. It was intense. There was even cake, no joke! ## Timestamps (00:00:00) INTRO (00:02:05) PART 1: THE CAKE IS NOT A LIE (00:04:09) Interview with sprint organizer Itamar Oren (00:07:48) Surprise waffle machine (00:08:36) PART 2: DEVELOPERS IN THEIR OWN WORDS (00:08:47) Brandt Bucher (00:10:49) Tian Gao (00:12:31) Hugo van Kemenade (00:16:35) Thomas Wow-ters (00:24:08) Yury Selivanov (00:29:33) Mark Shannon (00:30:56) Raymond Hettinger (00:32:06) Russell Keith-Magee (00:33:18) Mariatta (00:41:41) Larry Hastings (00:46:07) Barry Warsaw (00:53:10) Jelle Zijlstra (00:56:20) Alex Waygood (00:57:39) Gregory P. Smith (01:04:08) Petr Viktorin (01:04:49) Emily Morehouse-Valcarcel (01:08:27) Interlude: Łukasz was right for once (01:09:28) Lysandros Nikolaou (01:17:38) Dino Viehland (01:23:22) Carl Meyer (01:24:29) Eric Smith (01:32:00) Eric Snow (01:41:46) Janitor 1 (01:50:01) Janitor 2 (01:53:03) OUTRO…

core.py

1
Episode 14: Integration Events 1:30:50

46 weeks ago1:30:50

1:30:50

We’ve been gone all Summer, visiting two European conferences in the mean time. In this episode we’re talking about them both, talks we liked, as well as our own talks at those events. In a rare turn of events, this one was recorded in person at Łukasz’s home studio in Poznań! ## Outline (00:00:00) INTRO (00:01:30) PART 1: EuroPython highlights (00:02:03) Maintaining pyrepl forward with pypy (00:05:51) Mai Giménez and her keynote (00:09:30) Yuliia Barabash and Laysa Uchoa talk memory management (00:11:03) Core developer panel and sprints (00:11:56) The abyss stares back at Pablo (00:18:21) Disappointing (00:25:50) Łukasz and the all-singing all-dancing pyrepl (00:33:52) Hackable REPL = new contributors (00:40:24) PART 2: PyCon PL (00:42:24) An integration event (00:44:45) Say No To Notebooks (00:48:32) Pablo's grandfather's axe (00:56:00) LL1 made Python easy to read for humans, too (00:59:29) Python and triangles in the browser (01:06:16) PART 3: PR OF THE WEEK (01:13:08) PART 4: WHAT'S GOING ON IN CPYTHON (01:13:32) HACL* blake2 (01:15:29) New methods on pathlib.Path objects (01:16:28) PyIter_GetNext (01:17:20) PyLong_FromInt64 (01:18:07) CALL_ALLOC_AND_ENTER_INIT suitable for Tier 2 (01:19:09) Tier 2 support for BINARY_SUBSCR_GETITEM (01:19:26) Removal of the CHECK_EVAL_BREAKER macro (01:20:11) Don't be a jerk on GitHub (01:21:30) Per-thread heap type refcounts (01:22:29) Pickling by name for object without __module__ is now faster (01:23:12) CVE-2024-7592 fixed (01:24:11) Free threading memory leaks at interpreter shutdown (01:25:34) The "What's New in 3.13" document getting some love (01:26:32) Recursion-guarding code improved in the compiler (01:27:13) A new Android test script for running with emulators (01:27:28) Core sprint week at Meta coming up! (01:29:53) OUTRO…

core.py

1
Episode 13: A Legit Episode 1:51:55

1 year ago1:51:55

1:51:55

In this lucky episode we're interviewing fellow core developer Brandt Bucher to talk about Justin, Swedish warships, and the n-body benchmark. We're also breaking the duration record with this one. We promise we'll get faster in future releases! ## Outline (00:00:00) INTRO (00:01:43) PART 1: BRANDT BUCHER INTERVIEW (00:03:04) Beginnings of contribution (00:06:29) Sticking around (00:09:38) PEP work: pattern matching, dict unions, weird decorators (00:13:07) Implementing pattern matching, we like parsers (00:19:41) First tasks with the Faster Python team (00:20:59) It's always pytest with these things (00:28:55) Pepe Silvia and generators (00:30:12) The paper that inspired the JIT (00:32:01) The n-body benchmark is a joke (00:35:33) What even is a JIT? (00:38:11) Advantages of copy & patch (00:40:27) The Vasa Question (00:45:30) When are we getting faster? (00:49:09) Using pure Python versions of libraries... for speed? (00:52:18) The weirdest bug so far (00:55:12) How did removal of the GIL complicate your life? (00:57:53) Naming things is hard (00:59:55) Collaborating and mentoring others (01:06:19) The Linker Connoisseur Question (01:08:53) PART 2: PR OF THE WEEK (01:14:04) PART 3: WHAT'S GOING ON IN CPYTHON (01:14:40) Jelle is implementing PEP 649 and PEP 749 (01:15:08) Petr's battle with string interning (01:16:24) Ruben Vorderman makes str.count 2X faster (01:16:54) Ken Jin folds constants in entire attribute loads (01:18:07) neonene and Eric Snow make datetime work better with subinterpreters (01:20:18) pickle protocol 5 will be the default in 3.14 (01:21:58) Tian Gao improves pdb (01:23:42) Free-threading changes galore (01:27:34) Victor exposes PyUnicodeWriter in the C API (01:28:18) PyREPL changes & going off the rails…

core.py

1
Episode 12: WTF Python 1:24:55

1 year ago1:24:55

1:24:55

You think you know Python? We thought so, too. Join us for an episode of surprises. You might know some of those, but you sure don't know all of them. ## Outline (00:00:00) INTRO (00:02:22) Integer interning with a twist (00:10:58) Return in finally (00:15:32) all([[]]) (00:20:06) Lists, iterators and hashing shenanigans (00:27:08) hash(-1) (00:31:30) String interning (00:34:23) PR OF THE WEEK (00:34:56) asyncio REPL now uses pyrepl, too (00:44:06) PyOS_InputHook (00:51:56) WHAT'S GOING ON IN CPYTHON (00:52:05) New core developers! (00:55:10) 3.13 beta 2 & 3.12.4 released (00:56:04) Pablo's top 3 favorite pyrepl improvements in Beta 2 (00:59:50) PEP 667 implemented! (01:02:42) Tian Gao's improvements to pdb (01:06:31) Uncle Tim's crusade to make int(some_string) asymptotically faster (01:08:50) datetime now uses interpreter-local static types (01:09:55) PEP 749 enters the chat, paving the way to a PEP 649 future (01:11:35) os.fwalk, os.walk, shutil.rmtree now support arbitrary depth directory trees (01:14:13) Free-threading changes (01:22:15) OUTRO…

core.py

1
Episode 11: Live from PyCon 2024 30:30

1 year ago30:30

30:30

Who says we can't do short episodes? Well, it was a challenge! But with the help of some gentle conference schedule pressure, here's our first sub-hour episode. We're discussing the language summit, answering audience questions, and behaving like serious professionals, as usual. ## Outline (00:00:00) INTRO (00:02:01) PEP 602 tweaks: 2 years of bug fix releases (00:02:45) CalVer for Python versions? (00:04:51) In Python there's this thing called the C API (00:08:51) More about PyREPL (00:12:17) Evolving pdb (00:14:49) Memhive (00:16:18) The boring 3.13 (00:18:16) Q&A (00:18:43) Fashion + favorite bug (00:22:12) What happens in 2100 with CalVer? (00:23:02) Subinterpreters or free-threading? (00:23:41) Where else to find information on current development? (00:25:40) Will distros provide free-threading builds? (00:26:23) Will the podcast run out of content? (00:28:37) OUTRO…

core.py

1
Episode 10: The Interactive REPL 1:22:51

1 year ago1:22:51

1:22:51

Oof, no episode in April, huh? Yeah, we're getting close to Python 3.13 beta 1. PyCon US is also coming up real soon. Let's use this opportunity then to talk about a feature we're teaming up on: a better interactive interpreter! ## Outline (00:00:00) INTRO (00:01:53) PART 1: History of Terminals (00:03:20) /dev/tty (00:04:51) The first cool word (00:05:45) Chrząszcz (00:06:20) Control code characters in ASCII (00:11:54) PART 2: Python REPL Today (00:12:34) There is no REPL (00:15:28) So what is there instead? (00:19:13) readline (00:25:38) Source in the REPL (00:31:13) Implementing a REPL from scratch? Prepare to support arg: 5 (00:36:09) PART 3: PR OF THE WEEK (00:37:09) Introducing: Complaining Pablo (00:38:23) Tests are always green if you skip them (00:39:57) Getting dirty with escape sequences (00:41:28) Typing finds bugs (00:42:29) Shiny new features of the new REPL (00:45:55) Contributing back to PyPy (00:48:10) We still have two weeks, right? (00:49:59) Is Python synthwave enough? (00:51:57) Do we have a bug? (00:55:31) What's lurking in pydoc? (00:59:38) PART 4: WHAT'S HAPPENING IN CPYTHON? (01:02:39) PEP 744: The JIT (01:06:05) Incremental GC is now actually in (01:08:21) Tier 2 interpreter updates (01:10:29) Python supported on iOS with PEP 730 (01:13:11) Better error messages for name shadowing (01:15:17) Queue.shutdown() (01:17:14) ctypes adopts heap types (01:18:26) Free-threading updates (01:20:14) Dataclass creation is faster (01:20:44) OUTRO…

core.py

1
Episode 9: Py Day with Emily Morehouse-Valcarcel 1:09:44

1 year ago1:09:44

1:09:44

Let's talk about the Steering Council, running a small consultancy business, the Walrus, and pet peeves with our special guest today! ## Outline (00:00:00) INTRO (00:00:56) PART 1: Emily Morehouse (00:02:15) Running a small consultancy business (00:04:39) What features of JS do you miss in Python? (00:05:50) Łukasz outnumbered in a world of Steering Council members (00:06:12) Upgrading to new Python versions (00:07:00) It depends on who deployed the project (00:09:44) Second term as a Steering Council member (00:11:33) Barry, play some bass for us (00:13:04) Let's hear a recent war story (00:15:17) Is this progress bar even working? (00:17:40) The Villain Origin Story (00:21:37) Emily, The Bringer of Doom (00:22:37) Consensus within the Steering Council (00:25:52) Syntax changes in Python are rare, right? Right? (00:28:22) On implementing PEP 572 (00:32:52) How would PyCon 2020 in Pittsburgh feel? (00:34:18) How can you be mad about the Walrus? (00:36:10) Favorite parts of the standard library (00:38:10) Is hacking on Python a good experience to newcomers? (00:40:26) Emily's pet peeve about Python, take 1 (00:42:17) Emily's favorite change in Python in recent years (00:44:34) Emily's pet peeve about Python, take 2 (00:46:34) Łukasz's pet peeve (00:48:25) Surprise extra question (00:49:42) At core.py we are professionals (00:51:00) PART 2: PR of the Week (00:54:00) CALL TO ACTION: Upgrade Python.org to Django 4! (00:56:22) PART 3: What's Going On in CPython? (00:56:38) Faster Python updates (01:00:10) Free threading: GIL can be disabled but we're not done yet! (01:04:17) New defaults for SSL context flags (01:05:39) python -m asyncio and sys.__interactivehook__ (01:06:24) Surprise question: what is sys.__interactivehook__ even doing? (01:08:11) OUTRO…

core.py

1
Episode 8: The New Parser 1:42:36

1 year ago1:42:36

1:42:36

The suspense was killing us! OK, the old parser was then... but what about NOW? We're finally answering this question... in more detail than you dared to ask for. PEG, memoization, funky secrets, and how a certain auto-formatter self-inflicted an existential crisis on itself. It's all there, told in barely 100 minutes! Can you believe it? # Timestamps (00:00:00) INTRO (00:00:54) PART 1: What even is PEG? (00:04:02) You can't prove anything! (00:05:03) What's a "parsing expression"? (00:08:23) Our old LL1 parser wasn't doing its job (00:09:37) "Soft keywords" in LL1: A Horror Story (00:13:16) PART 2: How PEG was adopted by Python (00:17:10) Why not LALR? (00:22:11) The PEG paper wasn't enough either, if we're honest (00:26:26) Less obvious advantages of the new parser (00:31:28) Black is stuck with LL1, can it cope? (00:36:24) Hedging against Łukasz, the bringer of doom (00:41:14) PART 3: How does the PEG parser of CPython work? (00:44:30) Pedantic Pablo on "exponential" (00:45:14) Fresh news from literally yesterday last week (00:46:39) Pedantic Pablo on "infinite" (00:47:32) Memoization in the PEG parser (00:50:41) Parse once, and if it fails, try again! (00:52:14) How to model a grammar of programming mistakes? (00:56:36) Why is there C code in my grammar file? (00:59:57) Bro, do you even lift? (01:01:45) How soft keywords work today: it's not free lunch (01:04:29) Funky grammar secrets (01:09:07) PART 4: PR OF THE WEEK (01:09:15) audioop.c license shenanigans (01:14:56) The secret profiler inside CPython (tests) (01:22:45) PART 5: WHAT'S GOING ON IN CPYTHON? (01:23:30) Free-threading changes (01:28:15) Faster Python changes (01:35:39) End of an era: docs get rid of Python 2 migration info (01:36:45) Python --help output is now nicer (01:38:43) SQLite as a dbm backend (01:41:08) OUTRO…

core.py

1
Episode 7: The Old Parser 1:23:24

1 year ago1:23:24

1:23:24

Context-free grammars, non-deterministic finite automatons, left-to-right leftmost derivations... what even is all that?! Today we're talking about how Python parses your source code. We start gently with how this worked in the past. Come listen to Łukasz's high-level explanations and Pedantic Pablo's "well actuallys". # Timestamps (00:00:00) INTRO (00:01:35) You can still download Python 1.0! (00:02:19) The original tokenizer (00:03:10) What even is a tokenizer? (00:04:08) FUN FACTS ABOUT THE TOKENIZER (00:04:34) Circumflex (00:05:16) Python's invisible braces (00:08:29) Backticks in the syntax (00:11:00) Where are the comments stored? (00:12:27) GRAMMAR (00:13:37) What is a grammar? (00:16:25) The long-forgotten 'access' keyword (00:20:25) Making LL1 do things it wasn't meant to do (00:23:24) SURPRISE QUESTION 1: soft keywords (00:24:46) What's a context-free grammar? (00:26:51) A note about backslashes (00:29:33) The Dragon Book(s) (00:31:27) PARSING: What is it? (00:35:23) How to generate a parser? (00:39:00) LL Cool Parser (00:41:15) What if we used LR? (00:44:01) Let's have three tokenizers! (00:47:50) 2to3 and its legacy (00:52:38) Black and its blib2to3 (00:54:04) The pesky 'with' statement and the death of LL1 (01:00:05) PR OF THE WEEK: GH-113745 (01:05:41) SURPRISE QUESTION 2: Subclasses of SyntaxError (01:07:02) WHAT'S GOING ON IN CPYTHON? (01:09:16) Sam Gross nominated as a core dev (01:10:13) Free-threading progress (01:13:11) Faster CPython changes (01:17:29) ntpath.isreserved() (01:20:11) Pablo and the DWARF (01:22:02) OUTRO…

core.py

1
Episode 6 - An Exceptional Episode 1:31:25

2 years ago1:31:25

1:31:25

Happy New Year! In this episode we're talking about exceptions, how they work, and how they evolved. Expect the unexpected. # Timestamps (00:00:00) INTRO (00:01:43) How does a 'try' block work? (00:04:00) How many 'try' blocks can you fit on a bus? (00:05:56) How does Python store the current exception? (00:09:30) Pre-history: exceptions as strings (00:12:59) Try out string exceptions with CentOS 5 (00:14:28) PEP 341: Combining 'finally' and 'except' in one 'try' block (00:16:15) Core Hacker in Residence (00:16:51) PEP 3109: Raising exceptions in Python 3K (00:19:17) Automatic tuple unpacking for raised exceptions?! (00:21:55) PEP 3110: Catching exceptions in Python 3K (00:26:00) Foreshadowing: exception groups (00:27:10) PEP 3134: Exception chaining (00:29:12) __cause__, __context__, __traceback__ (00:31:50) Back in the day we had to walk uphill both ways (00:32:56) PEP 409: Suppressing exception chaining (00:34:44) Raise from None? Or raise from Ellipsis? (00:37:11) __supress_context__ (00:38:13) Semantic difference between 'pass', 'None', and '....' (00:41:02) NotImplemented vs NotImplementedError (00:43:02) Zero-cost exceptions in Python 3.11 (00:51:12) Reconstructing exception table entries dynamically (00:52:51) Objects/exception_handling_notes.txt (00:54:19) PEP 654: Historical context (00:56:58) PEP 654: BaseExceptionGroup and ExceptionGroup (00:58:29) PEP 654: except* (01:03:23) PEP 678: Exception notes (01:06:09) PEP 657: Cooler Errors (01:08:36) A message to language implementers (01:09:15) Fine-grained error locations in tracebacks (01:12:05) This is useful for tracing coverage and Specializator (01:13:50) Hacker in Residence fired? (01:16:25) WHAT'S GOING ON IN CPYTHON (01:16:58) Copy&patch JIT PR open (01:21:30) Free-threading progress: GC split, stop-the-world (01:24:22) The buildbots are red (01:25:27) Faster CPython changes: interpreter code generator refactors (01:26:27) Eval Game crash fix (01:28:41) Three developers in residence! (01:29:54) OUTRO # Links https://compilercrim.es/rust-np/…

core.py

1
Episode 5 - Cinder with Carl Meyer 1:21:19

2 years ago1:21:19

1:21:19

This time we're hosting a special guest: Carl Meyer from Meta. What is Cinder, how does it work, and how does it intersect with the future of Python 3? Find out in today's episode. 100% serious stuff! # Timestamps (00:00:00) INTRO (00:00:53) Carl Meyer's war story (00:02:27) CINDER (00:03:22) Static Python makes things significantly faster (00:08:15) Cinder JIT and how it's tuned for Instagram (00:11:57) Strict Python and the joy of import side effects (00:16:35) The static typing controversy (00:18:52) Upstreaming changes from Cinder? (00:22:53) PEP 709: Comprehension inlining (00:28:35) pip install CinderX (00:31:19) Immortal instances (00:35:15) asyncio.eager_task_factory() (00:39:39) Carl's pet peeve with Python (00:44:49) PR OF THE WEEK: PyPy's REPL in CPython (00:52:07) WHAT'S GOING ON IN CPYTHON (00:52:22) Python 3.12.1 (00:53:17) Python 3.11.7 (00:54:45) multiprocessing.SharedMemory track (00:56:49) Fine-grained error locations for multi-line expressions (01:00:03) libedit tab completion is fixed (01:02:14) Colored exception tracebacks (01:05:11) Removing testing modules from sys.modules, correctly (01:06:47) SBOMs are a very serious matter (01:09:08) Arrays by value on ARM (01:12:24) Remove development environments and CAPS LOCK (01:15:30) Interpreter cases generator refactored (01:16:17) Free-threading news (01:20:01) OUTRO…

core.py

1
Episode 4 - Frame Evaluation 1:13:20

2 years ago1:13:20

1:13:20

What makes Python an interpreter? Today we're talking about ceval.c, the wonders of frame evaluation, and how it changed over the years. # Timestamps (00:00:00) INTRO (00:00:59) BACK TO PYTHON 2.6 (00:02:53) Stack virtual machine (00:04:41) First encounter with opcodes (00:08:06) What even is frame evaluation? (00:12:51) Stack! Which stack? (00:15:46) PRESENT DAY (00:16:41) Computed gotos (00:21:22) PEP 523: JIT me, maybe (00:26:53) Let's generate the interpreter (00:29:08) The JIT is coming (00:33:13) Python function call inlining (00:37:23) Instrumentation: DTrace, PEP 669 (00:41:50) lltrace and pystats (00:44:02) Eval breaker (00:47:54) Signal handling (00:50:47) Recursion limits (00:54:27) String concatenation special case (00:58:24) WHAT'S GOING ON IN CPYTHON? (00:58:42) 3.12.0a2 (00:59:12) Critical section API adoption (00:59:34) PyOnceFlag (01:00:28) PyDict_GetItemRef() (01:03:36) PyList_Extend() and PyDict_Pop() (01:04:18) Parser: better error messages for non-matching elif/else (01:05:39) glob.translate() (01:07:22) TLS-PSK in the ssl module (01:08:35) IDLE debugger improvements (01:10:50) First micro-op in the Tier 2 interpreter (01:11:18) OUTRO…

core.py

1
Episode 3 - Imports, frozen modules, Python news 1:11:44

2 years ago1:11:44

1:11:44

What happens when you type “import abc”? Why does it say the module is frozen? What significant changes landed in Python in the past two weeks? And why does the “PR of the Week” jingle go so hard? Find out in this week’s episode! # Timestamps (00:00:00) INTRO (00:01:12) IMPORTS (00:02:21) Here be dragons (00:02:42) High level summary (00:05:12) Finders and Loaders (00:06:29) Loading encrypted modules (00:08:13) Debugging by altering someone else's code (00:10:08) Support for site-packages (00:14:22) Nefarious uses of imports (00:17:28) Tagged strings? (00:21:00) FROZEN MODULES (00:23:38) Deepfreeze origins: importlib (00:26:01) Deepfreeze evolution (00:31:32) First world problems with build speed (00:33:05) WHAT'S GOING ON IN CPYTHON (00:34:16) Biased reference counts landed, buildbots (00:35:51) There is no nogil, there is only free-threaded (00:36:54) mimalloc landed (00:42:29) Critical sections landed (00:45:04) Tuples can now be shared across subinterpreters (00:48:31) JIT beginnings: Tier 2 interpreter merged with Tier 1 (00:50:54) perf support with DWARF: look ma, no frame pointers (00:55:05) Yo DAWG, I herd you liked codepoints (00:58:46) Thread.join() waits for the OS thread now (01:01:04) PR OF THE WEEK (01:02:06) Pablo: gh-109181 -- traceback object creation is now lazy = much faster (01:05:25) Łukasz: gh-111710 -- stdlib coverage now gathered with sys.monitoring (01:10:20) WRAP UP…

core.py

1
Episode 2 - PEP 703: Removing the GIL 1:14:39

2 years ago1:14:39

1:14:39

We've read the PEP on making the Global Interpreter Lock optional so you don't have to. Timestamps (00:00:00) Intro (00:00:50) CURRENT STATE OF THINGS (00:00:58) Reference counting (00:01:35) Garbage collection (00:02:33) What is the Global Interpreter Lock? (00:03:57) The GIL and threading (00:07:24) Current ways around the GIL (00:09:26) HISTORICAL ATTEMPTS TO REMOVE THE GIL (00:09:46) 1999: Greg Stein's attempt at Python 1.6 (00:10:46) Jython doesn't have the GIL (00:11:19) 2015: Larry Hastings' Gilectomy at Python 3.5 (00:12:34) Pablo says removing the GIL is actually very easy (00:13:24) Łukasz is skeptical (00:14:20) 2021: Sam Gross' nogil at Python 3.9 (00:18:22) 2023: PEP 703 for Python 3.13 (00:20:05) PEP 703 IN DETAIL (00:20:20) Biased Reference Counting (00:25:19) Other needed speedups: deferred refcounts, immortalization, GC (00:28:41) mimalloc (00:34:46) More GC changes (00:38:20) eval breaker (00:42:08) Thread-safe standard collections (00:45:26) Fast paths vs. slow paths (00:49:47) Reading freed memory with mimalloc is kinda okay? (00:58:06) Specializations become harder to implement without the GIL (01:01:27) PEP 703 terms of acceptance (01:04:08) No free lunch (01:09:25) It's now or never (01:11:59) Outro…

core.py

1
Episode 1 - Core Sprint in Brno & Python 3.13.0 alpha 1 1:11:59

2 years ago1:11:59