Best Kafka Cloud Native Podcasts (2025)

1
Anna: A KVS For Any Scale 19:01

17d ago19:01

19:01

This research paper introduces Anna, a key-value store (KVS) designed for scalable performance across diverse computing environments, from single multi-core machines to globally distributed cloud deployments. Anna achieves high performance and adaptability through a partitioned, multi-master architecture utilizing wait-free execution and coordinati…

1
Conflict-free Replicated Data Types 26:17

25d ago26:17

26:17

This academic paper introduces Conflict-free Replicated Data Types (CRDTs), which are abstract data types designed for distributed systems where data is replicated across multiple locations. CRDTs allow any replica to be modified without needing immediate coordination with other replicas, ensuring high availability and low latency. The core concept…

1
CAP Twelve Years Later: How the "Rules" Have Changed 29:47

1M ago29:47

29:47

This content from InfoQ provides insights for software architects and developers through various formats like newsletters, articles, and conference information. It highlights topics in architecture, AI, data engineering, culture, methods, and DevOps. Featured pieces discuss Slack's cellular architecture, data stream processing patterns, cultivating…

1
Raft versus Paxos: An Understandable Consensus Algorithm 33:14

1M ago33:14

33:14

Raft, a consensus algorithm designed for managing a replicated log in distributed systems. It aims to be more understandable than Paxos, a widely used but complex alternative, while achieving equivalent efficiency and safety. Raft separates key consensus elements like leader election, log replication, and safety, using techniques such as problem de…

1
Neo4j Architecture: Graph Database Internals, Performance, and Optimization 17:42

2M ago17:42

17:42

This compilation of resources offers a comprehensive examination of Neo4j's graph database architecture. It explains how Neo4j differs from relational and document-oriented databases through its native graph storage. The materials describe how nodes, relationships, and properties are stored and indexed for efficient traversal and query processing. …

1
Sentry: Error Monitoring at Scale - Design Principles Analysis 15:48

2M ago15:48

15:48

Sentry is a large-scale, open-source error monitoring platform designed for modern distributed systems. It prioritizes actionable insights by focusing on exceptions and crashes, enriching errors with contextual data, and using features such as breadcrumbs and error grouping. Sentry's architecture employs modular and decoupled components like Relay …

1
Istio Service Mesh: Architecture, Security, and Traffic Management 33:58

2M ago33:58

33:58

These excerpts offer a detailed look at Istio's service mesh architecture, a critical component for managing microservices in cloud-native environments. The architecture is divided into a control plane and data plane, emphasizing security through automated mTLS and traffic management with advanced load balancing techniques. Observability is achieve…

1
CockroachDB: SQL for Global Scale Design Principles 14:33

2M ago14:33

14:33

CockroachDB is a distributed SQL database designed for global scalability and resilience. The database achieves this through a unique architecture built on a monolithic key-value store, Raft-based replication, and hybrid logical clocks. Transaction management is optimized for global workloads using a non-blocking commit protocol and multi-region ca…

1
Snowflake: Revolutionizing Cloud Data Warehousing and Analytics 17:21

2M ago17:21

17:21

Snowflake, a cloud-native data warehouse, revolutionizes modern analytics through its unique architecture and capabilities. The platform separates compute and storage layers, enabling independent scaling and optimized performance. Its three-layer design encompasses cloud services, a compute layer using virtual warehouses, and a storage layer levera…

1
Kubernetes: Container Orchestration, Architecture, and Evolution 25:56

3M ago25:56

25:56

This collection of excerpts comprehensively examines Kubernetes, the leading container orchestration platform. It traces the historical evolution of container orchestration and highlights Kubernetes' architectural foundations, including its control plane and node components. Scalability mechanisms like horizontal pod autoscaling and cell-based arch…

1
Elasticsearch: Architecture, Applications, and Emerging Trends 18:13

3M ago18:13

18:13

This compilation of excerpts thoroughly examines Elasticsearch, focusing on its architecture, applications, and future trends. The core architecture and its integration within the Elastic Stack are highlighted, emphasizing scalability and real-time analytics. Various specialized applications are discussed, including maritime data storage, academic …

1
Ray: A Distributed Framework for Emerging AI Applications 19:40

3M ago19:40

19:40

This research paper introduces Ray, a distributed framework designed for emerging AI applications, particularly those involving reinforcement learning. It addresses the limitations of existing systems in handling the complex demands of these applications, which require continuous interaction with the environment. Ray unifies task-parallel and actor…

1
Zanzibar: Google's Global Authorization System 27:21

3M ago27:21

27:21

This paper details Zanzibar, Google's globally distributed authorization system, designed to manage access control lists (ACLs) at a massive scale. Zanzibar uses a flexible data model and configuration language to handle diverse access control policies for numerous Google services, achieving high availability and low latency. The system maintains e…

1
Google Mesa: A Geo-Replicated, Near Real-Time Data Warehouse 15:02

4M ago15:02

15:02

**Mesa** is a highly scalable, geo-replicated data warehousing system developed at Google to handle petabytes of data related to its advertising business. **Designed for near real-time data ingestion and querying**, it processes millions of updates per second and serves billions of queries daily. **Key features include strong consistency, high avai…

1
Time, Clocks, and the Ordering of Events in a Distributed System 13:50

4M ago13:50

13:50

This paper, "Time, Clocks, and the Ordering of Events in a Distributed System," explores the challenges of defining and managing time in distributed systems. It introduces the concept of a "happened before" relation to partially order events and presents an algorithm for creating a consistent total ordering using logical clocks. The paper then exte…

1
ZooKeeper: Wait-Free Coordination for Internet-Scale Systems 26:38

4M ago26:38

26:38

This paper details the design and implementation of ZooKeeper, a high-performance coordination service for large-scale distributed systems. ZooKeeper provides a simple, wait-free API enabling developers to build various coordination primitives, such as locks and group membership, without server-side modifications. It achieves high throughput throug…

1
TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems 17:02

4M ago17:02

17:02

This paper details TensorFlow, a large-scale machine learning system developed by Google. TensorFlow uses dataflow graphs to represent computation and manages state across diverse hardware, including CPUs, GPUs, and TPUs. It offers a flexible programming model, allowing developers to experiment with novel optimizations and training algorithms beyon…

1
Firestore: A Serverless NoSQL Database 27:46

5M ago27:46

27:46

This paper details Google Firestore, a NoSQL serverless database built on Spanner. It highlights Firestore's ease of use, scalability, real-time query capabilities, and support for disconnected operations. The architecture, which enables multi-tenancy and efficient handling of large datasets, is explained. Performance benchmarks and practical lesso…

1
Apache Flink: Stream and Batch Processing in a Single Engine 18:12

5M ago18:12

18:12

This research paper details Apache Flink, an open-source system unifying stream and batch data processing. Flink uses a dataflow model to handle various data processing needs, including real-time analytics and batch jobs, within a single engine. The paper explores Flink's architecture, APIs (including DataStream and DataSet APIs), and fault-toleran…

1
Kafka: A Distributed Messaging System for Log Processing 16:52

5M ago16:52

16:52

This paper introduces Kafka, a novel distributed messaging system designed for high-throughput log processing. Kafka addresses limitations in existing messaging systems and log aggregators by offering a scalable, efficient architecture with a simple API. Key features include a pull-based consumption model, efficient storage and data transfer mechan…

1
LinkedIn: Using Set Cover to Optimize a Large-Scale Low Latency Distributed Graph 12:59

5M ago12:59

12:59

This research paper details LinkedIn's solution for optimizing low-latency graph computations within their large-scale distributed graph system. To improve performance, they implemented a modified greedy set cover algorithm to minimize the number of machines needed for processing second-degree connection queries. This optimization significantly red…

1
Monolith: A Real-Time Recommendation System 20:25

6M ago20:25

20:25

This research paper details Monolith, a real-time recommendation system developed by Bytedance. Monolith addresses challenges in building scalable recommendation systems, such as sparse and dynamic data, and concept drift, by employing a collisionless embedding table and an online training architecture. Key innovations include a Cuckoo HashMap for …

1
Meta FlexiRaft: Flexible Quorums for Raft Consensus 24:27

6M ago24:27

24:27

This research paper details FlexiRaft, a modified Raft consensus algorithm designed for Meta's petabyte-scale MySQL deployments. The core improvement is the introduction of flexible quorums, allowing configurable trade-offs between latency, throughput, and fault tolerance. Two quorum modes are presented: static and dynamic. The paper explores the a…

1
Spanner: Google’s Globally Distributed Database 13:28

6M ago13:28

13:28

This research paper details Spanner, Google's globally-distributed database system. Spanner achieves strong consistency across its geographically dispersed data centers using a novel TrueTime API that accounts for clock uncertainty. The system features automatic sharding, failover, and a semi-relational data model, addressing limitations of previou…

1
Meta Minesweeper: Scalable Statistical Root Cause Analysis on App Telemetry 17:54

7M ago17:54

17:54

This research paper introduces Minesweeper, a novel technique for automated root cause analysis (RCA) of software bugs at scale. Leveraging telemetry data, Minesweeper efficiently identifies statistically significant patterns in user app traces that correlate with bugs, even in the absence of detailed debugging information. The method uses sequenti…

1
Cassandra- A Decentralized Structured Storage System 14:27

7M ago14:27

14:27

This paper details Cassandra, a decentralized structured storage system designed for managing massive amounts of structured data across numerous commodity servers. High availability and scalability are key features, achieved through techniques like consistent hashing for data partitioning and replication strategies across multiple data centers to h…

1
FoundationDB: A Distributed Unbundled Transactional Key Value Store 21:02

7M ago21:02

21:02

The provided text is an excerpt from a research paper on FoundationDB, an open-source, distributed transactional key-value store. The paper details FoundationDB's design principles, architecture, and key features, including its unbundled architecture, strict serializability through a combination of optimistic concurrency control (OCC) and multi-ver…

1
Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases 18:56

7M ago18:56

18:56

This document describes the design of Amazon Aurora, a cloud-native relational database service built to handle high-throughput, online transaction processing (OLTP) workloads. The paper highlights the challenges of traditional database architectures in cloud environments, specifically the I/O bottleneck created by network traffic. Aurora addresses…

1
Pregel: A System for Large-Scale Graph Processing 22:39

7M ago22:39

22:39

The article is a paper published in 2010 by researchers at Google that introduces Pregel, a large-scale graph processing system. Pregel is designed for processing graphs with billions of vertices and trillions of edges, and it uses a vertex-centric approach where vertices are assigned to individual machines and communicate with each other through m…

1
Dapper, a Large-Scale Distributed Systems Tracing Infrastructure 20:42

7M ago20:42

20:42

This paper from Google describes the design and implementation of Dapper, Google’s system for tracing requests in distributed systems. The authors explain why they chose a distributed tracing system, the design decisions they made for Dapper, and how the Dapper infrastructure has been used in practice. They also discuss the impact of Dapper on appl…

1
Google: The Chubby lock service for loosely-coupled distributed systems 51:11

8M ago51:11

51:11

This document describes the development and implementation of Google's Chubby lock service, a highly available and reliable system that provides coarse-grained locking and storage for distributed systems. The authors discuss the design choices behind Chubby, including its emphasis on availability over performance, and the use of a file system-like …

1
Megastore: Providing Scalable, Highly Available Storage for Interactive Services 18:19

8M ago18:19

18:19

The provided text describes the architecture and design of Megastore, a Google-developed storage system designed to meet the needs of interactive online services. Megastore blends the scalability of NoSQL datastores with the convenience of traditional relational databases, offering high availability and strong consistency guarantees. It achieves th…

1
Bigtable: A Distributed Storage System for Structured Data 15:26

8M ago15:26

15:26

The article, “Bigtable: A Distributed Storage System for Structured Data,” describes a large-scale distributed data storage system developed at Google, capable of handling petabytes of data across thousands of servers. Bigtable uses a simple data model that allows clients to dynamically control data layout and format, making it suitable for various…

1
MapReduce: Simplified Data Processing on Large Clusters 14:15

8M ago14:15

14:15

MapReduce is a programming model that simplifies the process of processing large datasets on clusters of commodity machines. It allows users to define two functions: Map and Reduce, which are then automatically parallelized and executed across the cluster. The Map function processes key/value pairs from the input data and generates intermediate key…

1
The Google File System 32:26

8M ago32:26

32:26

The source is a technical paper that describes the Google File System (GFS), a scalable distributed file system designed to meet Google's data processing needs. The paper discusses the design principles behind GFS, including its focus on handling component failures, managing large files, and optimizing for append-only operations. It also details th…

1
TAO: Facebook’s Distributed Data Store for the Social Graph 18:46

8M ago18:46

18:46

Facebook developed a distributed data store called TAO to efficiently serve the social graph data. TAO prioritizes read optimization, availability, and scalability over strict consistency, handling billions of reads and millions of writes per second. TAO utilizes a simplified data model based on objects and associations, offering a specialized API …

1
Scaling Memcache at Facebook 13:29

8M ago13:29

13:29

This document details how Facebook engineers scaled Memcached, a popular open-source in-memory caching solution, to accommodate the demands of the world's largest social network. The paper outlines the development of Facebook's Memcached architecture, starting with a single cluster of servers and progressing through geographically distributed clust…

1
Monarch: Google’s Planet-Scale In-Memory Time Series Database 20:26

8M ago20:26

20:26

This technical paper details the architecture and design of Monarch, a planet-scale in-memory time series database developed at Google. Monarch is used to monitor the performance and availability of massive, globally distributed systems like YouTube, Google Maps, and Gmail. The paper discusses the system's novel features, including its regionalized…

1
Gorilla: A Fast, Scalable, In-Memory Time Series Database 9:52

8M ago9:52

9:52

The provided text describes the architecture and functionality of Gorilla, Facebook's in-memory time series database. Gorilla was developed to address the challenges of monitoring and analyzing massive amounts of time series data generated by Facebook's vast infrastructure. The system prioritizes high availability for writes and reads, even in the …

1
Building a three-tier architecture on a budget 16:22

8M ago16:22

16:22

This document, an AWS blog post, guides users through the process of building a cost-effective, three-tier architecture using serverless technologies within the AWS Free Tier. It begins by explaining the benefits and capabilities of AWS serverless services and then provides a detailed walkthrough of how to construct each tier (presentation, busines…

1
Saas Lens: Deploy multi-tenant SaaS workloads using AWS services 21:18

8M ago21:18

21:18

This whitepaper outlines the AWS Well-Architected Framework specifically for Software as a Service (SaaS) applications. It examines how to design and deploy multi-tenant SaaS workloads using AWS services, detailing best practices in operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. The whi…

1
Streaming Media Lens 50:03

8M ago50:03

50:03

This document is a white paper about the AWS Well-Architected Framework, particularly focusing on its application to streaming media workloads. It defines key components within a streaming media architecture, including ingest, processing, origin, delivery, and the client. The paper then outlines best practices for designing and implementing streami…

1
Dynamo: Amazon’s Highly Available Key-value Store 17:43

8M ago17:43

17:43

This technical paper details the design and implementation of Dynamo, a highly available and scalable key-value storage system developed by Amazon.com. The paper outlines the challenges of maintaining reliability at a massive scale in an e-commerce environment and explains how Dynamo addresses these challenges by sacrificing consistency in favor of…

1
Optimizing Cloud Costs for SaaS Startups 29:35

1y ago29:35

29:35

You can't manage what you don't measure, and this includes your cloud costs. But how detailed should this measurement be? And how will the data translate into impact?I sat down with Adam Shugar, co-founder and CTO of Dashdive, to discuss his approach to cloud costs. He shared his advice, not only on cost cutting but also technology, growing a start…

1
SaaS: More than just a business model 52:04

1+ y ago52:04

52:04

Bill Tarr has the most interesting job in the world. He and his team of SaaS Evangelists at the AWS SaaS Factory work with companies large and small to help them build SaaS on AWS. In this conversation, we discuss the different ways technology and business interact when building SaaS - from cool technical options available only to SaaS companies to…

1
Building Streaming on S3 58:11

1+ y ago58:11

58:11

Ryan Worl, co-founder and CTO of WarpStream, is on a mission to re-engineer fundamental infrastructure on top of S3. Starting with metrics at Uber, continuing with Husky - DataDog's platform for events, logs and everything except metrics and then... Ryan thought "it will be cool to re-build Kafka on S3", reached out to other developers to hear thei…

1
Kora: Cloud Native Platform for Kafka 42:49

1+ y ago42:49

42:49

We invited Anna Povzner, director of engineering at Confluent, to the show to discuss Kora. Kora is a Cloud Native platform based on Apache Kafka, which her team at Confluent built. Anna and her team recently published a paper about Kora in VLDB 2023, and it won the industry's best paper award.Kora VLDB paper: https://www.vldb.org/pvldb/vol16/p3822…

1
Cell Based Architecture for Early Stage SaaS 55:12

1+ y ago55:12

55:12

In the last year or two I started hearing a lot about cell-based architectures. Usually in the form of “We had a lot of issues scaling our infrastructure, but then we moved to cell-based architectures” and “I wish I’ve learned about cell based architectures earlier, it would have saved me a lot of pain”. As a result, I’ve wanted to share knowledge …

1
Building a Serverless Streaming Platform 42:23

1+ y ago42:23

42:23

Krishna Raman has many years of experience building platforms for developers. Now he's applying this experience at Delta Streams to build a serverless platform for stream processing in SQL. In our conversation, we discussed the serverless developer experience, some of the secret sauce behind Delta Stream's Flink Operator, how to deliver a bring-you…

1
Never Rewrite! And other advice for SaaS Developers 36:31

2y ago36:31

36:31

You are a founding engineer at a SaaS startup. You built the MVP, and to everyone's great delight - usage is picking up. What's next? In this episode, Jeffrey shares how a messy MVP can gradually evolve into a scalable SaaS product. We discuss the critical design decisions engineers have to make in the early days of building the product: Tabs or sp…

Podcasts Worth a Listen

Kafka Cloud Native Podcasts

Podcasts Worth a Listen

Quick Reference Guide