32 subscribers
Go offline with the Player FM app!
Podcasts Worth a Listen
SPONSORED


Generative AI on Kubernetes
Manage episode 406140511 series 3332465
In this episode of the Kubernetes Bytes podcast, Ryan and Bhavin sit down with Janakiram MSV - an advisor, analyst and architect to talk about how users can run Generative AI models on Kubernetes. The discussion revolves around Jani's home lab and his experimentation with different LLM models and how to get them running on NVIDIA GPUs. Jani has spent the past year becoming a subject matter expert in GenAI, and this discussion highlights all the different challenges he faced and what lessons he learnt from them.
Check out our website at https://kubernetesbytes.com/
Episode Sponsor: Elotl
- https://elotl.co/luna
- https://www.elotl.co/luna-free-trial
Timestamps:
- 02:02 Cloud Native News
- 15:31 Interview with Jani
- 01:11:00 Key takeaways
Cloud Native News:
- https://www.techerati.com/press-release/octopus-deploy-acquires-codefresh-to-boost-kubernetes-and-cloud-native-delivery/
- https://www.civo.com/blog/kubefirst-joins-civo
- https://cast.ai/kubernetes-cost-benchmark
- https://www.techradar.com/pro/vmware-customers-are-jumping-ship-as-broadcom-sales-continue-heres-where-theyre-moving-to
- https://cloudonair.withgoogle.com/events/techbyte-making-ai-ml-scalable-cost-effective-gke
- https://dok.community/dok-events/dok-day-kubecon-paris/
- https://training.linuxfoundation.org/certification/certified-argo-project-associate-capa
Show Links:
- https://www.youtube.com/janakirammsv
- https://www.linkedin.com/in/janakiramm/
- - NVIDIA Container Toolkit - https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/index.html
- NVIDIA Device Plugin - https://github.com/NVIDIA/k8s-device-plugin
- NVIDIA Feature Discovery - https://github.com/NVIDIA/gpu-feature-discovery
- Hugging Face Text Gen Inference - https://huggingface.co/docs/text-generation-inference/index
- Hugging Face Text Embeddings Inference - https://huggingface.co/docs/text-embeddings-inference/index
- ChromaDB - https://www.trychroma.com/
88 episodes
Manage episode 406140511 series 3332465
In this episode of the Kubernetes Bytes podcast, Ryan and Bhavin sit down with Janakiram MSV - an advisor, analyst and architect to talk about how users can run Generative AI models on Kubernetes. The discussion revolves around Jani's home lab and his experimentation with different LLM models and how to get them running on NVIDIA GPUs. Jani has spent the past year becoming a subject matter expert in GenAI, and this discussion highlights all the different challenges he faced and what lessons he learnt from them.
Check out our website at https://kubernetesbytes.com/
Episode Sponsor: Elotl
- https://elotl.co/luna
- https://www.elotl.co/luna-free-trial
Timestamps:
- 02:02 Cloud Native News
- 15:31 Interview with Jani
- 01:11:00 Key takeaways
Cloud Native News:
- https://www.techerati.com/press-release/octopus-deploy-acquires-codefresh-to-boost-kubernetes-and-cloud-native-delivery/
- https://www.civo.com/blog/kubefirst-joins-civo
- https://cast.ai/kubernetes-cost-benchmark
- https://www.techradar.com/pro/vmware-customers-are-jumping-ship-as-broadcom-sales-continue-heres-where-theyre-moving-to
- https://cloudonair.withgoogle.com/events/techbyte-making-ai-ml-scalable-cost-effective-gke
- https://dok.community/dok-events/dok-day-kubecon-paris/
- https://training.linuxfoundation.org/certification/certified-argo-project-associate-capa
Show Links:
- https://www.youtube.com/janakirammsv
- https://www.linkedin.com/in/janakiramm/
- - NVIDIA Container Toolkit - https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/index.html
- NVIDIA Device Plugin - https://github.com/NVIDIA/k8s-device-plugin
- NVIDIA Feature Discovery - https://github.com/NVIDIA/gpu-feature-discovery
- Hugging Face Text Gen Inference - https://huggingface.co/docs/text-generation-inference/index
- Hugging Face Text Embeddings Inference - https://huggingface.co/docs/text-embeddings-inference/index
- ChromaDB - https://www.trychroma.com/
88 episodes
All episodes
×
1 Database as a service with Percona Everest 1:02:44



1 Monolith to Microservices using Kubernetes at Guidewire 1:06:28

1 Inference in Action: Scaling Al Smarter with Inferless 55:17


1 Dagger.io Deep Dive with Co-Founder Sam Alba 1:06:24


1 Building scalable data platforms using Data on EKS 1:02:20

1 Deploy and fine-tune LLM models on Kubernetes using KAITO 44:17

1 The business case for cloud-native and Kubernetes 54:24

1 Building the AI Hyperscaler with Kubernetes 54:56

1 Shifting Minds: Exploring OpenShift's AI Landscape 1:05:07

1 Training Machine Learning (ML) models on Kubernetes 55:29

1 The evolution of service mesh technologies 1:08:00




1 Ops Ops Hooray! Navigating IDPs from an Ops perspective 58:17


1 IDPs Unveiled: Accelerating Deployment on Kubernetes 59:52


1 Running multi-tenant Kubernetes clusters using vCluster 57:58


1 Byte-sized: Exploring the Basics of AI in Plain English 1:00:18

1 Kubecon North America 2023: Highlights, Themes and Key Takeaways 57:41

1 Universal Control Planes for Kubernetes and Beyond 59:36

1 DevOpsDays Boston - Helping developers be more productive in a multi-cloud world 35:02

1 DevOpsDays Boston - Platform Engineering and Internal Developer Platforms 31:12

1 DevOpsDays Boston - Real value of community 42:41

1 How Chick-fil-A adopts GitOps and K3s at the Edge 1:19:16

1 Nodeless Kubernetes - Optimizing costs with just in time compute 1:02:20

1 Solving Multicloud with Seamless Connectivity and AI - with Rob Croteau 58:04


1 Generative AI: The New Frontier in Kubernetes Problem-Solving 1:04:58

1 From Manual to Automatic: Revolutionizing Cloud Native Stack Deployment with Argonaut 1:08:16

1 Accelerating Kubernetes Adoption: Unleashing the Power of GitOps using Kubefirst 1:00:47

1 Continuous Security: Keeping Pace in the DevOps Lifecycle w/ ARMO 1:01:44

1 Unleashing the power of KubeVirt - Running Containers and VMs on Kubernetes 1:14:39

1 Breaking Down the Diamond: A Look at MLB's Kubernetes-Powered Analytics 53:36

1 Kubecon Europe 2023: Highlights and Key Takeaways 46:35

1 Kubernetes Community Corner with Michael O'Leary: Exploring the Intersection of Learning and Collaboration 48:33


1 What is Platform Engineering with Luca Galante 1:07:11

1 Cloud Native WebAssembly with Nigel Poulton 1:06:07

1 Kubernetes Security Posture Management with Mondoo 53:22

1 Unified application deployment platform for Kubernetes with Plural.sh 54:38

1 GitOps, DevSecOps & Kubernetes w/ GitLab 1:00:15

1 Kubernetes Alternatives - when NOT to use Kubernetes! 57:02

1 Understanding the cost of Kubernetes w/ Kubecost 54:48

1 Part 2 - Live from Kubecon North America 2022 - Interviews with Redis, Teleport, Instruqt, and Pulumi 41:28

1 Part 1 - Live from Kubecon North America 2022 - Interviews with Percona, EDB, Dell, and Akamai 41:29

1 Powering Decentralized Cloud with Kubernetes 58:23

1 Kubernetes Security 101 - 4C's of Cloud Native Security 59:44

1 Community, Opensource and Kubernetes with Brendan Burns and Ganesh Ashokavardhanan 54:35


1 MongoDB Kubernetes Operators with Joel Lord & Cedric Clyburn 50:20








1 Kubernetes Observability using Promscale and tobs 51:27

1 Kubernetes SIG Storage - Intro and Deep Dive with Xing Yang 42:56

1 Intro to distributed databases on Kubernetes 37:03


1 What Kubernetes objects use persistent storage? 40:31

1 Let's talk Data Protection & Disaster Recovery with Michael Cade 45:51



1 Databases on Kubernetes, Why Database-as-a-Service matters 47:21

1 Data management on various Kubernetes orchestration systems with Andy Gower 36:32

1 Cloud Native Storage and Traditional Storage: What's the difference? 42:45


Welcome to Player FM!
Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.