32 subscribers
Go offline with the Player FM app!
Podcasts Worth a Listen
SPONSORED


Deploy and fine-tune LLM models on Kubernetes using KAITO
Manage episode 433011321 series 3332465
In this episode of the Kubernetes Bytes podcast, Bhavin sits down with Sachi Desai, Product Manager and Paul Yu, Sr. Cloud Advocate at Microsoft to talk about the open source KAITO project. KAITO is the Kubernetes AI Toolchain Operator that enables AKS users to deploy open source LLM models on their Kubernetes clusters. They discuss how KAITO helps with running AI-enabled applications alongside the LLM models, how it helps users bring their own LLM models and run them as containers, and how KAITO helps them fine-tune open source LLMs on their Kubernetes clusters.
Check out our website at https://kubernetesbytes.com/
Cloud Native News:
- https://azure.github.io/AKS/2024/07/30/azure-container-storage-ga
- https://github.blog/news-insights/product-news/introducing-github-models/
Show links:
- Azure/kaito: Kubernetes AI Toolchain Operator - https://github.com/Azure/kaito/tree/main
- https://www.youtube.com/watch?v=3cGmHDjR_3I&list=PLc3Ep462vVYtgN4rP1ThTJd2UlsBc2sou&index=2
- https://aka.ms/cloudnative/learnlive/intelligent-apps-on-aks/episode-2
- Jumpstart AI Workflows With Kubernetes AI Toolchain Operator - The New Stack - https://thenewstack.io/jumpstart-ai-workflows-with-kubernetes-ai-toolchain-operator
- https://paulyu.dev/article/soaring-with-kaito/
- Concepts - Fine-tuning language models for AI and machine learning workflows - Azure Kubernetes Service | Microsoft Learn - https://learn.microsoft.com/en-us/azure/aks/concepts-fine-tune-language-models
- Keep up to date on the most recent announcements by following some of the KAITO engineers on LinkedIn:
- Fei Guo - https://www.linkedin.com/in/fei-guo-a48319a/
- Ishaan Sehgal - https://www.linkedin.com/in/ishaan-sehgal/
Timestamps:
- 00:02:15 Cloud Native News
- 00:05:34 Interview with Sachi and Paul
- 00:42:08 Key takeaways
88 episodes
Manage episode 433011321 series 3332465
In this episode of the Kubernetes Bytes podcast, Bhavin sits down with Sachi Desai, Product Manager and Paul Yu, Sr. Cloud Advocate at Microsoft to talk about the open source KAITO project. KAITO is the Kubernetes AI Toolchain Operator that enables AKS users to deploy open source LLM models on their Kubernetes clusters. They discuss how KAITO helps with running AI-enabled applications alongside the LLM models, how it helps users bring their own LLM models and run them as containers, and how KAITO helps them fine-tune open source LLMs on their Kubernetes clusters.
Check out our website at https://kubernetesbytes.com/
Cloud Native News:
- https://azure.github.io/AKS/2024/07/30/azure-container-storage-ga
- https://github.blog/news-insights/product-news/introducing-github-models/
Show links:
- Azure/kaito: Kubernetes AI Toolchain Operator - https://github.com/Azure/kaito/tree/main
- https://www.youtube.com/watch?v=3cGmHDjR_3I&list=PLc3Ep462vVYtgN4rP1ThTJd2UlsBc2sou&index=2
- https://aka.ms/cloudnative/learnlive/intelligent-apps-on-aks/episode-2
- Jumpstart AI Workflows With Kubernetes AI Toolchain Operator - The New Stack - https://thenewstack.io/jumpstart-ai-workflows-with-kubernetes-ai-toolchain-operator
- https://paulyu.dev/article/soaring-with-kaito/
- Concepts - Fine-tuning language models for AI and machine learning workflows - Azure Kubernetes Service | Microsoft Learn - https://learn.microsoft.com/en-us/azure/aks/concepts-fine-tune-language-models
- Keep up to date on the most recent announcements by following some of the KAITO engineers on LinkedIn:
- Fei Guo - https://www.linkedin.com/in/fei-guo-a48319a/
- Ishaan Sehgal - https://www.linkedin.com/in/ishaan-sehgal/
Timestamps:
- 00:02:15 Cloud Native News
- 00:05:34 Interview with Sachi and Paul
- 00:42:08 Key takeaways
88 episodes
All episodes
×
1 Database as a service with Percona Everest 1:02:44



1 Monolith to Microservices using Kubernetes at Guidewire 1:06:28

1 Inference in Action: Scaling Al Smarter with Inferless 55:17


1 Dagger.io Deep Dive with Co-Founder Sam Alba 1:06:24


1 Building scalable data platforms using Data on EKS 1:02:20

1 Deploy and fine-tune LLM models on Kubernetes using KAITO 44:17

1 The business case for cloud-native and Kubernetes 54:24

1 Building the AI Hyperscaler with Kubernetes 54:56

1 Shifting Minds: Exploring OpenShift's AI Landscape 1:05:07

1 Training Machine Learning (ML) models on Kubernetes 55:29

1 The evolution of service mesh technologies 1:08:00




1 Ops Ops Hooray! Navigating IDPs from an Ops perspective 58:17


1 IDPs Unveiled: Accelerating Deployment on Kubernetes 59:52


1 Running multi-tenant Kubernetes clusters using vCluster 57:58


1 Byte-sized: Exploring the Basics of AI in Plain English 1:00:18

1 Kubecon North America 2023: Highlights, Themes and Key Takeaways 57:41

1 Universal Control Planes for Kubernetes and Beyond 59:36

1 DevOpsDays Boston - Helping developers be more productive in a multi-cloud world 35:02

1 DevOpsDays Boston - Platform Engineering and Internal Developer Platforms 31:12

1 DevOpsDays Boston - Real value of community 42:41

1 How Chick-fil-A adopts GitOps and K3s at the Edge 1:19:16

1 Nodeless Kubernetes - Optimizing costs with just in time compute 1:02:20

1 Solving Multicloud with Seamless Connectivity and AI - with Rob Croteau 58:04


1 Generative AI: The New Frontier in Kubernetes Problem-Solving 1:04:58

1 From Manual to Automatic: Revolutionizing Cloud Native Stack Deployment with Argonaut 1:08:16

1 Accelerating Kubernetes Adoption: Unleashing the Power of GitOps using Kubefirst 1:00:47

1 Continuous Security: Keeping Pace in the DevOps Lifecycle w/ ARMO 1:01:44

1 Unleashing the power of KubeVirt - Running Containers and VMs on Kubernetes 1:14:39

1 Breaking Down the Diamond: A Look at MLB's Kubernetes-Powered Analytics 53:36

1 Kubecon Europe 2023: Highlights and Key Takeaways 46:35

1 Kubernetes Community Corner with Michael O'Leary: Exploring the Intersection of Learning and Collaboration 48:33

Welcome to Player FM!
Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.