Inference, Guardrails, and Observability for LLMs with Jonathan Cohen

AI Explained

6:24

Grumpy helps a reader choose the best hydrangea for their hometown. Plus, Grumpy’s gripe of the week. You can find us online at southernliving.com/askgrumpy Ask Grumpy Credits: Steve Bender aka The Grumpy Gardener - Host Nellah McGough - Co-Host Krissy Tiglias - GM, Southern Living Lottie Leymarie - Executive Producer Michael Onufrak - Audio Engineer/Producer Isaac Nunn - Recording Tech Learn more about your ad choices. Visit podcastchoices.com/adchoices…

about a year ago 53:10

MP3•Episode home

In this episode of AI Explained, we are joined by Jonathan Cohen, VP of Applied Research at NVIDIA.

We will explore the intricacies of NVIDIA's NeMo platform and its components like NeMo Guardrails and NIMS. Jonathan explains how these tools help in deploying and managing AI models with a focus on observability, security, and efficiency. They also explore topics such as the evolving role of AI agents, the importance of guardrails in maintaining responsible AI, and real-world examples of successful AI deployments in enterprises like Amdocs. Listeners will gain insights into NVIDIA's AI strategy and the practical aspects of deploying large language models in various industries.

14 episodes

#Tech #AI Explained

Inference, Guardrails, and Observability for LLMs with Jonathan Cohen

AI Explained

12 subscribers

published about a year ago

MP3•Episode home

In this episode of AI Explained, we are joined by Jonathan Cohen, VP of Applied Research at NVIDIA.

14 episodes

#Tech #AI Explained

All episodes

1
AI Observability and Security for Agentic Workflows with Karthik Bharathy 44:01

8 weeks ago44:01

44:01

In this episode of AI Explained, we are joined by Karthik Bharathy, General Manager, AI Ops & Governance for Amazon SageMaker AI at AWS. He discusses the critical aspects of AI security and observability for agentic workflows. He covers the evolution of AI Ops, end-to-end observability, human oversight, the current state of AI in enterprises, and the ways agentic AI systems are transforming business operations. He also dives into the challenges of implementing AI security, evaluating AI decisions, and ensuring transparency and compliance.…

1
GenAI Use Cases and Challenges in Healthcare with Dr. Girish Nadkarni 39:47

11 weeks ago39:47

39:47

In this episode of AI Explained, Dr. Girish Nadkarni from Icahn School of Medicine at Mount Sinai. He discusses the implementation and impact of AI, specifically generative AI, in healthcare. He covers topics such as clinical implementation, risk prediction, the interplay between predictive and generative AI, the importance of governance and ethical considerations in AI deployment, and the future of personalized medicine.…

1
GRC in Generative AI with Navrina Singh 56:43

23 weeks ago56:43

56:43

In this episode of AI Explained, we are joined by Navrina Singh, Founder and CEO at Credo AI. We will discuss the comprehensive need for AI governance beyond regulated industries, the core principles of responsible AI, and the importance of AI governance in accelerating business innovation. The conversation also covers the challenges companies face when implementing responsible AI practices and dives into the latest regulations like the EU AI Act and state-specific laws in the U.S.…

1
Inference, Guardrails, and Observability for LLMs with Jonathan Cohen 53:10

27 weeks ago53:10

53:10

In this episode of AI Explained, we are joined by Jonathan Cohen, VP of Applied Research at NVIDIA. We will explore the intricacies of NVIDIA's NeMo platform and its components like NeMo Guardrails and NIMS. Jonathan explains how these tools help in deploying and managing AI models with a focus on observability, security, and efficiency. They also explore topics such as the evolving role of AI agents, the importance of guardrails in maintaining responsible AI, and real-world examples of successful AI deployments in enterprises like Amdocs. Listeners will gain insights into NVIDIA's AI strategy and the practical aspects of deploying large language models in various industries.…

1
What the EU AI Act Really Means with Kevin Schawinski 45:47

29 weeks ago45:47

45:47

On this episode, we’re joined by Kevin Schawinski, CEO and Co-Founder at Modulos AG The EU AI Act was passed to redefine the landscape for AI development and deployment in Europe. But what does it really mean for enterprises, AI innovators, and industry leaders? Schawinski will share actionable insights to help organizations stay ahead of the EU AI Act, and discuss risk implications to meeting transparency requirements, while advancing responsible AI practices.…

1
Productionizing GenAI at Scale with Robert Nishihara 48:29

42 weeks ago48:29

48:29

In this episode, we’re joined by Robert Nishihara, Co-founder and CEO at Anyscale. Enterprises are harnessing the full potential of GenAI across various facets of their operations for enhancing productivity, driving innovation, and gaining a competitive edge. However, scaling production GenAI deployments can be challenging due to the need for evolving AI infrastructure, approaches, and processes that can support advanced GenAI use cases. Nishihara will discuss reliability challenges, building the right AI infrastructure, and implementing the latest practices in productionizing GenAI at scale.…

1
Metrics to Detect Hallucinations with Pradeep Javangula 58:39

1 year ago58:39

58:39

In this episode, we’re joined by Pradeep Javangula, Chief AI Officer at RagaAI Deploying LLM applications for real-world use cases requires a comprehensive workflow to ensure LLM applications generate high-quality and accurate content. Testing, fixing issues, and measuring impact are critical steps of the workflow to help LLM applications deliver value. Pradeep Javangula, Chief AI Officer at RagaAI will discuss strategies and practical approaches organizations can follow to maintain high performing, correct, and safe LLM applications.…

1
AI Safety and Alignment with Amal Iyer 57:17

1 year ago57:17

57:17

In this episode, we’re joined by Amal Iyer, Sr. Staff AI Scientist at Fiddler AI. Large-scale AI models trained on internet-scale datasets have ushered in a new era of technological capabilities, some of which now match or even exceed human ability. However, this progress emphasizes the importance of aligning AI with human values to ensure its safe and beneficial societal integration. In this talk, we will provide an overview of the alignment problem and highlight promising areas of research spanning scalable oversight, robustness and interpretability.…

1
Managing the Risks of Generative AI with Kathy Baxter 57:21

1 year ago57:21

57:21

On this episode, we’re joined by Kathy Baxter, Principal Architect of Responsible AI & Tech at Salesforce. Generative AI has become widely popular with organizations finding ways to drive innovation and business growth. The adoption of generative AI, however, remains low due to ethical implications and unintended consequences that negatively impact the organization and its consumers. Baxter will discuss ethical AI practices organizations can follow to minimize potential harms and maximize the social benefits of AI.…

1
Legal Frontiers of AI with Patrick Hall 58:49

1 year ago58:49

58:49

On this episode, we’re joined by Patrick Hall, Co-Founder of BNH.AI. We will delve into critical aspects of AI, such as model risk management, generating adverse action notices, addressing algorithmic discrimination, ensuring data privacy, fortifying ML security, and implementing advanced model governance and explainability.…

1
Building Generative AI Applications for Production with Chaoyu Yang 59:07

2 years ago59:07

59:07

On this episode, we’re joined by Chaoyu Yang, Founder and CEO at BentoML. AI-forward enterprises across industries are building generative AI applications to transform their businesses. While AI teams need to consider several factors ranging from ethical and social considerations to overall AI strategy, technical challenges remain to deploy these applications into production. Yang, will explore key aspects of generative AI application development and deployment.…

1
Graph Neural Networks and Generative AI with Jure Leskovec 52:05

2 years ago52:05

52:05

On this episode, we’re joined by Jure Leskovec, Stanford professor and co-founder at Kumo.ai. Graph neural networks (GNNs) are gaining popularity in the AI community, helping ML teams build advanced AI applications that provide deep insights to tackle real-world problems. Stanford professor and co-founder at Kumo.AI, Jure Leskovec, whose work is at the intersection of graph neural networks, knowledge graphs, and generative AI, will explore how organizations can incorporate GNNs in their generative AI initiatives.…

1
Machine Learning for High Risk Applications with Parul Pandey 54:40

2 years ago54:40

54:40

On this episode, we’re joined by Parul Pandey, Principal Data Scientist at H2O.ai and co-author of Machine Learning for High-Risk Applications. Although AI is being widely adopted, it poses several adversarial risks that can be harmful to organizations and users. Listen to this episode to learn how data scientists and ML practitioners can improve AI outcomes with proper model risk management techniques.…

1
AI Safety in Generative AI with Peter Norvig 39:58

2 years ago39:58