Artwork

Content provided by Peter Yang. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Peter Yang or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://ppacc.player.fm/legal.
Player FM - Podcast App
Go offline with the Player FM app!

The AI Skill That Will Define Your PM Career in 2025 | Aman Khan (Arize)

46:26
 
Share
 

Manage episode 460706171 series 3621237
Content provided by Peter Yang. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Peter Yang or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://ppacc.player.fm/legal.

My guest today is Aman Khan.

Aman is the Director of Product at Arize AI. Last year, we both heard the Chief Product Officers of OpenAI and Anthropic share that AI evaluations will be the most important skill for PMs in 2025. Aman gave me a crash course on how to build this critical skill in our interview.

Timestamps:

  • (00:00) Evals force you to get into your user's shoes

  • (03:42) 5 skills to build right now to become a great AI PM

  • (07:43) How curiosity leads to better AI products

  • (10:20) You can build AI without the job title

  • (17:00) A deep dive into why AI evals are so important

  • (24:53) Example of running evals for an AI customer support agent

  • (31:32) When humans need to be in the loop for evals

  • (35:46) How to get better at writing evals right now

  • (39:39) My personal prompt and eval for transcripts

  • (44:23) Catching the AI wave is alot like surfing

    Get the takeaways: https://creatoreconomy.so/p/the-ai-skill-that-will-define-your-pm-career-aman-khan

    Where to find Aman:

    LinkedIn: https://www.linkedin.com/in/amanberkeley/

    Arize: https://arize.com/

    📌 Subscribe to this channel – more interviews coming soon!

      continue reading

    65 episodes

    Artwork
    iconShare
     
    Manage episode 460706171 series 3621237
    Content provided by Peter Yang. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Peter Yang or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://ppacc.player.fm/legal.

    My guest today is Aman Khan.

    Aman is the Director of Product at Arize AI. Last year, we both heard the Chief Product Officers of OpenAI and Anthropic share that AI evaluations will be the most important skill for PMs in 2025. Aman gave me a crash course on how to build this critical skill in our interview.

    Timestamps:

  • (00:00) Evals force you to get into your user's shoes

  • (03:42) 5 skills to build right now to become a great AI PM

  • (07:43) How curiosity leads to better AI products

  • (10:20) You can build AI without the job title

  • (17:00) A deep dive into why AI evals are so important

  • (24:53) Example of running evals for an AI customer support agent

  • (31:32) When humans need to be in the loop for evals

  • (35:46) How to get better at writing evals right now

  • (39:39) My personal prompt and eval for transcripts

  • (44:23) Catching the AI wave is alot like surfing

    Get the takeaways: https://creatoreconomy.so/p/the-ai-skill-that-will-define-your-pm-career-aman-khan

    Where to find Aman:

    LinkedIn: https://www.linkedin.com/in/amanberkeley/

    Arize: https://arize.com/

    📌 Subscribe to this channel – more interviews coming soon!

      continue reading

    65 episodes

    All episodes

    ×
     
    Loading …

    Welcome to Player FM!

    Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.

     

    Quick Reference Guide

    Copyright 2025 | Privacy Policy | Terms of Service | | Copyright
    Listen to this show while you explore
    Play