Navigating AI for Testing: Insights on Context and Evaluation with Sourcegraph

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

#Tech #Tessl #AI Native Software Development #Ai Developments #Ai Software Development #AI Native #Ai Assistants #Ai Code Completion

50:30

Episode Notes [00:00] The Importance of Questioning [01:21] Introduction to Curated Questions [02:20] Meet Kevin Kelly [03:56] Kevin Kelly's Mentor: Stewart Brand [05:33] The Role of Questions in Intellectual Traditions [06:47] Disequilibrium and Growth [10:21] Embodied Questions and Exploration [11:11] Balancing Exploration and Exploitation [11:50] The Inefficiency of Questioning [15:53] The Abundance Mindset [18:39] The Inevitable and Quality Questions [19:26] Hill Climbing vs. Hill Making [22:28] The Challenge of Innovation [24:13] The Beauty of Engineering and Innovation [24:34] Navigating the Frontier of New Technologies [25:33] The Role of AI in Question Formulation [26:43] Challenges in Advancing AI Capabilities [29:11] The Long Now Foundation and the 10,000 Year Clock [29:56] Transmitting Values Over Time [31:03] Ethics in AI and Self-Driving Cars [33:26] The Art of Questioning [34:04] Photography: Capturing vs. Creating [36:12] The Inefficiency of Exploration [38:36] Daily Practice and Long-Term Success [40:17] The Importance of Quantity for Quality [43:22] Final Thoughts and Encouragement on Questioning [46:24] Summary Takeaways Resources Mentioned Wired Magazine Whole Earth Review WELL Hackers Conference What Technology Wants by Kevin Kelly The Inevitable by Kevin Kelly Cool Tools Project Long Now Foundation Stewart Brand Socratic Method Koan René Descartes Conde Nast Vouge Olivetti Typewriter Trolley Problem Terry Gross Lex Friedman Tim Ferriss KK.org Kevin2Kelly on Instagram Recomendo Newsletter Excellent Advice for Living Beauty Pill Producer Ben Ford Questions Asked When did you first understand the power of questions? Can I do that? Can that be something that you can learn? How did questions function differently between Eastern versus Western intellectual traditions? What role do you think embodied questions those we explore through doing rather than thinking play in developing wisdom? What's on the other side of the hill? What happens if you go to the end? What's the origin of this? How should one think about the exploratory in one's life? Is there anything that you would add to your list of 15 statements that define what makes a quality question? Is there a qualitative difference between the questions humans are asking and the questions our AI systems are beginning to formulate? What do you think would help them get there? Any idea on a forcing function on how we get them [LLMs] to ask the better questions so that they might improve in that direction? What were some novel questions that broke your brain at the time in thinking about this 10,000-year clock or beyond? What's it good for? What would you use it for? What else could you do over the long term for 10,000 years? How do you transmit values over time? How do you evolve values that need to change, and how do you make a difference? How do even know what you don't want to change? What do you want to continue? What's the most essential aspects of our civilization that we don't want to go away? What are the rules? What is the system? How do you pass things along in time and not change the ones you don't wanna change, and make sure you change the ones that are more adaptable so they can adapt? What do you think about questioning itself as an art form? How has being a photographer influenced the way you question reality, visually compared to verbally? Are you a photographer that takes photos or makes photos? What will happen? What will happen next? What are your right now questions that you are wrestling with or working with in your life? Can someone else do what I'm trying to do here? Am I more me in doing my art or more me in doing the writing? Do you have any other thoughts or encouragement about questions that we haven't explored? What makes a good question? How do you ask a good question? What questions do you dwell on to be in purposeful imbalance? What is your practice in embracing the inefficient nature of questions to achieve breakthroughs? What are the new hills you can build and frontiers you can explore? How can you use your curiosity and humanity to pursue questions that trend toward the fringes?…

about a year ago 52:51

MP3•Episode home

In this episode, Simon Maple dives into the world of AI testing with Rishabh Mehrotra from Sourcegraph. Together, they explore the essential aspects of AI in development, focusing on how models need context to create effective tests, the importance of evaluation, and the implications of AI-generated code. Rishabh shares his expertise on when and how AI tests should be conducted, balancing latency and quality, and the critical role of unit tests. They also discuss the evolving landscape of machine learning, the challenges of integrating AI into development workflows, and practical strategies for developers to leverage AI tools like Cody for improved productivity. Whether you're a seasoned developer or just beginning to explore AI in coding, this episode is packed with insights and best practices to elevate your development process.

Join the AI Native Dev Community on discord: https://tessl.co/4ghikjh
Ask us questions: podcast@tessl.io

54 episodes

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

Navigating AI for Testing: Insights on Context and Evaluation with Sourcegraph

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

published about a year ago

MP3•Episode home

Join the AI Native Dev Community on discord: https://tessl.co/4ghikjh
Ask us questions: podcast@tessl.io

54 episodes

#Tech #Tessl #AI Native Software Development #Ai Developments #Ai Software Development #AI Native #Ai Assistants #Ai Code Completion

All episodes

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

1
Vibe Coding SimCity II: Injecting Chaos with Natural Disasters and AI Tools 1:00:42

2 days ago1:00:42

1:00:42

Simon Maple and Joe Holdcroft swap their AI-generated apps and put Cursor and Windsurf to the test by adding natural disasters like earthquakes and lightning strikes. Discover how AI coding tools handle randomness, spreading effects, animation, and bugs—plus the devs’ thoughts on debugging AI-generated code. Subscribe for deep dives into modern dev workflows and AI-powered coding experiments. Watch the episode on YouTube: https://youtu.be/dzXGY8YUx-8 Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

1
Vibe Coding SimCity: Prototyping Tiny Towns with AI Dev Tools 44:00

10 days ago44:00

44:00

In this hands-on episode of AI Native Dev , Simon Maple and Joe Holdcroft go head-to-head in a “vibe coding relay” challenge—building a tiny town simulator from scratch using AI-powered dev tools BASE44 and Bolt.new. With only 10 minutes, vague prompts, and no prep, they put these rapid-prototyping tools to the test. Simon leans into SimCity nostalgia while Joe battles ghost-town bugs and infinite money glitches. Hosted by Macey Baker, the episode dives deep into AI-assisted UI generation, event simulation, and how far these tools can really go with minimal human input. Part 2 continues next week with Cursor and Windsurf as Simon and Joe swap codebases to iterate on each other’s AI-born towns. Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

1
Exploring LLM Observability with Traceloop's Gal Kleinman 40:26

17 days ago40:26

40:26

In this episode, Simon Maple converses with Gal Kleinman, co-founder of Traceloop, to delve into the realm of LLM observability. Kleinman shares his expertise on constructing evaluation suites and overcoming the challenges associated with LLM applications. Learn about innovative solutions like Open LLMetry and gain insights into best practices for maintaining robust observability systems. Subscribe to the AI Native Developer podcast for more insights on AI and development. Watch the episode on YouTube: https://youtu.be/TD-wjNStTVM Check out Traceloop: https://www.traceloop.com/ Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

1
From Builder to Orchestrator—Confronting the Software Engineer’s Identity Crisis 54:37

24 days ago54:37

54:37

In this episode, host Simon Maple engages in a compelling discussion with Annie Vella, a skilled software engineer and manager, about the integration of AI into software development. Annie shares her insights on how AI tools are reshaping coding practices, the challenges engineers face in transitioning to management roles, and the skills required to thrive in this evolving landscape. This episode is crucial for anyone looking to understand the implications of AI in their work. Subscribe to the AI Native Developer podcast for more insights on AI and development. Watch the episode on YouTube: https://youtu.be/GI2ywMl2gQk Read Annie's blog : https://annievella.com/posts/the-software-engineering-identity-crisis/ Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

1
Is Code Dead & The $1B Solo Startup Myth - 5 AI Realities with Tessl's Guy Podjarny 29:03

4 weeks ago29:03

29:03

In this episode, Guy Podjarny sits down with Tom Hulme to discuss the transformative shift from a code-centric to a spec-centric approach in software development. They explore the advantages of cloud-native practices, the evolving tool ecosystem, and the future role of AI and LLMs. Gain expert insights into the trends that are shaping the future of the industry. Subscribe to the AI Native Developer podcast for more insights on AI and development! Watch the episode on YouTube: https://youtu.be/reU8W2aH30Y Read the GV Event Blog: gv.com/news/five-ai-realities-with-tessl-guy-podjarny Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

1
AI's Transformative Impact on Development with Alex Komorske 31:25

5 weeks ago31:25

31:25

In this episode, Alex Komoroske delves into the transformative role of AI in the tech industry. As an expert in web platform development, Alex discusses AI's potential to redefine technology and creativity. Gain insights into how AI serves as a disruptive innovation and explore its implications for developers. Subscribe to the AI Native Developer podcast for more insights on AI and development! Watch the episode on YouTube: https://youtu.be/utzvGealvrE Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

1
Datadog CEO Olivier Pomel on AI Security, Trust, and the Future of Observability 59:59

6 weeks ago59:59

59:59

Join Guy Podjarny as he hosts Olivier Pomel, CEO of Datadog, in a compelling discussion on the evolution of observability and AI's role in modern technology. Olivier offers his expertise on AI-powered applications, the security challenges they face, and the future of AI interactions. This episode provides crucial insights for tech professionals and developers seeking to understand AI's impact on cloud computing and security. Subscribe to the AI Native Developer podcast for more insights on AI and development! Watch the episode on YouTube: https://youtu.be/-5N53Xq5DX4 Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

1
Intent-Driven Development: Insights from Patrick Debois 45:11

7 weeks ago45:11

45:11

In this episode of the Tessl podcast, hosts Patrick Debois and Simon Maple explore the transformative landscape of software development practices. Join them as they discuss emerging patterns that are reshaping the industry. Patrick, a DevOps pioneer, shares valuable insights on aligning development work with business intents, while Simon brings his expertise in Java, cloud computing, and AI integration in software to the table. This engaging discussion is a must-listen for anyone interested in the future of development. What You'll Learn: Identify emerging patterns in software development that enhance decision-making. Shift your focus from implementation to understanding business intent for more meaningful outcomes. Utilize spec centricity to create clearer specifications that bridge technical and non-technical teams. Leverage the right tools to facilitate innovative practices and improve productivity. Recognize the significance of clean data in enhancing software requirements and decision-making. This episode is perfect for developers, project managers, and anyone in the tech field looking to stay ahead of the curve in evolving development practices. It's an invaluable resource for those seeking to understand how to better align their work with business goals and improve collaboration across teams. 💡 What are your thoughts on the future of development practices? Join the conversation and share your insights in the comments! 👇 Watch the episode on YouTube: https://youtu.be/kMRHuc36AK4 Blog post: https://ainativedev.io/news/the-4-patterns-of-ai-native-dev-overview Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

1
How Attackers Trick AI: Lessons from Gandalf’s Creator 54:35

8 weeks ago54:35

54:35

🔒 How Secure is AI? Gandalf’s Creator Exposes the Risks 🔥 AI security is under attack, and hackers are finding new ways to manipulate AI systems. In this episode, Guy Podjarny sits down with Mateo Rojas-Carulla, co-founder of Lakera and creator of Gandalf, to break down the biggest threats facing AI today—from prompt injections and jailbreaks to data poisoning and agent manipulation. What You’ll Learn: - How attackers exploit AI vulnerabilities in real-world applications - Why AI models struggle to separate instructions from external data - How Gandalf’s 60M+ attack attempts revealed shocking insights - What the Dynamic Security Utility Framework (DSEC) means for AI safety - Why red teaming is critical for preventing AI disasters Whether you’re a developer, security expert, or just curious about AI risks, this episode is packed with must-know insights on keeping AI safe in an evolving landscape. 💡 Can AI truly be secured? Or will attackers always find a way? Drop your thoughts in the comments! 👇 Watch the episode on YouTube: https://youtu.be/RKCvlJT_r4s Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

1
AI Gone Rogue? LLM Werewolf Showdown 54:43

9 weeks ago54:43

54:43

What happens when AI learns to lie? In this electrifying episode of AI Native Dev , Simon Maple is joined by the brilliant Macey Baker, Community Engineer at Tessl, to unravel the wild, unpredictable, and sometimes downright hilarious world of Large Language Models (LLMs) in social deception games. From the psychological mind games of Werewolf to the cutthroat negotiations of Split or Steal , Macey spills the details on how AI models like OpenAI’s GPT-4o, Anthropic’s Sonnet, Llama, and DeepSeek R1 navigate deception, trust, and ethics when the stakes are high. Can an AI manipulate, bluff, or betray? Which model is the sneakiest, and which one folds under pressure? Prepare for shocking twists, unexpected strategies, and an eye-opening look at the ethical implications of AI in interactive gameplay. This is one episode you won’t want to miss—especially if you think you can outwit an AI! Watch the episode on YouTube: https://youtu.be/9hVeVYlqyBc Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

1
Monthly Roundup: AI Model Wars, GPT-4.5 vs. Sonnet 3.7, and the Future of AI Dev Tools 42:09

10 weeks ago42:09

42:09

In this episode, Guy and Simon dive deep into February’s wave of AI model releases, focusing heavily on the showdown between OpenAI’s GPT-4.5 and Anthropic’s Sonnet 3.7. They discuss GPT-4.5's move toward higher emotional intelligence (EQ), how it compares with Claude, and whether it’s actually meeting developers' expectations—or falling short. Anthropic’s introduction of dynamic reasoning with Sonnet 3.7 gets special attention as Guy and Simon explore what this new feature means practically for developers, referencing Bolt.new’s recent implementation as an example. They also weigh the strategic choices these AI labs face—when to build tools themselves and when to act purely as platforms—highlighting the delicate balance companies like Anthropic and ElevenLabs must strike. The conversation also covers practical developer workflows, featuring insights from recent interviews with Farhath, who leverages multiple AI dev tools simultaneously, and Mati from ElevenLabs, who shares how his company navigates scaling and research while remaining nimble. Finally, Guy and Simon announce AI Native Dev Con: Spring 2025, and hint at exciting upcoming episodes, rounding out a packed discussion that goes straight to the heart of what matters for AI-native developers today. Watch the episode on YouTube: https://youtu.be/kjAs_isf-Vs Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

1
The Future of Audio AI: Insights from Mati Staniszewski of ElevenLabs 1:02:52

11 weeks ago1:02:52

1:02:52

In this episode of AI Native Dev, hosts Guy Podjarny sits down with Mati Staniszewski, the visionary CEO and co-founder of ElevenLabs, a leader in AI audio technology. Mati shares the origin story of ElevenLabs, detailing how a frustration with subpar dubbing in Polish movies sparked a mission to revolutionize audio processing. The conversation delves into the technical challenges and breakthroughs that have positioned ElevenLabs at the forefront of AI-powered audio experiences. Mati also discusses the company's unique organizational culture and its commitment to pushing the boundaries of audio AI. Whether you're a developer, entrepreneur, or AI enthusiast, this episode offers valuable insights into the future of audio technology and the role ElevenLabs is playing in shaping it. Watch the episode on YouTube: https://youtu.be/ImknDhWdDkQ Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

1
Building the Ultimate AI-Powered Development Environment with Farhath Razzaque 36:16

12 weeks ago36:16

36:16

In this episode of the AI Native Dev podcast, Simon Maple sits down with Farhath Razzaque, a freelance software AI engineer who has been extensively using AI tools in his development workflow. Farhath has built a reputation as a trusted voice in the AI and software development communities, thanks to his rich background and experience. He has worked on multiple projects that integrate cutting-edge AI technologies, helping companies streamline their processes and innovate effectively. His passion for AI and development is evident through his active participation in various tech forums and his contributions to open-source projects. Farhath's insights into his ideal development environment, the tools he frequently uses, and his thoughts on the future of AI in software development offer invaluable guidance for developers looking to integrate AI into their processes. Watch the episode on YouTube: https://youtu.be/7eE8rzzcrZM Join the AI Native Dev Community on discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

1
DeepSeek R1: Ask Me Anything - Open Weights, MoE innovations, Model Distillation and more! 33:18

13 weeks ago33:18

33:18

Join Simon Maple as he hosts a compelling discussion at the State of Open Con, featuring Amanda Brock, CEO of OpenUK, Guy Podjarny, Founder and CEO of Tessl, and Richard Sikang Bian, Head of Open Source from Ant Group. This episode delves into the impact of DeepSeek R1, a groundbreaking AI model that has taken the market by storm due to its cost-effective and innovative approach. The panel explores the broader implications for open source AI, discussing legal, ethical, and collaborative aspects. Amanda Brock provides insights into the open source legal frameworks, while Guy Podjarny shares his expertise on transparency and innovation. Richard Sikang Bian offers a unique perspective on company culture and AI development strategies. Tune in to understand how DeepSeek R1 is setting new standards in AI and what the future holds for open source technology. Join the AI Native Dev Community on discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

1
Live Monthly: News on DeepSeek, Stargate, StackBlitz (Bolt.new) funding, prompting, and more 51:46

14 weeks ago51:46