Join the AgileData Podcast where we discuss how data and analytics teams can apply agile, product and data patterns to change the way they work.
…
continue reading
Real Stories. Real Strategy. No Fluff. Data With Direction is the show where Chris Gambill, data engineering consultant and educator, cuts through the noise to help you build resilient data systems, tackle real-world challenges, and future-proof your analytics career. Each episode delivers practical guidance, industry insights, and the war stories you won’t find in textbooks, whether you’re tuning in live or catching up on the go.
…
continue reading

1
Activity Event Tile, AgileData Engineering Pattern #6 - Episode #72
11:52
11:52
Play later
Play later
Lists
Like
Liked
11:52Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern for the AgileData Activity Event Tile. The Activity Event Tile pattern tracks a sequence of events (e.g., subscription signups, subscription payments) for a core business concept by storing minimal event data and using a hydration layer to automaticall…
…
continue reading

1
Negotiate Like a Pro or Stay Underpaid in 2025
9:50
9:50
Play later
Play later
Lists
Like
Liked
9:50Send us a text Negotiate Like a Pro or Stay Underpaid in 2025 Unlock the secrets to salary negotiation as a data engineer in 2025! In this video, you'll learn exactly what to say, when to say it, and how to anchor your value so you never leave money on the table again. Whether you're entry-level or a seasoned pro, these strategies will help you max…
…
continue reading

1
Data Engineering Patterns for the AgileData Event Tile, AgileData Engineering Pattern #5 - Episode #71
16:22
16:22
Play later
Play later
Lists
Like
Liked
16:22Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern for the AgileData Event Tile. The Event Tile Data Modeling pattern captures business processes and transactions within AgileData's three-layered data architecture, modeling "who does what" at a specific point in time. It achieves this by storing only a…
…
continue reading

1
Real-World Lessons with Kyle Stratis (Author, AI Agents with MCP)
1:04:07
1:04:07
Play later
Play later
Lists
Like
Liked
1:04:07Send us a text What’s it really like building data solutions from the ground up in a startup? Join Chris Gambill (Founder, Gambill Data) for a candid live conversation with Kyle Stratis, founder of Stratis Data Labs, O’Reilly author, and hands-on data leader in the trenches of startup growth. Kyle’s not only shaping the future of AI automation with…
…
continue reading

1
Automated Load Patterns based on Source Data Profiles, AgileData Engineering Pattern #4 - Episode #70
11:42
11:42
Play later
Play later
Lists
Like
Liked
11:42Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern of Automated Load Patterns based on Source Data Profiles. The Automated Load Patterns based on Source Data Profiles pattern automatically profiles incoming data to determine its optimal loading method. It classifies data as either event data (immutable…
…
continue reading

1
10 AI Skills You Didn’t Know You Already Had, With AI Architect Ramona Truta!
1:37:14
1:37:14
Play later
Play later
Lists
Like
Liked
1:37:14Send us a text Think AI is only for machine learning experts and prompt engineering pros? Think again. In this episode of Data With Direction, Chris Gambill joined by AI Architect Ramona Truta breaks down 10 essential AI skills that you probably already have—and shows you how to level them up to stay relevant in the rapidly shifting world of data a…
…
continue reading

1
Career Mistakes Costing Data Engineers $50K Per Year
4:24
4:24
Play later
Play later
Lists
Like
Liked
4:24Send us a text Are you a data engineer stuck in a mid-level role, wondering why your career isn’t advancing as fast as your skills? In this video, Chris Gambill breaks down the three invisible mistakes that hold back 80% of data engineers from landing senior positions and big salary jumps. Learn how to avoid the technology trap, visibility gap, and…
…
continue reading

1
Trust Rules to validate your data, AgileData Engineering Pattern #3 - Episode #69
12:01
12:01
Play later
Play later
Lists
Like
Liked
12:01Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern of Trust Rules to validate data. The Trust Rules pattern provides automated data validation to ensure all incoming data is fit for purpose and trustworthy. It bakes in essential checks such as unique business keys and business effective dates, which ru…
…
continue reading

1
Union two or more tables together automatically, AgileData Engineering Pattern #2 - Episode #68
10:32
10:32
Play later
Play later
Lists
Like
Liked
10:32Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern of Unioning two or more tables together automatically. The Automated Table Unioning pattern automatically combines two or more tables by intelligently looking up column names and data types to generate safe SQL under the covers. It supports disparate d…
…
continue reading

1
Orchestrating Dynamic Data Flows, AgileData Engineering Pattern #1 - Episode #67
22:19
22:19
Play later
Play later
Lists
Like
Liked
22:19Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern of Orchestrating Dynamic Data Flows. The Dynamic Data Flow Orchestration pattern dynamically generates and self-heals data flow manifests (DAGs) at runtime from a central context database, enabling adaptive, cost-effective, and low-latency data process…
…
continue reading

1
Data Architecture Principles That Skyrocket Your Career!
1:19:36
1:19:36
Play later
Play later
Lists
Like
Liked
1:19:36Send us a text Join Ramona and me on Data With Direction as we discuss architecture patterns that set you on a career trajectory beyond your greatest imagination. You want to earn more? Show you have the skills to move the needle! I remember a time when I created manual triggers that I had to babysit and watch for an hour, only to walk away at a cr…
…
continue reading

1
$180k Data Engineers Don’t Make These Python Mistakes in Databricks!
6:09
6:09
Play later
Play later
Lists
Like
Liked
6:09Send us a text Are you making these common mistakes in Databricks? In this video, we break down the top 10 pitfalls that data engineers, both new and experienced, fall into when working with Databricks and Spark. From treating Databricks like a simple Jupyter Notebook to neglecting the power of Delta Lake and proper version control, we cover the es…
…
continue reading

1
From Teacher to Data Master | Learn the Secrets of Data-Guided Success!
1:05:02
1:05:02
Play later
Play later
Lists
Like
Liked
1:05:02Send us a text Mid-Career Changes to data with CJ Watkins. We will be talking about work/study/home life balance, what it looks like moving from a career in education to a career in data, as well as CJ's unique take on data-guided decisions vs data-driven decisions. Links from CJ from our video: Measure what Matters: https://amzn.to/40pNxLZ Stats B…
…
continue reading

1
You Are NOT A Data Engineer If You Are Doing This!
1:16:07
1:16:07
Play later
Play later
Lists
Like
Liked
1:16:07Send us a text We have Data Analysts out there and Analytics Engineers claiming to be doing ETL in Power BI and Tableau. Find out why it isn't true, and why it will stall your career if you continue to do it! Support the show Chris Gambill is a data engineering consultant and educator with 25+ years of experience helping organizations modernize the…
…
continue reading

1
From Navy Corpsman to 6 Figure Tech Career!
1:33:06
1:33:06
Play later
Play later
Lists
Like
Liked
1:33:06Send us a text Join us for a special live interview with a Navy Corpsman-turned-software engineer! We'll dive into his journey from military life to the world of tech. We'll explore how his time in the Navy prepared him for a career in software engineering, the challenges he faced, and the resources that helped him succeed! Whether you're a veteran…
…
continue reading

1
Leveraging Your Passion To Break Into Data in 2025!
1:11:48
1:11:48
Play later
Play later
Lists
Like
Liked
1:11:48Send us a text What happens when you mix a love for flying with a knack for data? In this episode, I’m joined by Ridah, one of my mentees and the founder of Aviadata, a company using analytics to revolutionize how flight schools operate. We’ll dive into Ridah’s learning journey, how he developed his skills in data engineering, and how he’s applying…
…
continue reading

1
Data Engineering & Security: Your Questions Answered
1:16:43
1:16:43
Play later
Play later
Lists
Like
Liked
1:16:43Send us a text Today I'll be jumping on for about 60 minutes to do a short AMA and talk about how to ensure you are securing your secrets in your pipelines Join Codecademy while supporting the channel! If you are a high school or college student you get 35% off with this link: https://www.pntrs.com/t/2-591852-361892-213588 Not a student, but want a…
…
continue reading

1
Data Engine Thinking Patterns with Roelant Vos - Episode #66
52:03
52:03
Play later
Play later
Lists
Like
Liked
52:03Join Shane Gibson as he chats with Roelant Vos about a number of Data Engine Thinking patterns and his new book Data Engine Thinking. You can get in touch with Roelant via LinkedIn or over at https://roelantvos.com/blog/ Or you can grab a copy of the Data Engine Thinking book on Amazon or at https://dataenginethinking.com/en/ If you want want to re…
…
continue reading

1
5 (More!) Rookie Mistakes in Data Engineering!
1:08:04
1:08:04
Play later
Play later
Lists
Like
Liked
1:08:04Send us a text Are you new to data engineering and worried about common pitfalls derailing your projects? Join me LIVE tonight as we dive into the top 5 rookie mistakes every fresher makes and—more importantly—how you can avoid them! ✅ Learn from real-world examples and save hours of headaches. ✅ Engage in interactive Q&A to solve your specific cha…
…
continue reading

1
Ethical AI and Data Governance | Protecting Your Business from Data Compliance Nightmares
27:46
27:46
Play later
Play later
Lists
Like
Liked
27:46Send us a text Join me live as we dive into “Ethical AI and Data Governance” ... why your data quality isn’t just a technical checkbox but an ethical imperative. We’ll unpack real‐world failures (from biased hiring tools to misconfigured AI inspections), explore the three pillars of governance (Transparency, Accountability, Fairness), and walk thro…
…
continue reading

1
5 Data Engineering Mistakes That’ll Haunt You (and How to Avoid Them)
40:55
40:55
Play later
Play later
Lists
Like
Liked
40:55Send us a text Is your team spending time repairing broken data pipelines? In this live session, I’ll walk you through 5 of the most common—and costly—mistakes I’ve seen in 25+ years of data engineering. From schema drift disasters to overengineered tech stacks, I’ll break down real-world examples and teach you how to build smarter, cleaner, and mo…
…
continue reading

1
Data Engineering 2025: The Future Awaits
1:09:28
1:09:28
Play later
Play later
Lists
Like
Liked
1:09:28Send us a text Tonight I'm going to talk about the last couple videos we did which were the 5 skill every data engineer has to have and then 6 Skills that Separate you from your Competition! I'll also discuss data consulting and how I got into it and how you can too! As always we will be answering questions about data engineering and cover anything…
…
continue reading

1
Future Proofing Your Career With Shane Gibson and Chris Gambill
1:16:57
1:16:57
Play later
Play later
Lists
Like
Liked
1:16:57Send us a text Futureproof your career and master the idea of T-Skills! As Data Engineers we have a choice to make right now, given the new AI world: Go deeper into their Engineering skills and become the 5% who still apply their craft to solve the very complex data use cases. Go broad and increase their other skills, because (per Shane) 95% of the…
…
continue reading

1
Data Engineering Patterns with Chris Gambill - Episode #65
1:04:28
1:04:28
Play later
Play later
Lists
Like
Liked
1:04:28Join Shane Gibson as he chats with Chris Gambill about a number of Data Engineering patterns. You can get in touch with Chris via LinkedIn or view his YouTube channel https://www.youtube.com/@GambillDataEngineering If you want want to read a summary generated with GenAI, head over to: https://agiledata.substack.com/i/165842333/google-notebooklm-bri…
…
continue reading

1
The pattern of writing a data book with Shane Gibson and Ramona C Truta - Episode #64
1:04:23
1:04:23
Play later
Play later
Lists
Like
Liked
1:04:23Join Ramona C Truta as she interviews Shane Gibson about the patterns of writing a data book (and they discuss a whole lot of other data things) You can get in touch with Ramona via LinkedIn or read more on substack https://ramonatruta.substack.com/ Or buy a copy of Shanes book "an Agile Data Guide to Information Product Canvas", and finally fix th…
…
continue reading

1
Breaking Things: The Secret to Data Engineering Career Success
8:00
8:00
Play later
Play later
Lists
Like
Liked
8:00Send us a text Data engineering in 2025 isn't about cool tools but about survival, as most current data engineers risk becoming replaceable commodities if they can't demonstrate real-world problem-solving skills and business understanding. Successful data engineers need to prove they've overcome significant challenges and understand the business im…
…
continue reading

1
Data Engineering 101: Learn to Scale with Modern Data Modeling
11:53
11:53
Play later
Play later
Lists
Like
Liked
11:53Send us a text What is the modern data stack—and how do tools like Spark, dbt, and Airflow actually work together? In this lesson, I’ll break it down step by step with real-world insights. In Lesson 2 of the Data Engineering 101 series, we take a deep dive into the Modern Data Stack—what it is, how the components work together, and why data modelin…
…
continue reading
Send us a text Are you thinking about jumping into DataOps Or just curious what all the hype is about In this episode, Chris dives deep into why DataOps is not all sunshine and rainbows. From the real challenges of the job to the toolbox every DataOps engineer relies on, you’ll get the honest truth... and a few laughs along the way. Timestamps 00:0…
…
continue reading

1
Data World Shaken: Salesforce x Informatica & DuckLake Unveiled!
54:57
54:57
Play later
Play later
Lists
Like
Liked
54:57Send us a text 🚨 Major News in Data Engineering! 🚨 Today, two industry-shaking headlines dropped: 1️⃣ Salesforce announces its $8B acquisition of Informatica. 2️⃣ DuckDB releases DuckLake, their open-source lakehouse format! Join me LIVE as I break down what these moves mean for data engineers, the enterprise stack, and the future of open-source an…
…
continue reading

1
Data Engineering in the Real World (From Career to Consulting)
1:28:22
1:28:22
Play later
Play later
Lists
Like
Liked
1:28:22Send us a text Behind the Scenes: How I Work Full-Time & Still Help Clients Support the show Chris Gambill is a data engineering consultant and educator with 25+ years of experience helping organizations modernize their data stacks. As founder of Gambill Data, he specializes in data strategy, cloud migration, and building resilient analytics platfo…
…
continue reading

1
The Silent Killers of Data Engineering (And How to Catch Them Before It’s Too Late)
28:41
28:41
Play later
Play later
Lists
Like
Liked
28:41Send us a text Are hidden problems lurking in your data pipelines? In both medicine and data engineering, “silent killers” can do major damage... quietly, with no warning, until it’s almost too late. In this live session, Chris Gambill breaks down the three hidden threats every data engineering team needs to catch before disaster strikes. You’ll le…
…
continue reading

1
The Hook data modeling pattern with Andrew Foad - Episode #63
47:56
47:56
Play later
Play later
Lists
Like
Liked
47:56Join Shane Gibson as he chats with Andrew Foad on his data modeling pattern "Hook" You can get in touch with Andrew via LinkedIn or read more on substack https://hookcookbook.substack.com If you want to download the transcript for the podcast, head over to: https://agiledata.io/podcast/agiledata-podcast/the-hook-data-modeling-pattern-with-andrew-fo…
…
continue reading

1
Increasing the perceived value of your data team with Aaron Wilkerson - Episode #62
1:03:48
1:03:48
Play later
Play later
Lists
Like
Liked
1:03:48Join Shane Gibson as he chats with Aaron Wilkerson on ways data teams can increase the perception of the value they add to their organisation. You can get in touch with Aaron via LinkedIn or read his newsletter on substack https://aaronwilkerson.substack.com If you want to read the transcript for the podcast head over to: https://agiledata.io/podca…
…
continue reading

1
ELM Patterns Templates with Remco Broekmans - Episode #61
58:10
58:10
Play later
Play later
Lists
Like
Liked
58:10Join Shane Gibson as he chats with Remco Broekmans about the pattern templates that are part of the Ensemble Logical Modeling (ELM) patterns. You can find more about ELM and its templates over at: https://www.elmstandards.com/ Find Remco's book on Amazon: https://www.amazon.com/Stories-Solutions-Redefining-Communication-Professionals-ebook/dp/B0DXQ…
…
continue reading

1
Data Operating Models patterns with Dylan Anderson - Episode #60
1:09:24
1:09:24
Play later
Play later
Lists
Like
Liked
1:09:24Join Shane Gibson as he chats with Dylan Anderson about the patterns required to define a Data Operating Model. This conversation was based on this article written by Dylan: https://thedataecosystem.substack.com/p/issue-13-defining-the-data-operating And specifically this pattern diagram: You can get in touch with Dylan via LinkedIn or subscribe to…
…
continue reading

1
DataOps Patterns with Chris Bergh - Episode #59
1:15:19
1:15:19
Play later
Play later
Lists
Like
Liked
1:15:19Join Shane Gibson as he chats with Chris Bergh on improving your teams way of working by using DataOps patterns. You can get in touch with Chris via LinkedIn or over at https://datakitchen.io If you want to read the transcript for the podcast head over to: https://agiledata.io/podcast/agiledata-podcast/dataops-patterns-with-chris-bergh/#read Listen…
…
continue reading