Building a Data Lake with Adam Ferrari

Data Archives - Software Engineering Daily

Please click here to see the transcript of this episode.

36:16

DANMMMMM…Have I got a show for you! First, a lot of Sister Wives tea - new rumors have surfaced Janelle Brown is leaving the show. Plus, Gabe Brown gives a life update after losing and tragically finding his brother Garrison dead. Sadly, Garrison took his own life in March 2024. Then we head over to discuss the new Welcome To Plathville tea. The first pictures of Micah Plath have surfaced after being beat up by his brother Issac and it doesn’t look good for the future of his modeling career. Lastly, we discuss the latest in the Justin Baldoni v Blake Lively case, Justin is back on social media and it was the perfect social media return. Timestamps: 00:00:00 - Open and new Sister Wives news 00:05:43 - Janelle Brown leaving the show? Sister Wives Closet is officially closed 00:12:45 - A new pic of Micah Plath’s broken nose has surfaced 00:18:18 - Justin Baldoni back on social media and Taylor Swifts team is pissed at Justin Baldoni MY Go Big Podcasting Courses Are Here! Purchase Go Big Podcasting and learn to start, monetize, and grow your own podcast. USE CODE: MOM15 for 15% OFF (code expires May 11th, 2025) **SHOP my Amazon Marketplace - especially if you're looking to get geared-up to start your own Podcast!!!** https://www.amazon.com/shop/thesarahfrasershow Show is sponsored by: Download Cash App & sign up! Use our exclusive referral code TSFS in your profile, send $5 to a friend within 14 days, and you’ll get $10 dropped right into your account. Terms apply Horizonfibroids.com get rid of those nasty fibroids Gopurebeauty.com science backed skincare from head to toe, use code TSFS at checkout for 25% OFF your order Nutrafol.com use code TSFS for FREE shipping and $10 off your subscription Rula.com/tsfs to get started today. That’s R-U-L-A dot com slash tsfs for convenient therapy that’s covered by insurance. SkylightCal.com/tsfs for $30 OFF your 15 inch calendar Quince.com/tsfs for FREE shipping on your order and 365 day returns Warbyparker.com/tsfs make an appointment at one of their 270 store locations and head to the website to try on endless pairs of glasses virtually and buy your perfect pair Follow me on Instagram/Tiktok: @thesarahfrasershow ***Visit our Sub-Reddit: reddit.com/r/thesarahfrasershow for ALL things The Sarah Fraser Show!!!*** Advertise on The Sarah Fraser Show: thesarahfrasershow@gmail.com Got a juicy gossip TIP from your favorite TLC or Bravo show? Email: thesarahfrasershow@gmail.com Learn more about your ad choices. Visit megaphone.fm/adchoices…

about a year ago 46:19

MP3•Episode home

Starburst is a data lake analytics platform. It’s designed to help users work with structured data at scale, and is built on the open source platform, Trino.

Adam Ferrari is the SVP of Engineering at Starburst. He joins the show to talk about Starburst, data engineering, and what it takes to build a data lake.

Full Disclosure: Starburst is a sponsor of Software Engineering Daily

Sean’s been an academic, startup founder, and Googler. He has published works covering a wide range of topics from information visualization to quantum computing. Currently, Sean is Head of Marketing and Developer Relations at Skyflow and host of the podcast Partially Redacted, a podcast about privacy and security engineering. You can connect with Sean on Twitter @seanfalconer .

Watch the video episode here

Sponsorship inquiries: sponsor@softwareengineeringdaily.com

The post Building a Data Lake with Adam Ferrari appeared first on Software Engineering Daily.

231 episodes

#Data Science #Science #Tech News #News #Tech #Data Archives Software Engineering Daily

Data Archives - Software Engineering Daily

Building a Data Lake with Adam Ferrari

Data Archives - Software Engineering Daily

300 subscribers

published about a year ago

MP3•Episode home

Starburst is a data lake analytics platform. It’s designed to help users work with structured data at scale, and is built on the open source platform, Trino.

Adam Ferrari is the SVP of Engineering at Starburst. He joins the show to talk about Starburst, data engineering, and what it takes to build a data lake.

Full Disclosure: Starburst is a sponsor of Software Engineering Daily

Please click here to see the transcript of this episode.

Watch the video episode here

Sponsorship inquiries: sponsor@softwareengineeringdaily.com

The post Building a Data Lake with Adam Ferrari appeared first on Software Engineering Daily.

231 episodes

#Data Science #Science #Tech News #News #Tech #Data Archives Software Engineering Daily

All episodes

Data Archives - Software Engineering Daily

1
Hyperscaling SQL with Sam Lambert

46 weeks ago

—

Databases underpin almost every user experience on the web, but scaling a database is one of the most fundamental infrastructure challenges in software development. PlanetScale offers a MySQL platform that is managed and highly scaleable. Sam Lambert is the CEO of PlanetScale and he joins the show to talk about why he started the platform, scaling databases, using Vitess for SQL shard orchestration, and more. This episode is hosted by Lee Atchison. Lee Atchison is a software architect, author, and thought leader on cloud computing and application modernization. His best-selling book, Architecting for Scale (O’Reilly Media), is an essential resource for technical teams looking to maintain high availability and manage risk in their cloud environments. Lee is the host of his podcast, Modern Digital Business , an engaging and informative podcast produced for people looking to build and grow their digital business with the help of modern applications and processes developed for today’s fast-moving business environment. Listen at mdb.fm . Follow Lee at softwarearchitectureinsights.com , and see all his content at leeatchison.com . Please click here to see the transcript of this episode. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Hyperscaling SQL with Sam Lambert appeared first on Software Engineering Daily .…

Data Archives - Software Engineering Daily

1
Iceberg at Netflix and Beyond with Ryan Blue 47:37

1 year ago47:37

47:37

Apache Iceberg is an open source high-performance format for huge data tables. Iceberg enables the use of SQL tables for big data, while making it possible for engines like Spark and Hive to safely work with the same tables, at the same time. Iceberg was started at Netflix by Ryan Blue and Dan Weeks, and was open-sourced and donated to the Apache Software Foundation in November 2018. It has now been adopted at many other companies including Airbnb, Apple, and Lyft. Ryan Blue joins the podcast to describe the origins of Iceberg, how it works, the problems it solves, collaborating with Apple and others to open-source it, and more. This episode is hosted by Lee Atchison. Lee Atchison is a software architect, author, and thought leader on cloud computing and application modernization. His best-selling book, Architecting for Scale (O’Reilly Media), is an essential resource for technical teams looking to maintain high availability and manage risk in their cloud environments. Lee is the host of his podcast, Modern Digital Business , an engaging and informative podcast produced for people looking to build and grow their digital business with the help of modern applications and processes developed for today’s fast-moving business environment. Listen at mdb.fm . Follow Lee at softwarearchitectureinsights.com , and see all his content at leeatchison.com . Please click here to see the transcript of this episode. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Iceberg at Netflix and Beyond with Ryan Blue appeared first on Software Engineering Daily .…

Data Archives - Software Engineering Daily

1
Building a Data Lake with Adam Ferrari 46:19

1 year ago46:19

46:19

Starburst is a data lake analytics platform. It’s designed to help users work with structured data at scale, and is built on the open source platform, Trino . Adam Ferrari is the SVP of Engineering at Starburst. He joins the show to talk about Starburst, data engineering, and what it takes to build a data lake. Full Disclosure: Starburst is a sponsor of Software Engineering Daily Sean’s been an academic, startup founder, and Googler. He has published works covering a wide range of topics from information visualization to quantum computing. Currently, Sean is Head of Marketing and Developer Relations at Skyflow and host of the podcast Partially Redacted, a podcast about privacy and security engineering. You can connect with Sean on Twitter @seanfalconer . Please click here to see the transcript of this episode. Watch the video episode here Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Building a Data Lake with Adam Ferrari appeared first on Software Engineering Daily .…

Data Archives - Software Engineering Daily

1
Rama with Nathan Marz 45:06

1 year ago45:06

45:06

Building scalable software applications can be complex and typically requires dozens of different tools. The engineering often involves handling many arcane tasks that are distant from actual application logic. In addition, a lack of a cohesive model for building applications can lead to substantial engineering costs. Nathan Marz is the creator of Rama , which is a platform for building end-to-end, scalable backends. Previously Nathan led engineering at BackType which was acquired by Twitter in 2011. In addition, Nathan created the Apache Storm project and is the author of the book Big Data: Principles and best practices of scalable realtime data systems. Nathan joins the podcast today to talk about Rama, and how to build scalable software applications. This episode is hosted by Lee Atchison. Lee Atchison is a software architect, author, and thought leader on cloud computing and application modernization. His best-selling book, Architecting for Scale (O’Reilly Media), is an essential resource for technical teams looking to maintain high availability and manage risk in their cloud environments. Lee is the host of his podcast, Modern Digital Business , an engaging and informative podcast produced for people looking to build and grow their digital business with the help of modern applications and processes developed for today’s fast-moving business environment. Listen at mdb.fm . Follow Lee at softwarearchitectureinsights.com , and see all his content at leeatchison.com . Please click here to see the transcript of this episode. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Rama with Nathan Marz appeared first on Software Engineering Daily .…

Data Archives - Software Engineering Daily

1
Bonus Episode: SurrealDB with Tobie Morgan Hitchcock 57:17

1 year ago57:17

57:17

SurrealDB is the result of a long-time collaboration between brothers Tobie and Jaime Morgan Hitchcock. The project has modest origins and started merely to support other projects the brothers were working on. However, over time the project grew and in 2021 they started working on it full-time. Since then the project has gained serious adoption. What’s makes SurrealDB so special? Tobie Morgan Hitchcock is the CEO of SurrealDB and he joins the show to talk about his multimodal database, support for graph and time series data, why they rewrote the entire project in Rust, and more. Starting her career as a software developer, Jocelyn Houle is now a Senior Director of Product Management at Securiti.ai, a unified data protection and governance platform. Before that, she was an Operating Partner at Capital One Ventures investing in data and AI startups. Jocelyn has been a founder of two startups and a full life cycle, technical product manager at large companies like Fannie Mae, Microsoft and Capital One. Follow Jocelyn on LinkedIn or Twitter @jocelynbyrne. Please click here to see the transcript of this episode. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Bonus Episode: SurrealDB with Tobie Morgan Hitchcock appeared first on Software Engineering Daily .…

Data Archives - Software Engineering Daily

1
Tracking Drug Smugglers and Migrating Databases with Benny Keinan and Lior Resisi 50:40

1 year ago50:40

50:40

Maritime logistics is the process organizing the movement of goods across the ocean. Historically, this has been a challenging problem because of the multinational nature of shipping, as well as piracy, smuggling, and legacy technology. It’s also profoundly important for security reasons, and because 90% of what we buy travels over the oceans. Ocean vessels produce a lot of CO2, which adds climate change and energy dimensions to maritime logistics. Windward AI is a maritime logistics platform that was started 13 years ago by two ex-Israeli naval officers. The idea for the company came from the observation that, at that time, it was hard or impossible to know what’s happening on the deep sea. Benny Keinan is the VP of R&D and Lior Resisi is the Data Platforms Group Lead at Windward AI. They join the podcast today to talk about the technical and practical challenges of maritime logistics, why Rockset was the right database for their unique datasets, the impact of the Ukraine war, and more. This episode is hosted by Lee Atchison. Lee Atchison is a software architect, author, and thought leader on cloud computing and application modernization. His best-selling book, Architecting for Scale (O’Reilly Media), is an essential resource for technical teams looking to maintain high availability and manage risk in their cloud environments. Lee is the host of his podcast, Modern Digital Business , an engaging and informative podcast produced for people looking to build and grow their digital business with the help of modern applications and processes developed for today’s fast-moving business environment. Listen at mdb.fm . Follow Lee at softwarearchitectureinsights.com , and see all his content at leeatchison.com . Please click here to see the transcript of this episode. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Tracking Drug Smugglers and Migrating Databases with Benny Keinan and Lior Resisi appeared first on Software Engineering Daily .…

Data Archives - Software Engineering Daily

1
The Right to Be Forgotten with Gal Ringel 47:45

1 year ago47:45

47:45

Data breaches at major companies are so now common that they hardly make the news. The Wikipedia page on data breaches lists over 350 between 2004 and 2023. The Equifax breach in 2017 was especially notable because over 160 million records were leaked, and much of the data was acquired by Equifax without individuals’ knowledge or consent. Data breaches are increasingly costly to companies and to affected users who must deal with the ensuing identity theft. In 2018 the European Union implemented the General Data Protection Regulation, or GDPR. Despite its mild name, the GDPR had major consequences for individuals’ rights to control their data, and for companies that operate in the EU. Among other things, it gives the right to have personal data removed from a company’s records. This is the so-called “right to be forgotten”. Gal Ringel is the Co-Founder and CEO of Mine , which allows users to identify which companies have their data, and it automates the process of removing data on behalf of its users. In this way, Mine aims to reduce online exposure and minimize risk for anyone using online services. Gal joins the show today to talk about his company, the impact of GDPR, and how his experience in military intelligence, venture capital, and tech led him to co-found the company. Jordi Mon Companys is a product manager and marketer that specializes in software delivery, developer experience, cloud native and open source. He has developed his career at companies like GitLab, Weaveworks, Harness and other platform and devtool providers. His interests range from software supply chain security to open source innovation. You can reach out to him on Twitter at @jordimonpmm Please click here to see the transcript of this episode . Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post The Right to Be Forgotten with Gal Ringel appeared first on Software Engineering Daily .…

Data Archives - Software Engineering Daily

1
Sofascore with Josip Stuhli 49:48

1 year ago49:48

49:48

If you’re a sports fan and like to track sports statistics and results, you’ve probably heard of Sofascore. The website started in 2010 and ran on a modest single server. It now has 25 million monthly active users, covers 20 different sports, 11,000 leagues and tournaments, and is available in over 30 languages. Josip Stuhli has been with Sofascore for 13 years. He started there as an engineer and is currently CTO. Josip joins the show today to talk about the challenges Sofascore encountered over the years, and how the team solved them. He discusses dealing with traffic spikes from game days, structuring and restructuring the codebase, organizing the frontend and backend, and much more. Sean’s been an academic, startup founder, and Googler. He has published works covering a wide range of topics from information visualization to quantum computing. Currently, Sean is Head of Marketing and Developer Relations at Skyflow and host of the podcast Partially Redacted, a podcast about privacy and security engineering. You can connect with Sean on Twitter @seanfalconer . Please clicke here for the full transcript of this episode Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Sofascore with Josip Stuhli appeared first on Software Engineering Daily .…

Data Archives - Software Engineering Daily

1
Daytona with Ivan Burazin 47:50

1 year ago47:50

47:50

Cloud-based software development platforms such as GitHub Codespaces continue to grow in popularity. These platforms are attractive to enterprise organizations because they can be managed centrally with security controls. However, many, if not most, developers prefer a local IDE. Daytona is aiming to bridge that gap. It’s a layer between a local IDE and a backend server, so developers can work locally while interfacing invisibly with a remote environment. Ivan Burazin is the CEO and Co-Founder at Daytona , and he joins the show today to talk about how Daytona works, Spotify as an inspiration for his product, and more. Jordi Mon Companys is a product manager and marketer that specializes in software delivery, developer experience, cloud native and open source. He has developed his career at companies like GitLab, Weaveworks, Harness and other platform and devtool providers. His interests range from software supply chain security to open source innovation. You can reach out to him on Twitter at @jordimonpmm Please click here to see the transcript of this episode. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Daytona with Ivan Burazin appeared first on Software Engineering Daily .…

Data Archives - Software Engineering Daily

1
GraphAware with Luanne Misquitta 57:51

1 year ago57:51

57:51

Knowledge graphs are an intuitive way to define relationships between objects, events, situations, and concepts. Their ability to encode this information makes them an attractive database paradigm. Hume is a graph-based analysis solution developed by GraphAware . It represents data as a network of interconnected entities and provides analysis capabilities to extract insights from the data. Luanne Misquitta is VP of Engineering at GraphAware and she joins the show today to talk about graph databases, and the engineering of Hume. Starting her career as a software developer, Jocelyn Houle is now a Senior Director of Product Management at Securiti.ai, a unified data protection and governance platform. Before that, she was an Operating Partner at Capital One Ventures investing in data and AI startups. Jocelyn has been a founder of two startups and a full life cycle, technical product manager at large companies like Fannie Mae, Microsoft and Capital One. Follow Jocelyn on LinkedIn or Twitter @jocelynbyrne. Please click here to view this show’s transcript. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post GraphAware with Luanne Misquitta appeared first on Software Engineering Daily .…

Data Archives - Software Engineering Daily

1
Chronosphere with Martin Mao 48:08

2 years ago48:08

48:08

Observability software helps teams to actively monitor and debug their systems, and these tools are increasingly vital in DevOps. However, it’s not uncommon for the volume of observability data to exceed the amount of actual business data. This creates two challenges – how to analyze the large stream of observability data, and how to keep down the compute and storage costs for that data. Chronosphere is a popular observability platform that works by identifying the data that’s actually being used to power dashboards and metrics. It then shows the cost for each segment of data, and allows users to decide if a metric is worth that cost. In this way, technical teams can manage costs by dynamically adjusting which data is analyzed and stored. Martin Mao is the Co-founder and CEO of Chronosphere and he joins the podcast today to talk about the growing challenge of managing observability data, and the design of Chronosphere. This episode is hosted by Lee Atchison. Lee Atchison is a software architect, author, and thought leader on cloud computing and application modernization. His best-selling book, Architecting for Scale (O’Reilly Media), is an essential resource for technical teams looking to maintain high availability and manage risk in their cloud environments. Lee is the host of his podcast, Modern Digital Business , an engaging and informative podcast produced for people looking to build and grow their digital business with the help of modern applications and processes developed for today’s fast-moving business environment. Listen at mdb.fm . Follow Lee at softwarearchitectureinsights.com , and see all his content at leeatchison.com . Please click here to see the transcript for this episode. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Chronosphere with Martin Mao appeared first on Software Engineering Daily .…

Data Archives - Software Engineering Daily

1
Streamlit with Amanda Kelly 47:06

2 years ago47:06

47:06

The importance of data teams is undeniable. Most companies today use data to drive decision-making on anything from software feature development to product strategy, hiring and marketing. In some companies data is the product, which can make data teams even more vital. But there’s a common problem – analyzing data is hard and time consuming. Lots of people have questions they want to answer with data, but data teams often don’t have the resources to move quickly. This can create a pernicious effect where organizations stop asking questions about their own data. Amanda Kelly thinks a lot about data and the dynamics of data teams inside organizations. She’s worked at Google X, and on self-driving cars and cybersecurity. Her experiences on data teams inspired her to co-found Streamlit, which is an open source Python library that gives primitives to assemble a data app for rapid data visualization and interaction. Her goal was to accelerate the iteration loop to go from a question to a data-driven answer. Amanda is currently the COO of Streamlit and a Product Director at Snowflake, and she joins us today to talk all about data and how she’s building Streamlit. Sean’s been an academic, startup founder, and Googler. He has published works covering a wide range of topics from information visualization to quantum computing. Currently, Sean is Head of Marketing and Developer Relations at Skyflow and host of the podcast Partially Redacted, a podcast about privacy and security engineering. You can connect with Sean on Twitter @seanfalconer. Please click here to view this show’s transcript. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Streamlit with Amanda Kelly appeared first on Software Engineering Daily .…

Data Archives - Software Engineering Daily

1
Modern Web Scraping with Erez Naveh 57:29

2 years ago57:29

57:29

Today it’s estimated there are over 1 billion websites on the internet. Much of this content is optimized to be viewed by human eyes, not consumed by machines. However, creating systems to automatically parse and structure the web greatly extends its utility, and paves the way for innovative solutions and applications. The industry of web scraping has emerged to do just that. However, many websites erect obstacles to hinder web scraping. This has created a new kind of arms race between developers and anti-scraping software. Bright Data has developed some of the most sophisticated consumer tools available to scrape public web data. Erez Naveh is an entrepreneur and former engineer at Meta. He is currently the VP of Product at Bright Data. Erez joins us in this episode to talk about Bright Data’s mission to structure the open web, and the toolkit they’ve developed to make this possible. Full Disclosure: Bright Data is a sponsor of Software Engineering Daily Paweł is the founder at flat.social the world’s first ‘flatverse’ start-up and glot.space , an AI-powered language learning app. Pawel’s background is as a full-stack software engineer with a lean and experimental approach towards product development. With a strong grounding in computing science, he spent the last decade getting early-stage products off the ground – both in startup and corporate settings. Follow Paweł on Twitter , LinkedIn and his personal website – pawel.io . Please click here to view this show’s transcript . Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Modern Web Scraping with Erez Naveh appeared first on Software Engineering Daily .…

Data Archives - Software Engineering Daily

1
Observability with Eduardo Silva 44:47

2 years ago44:47

44:47

There are hundreds of observability companies out there, and many ways to think about observability, such as application performance monitoring, server monitoring, and tracing. In a production application, multiple tools are often needed to get proper visibility on the application. This creates some challenges. Applications can produce lots of different observatory observability data, but how should the data be routed to the various downstream tools? In addition, how can data be selectively sent to different storage tiers to minimize costs? Calyptia is a service that helps manage observability data from source to destination. Eduardo Silva is the founder and CEO of Calyptia and he joins us in this episode. This episode is hosted by Lee Atchison. Lee Atchison is a software architect, author, and thought leader on cloud computing and application modernization. His best-selling book, Architecting for Scale (O’Reilly Media), is an essential resource for technical teams looking to maintain high availability and manage risk in their cloud environments. Lee is the host of his podcast, Modern Digital Business , an engaging and informative podcast produced for people looking to build and grow their digital business with the help of modern applications and processes developed for today’s fast-moving business environment. Listen at mdb.fm . Follow Lee at softwarearchitectureinsights.com , and see all his content at leeatchison.com . Please click here to view this show’s transcript . Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Observability with Eduardo Silva appeared first on Software Engineering Daily .…

Data Archives - Software Engineering Daily

1
AI and Business Analytics with John Adams 30:03

2 years ago30:03