Snowflake vs Databricks - And the Battle For Iceberg
HTML-код
- Опубликовано: 23 окт 2024
- Can Iceberg become the single data layer where all our analytical processing can occur on top of?
If you enjoyed this video, check out some of my other top videos.
Don't Lead A Data Team Before Watching This - 5 Lessons You Need To Know As A Head of Data
• Don't Lead A Data Team...
Data Modeling Where Theory Meets Reality - How Different Companies I Worked At Modeled Their Data
• Data Modeling Where Th...
Also, if you're looking to ingest data into your data warehouse, consider checking out Estuary!
bit.ly/3Svmeen
If you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.
seattledataguy...
Or check out my blog
www.theseattle...
And if you want to support the channel, then you can become a paid member of my newsletter
seattledataguy...
Tags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio
_____________________________________________________________
Subscribe: / @seattledataguy
_____________________________________________________________
About me:
I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.
*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
During the entire talk... "Master Data Management" and "Data Virtualization" were common echoes... Silos should be preserved for most important operations and then for analytics outside of production/operational dashboards should be replicated out to something like a lakehouse or other systems where the data democratization can shine.
He is making so much sense!!!!
Dude, I was living under the rock.. I discovered your channel so late…
Glad you found the video helpful!
Really loved that video Ben, it's somehow really timely for what I'm working on these days ! Thanks 🎉
Loving your content. As a headphones listener, it's time to splurge on a nice mic. Your presentation is so good, and definitely structured enough to podcast it. But it's hard to *just* listen because the audio can get a bit garbled in that mic.
Confused about your statement where you say copying from oltp to olap is another silo but then acknowledging that you can do both in one database.
That’s why you add starburst or another federated access point to the data such that everything can come together in one place. Some of the teams may use iceberg for analytics, some other teams may use postgres. Who cares, it can all come together in one place. Plus a well constructed data mesh can help to ensure teams do not double up the responsibilities or data sets, which all can be accessed in the federated data store.
NICEEEEEEEEEEEEEE