Data Lakehouses Explained
HTML-код
- Опубликовано: 28 июн 2024
- Learn more about watsonx → ibm.biz/BdvxRh
Have you ever thought about how the process of moving food ingredients from farm to table could relate to how organizations store and eventually evaluate data - through data lakes, data warehouses and now a trending architecture, known as data lakehouse?
In this video, Luv Aggarwal explains that analogy, and how a data lakehouse delivers on the benefits of data lakes and warehouses, and more!
#datalake #datalakehouse #datawarehouse #watsonX
![Remble - Colors (ft. Mozzy & Stoneda5th) [Official Music Video]](http://i.ytimg.com/vi/KF-3AU9g9xg/mqdefault.jpg)
Great video Luv. I like the analogy of food service prep that you used also.
In future eposiode , can you cover comparison between Data Lake & Data Mesh ?
Gran forma de explicar con simpleza el uso que le podemos dar a los datos
Loading dock example was a great way to illustrate the concept, thanks!
Brilliant analogy! Invaluable info. Thank you.
Great metaphore ! Well done !
Great video Luv! Amazing explanation!
In a nutshell, data lakes stores all kind of data coming into the organization in cost effective manner as it utilises cloud object storage which is infinitely scalable.. It is equivalent to data swamps as data stroed inside also can be inaccurate, duplicate or inaccurate data which can not be used for querying or for Business Intelligence.
In order to use this data, Data is cleaned first and then loaded into Data Warehouse through ETL process. It is easily queryable and can be used for BI and report generation. But it has two disadvantages :-
1. The cost of data warehouse is too high
2. Apps wants to consume fresh data may not get it from Data warehouse as it ETL process takes time to load data into warehoulse.
Hence to solve the shortcomings of both Data Lake and Data Warehouse, concept of data lakehouse is introduced
brilliant video. best explained data lakehouse in almost 8 minutes. Thank you :)
Amazing video explaining the Data structure using simple method
Excellent presentation about DataLakeHouse
nice explanation, not too technical but really clear
Excellent video. Thanks!
Great analogy, thanks Luv!!
It was a wonderful explaination !! Thanks !
Excellent video.. thanks
Absolutely loved this!
Hey ,
very cleary and simple explanation. One big question from me , then i used both termin's synonymous,. And I think is not realy correct. Is the statging area equal to the Data lake, even not, what is the main difference between thus both ?
Thanks
You are the Best Luv!
The video is very clear in explaining the concepts. But one question that comes to my mind is in which situation would a data warehouse still be viable as a final destination for some of the tables built. Could a use case be optimized query performance that the lakehouse may lake?
Good analogy thx for explain it !
great video *^^* thank you!
REAL-GOOD VIDEO❗😃
Great vid - would love to know how a data lakehouse works though
Data lakehouse architecture explainer coming soon!
Great vid!
Good video. keep em' coming!
If the data Is coming from the api and I want to store it in the database and I wanted to ask how will give and access to data load like an validation
any future videos showing real life examples?
- 5:43 the data doesn't lose its value per se (on the same way at least as food does when it expires). E.g. if it's not "found" (not labelled so nobody knows what it is) and when it is recognized that it's a duplicate of something else are not the same things. In the first case you don't know what the value is, and in the 2nd case the actual/original data has the same value as before and the copy of it has no value.
- well, when it comes to have a lakehouse, the restaurant could force the supplier to dock at a special place to load ONLY vegetables or ONLY meat, so reducing the amount of "labeling" (obviously it has some additional costs to build different docks and certain restaurants (small ones) may not be able to afford that) so on the same way a data lake could apply some data warehouse "principles" to increase the structured-ness and the possibility of "governance".
- It reminds me the sci-fi writer Stanislav Lem's novel where he describes how the wireless communication was "invented": "the engineers made the diameter of the wire by which the communication was done smaller... and then even smaller... and then a bit more... and at one point... there was no wire..." 🙂
Thanks
ok used bard to help: Data Lakehouse:
Unifies the advantages of both data lakes and data warehouses, creating a single platform for all data needs.
Stores all data, structured and unstructured, in low-cost object storage like a data lake.
Applies metadata and schema to the data, like a data warehouse, enabling efficient querying and analysis.
Offers cost-effective storage, flexibility for exploration, and structured data for analysis.
Thank u very much❤❤
Great!
Can you please explain about data mesh??
Awesome and very clear video! By the way, how can you write backwards? 😅
I would like to focus on my meal really :D
JK. Amazing video. Keep up the good work.
❤
The god bless you
Informative, but not an explanation of Data Lakehouses. It explains _why_ there aree Data Lakehouse, not _what_ they are.
Am I the only one mesmerized by how he can write backwards, while talking about complex concepts?
Haha, he doesn't. The video is flipped in post processing.
Summary:
We encounter various types of data-unstructured, semi-structured, and structured-in our data lakes, sourced from different databases and various channels.
Our need extends to powerful dashboards, business intelligence, and reports. Subsequently, we establish an ETL path to transform this data into our enterprise warehouses, which contain domain-specific data tailored for particular use cases.
However, two critical issues arise concerning data governance and data quality, creating what can be likened to data swarms.
To address these challenges, developers contemplate a solution that combines both aspects, known as a lake house. This approach provides a cost-effective, flexible, and high-performance structure, bundling everything into one cohesive system. This integrated system can be utilized for both business intelligence and machine learning processes.
Is he writing backwards? How is this filmed??
See ibm.biz/write-backwards
All the time i am thinking how is writing like this , its better to watch it at 1.5 speed
It would be more efficient to use graphics instead of the painfully slow drawings
data lake is such a useless term. what does this mean in tech terms ?
I wish these videos went straight to the topic.... meals? Trucks?.... im out