Видео

🤣🤣🤣

Spot on!

Agreed.

So what in your mind is the issue to a very real enterprise problem?

@@morespinach9832 Advancing the data management industry ought to be done with ideas closely coupled with technology. Good examples are star schemas and event driven architecture. Both are ideas backed by intuitive, strict technical requirements.

Yes after a lot of data (re)modeling and deciding/agreeing who (domain) are the owners. Then implement technically (which was not talked about in this theoretical video). Technically, as in where the each domain data will reside and how each will be accessed relationally.

Data mesh restructuring requires multi-million-dollar investments depending on the size of its enterprise, its previous architecture, and tech adoption readiness. Siloed architecture means gatekeepers such as either one central IT team does data access and distribution, or decentralized silos where each team has its own independent systems, and you need to ask each systems/gatekeepers for data that you may require. Instead, data mesh promotes the idea that, instead of having independent data systems or one central IT for access and distribution, why not bring data into decentralized data ownership and management, with shared governance and interoperability? In simple terms, data mesh says, let's bring all the data from these multiple siloed or centrally IT-owned systems into something like a data lakehouse. Then, let teams who need that data take whatever they need themselves. And now come the domain product owners. While learning this concept from an enterprise standpoint, a data product owner who owns the Customer domain may have 200 attributes in his customer master data. For a company like Meta, for advertisers, there can be a million identifiers of the attribute because so many people advertise, right? Ad impressions can be a table with hundreds of millions of fields. The Advertising analytics team says that while certain data can be necessary or useful, there are 20 attributes that I absolutely need to function. These may consist of certain attributes of the Customer Domain, certain attributes of Product, certain attributes of Marketing and advertising tables, and such. The traditional silo system makes this a very difficult and slow process for advertising analytics teams to get the data they need, having to go through multiple siloes to get all the data they need. However, with data mesh, this problem is now eliminated because the data they need is egressed into a lakehouse and enabled with federated queries/ SQL/SparkSQL, allowing the Forecasting team to take the data they need. The data product owner's job here will be to bring all the data that the team needs from the source system into the local system and maintain it in such a way that it is ready to be consumed by the advertising analytics team to enable them.