To some, Zhamak Dehghani is the Godfather of Data Mesh( or Godmother if want to be grammatically correct), the least Stanford could do was produce a better quality video.
Yes after a lot of data (re)modeling and deciding/agreeing who (domain) are the owners. Then implement technically (which was not talked about in this theoretical video). Technically, as in where the each domain data will reside and how each will be accessed relationally.
Data mesh restructuring requires multi-million-dollar investments depending on the size of its enterprise, its previous architecture, and tech adoption readiness. Siloed architecture means gatekeepers such as either one central IT team does data access and distribution, or decentralized silos where each team has its own independent systems, and you need to ask each systems/gatekeepers for data that you may require. Instead, data mesh promotes the idea that, instead of having independent data systems or one central IT for access and distribution, why not bring data into decentralized data ownership and management, with shared governance and interoperability? In simple terms, data mesh says, let's bring all the data from these multiple siloed or centrally IT-owned systems into something like a data lakehouse. Then, let teams who need that data take whatever they need themselves. And now come the domain product owners. While learning this concept from an enterprise standpoint, a data product owner who owns the Customer domain may have 200 attributes in his customer master data. For a company like Meta, for advertisers, there can be a million identifiers of the attribute because so many people advertise, right? Ad impressions can be a table with hundreds of millions of fields. The Advertising analytics team says that while certain data can be necessary or useful, there are 20 attributes that I absolutely need to function. These may consist of certain attributes of the Customer Domain, certain attributes of Product, certain attributes of Marketing and advertising tables, and such. The traditional silo system makes this a very difficult and slow process for advertising analytics teams to get the data they need, having to go through multiple siloes to get all the data they need. However, with data mesh, this problem is now eliminated because the data they need is egressed into a lakehouse and enabled with federated queries/ SQL/SparkSQL, allowing the Forecasting team to take the data they need. The data product owner's job here will be to bring all the data that the team needs from the source system into the local system and maintain it in such a way that it is ready to be consumed by the advertising analytics team to enable them.
A Data Mesh is a collection of subject specific data warehouses. There's no technology behind the idea. Unpopular opinion perhaps, I think the Data Mesh exists to bill consulting hours.
@@morespinach9832 Advancing the data management industry ought to be done with ideas closely coupled with technology. Good examples are star schemas and event driven architecture. Both are ideas backed by intuitive, strict technical requirements.
This doesn't seem novel. Only the author (and her rather kind colleagues at Thoughtworks) seems to be ve promoting it as a concept, for self advancement reasons.
To some, Zhamak Dehghani is the Godfather of Data Mesh( or Godmother if want to be grammatically correct), the least Stanford could do was produce a better quality video.
We've got 2024. What kind of sound quality is this? Gi!
Great approach
What a superb and talented person is Zhamak Dehghani, and she is very beautiful as well.
Good one
Is possible build this kind of infraestructure starting form a silos architechture?
Yes after a lot of data (re)modeling and deciding/agreeing who (domain) are the owners. Then implement technically (which was not talked about in this theoretical video). Technically, as in where the each domain data will reside and how each will be accessed relationally.
Data mesh restructuring requires multi-million-dollar investments depending on the size of its enterprise, its previous architecture, and tech adoption readiness. Siloed architecture means gatekeepers such as either one central IT team does data access and distribution, or decentralized silos where each team has its own independent systems, and you need to ask each systems/gatekeepers for data that you may require. Instead, data mesh promotes the idea that, instead of having independent data systems or one central IT for access and distribution, why not bring data into decentralized data ownership and management, with shared governance and interoperability?
In simple terms, data mesh says, let's bring all the data from these multiple siloed or centrally IT-owned systems into something like a data lakehouse. Then, let teams who need that data take whatever they need themselves. And now come the domain product owners. While learning this concept from an enterprise standpoint, a data product owner who owns the Customer domain may have 200 attributes in his customer master data. For a company like Meta, for advertisers, there can be a million identifiers of the attribute because so many people advertise, right? Ad impressions can be a table with hundreds of millions of fields. The Advertising analytics team says that while certain data can be necessary or useful, there are 20 attributes that I absolutely need to function. These may consist of certain attributes of the Customer Domain, certain attributes of Product, certain attributes of Marketing and advertising tables, and such.
The traditional silo system makes this a very difficult and slow process for advertising analytics teams to get the data they need, having to go through multiple siloes to get all the data they need. However, with data mesh, this problem is now eliminated because the data they need is egressed into a lakehouse and enabled with federated queries/ SQL/SparkSQL, allowing the Forecasting team to take the data they need.
The data product owner's job here will be to bring all the data that the team needs from the source system into the local system and maintain it in such a way that it is ready to be consumed by the advertising analytics team to enable them.
A Data Mesh is a collection of subject specific data warehouses. There's no technology behind the idea. Unpopular opinion perhaps, I think the Data Mesh exists to bill consulting hours.
Spot on!
Agreed.
So what in your mind is the issue to a very real enterprise problem?
@@morespinach9832 Advancing the data management industry ought to be done with ideas closely coupled with technology. Good examples are star schemas and event driven architecture. Both are ideas backed by intuitive, strict technical requirements.
No connection to data mesh group?
This doesn't seem novel. Only the author (and her rather kind colleagues at Thoughtworks) seems to be ve promoting it as a concept, for self advancement reasons.
LOL STANFORD! Wallow in your disgrace, you were caught cheating too much!
I really envy people who can create a presentation about nothing, write a book, and talk without conveying anything new besides a few slogans.
🤣🤣🤣