Materialized Views: Tips, Tricks, and Use Cases

Materialized views are one of StarRocks’ most popular and powerful features, but are you getting the most out of them? Murphy Wang, the technical mind behind the project’s materialized views, is ready to share all the latest tips and tricks to help you get the best query performance for your data pipeline.
Session Highlights:
🌟Best practices for rolling out materialized views: Learn what causes slow queries and how StarRocks offers the most optimal solution.
🌟Actionable use cases: Explore a variety of use cases where materialized views excel, from accelerating data lake analytics to optimizing complex BI queries.
🌟Materialized view tips and tricks: Dive into the fundamentals of setting up and managing materialized views, and the steps you should take to get the most out of them.
----------------------------------------------------------------------------------------------------------------------
Timestamps
00:00 Intro and Agenda
00:57 What Specific Challenges Do Materialized Views Address for Data Pipelines in Data Lakes?
01:59 What Is a Materialized View and What Are the Advantages?
02:39 How Can Materialized Views Simplify Lakehouse Data Pipelines?
03:23 How Can Materialized Views Enable Seamless BI Acceleration
04:53 TPC-H Benchmark as an Example
05:50 Materialized Views: Real-World Use Cases
05:55 Aggregation Layer - Customer: podcast web application; Use Case: daily dashboard for tiered downloads bandwidth
07:27 Real-time Dashboard
Customer: Didi (ride-sharing); high concurrency real-time analytics supporting hundreds of concurrent users querying billions of records daily.
09:36 Dashboard - Trino Replacement
Customer: Gaming Company; Issues: slow query performance and pipeline complexity
11:17 BI Platform
Customer: Trip.com; Issues: slow query performance and complex BI queries
13:13 Metric Layer
Customer: Financial Institution; Cube replacement
14:29 How to Use Materialized Views
14:36 Materialized Views Storage
15:35 Materialized Views SQL
16:31 Materialized Views Refresh Task
17:47 Partitioning
21:06 Partitioning: Refresh
22:35 Schema Change
24:08 Operations
26:30 Auto-Rewrite Query to Materialized Views
30:09 Auto-Rewrite: Dimension Modeling
31:48 Auto-Rewrite: View Modeling
33:05 Auto-Rewrite: Union Rewrite
33:48 Auto MV (Only Available in CelerData Cloud Version)
----------------------------------------------------------------------------------------------------------------------
Learn more at starrocks.com/
Connect with us:
LinkedIn: / celerdata
Twitter: / celerdata
CelerData Website: celerdata.com/
StarRocks GitHub: github.com/StarRocks/StarRocks
StarRocks Website: www.starrocks.io/
Slack: try.starrocks.com/join-starro...
#DataAnalytics #DataEngineering #DataLakeAnalytics #OLAP #DataAnalyst #DataEngineer #DataInfrastructure #UserFacingAnalytics #Database #AnalyticalDatabase #DataLake #DataLakeHouse #DataWarehouse #datasciencebasics

[@andriifadieiev9757]

Great episode, keep going!