Big Data is Dead | MotherDuck

This talk will make the case that the era of Big Data is over. Now we can stop worrying about data size and focus on how we’re going to use it to make better decisions.
The data behind the graphs shown in this talk come from Jordan Tigani having analyzed query logs, deal post-mortems, benchmark results (published and unpublished), customer support tickets, customer conversations, service logs, and published blog posts, plus a bit of intuition.
ABOUT THE SPEAKER:
Jordan Tigani is co-founder and chief duck-herder at MotherDuck, a startup building a serverless analytics platform based on DuckDB. He helped create Google BigQuery, wrote two books on it, and led first the engineering team then the product team through its first $1B or so in revenue.
👉 Sign up for our “No BS” Newsletter to get the latest technical data & AI content: datacouncil.ai/newsletter
ABOUT DATA COUNCIL:
Data Council (www.datacouncil.ai/) is a community and conference series that provides data professionals with the learning and networking opportunities they need to grow their careers.
Make sure to subscribe to our channel for the most up-to-date talks from technical professionals on data related topics including data infrastructure, data engineering, ML systems, analytics and AI from top startups and tech companies.
FOLLOW DATA COUNCIL:
Twitter: / datacouncilai
LinkedIn: / datacouncil-ai

[@maskedvillainai]

So when you…actually work with production data, big data doesn’t just die simply because you wanna make it so.
When you load any data locally faster. This isn’t a duckdb thing it’s a server thing…
Just like when load SQL or pandas locally. It’s not SQL itself that’s slow, it’s connecting to it. Because it’s a static API inferences through a driver vs a URL. Like if duckdb even had the option to connect to it remotely, it would become the next “thing that’s dead” apparently.

[@EliMayost]

The correct analogy would be DuckDB querying data residing in a Data Lake.
Also a DuckDB in a k8s pod on a remote cluster could be queried quite easily, and you can call it remotely accessible. Just a different way of doing things.

[@tessafelice2181]

I love the name mother duck. I feel it’s a respectful tribute to the female source of life and code.

[@estebanrojas1322]

this felt like a product ad with extra steps...

[@_rd_kocaman]

I mean he founded a company emerged from the idea he has been believing/growing for years. this felt like hearing his motives and idea behind the product

[@btbytes]

well said. the idea that the tools are bigger than the data they are processing is not new. eg: see "Scalability, at what cost!" by Mcsherry etc (2015).

[@PimpoDiabolo]

Copying data on laptops is a privacy nightmare!! Because of GDPR you don't want to do this...

[@EliMayost]

DuckDB can query data residing in a Data Lake. None of the data has to be on your laptop.