The Geek Narrator
The Geek Narrator
  • Видео 158
  • Просмотров 371 590
Learnings from building Open Source Distributed Systems with Kishore Gopalakrishna
In this episode of The Geek Narrator podcast, hosted by Kaivalya Apte, we welcome a special guest, Kishore Gopalakrishna from StarTree, co-author of Apache Pinot and other notable projects. Kishore shares his extensive experience in building real-time analytics and streaming systems, including Apache Pino, Espresso, Apache Helix, and Third Eye. The episode delves into the motivations and challenges behind creating these systems, the innovations they brought to distributed systems, and the impact of community on open-source projects. Kishore also discusses the evolution of testing methodologies, cost optimizations in transactional and analytical systems, and key considerations for companie...
Просмотров: 2 284

Видео

WarpStream: A drop-in replacement for Kafka
Просмотров 864Месяц назад
In this episode of The GeekNarrator podcast, host Kaivalya Apte interviews Ryan and Richie, the founders of WarpStream. They discuss the architecture, benefits, and core functionalities of WarpStream, a drop-in replacement for Apache Kafka. The conversation covers their experience with Kafka, the design decisions behind WarpStream, and the operational challenges it addresses. They also delve in...
XTDB - An Immutable SQL Database
Просмотров 11 тыс.2 месяца назад
Exploring XTDB with Jeremy Taylor & Malcolm Sparks: An In-Depth Dive into Immutability and Database Internals In this episode of the Geek Narrator Podcast, host Kaivalya is joined by Jeremy Taylor and Malcolm Sparks from Juxt to explore XTDB, an immutable database designed to handle complex historical and financial data with precision. They delve into the architecture, internal mechanics, and u...
Turso - SQLite for production
Просмотров 17 тыс.2 месяца назад
Exploring Turso with Glauber Costa: Insights on SQLite for Production In this episode of The GeekNarrator podcast, host Kaivalya Apte interviews Glauber Costa, founder and CEO of TursoDB. They discuss the inception of TursoDB, Glauber's background in Linux kernel development, and the journey from unikernel projects to founding a database company. Glauber explains TursoDB's enhancements to SQLit...
Testing Distributed Systems the right way ft. Will Wilson
Просмотров 8 тыс.2 месяца назад
In this episode of The GeekNarrator podcast, host Kaivalya Apte dives into the complexities of testing distributed systems with Will Wilson from Antithesis. If you’re grappling with the challenges of testing databases, micro-services, and distributed systems, this episode is a must-watch. Will Wilson demystifies the concept of deterministic simulation testing, shares insights about its advantag...
Taking Postgres to the next level with Neon
Просмотров 18 тыс.3 месяца назад
Deep Dive into Serverless Databases with Neon: Featuring Heikki Linnakangas In this episode of the Geek Narrator podcast, host Kaivalya Apte is joined by Heikki Linnakangas, co-founder of Neon, to explore the innovative world of serverless databases. They discuss Neon's unique approach to separating compute and storage, the benefits of serverless architecture for modern applications, and dive i...
Scaling Derived Data for Planet-Scale Applications at Linkedin
Просмотров 4373 месяца назад
In this video I speak with Felix GV, who is a Principal Staff Engineer at Linkedin, and has done major contributions to the data infrastructure and Linkedin, including VeniceDB. This episode will give you a good understanding of why we need a new database for storing "Derived Data" in a low latency, high performance manner, which is very important for Machine Learning workloads. Chapters: 00:00...
SuperCharging PostgreSQL for Search and Analytics - ParadeDB (Philippe Noël)
Просмотров 18 тыс.3 месяца назад
In this video I speak with Philippe Noël, about ParadeDB, which is an Elasticsearch alternative built on Postgres, modernizing the features of Elasticsearch's product suite, starting with real-time search and analytics. I hope you will enjoy and learn about the product. Chapters: 00:00 Introduction 01:12 Challenges with Elasticsearch and the Need for ParadeDB 02:29 Why Postgres? 06:30 Technical...
Modern OLAP Database System Design with FDAP (Andrew Lamb)
Просмотров 14 тыс.3 месяца назад
In this video I speak with Andrew Lamb, Staff Software Engineer @Influxdb. We discuss FDAP (Flight, DataFusion, Arrow, Parquet) stack for modern OLAP database system design. Andrew shared some insights into why the FDAP stack is so powerful in designing and implementing a modern OLAP database. Chapters: 00:00 Introduction 01:48 Understanding Analytics: Transactional vs Analytical Databases 04:4...
The ultimate multi-model Database, SurrealDB with Pratim Bhosale
Просмотров 7223 месяца назад
The ultimate multi-model Database, SurrealDB with Pratim Bhosale
Demystifying Real-time Analytics, Search and Hybrid Search with Dhruba, CTO @Rockset
Просмотров 3,6 тыс.4 месяца назад
Demystifying Real-time Analytics, Search and Hybrid Search with Dhruba, CTO @Rockset
Rapidly Simulate Production Traffic ft. Michael Drogalis
Просмотров 5564 месяца назад
Rapidly Simulate Production Traffic ft. Michael Drogalis
Demystifying Stream Processing Fundamentals
Просмотров 6894 месяца назад
Demystifying Stream Processing Fundamentals
High Performance with GraalVM - Alina Yurenko
Просмотров 5 тыс.4 месяца назад
High Performance with GraalVM - Alina Yurenko
Taming TimeSeries Data with QuestDB - Javier Ramirez
Просмотров 3894 месяца назад
Taming TimeSeries Data with QuestDB - Javier Ramirez
Beat the CAP Theorem : Make Distributed consistency simple
Просмотров 1,5 тыс.5 месяцев назад
Beat the CAP Theorem : Make Distributed consistency simple
KuzuDB: The Future of Graph Databases (w/ The Founder of KuzuDB)
Просмотров 8625 месяцев назад
KuzuDB: The Future of Graph Databases (w/ The Founder of KuzuDB)
Restate - making distributed systems simple with Stephan Ewen
Просмотров 6015 месяцев назад
Restate - making distributed systems simple with Stephan Ewen
Volt Active Data: Low Latency Stream processing
Просмотров 4635 месяцев назад
Volt Active Data: Low Latency Stream processing
Stream Processing 101 - An Introduction
Просмотров 1,2 тыс.6 месяцев назад
Stream Processing 101 - An Introduction
TigerBeetle: World’s Fastest Financial Transactions Database
Просмотров 2,5 тыс.6 месяцев назад
TigerBeetle: World’s Fastest Financial Transactions Database
Clean Code Adventures with Uncle Bob
Просмотров 3,5 тыс.6 месяцев назад
Clean Code Adventures with Uncle Bob
Durable Async-Await with Dominik Tornow
Просмотров 6236 месяцев назад
Durable Async-Await with Dominik Tornow
Observability Engineering with Liz Fong-Jones
Просмотров 1,2 тыс.7 месяцев назад
Observability Engineering with Liz Fong-Jones
Messaging and Streaming with Apache Pulsar - with Matteo Merli
Просмотров 6707 месяцев назад
Messaging and Streaming with Apache Pulsar - with Matteo Merli
VictoriaMetrics Internals with Alex and Roman @victoriametrics
Просмотров 1,6 тыс.7 месяцев назад
VictoriaMetrics Internals with Alex and Roman @victoriametrics
TiDB internals with Li Shen
Просмотров 5947 месяцев назад
TiDB internals with Li Shen
AI Powered Database optimisation with Andy Pavlo, Ottertune
Просмотров 12 тыс.8 месяцев назад
AI Powered Database optimisation with Andy Pavlo, Ottertune
Timeplus Streaming and Analytics with Jove Zhong
Просмотров 3208 месяцев назад
Timeplus Streaming and Analytics with Jove Zhong
Clickhouse Internals with Tom and Tyler
Просмотров 2,3 тыс.8 месяцев назад
Clickhouse Internals with Tom and Tyler

Комментарии

  • @user-gb7cc3fv8c
    @user-gb7cc3fv8c 29 дней назад

    @TheGeedNarrator Great content, thanks! However, requesting you to not highlight the current word in the closed captions. It is distracting, especially in the selected color scheme. Had to zoom-in the browser window till the captions disappear.

    • @TheGeekNarrator
      @TheGeekNarrator 29 дней назад

      Thanks for watching and sharing your feedback. I have addressed that in all my recent videos.

    • @user-gb7cc3fv8c
      @user-gb7cc3fv8c 28 дней назад

      @@TheGeekNarrator Great, thanks! Please keep up the good work. Already subscribed!

    • @TheGeekNarrator
      @TheGeekNarrator 28 дней назад

      🙏🏻 thanks. Really appreciate it

  • @adityakirankorlepara4500
    @adityakirankorlepara4500 Месяц назад

    Brilliant one. I used to work on ES in FAANG, and can totally relate to this one

  • @prabhagaikwad4849
    @prabhagaikwad4849 Месяц назад

    Always lot of things to learn from Arpit 👆👆

  • @LeoLeo-nx5gi
    @LeoLeo-nx5gi Месяц назад

    Truly awesome insights!! Thanks a lot, these are really informative (love to understand more like these)

  • @adityakirankorlepara4500
    @adityakirankorlepara4500 Месяц назад

    Absolutely loved the conversation. Thank you

  • @abhilashbandi3866
    @abhilashbandi3866 Месяц назад

    Dhruba seems such a chill guy. Very passionate about his work too. Amazing podcast.

  • @AbhishekTripathi-o1j
    @AbhishekTripathi-o1j Месяц назад

    Very technical and insightful. Thanks for bringing great folks to the podcast. 🎉

  •  Месяц назад

    Is there a version without the blinking text at the bottom ? It is extremely distracting

    • @TheGeekNarrator
      @TheGeekNarrator Месяц назад

      Sorry for that. I have removed the captions from all my recent videos. Here is an alternative you can choose to listen, but watching is the same experience. podcasters.spotify.com/pod/show/kaivalya-apte/episodes/TigerBeetle-Worlds-Fastest-Financial-Transactions-Database-e2g5qur

    •  Месяц назад

      @@TheGeekNarrator Thank you ! The content is very good, i moved the video frame down so its out of sight. Btw the tech behind the subtitles is awesome and quality is great, it was just very distracting when trying to just watch.

  • @sudhanshusharma7975
    @sudhanshusharma7975 Месяц назад

    Great podcast, hats off to you for doing this consistently. One suggestion is to have some kind of architecture diagram maybe to help understand better while explaining things. I know it could be too much to ask but if it's feasible please try to do that.

    • @TheGeekNarrator
      @TheGeekNarrator Месяц назад

      Thanks Sudhanshu. Your feedback is noted.

  • @eliuddyn
    @eliuddyn 2 месяца назад

    Amazing 🔥🔥

  • @deepthich2543
    @deepthich2543 2 месяца назад

    I saw a small video of tiered storage in StarTree channel but this video is something that is much needed for in-depth understanding of what’s going on. Kudos to Neha for explaining it so clearly.

  • @notrab
    @notrab 2 месяца назад

    Great interview!

  • @VipulVaibhaw
    @VipulVaibhaw 2 месяца назад

    the initial story about how turso was started was awesome!

  • @andreagiammarchi6110
    @andreagiammarchi6110 2 месяца назад

    why couldn't LiteFS just observe writes to .wal and propagate only those like you are doing at Turso? Also ... if I understood correctly the primary is location based so you'll have fast primary in Europe, as example, but writings from US will inevitably need to end up there first, right? I think this is also common for other DBs, but I just would like to be confirmed reads are always fast (once in each replica) but writes are also "primary location latency" based. Is this correct? Last, but not least, please don't talk trash on JS, it's pretty awesome after all :P

  • @NaveenSiddareddy
    @NaveenSiddareddy 2 месяца назад

    this is great product .. i will give it a try.. also get quine folks @TheGeekNarrator . think of graph with triggers

  • @invgreat5608
    @invgreat5608 2 месяца назад

    Great video, Thank you!

  • @AngelMarquez-on8bo
    @AngelMarquez-on8bo 2 месяца назад

    ❤❤❤❤ great talk!

  • @ronyarmon210
    @ronyarmon210 3 месяца назад

    Great episode, highly informative regarding search, RDBMs, abd everything in between

  • @nosh3019
    @nosh3019 3 месяца назад

    Great episode, I’m a fan of arrow and datafusion

  • @LeoLeo-nx5gi
    @LeoLeo-nx5gi 3 месяца назад

    This was completely amazing!! Thanks a ton

  • @mst2421
    @mst2421 3 месяца назад

    What is the name of the app you using for drawings?

  • @VipulVaibhaw
    @VipulVaibhaw 3 месяца назад

    Super cool

  • @patrykkrzyzanski2159
    @patrykkrzyzanski2159 3 месяца назад

    Everything wonderful especial the quest. But please do not include this annoying subtitles. When I will then I enable the once from RUclips. You're can I not disable. And when I am trying to concentrate on the story, they really annoing

    • @TheGeekNarrator
      @TheGeekNarrator 3 месяца назад

      Thanks for watching and your feedback. This has been a feedback by many folks so I have removed the captions from the latest episodes. I hope this won’t be a problem going forward.

    • @yosiyosiro3866
      @yosiyosiro3866 3 месяца назад

      ​@@TheGeekNarratori disagree to him, the subtitle helps a lot for non-english speaker in understanding the podcast. If you will, at least, please insert it in the caption feature (not the auto-translate one), so we can still activate or deactivate the subtitle

  • @thecloudbaba8668
    @thecloudbaba8668 4 месяца назад

    Absolute fruitful discussion, Loved it. looking forward for master Template of Envelope of Estimation :)

  • @AtharvaRao0104
    @AtharvaRao0104 4 месяца назад

    What a Masterclass with the Master! Its a great summary of cassandra documentation and parts of it are covered in DDIA. Also, observe how the master crafts concise and clear explanations to the questions. great questions too Kaivalya - Loved it

  • @sampleshawn5380
    @sampleshawn5380 4 месяца назад

    thanks a lot for this one, keep going, you are doing really well

  • @VolodymyrPavlyshyn
    @VolodymyrPavlyshyn 4 месяца назад

    cozodb is also interesting

  • @AminSoftwareEngineer
    @AminSoftwareEngineer 4 месяца назад

    Thank you, looking forward for more videos about Cassandra under layers

  • @user-yv9fc4nb8s
    @user-yv9fc4nb8s 4 месяца назад

    Very good introduction. I definitely want to try your product!!!

  • @claytonstangeland
    @claytonstangeland 4 месяца назад

    Excellent in-depth information.

  • @jiekun-dev
    @jiekun-dev 5 месяцев назад

    Hi Can I re-upload this video to Bilibili with unmodified description and credits?

    • @TheGeekNarrator
      @TheGeekNarrator 5 месяцев назад

      Hi, Sure. Could you please send me an email at speakwithkv@gmail.com with more details?

  • @AryanSharma-ys7gr
    @AryanSharma-ys7gr 5 месяцев назад

    Awesome Content! Subscribed in a heartbeat, pun intended.

  • @ksramchandani
    @ksramchandani 5 месяцев назад

    What is the corresponding in-memory update for the log that is written? Log is also written to disk or is it in memory and then flushed to disk? What happens when log flushing to disk fails?

    • @TheGeekNarrator
      @TheGeekNarrator 5 месяцев назад

      A Log is flushed to disk, yes. Databases like Postgres also support fsync mode which waits for the log be flushed to the disk, which adds a little performance impact (but nothing comparable to updating data pages and indexes directly). Not using fsync mode OR in simple terms, not waiting for the OS to flush the log to the disk may result in lost committed transactions. Typically a database dealing with concurrent transactions can write log entries to disk with a single fsync which is very efficient. So choose your tradeoff, but the thumb rule I use is, enable fsync by default to ensure maximum reliability and tweak if performance ever become a problem. Does that answer your question?

    • @ksramchandani
      @ksramchandani 4 месяца назад

      @@TheGeekNarrator Yes. Thank you! Wasn't aware of the DB option to wait for log flushing to succeed.

  • @hamzaouni3009
    @hamzaouni3009 5 месяцев назад

    This was a great video, thanks a lot

  • @bkvinayakkumar9660
    @bkvinayakkumar9660 5 месяцев назад

    First time ever enjoyed listening to tech like a story felt living in that time... Fabulous! both of you.. The silly mistakes you covered were really spot on with that idea of taking baby steps I could feel the importance of it more now.. this is my take away

  • @sbera87
    @sbera87 5 месяцев назад

    Where is the commit log stored? I assume disk. If so, whats the cost of flushing data to disk vs appending to log on disk?

    • @TheGeekNarrator
      @TheGeekNarrator 5 месяцев назад

      Yes its on the disk. Appending to log is cheaper because it is sequential and hence no random access. Flushing actual data to disk requires random access which is slower and requires tons of IO depending on what you are inserting/updating/deleting.

    • @akashgoyal2567
      @akashgoyal2567 4 месяца назад

      But log is then also prone to getting lost in case of server crashing since they are stored in in-memory before being flushed to disk . What’s the advantage of creating log then structure then ?

    • @TheGeekNarrator
      @TheGeekNarrator 4 месяца назад

      @akashgoyal2567 Typically if durability is important, you would fsync the log (Databases have config for that) which means the log is persisted to the disk. Since it is a sequential write it is way faster and more importantly when DBs have high concurrency one fsync call can be used to write 100s if not 1000s of transaction log to disk. That’s when it becomes really light weight as compared to updating data pages. In short log isn’t lost if you use the right configuration.

  • @NaveenSiddareddy
    @NaveenSiddareddy 5 месяцев назад

    I think we are talking about radio technology here philosophically. i listen to radio and think world is in order or not

  • @vignaanjunior382
    @vignaanjunior382 5 месяцев назад

    is pub/sub and Kafka does the same ?

    • @TheGeekNarrator
      @TheGeekNarrator 5 месяцев назад

      Yes Kafka extends the Pub-Sub model of messaging.

    • @vignaanjunior382
      @vignaanjunior382 5 месяцев назад

      ok, thanks for the reply and I did subscribe to your channel. Thanks for your content@@TheGeekNarrator

  • @koteshwarraomaripudi1080
    @koteshwarraomaripudi1080 5 месяцев назад

    Loved the podcast. Questions and answers were spot on. 👌

  • @KhaPiano
    @KhaPiano 5 месяцев назад

    Great introduction - thanks!

  • @bhaskarchiru
    @bhaskarchiru 5 месяцев назад

    Very informative video. Wish we have few minutes explaining on-disk layout of data

  • @aus10d
    @aus10d 5 месяцев назад

    This was a really fantastic conversation. It was so interesting to hear so many of the details behind tiger beetle. And Joran is a very very good storyteller and the way he explained everything. Just made it a pleasure to listen to. This project is really exciting. Can't wait to see where and how it gets deployed across the world!

  • @TymurYarosh
    @TymurYarosh 5 месяцев назад

    Very insightful! For me, the main selling point of Restate is durable execution. However, how Stephan and team hide complexity of execution, communication and persistence into a single platform is a very promising approach to building distributed systems of the new era.

  • @riteshsingh112
    @riteshsingh112 5 месяцев назад

    Nice talk and Great initiative. Question about the write path where Broker writes on the page cache. So that means that if leader node fails, data in OS cache will be lost. Does that means that storage devices being used should have the power supplies that can at least let kernel flush all it relevant caches to devices.

  • @jbeasley1880
    @jbeasley1880 5 месяцев назад

    'Promo sm' 😚

  • @tedspens
    @tedspens 6 месяцев назад

    Dude, he's drunk 🍾🍷😅

  • @oussamaabdoune4170
    @oussamaabdoune4170 6 месяцев назад

    love how he is so energetic and humble thanks for sharing this information

  • @emmanuelbyiringiro7207
    @emmanuelbyiringiro7207 6 месяцев назад

    I like how is so energetic at his age. You can hate his clean code opinions not his personality

  • @swanv951
    @swanv951 6 месяцев назад

    Did support for these transactions make it to Cassandra 5.0? I couldn't find it in the list of new features.

  • @mohamedoueslati462
    @mohamedoueslati462 6 месяцев назад

    Thank you for this introduction :)