TigerBeetle
TigerBeetle
  • Видео 114
  • Просмотров 231 737
Episode 043: The End of the Compaction (Finally!)
Join live on Twitch on Thursday, 17:00 UTC at www.twitch.tv/tigerbeetle !
Source code: github.com/tigerbeetle/tigerbeetle/tree/IronBeetle
Playlist: ruclips.net/p/PL9eL-xg48OM3pnVqFSRyBFleHtBBw-nmZ
Просмотров: 175

Видео

Rethinking Authentication by Frank Denis
Просмотров 1,6 тыс.2 часа назад
In distributed systems, authentication is vital for ensuring security. However, traditional systems frequently struggle to provide adequate performance, and they often remain vulnerable to real-world attacks. We’re going to explore simple and practical authentication schemes that address these limitations, significantly enhancing performance, latency, and privacy while strenghtening overall sec...
Garden of Forking Paths by Alex Petrov
Просмотров 9977 часов назад
Traditional randomized testing, both in the field of Databases and outside of it, might be a somewhat inefficient way to hunt for bugs. However, in this talk, we’re going to empower you with the knowledge of how to make your tests cover more grounds. By controlling the inputs of random number generation, we can make our software try less of the same thing and instead, explore new, creative ways...
Iron Beetle 042: Compaction MERGE!
Просмотров 33516 часов назад
Join live on Twitch on Thursday, 17:00 UTC at www.twitch.tv/tigerbeetle ! Source code: github.com/tigerbeetle/tigerbeetle/tree/IronBeetle Playlist: ruclips.net/p/PL9eL-xg48OM3pnVqFSRyBFleHtBBw-nmZ
Systems Distributed '24 Lightning Talks
Просмотров 3,9 тыс.19 часов назад
Lightning talks from Systems Distributed ‘24 00:00 Intro 00:45 div, IDE, and Conquer by Loris Cro x.com/croloris 05:18 Zero Downtime: The Holy Grail of Infrastructure by Shivansh Vij x.com/confusedqubit 10:21 Sometimes You Don't Need A Distributed System by Amey Chaugule www.linkedin.com/in/ameychaugule/ 15:11 Data & Databases in Procedural Voxel Games by Nathan Bourgeois www.linkedin.com/in/nb...
5 Lessons from 5 Years of Building Databases at Scale by Sammy Steele
Просмотров 6 тыс.День назад
What is the fastest way to divide a group of systems engineers? Ask them to choose the best database. What actually unites most systems engineers? The idea that databases are hard. The CAP theorem isn’t just a theory. Database engineers deal with the realities of these trade offs every day. When debugging outages, they often simultaneously encounter scaling bottlenecks, availability impacts, ne...
Episode 041: Compaction Input
Просмотров 28014 дней назад
Join live on Twitch on Thursday, 17:00 UTC at www.twitch.tv/tigerbeetle ! Source code: github.com/tigerbeetle/tigerbeetle/tree/IronBeetle Playlist: ruclips.net/p/PL9eL-xg48OM3pnVqFSRyBFleHtBBw-nmZ
Being our own worst customer by Brian Lagoda & John Murray
Просмотров 1,1 тыс.14 дней назад
How do you deterministically test a deterministic testing system? After years of handpicking customers that were a good fit for our service, Antithesis faced our toughest customer yet: ourselves. We'll talk about why we did it, what went wrong, and what we learned along the way. Talk from Systems Distributed '24: systemsdistributed.com Join the chat at slack.tigerbeetle.com/invite
Systems Resurgent? by Amod Malviya
Просмотров 1,2 тыс.14 дней назад
The past 2 decades has seen technology take the center stage of life in modern society. Progress in accessibility of software development and open source, allowed one to produce useful software without necessarily understanding how systems worked underneath. Using anecdotes from my professional life, I do a review of the effect of these past developments on engineering culture, and make an argu...
Episode 040: Compaction Pipeline
Просмотров 33921 день назад
Join live on Twitch on Thursday, 17:00 UTC at www.twitch.tv/tigerbeetle ! Source code: github.com/tigerbeetle/tigerbeetle/tree/IronBeetle Playlist: ruclips.net/p/PL9eL-xg48OM3pnVqFSRyBFleHtBBw-nmZ
Bet Against SQL: Queries as Code by James Cowling & Sujay Jayakar
Просмотров 4,8 тыс.21 день назад
Traditional wisdom is to never bet against SQL but we’re here to convince you otherwise. The app development ecosystem accretes complexity year over year as a result of decades of bad design patterns and attempts to solve problems on the wrong layers of the stack. Consistent caching, dynamic subscriptions, and end-to-end type safety can all be provided automatically when databases are built wit...
Episode 039: Which Table to Compact?
Просмотров 24928 дней назад
Join live on Twitch on Thursday, 17:00 UTC at www.twitch.tv/tigerbeetle ! Source code: github.com/tigerbeetle/tigerbeetle/tree/IronBeetle Playlist: ruclips.net/p/PL9eL-xg48OM3pnVqFSRyBFleHtBBw-nmZ
Jepsen 15: What Even Are Transactions? by Kyle Kingsbury
Просмотров 4,7 тыс.28 дней назад
Databases often promise "ACID transactions", but what does that phrase actually mean? Using generative testing and fault injection techniques, we'll look at transaction safety in three systems which take different approaches not only to replication, but to transaction semantics themselves. We'll see how MySQL's "Repeatable Read" is not repeatable, and how MySQL clusters hosted in AWS could rout...
Distributed DDL Replication at Global Scale by Gwen Shapira
Просмотров 1,4 тыс.Месяц назад
When building SaaS, having multiple copies of the same database schema is common. Either because tenants are isolated to individual schema or for sharing. At Nile, we are building serverless Postgres for modern SaaS. We handle multi-tenant database design by isolating each tenant to a virtual Postgres database. We let users place the tenants on shared or dedicated compute nodes anywhere on the ...
Episode 038: The Number Of The Beat
Просмотров 754Месяц назад
Join live on Twitch on Thursday, 17:00 UTC at www.twitch.tv/tigerbeetle ! Source code: github.com/tigerbeetle/tigerbeetle/tree/IronBeetle Playlist: ruclips.net/p/PL9eL-xg48OM3pnVqFSRyBFleHtBBw-nmZ
Distributed Pure Functions by Richard Feldman
Просмотров 6 тыс.Месяц назад
Distributed Pure Functions by Richard Feldman
Lessons Learned from 2 Decades of Building Data Infra Products by Deepti Srivastava
Просмотров 2,3 тыс.Месяц назад
Lessons Learned from 2 Decades of Building Data Infra Products by Deepti Srivastava
Episode 037: Compaction Overview
Просмотров 322Месяц назад
Episode 037: Compaction Overview
Distributed Asynchronous Await • A new programming model for distributed applications - by D. Tornow
Просмотров 8 тыс.Месяц назад
Distributed Asynchronous Await • A new programming model for distributed applications - by D. Tornow
A Systems-Minded Approach to Creating a Music Player Application by Andrew Kelley
Просмотров 31 тыс.Месяц назад
A Systems-Minded Approach to Creating a Music Player Application by Andrew Kelley
Episode 037: The Mutable and the Immutable Tables
Просмотров 176Месяц назад
Episode 037: The Mutable and the Immutable Tables
Episode 036: A Cache That Always Hits
Просмотров 1,3 тыс.Месяц назад
Episode 036: A Cache That Always Hits
Durability and the Art of Consensus by Joran Dirk Greef
Durability and the Art of Consensus by Joran Dirk Greef
Episode 035: Write Path, the Beginning
Просмотров 5902 месяца назад
Episode 035: Write Path, the Beginning
Episode 034: Prefetching From Disk
Просмотров 9152 месяца назад
Episode 034: Prefetching From Disk
Episode 33: Prefetching From Memory
Просмотров 7622 месяца назад
Episode 33: Prefetching From Memory
Episode 2^5: Safety, Performance, Experience
Просмотров 7622 месяца назад
Episode 2^5: Safety, Performance, Experience
Episode 031 1/2: Journal vs. Checkpoint
Просмотров 4182 месяца назад
Episode 031 1/2: Journal vs. Checkpoint
Episode 031: The Forest and the Groove
Просмотров 1592 месяца назад
Episode 031: The Forest and the Groove
Episode 030: ManifestLog Compaction
Просмотров 1013 месяца назад
Episode 030: ManifestLog Compaction

Комментарии

  • @wireghoul
    @wireghoul День назад

    Insightful talk, I would love to see better alternatives to JWT become common. However server side sessions still seem like a solid alternative, with far fewer problems/attack surface than any of the signed token approaches discussed.

    • @pikzel
      @pikzel 3 часа назад

      Server side sessions are also a big headache, and more challenging to scale, especially cross-region.

  • @capability-snob
    @capability-snob День назад

    Fastly have done some fantastic work in this area, it's great to hear about it! I would like to warn about putting more than one capability into a credential. Once you do this, the lovely mathematical properties that object-capability systems grant you don't entirely hold, even if the weakness is nicely scoped. Being able to re-use one credential for a different resource is at the root of a wide variety of vulnerabilities, especially attacks involving enumeration or ibac.

  • @petenilson
    @petenilson День назад

    Isn’t the fact that he has a black card authorization and not authentication?

    • @capability-snob
      @capability-snob День назад

      This is a bit of a rabbit hole, as the distinction isn't always quite so clear. Yes, the card names a Subject, but at this point in the day, it's unlikely that staff are verifying that the human wearing the card is the subject so named. Indeed, we probably don't even care if they weren't - what we mostly care about is whether someone has paid the appropriate amount for a human to be there. The example more commonly used (and clearer) is the car key, but variety is the spice of life, I guess.

  • @Lemmy4555
    @Lemmy4555 3 дня назад

    yes but can i test a sum function with this thing? lol

  • @marvin_hansen
    @marvin_hansen 5 дней назад

    You have to look into meta data checksums in ZFS as it enables bitrot correction. Unlike Linux, Solaris and later BSD added ZFS exactly because of oblivious data loss caused by bitrot. And get this, ZFS was introduced with Solaris 10 a full decade before Fsyncgate… While you can’t easily update the IO used in a database, you can install a BSD distribution and format your database storage disk array with ZFS for handling all mission critical data. An enterprise would probably go with Oracle DB on Solaris . That said, for financial data, TigerBeetle is currently the single best option because not everyone has the time and expertise to gronk BSD / ZFS or the money for Oracle on Solaris.

  • @alasdairmacintyre9383
    @alasdairmacintyre9383 5 дней назад

    Dang we got a Borges reference... sign me up!

  • @mahkhi7154
    @mahkhi7154 8 дней назад

    You're Telling Me: Database Scaling - Because You Want Me to Believe its Cheap to Transmit Data on your Networks. I can come To America. build a Website in My Garage. Steal somethings from your Basement and Have a Successful E-Commerce Web Store. So You're Telling Me Database Scaling: "Come explore lessons learned from 5 years of building databases at petabyte scale at Dropbox and Figma" - You Want Me To Come for an Education at Dropbox/Figma. Where You'll Teach a Pr0stitute of Yours to Fu*ck Me.

  • @mahkhi7154
    @mahkhi7154 8 дней назад

    You're Telling Me: Database Scaling - To Hide or Prevent Me from Learning - How You've Built Your Satellite Broadband Networks.

  • @mahkhi7154
    @mahkhi7154 8 дней назад

    The Problem isnt Database Scaling. You Simply Shard the Database. Shard Means Split Up. Customers With surnames Beginning With A, are On Database 1. Customers With surnames Beginning With B are On Database 2. you Can Shard at The App Layer: PHP or ASP or You Can shard at The Database Layer: PGPool (Proxy DB). The Difficulty is Building The Network. You Have To Pay to Transmit Data on somebody else's Network. You've Turned Thailand into Slums to Make Your Satellite Broadband Networks. You Need Lots of Kerosine for your Top Gun Airforce (Jet engines) - You've Got Lots of Petrol/Diesel for Electricity And Piston Engine Aircraft. Such as a Cessna C172 or a Diamond DA42. Your Satellites are in The Clouds: Between 6,000 and 10,000 Feet. Piston Diesel Engines Work at that Altitude. You Need Kerosine for 15,000 Feet and above Jet Engines. The Difficulty is Building The Network. You Have To Pay to Transmit Data on somebody else's Network. You've Turned Thailand into Slums to Make Your Satellite Broadband Networks.

  • @troysperspective
    @troysperspective 8 дней назад

    rad Intro ;)

  • @capability-snob
    @capability-snob 8 дней назад

    Trying to abstract the client/server communication is a tale as old as MS ASP. It was leaky then, and it is leaky now. Much nicer to be explicit, and have to decide how to deal with network partition.

  • @UliTroyo
    @UliTroyo 8 дней назад

    Dang, I want to hear so much more about voxel data storage, what a great talk! Also I love Loris's HTML LSP; I'm already using it.

  • @nihil2501
    @nihil2501 8 дней назад

    juicy talks

  • @felipedelgado160
    @felipedelgado160 8 дней назад

    Ghostty ❤

  • @nathanfranck5822
    @nathanfranck5822 8 дней назад

    I love love love zig fmt's (zls') formatting rules for trailing commas resulting in each element on a new line. It's amazing. I get really frustrated everywhere else where this is not the default behavior of the formatter. Glad to see it coming to HTML :) Thanks loris!

  • @DanMackAlpha
    @DanMackAlpha 8 дней назад

    nice, high info density talk, very cool project

  • @loris683
    @loris683 8 дней назад

    Hi

  • @ikolomiets
    @ikolomiets 9 дней назад

    I enjoy Matklad's fluent code navigation and editing skills. Considering his deep knowledge in how to make IDE smart (LSP and rust-analyzer fame), I wish for special episode on his development setup, and effective use of VS Code for Zig programming and TigerBeetle.

  • @DoraClement-r9l
    @DoraClement-r9l 9 дней назад

    Lehner Views

  • @nathanaelmoh5848
    @nathanaelmoh5848 9 дней назад

    Every video I watch by this hero is amazing. Your contributions are invaluable.

  • @Kenbomp
    @Kenbomp 10 дней назад

    Well at least it works for its users

  • @Ffaarg7
    @Ffaarg7 10 дней назад

    Go slow to go fast. Love it!

  • @MartinCharles
    @MartinCharles 10 дней назад

    This is all true for transactional workloads

    • @MartinCharles
      @MartinCharles 10 дней назад

      This talk really glosses over a bunch of important details

  • @thrawn01
    @thrawn01 10 дней назад

    Big distributed systems are all about trade offs. I personally would not want to use PostgreSQL for such an in-house build, but if it makes sense, it makes sense. For this product, with these constraints, on this timeline, they did what was best for the product. Which is the way it should always be, but it often is not. Congratulations for a job well done! ♥️

  • @valeratrades
    @valeratrades 11 дней назад

    Amazing

  • @rotors_taker_0h
    @rotors_taker_0h 15 дней назад

    Does anyone have link to a "great talk about NES our CEO gave last year" at 10:35?

    •  15 дней назад

      It thinks it's this one: ruclips.net/video/m3HwXlQPCEU/видео.htmlsi=BSNzqbQxlm0TMEwB

    • @rotors_taker_0h
      @rotors_taker_0h 14 дней назад

      Thank you

  • @DeanBekkering
    @DeanBekkering 15 дней назад

    This video condenses 100s of arcane PhD research papers into a clear, concise, and entertaining 45-minute presentation any high schooler could understand. Unbelievable!

  • @YuFanLou
    @YuFanLou 15 дней назад

    10:59 I think SQL being a *query* language, it is maybe unfair to start off judging it with “hard to express application *behavior*”. It’s like judging a cake for not being savory. The fact that so many have been abusing SQL, which is designed to query for current time snapshot, to write time-varying behavior instead, is a testament to how good an abstraction it is at querying that it forces systems around it to conform to its shape.

    • @YuFanLou
      @YuFanLou 15 дней назад

      24:12 The point about performance predictability is cool, but this code example really does not show any superiority over SQL other than integrating with host language error reporting. The point is supposed to be something better than conversational transaction, but then it goes off into side effect management. Where is the demo for rolling back?

    • @maksimmuruev423
      @maksimmuruev423 5 дней назад

      Sql when you should put flat number before house.. this is not good for autocomplete and not intuitive.. general idea was use it by accountant.. but they wasn't happy with this instead devs are struggling.

  • @aus10d
    @aus10d 17 дней назад

    great talk. love the philosophy behind tigerbeetle. just awesome. you guys are killing it!

  • @kellymoses8566
    @kellymoses8566 19 дней назад

    A database is a base for your data. It is in the name!

  • @NostraDavid2
    @NostraDavid2 22 дня назад

    RE: Andy Pavlo Carnegie Mellon University has an _excellent_ freely available database course, presented by Andy BTW. "CMU Intro to Database Systems", here on YT.

  • @NostraDavid2
    @NostraDavid2 22 дня назад

    Current Relational Databases also completely ignore the concept of a sharable Domain; some DBs have them, but nobody is using them or teaching them either. For simplicity's sake think of a Domain as a datatype with extra constraints (either a subselection of specific values, or perhaps a range), that can be shared between columns (Column A from table X can have the same domain as column B from table Y).

    • @TapetBart
      @TapetBart 19 дней назад

      Clickhouse has something called Domains. I wonder kf that is the same.

  • @NostraDavid2
    @NostraDavid2 22 дня назад

    SQL is indeed a terrible language. I hope that in the next Stack Overflow Survey they'll introduce a separate "Query Language" question, instead of slapping SQL between regular programming languages. BTW, it's terrible because a Relational Query Language should ALWAYS return a Relation (a set of tuples), but SQL can return a table, a row, a column or a cell, which is kind of awful.

    • @YuFanLou
      @YuFanLou 15 дней назад

      but all of table, row, column, cell are indeed relations! lol I’ve come to accept that pragmatic QoL is a valid trade-off against conceptual simplicity speaking of relational language been looking at Alloy, albeit for modeling not querying

  • @NostraDavid2
    @NostraDavid2 22 дня назад

    I can't have you disrespecting The Coddfather (E. F. Codd) like that! The Relational Model was invented in 1969 (nice!) in a paper called "Derivability, Redundancy, and Consistency of Relations Stored in Large Data Banks", and took off in the second paper called "A Relational Model of Data for Large Shared Data Banks" in 1970. He also designed ALPHA in his paper "ALPHA: A Data Base Sublanguage Founded on the Relational Calculus", an unimplemented language that was based on Relational Calculus that could've been used to query data, instead of SQL, but alas. Oracle took SQL as basis and IBM had to follow suit if not to lose market share to Oracle, as the Relational Model was VASTLY superior to existing data types (trees and networks) that were being abused as databases - they were not even official models yet - The Relational Model predates whatever models IMS and IDS later on generated. There. Now I'm much more content.

    • @NostraDavid2
      @NostraDavid2 22 дня назад

      If you wish to check out all his papers (ordered in release date): thaumatorium dot com/articles/the-papers-of-ef-the-coddfather-codd/

    • @NostraDavid2
      @NostraDavid2 22 дня назад

      No ads, nor tracking on Thaumatorium BTW.

    • @JamesCowling
      @JamesCowling 21 день назад

      You won't find us taking Codd's name in vain over here! As far as abstractions go the RDBMS is one of the all-time greats. Transactions, relations, type safety and consistency are evergreen and will hopefully long outlive SQL.

  • @kellymoses8566
    @kellymoses8566 22 дня назад

    I worked at a company that uses Infor SyteLine ERP and the DBA was constantly complaining about all the long-running transactions it used and the issues they caused on SQL Server.

  • @kellymoses8566
    @kellymoses8566 22 дня назад

    Datalog is a fantastic query language.

  • @kellymoses8566
    @kellymoses8566 22 дня назад

    Git is a example of terrible abstractions.

    • @mattmurphy7030
      @mattmurphy7030 22 дня назад

      Explain?

    • @r.g.thesecond
      @r.g.thesecond 21 день назад

      @@mattmurphy7030 Well, perhaps the Git CLI. Its verbs don't help at all until you learn the underlying structure and use gitk a couple of times.

  • @trejohnson7677
    @trejohnson7677 22 дня назад

    this truly conveys that idea that databases are literally just shared space.

  • @keokawasaki7833
    @keokawasaki7833 22 дня назад

    The intro theme doesn't need to go this hard

  • @meme6793
    @meme6793 24 дня назад

    Chimera Linux mentioned!

  • @laughingvampire7555
    @laughingvampire7555 25 дней назад

    well, data is always immutable, what it is mutable is rw storage

  • @laughingvampire7555
    @laughingvampire7555 25 дней назад

    I wanna know if the Roc team will ever make a JavaScript platform or will Richard always say "Use Elm"

    • @asksage
      @asksage 9 дней назад

      I think he has said in one of the interviews that he is happy to continue to use Elm on the frontend.

  • @laughingvampire7555
    @laughingvampire7555 25 дней назад

    well, 8:25 thats obvious because when you parallelize you are adding communication problem, however, the reason you add concurrency is to get fault-tolerance and high availability.

    • @kellymoses8566
      @kellymoses8566 22 дня назад

      Supercomputers simply replace a compute limitation with a communication limitation.

  • @juliendebache8330
    @juliendebache8330 27 дней назад

    Great talk, great technique, great philosophy behind these techniques. Cheers to many more of these!

  • @nazikhuq
    @nazikhuq 29 дней назад

    Not sure this is the same guy almost took down mongodb & elastic pointing out data inconsistencies in write operations a decade ago

    • @shikharraje
      @shikharraje 29 дней назад

      Yup, that's him. I vividly remember because my entry into industry was around the same time when he published those findings. I've been wary of Mongo and Elastic ever since.

    • @aymanpatel5862
      @aymanpatel5862 29 дней назад

      can tou provide link to that talk

    • @paul_kemp
      @paul_kemp 24 дня назад

      @@aymanpatel5862 Look for Jepsen Analyses to find all the reports his company did, including for Elasticsearch and MongoDB.

    • @apslinux9056
      @apslinux9056 22 дня назад

      Also he had some conversations with redis

  • @YuruCampSupermacy
    @YuruCampSupermacy Месяц назад

    Lesssssgo

  • @danishmehmood6110
    @danishmehmood6110 Месяц назад

    waitingggg for this one

  • @canismajorisvy
    @canismajorisvy Месяц назад

    great talk, thank you very much

  • @Gilgamesh557
    @Gilgamesh557 Месяц назад

    Commutativity is not the important factor for the reduce function; assiciativity is.

  • @FranzBrummer
    @FranzBrummer Месяц назад

    Love the mixture of fun and useful info in Andrews talks