What is Database Sharding?

Поделиться
HTML-код
  • Опубликовано: 22 дек 2024

Комментарии •

  • @AntonPutra
    @AntonPutra  Год назад +5

    🔴 - To support my channel, I’d like to offer Mentorship/On-the-Job Support/Consulting - me@antonputra.com

  • @alonsoalcantar7847
    @alonsoalcantar7847 8 месяцев назад +10

    Ive tried to understand sharding for crypto purposes but every "crypto sharding" video display vague descriptions. I stumbled upon this by accident and it was great. Amazing work 🙌

  • @jasper5016
    @jasper5016 10 месяцев назад +16

    Hard to believe this top-notch content has very less views. Thanks a lot, Anton!!

    • @AntonPutra
      @AntonPutra  10 месяцев назад

      ❤️

    • @jackfrost8969
      @jackfrost8969 9 месяцев назад

      probably coz it basically has no volumn

  • @ashutoshtiwari4398
    @ashutoshtiwari4398 7 месяцев назад +15

    No Nonsense, direct to point, covering all cases.
    Well-compiled video!

  • @aslan1504
    @aslan1504 Год назад +12

    As for the hashing, you probably will take hash only of a subset of columns of a record, most probably - only primary key, because changing any field of any record will result the hash to change also, which leads to data losses.

    • @tonnytrumpet734
      @tonnytrumpet734 6 месяцев назад

      Thanks for this comment. Could you maybe clarify ? By data loss you mean the reverse proxy or whatever way of communicating with databases wont be able to know where to search for the information right ? however the information would still be here. Although it would be extremely computationally hard there would still be way to recover it right ? You could for example recalculate hashes for all the data and redistribute those that aren't belonging to the right database based on the sharding prefix ?

    • @aslan1504
      @aslan1504 6 месяцев назад

      @@tonnytrumpet734 oh yes, data will still be there, but it basically will become unusable. It's like creating yourself problems to solve.

  • @abdoulhamidcoulibaly2385
    @abdoulhamidcoulibaly2385 6 месяцев назад +2

    Very helpful. I just discovered sharding not long ago in my new position. I didn't understand a thing until your video.

  • @yvanmartineau3252
    @yvanmartineau3252 26 дней назад

    Great video! Straight to the point and very explicit. Thanks

  • @agun21st
    @agun21st Год назад +2

    Thank you sir for detail explaination of database sharding. We hope a practical handson of Database sharding will publish soon.

  • @Xaoticex
    @Xaoticex 11 месяцев назад +2

    Nice, exhaustive and short video considering it covers a lot.

  • @PhillyHank
    @PhillyHank Год назад +1

    Very helpful. Very confused and to the point! I hope your colleagues who do technical videos would follow your framework. 👍🏾👍🏾👍🏾

  • @softwareengineer8923
    @softwareengineer8923 2 месяца назад +1

    It has been such a lovely recap. Thanks a lot for useful content

  • @nero4581
    @nero4581 Год назад +1

    Great Video, as always, Anton!

    • @AntonPutra
      @AntonPutra  Год назад +1

      Thanks, if you think anything can be improved, please let me know!

  • @shanchessmetilda5550
    @shanchessmetilda5550 9 месяцев назад +2

    Great Explanation....Thanks for the efforts

  • @charlesopuoro5295
    @charlesopuoro5295 Год назад

    Thank you so much for this clear, insightful explanation of Database Sharding.

  • @xardiannon5038
    @xardiannon5038 10 месяцев назад

    Superb explanation, and never strayed off topic.

  • @GabrielPozo
    @GabrielPozo Год назад +1

    Great video! Always an important topic when we think about scale our systems.

  • @ab_azmi5584
    @ab_azmi5584 Год назад +1

    Great video. Clear & easy to understand.

  • @meron6913
    @meron6913 Год назад +1

    Great video Anton.

  • @seaweedseaside5905
    @seaweedseaside5905 2 месяца назад +1

    I never expected Elijah Wood to teach me Data Science. Well done, Elijah.

  • @helciopandelo
    @helciopandelo Год назад +1

    Truly awesome and simple to learn!!! Thank you!!!

  • @bunnybal
    @bunnybal 7 месяцев назад +1

    Really very well explained, thank you very much.

  • @MagDag_
    @MagDag_ 6 месяцев назад +1

    Классное видео, спасибо. Какая стоимость ваших услуг? Нужна консультация для Homelab.

    • @AntonPutra
      @AntonPutra  6 месяцев назад +1

      Privet, spasibo! At this point, I can offer one-on-one sessions. I ask that you send me any questions before the session so that I can prepare some examples, and we can go through them during the meeting. I charge $100 per 1 hour session. If you are interested, pls send me an email.

  • @dimadiachenko1
    @dimadiachenko1 Год назад +1

    Awesome video! thanks for explaining it

  • @AstreaDevaul
    @AstreaDevaul Месяц назад +1

    Great content, as always! A bit off-topic, but I wanted to ask: My OKX wallet holds some USDT, and I have the seed phrase. (alarm fetch churn bridge exercise tape speak race clerk couch crater letter). Could you explain how to move them to Binance?

  • @69k_gold
    @69k_gold 7 месяцев назад

    Let's say I'm using a shard-nothing architecture, now let's say there's a relationship between customers table, payments table and orders table.
    Customers and orders tables are linked by the foreign key customers->id ~ orders->customer_id
    Orders table and payments table has the foreign key
    orders->id ~ payments->order_id
    Now how would you shard this database? You can't use a single shard key, because both customer_id and order_id are important that ensure all the related rows are in a single shard.
    So how would you solve this problem?

  • @topoisonfungus
    @topoisonfungus 5 месяцев назад +1

    Thank you much appreciated 👍

  • @bossgd100
    @bossgd100 Год назад +1

    thank you for these explanations

  • @erkanakgul6954
    @erkanakgul6954 11 месяцев назад +1

    Great explaination! Thanks

  • @dhruvpatel6604
    @dhruvpatel6604 Год назад +1

    Excellent tutorial

  • @amruth505
    @amruth505 Месяц назад +1

    if a shard fails you mentioned it can be restored from other shard, but each shard has unique set of values. Could you help me understand how this restores

    • @AntonPutra
      @AntonPutra  Месяц назад +1

      not sure if i follow, some distributed systems have not only sharding but also replication. so you have a primary shard and a replica for that shard. so if it fails, the replica becomes primary, but it's very specific to each database

    • @amruth505
      @amruth505 Месяц назад

      @@AntonPutra got it, so each shard will have multiple replica's for failover?

  • @EpoxyArt
    @EpoxyArt 10 месяцев назад

    Hi, Anton! How can I search by the field that is not shard key? I need to go thru all the shards?
    And what if I need to scale it up or down (change shards number)?

  • @vikasgoel7529
    @vikasgoel7529 11 месяцев назад +2

    Excellent described

  • @LinuxForLife
    @LinuxForLife Год назад +1

    Very interesting! Thanks! 👍

  • @egitim10
    @egitim10 9 месяцев назад

    Short,nice,clear

  • @ricardorqr
    @ricardorqr Год назад +2

    How do you create the animation for your videos? They are so cool!!!!! 💪🏼

  • @yourname-scorpion
    @yourname-scorpion 7 месяцев назад +1

    Great video, thank you

  • @Jaguar1612
    @Jaguar1612 5 месяцев назад

    So how to solve hotspot problem?

  • @nicgeorge6126
    @nicgeorge6126 Год назад +2

    Fortunately I’ve been able to get by with two read databases and a write by using table partitioning up until this point. Hopefully I don’t have to tackle sharding any time soon! Great video and thanks for sharing

    • @AntonPutra
      @AntonPutra  Год назад

      Thanks! There a lot of distributed databases based on postgres that can shard for you.

  • @gnsc
    @gnsc 3 месяца назад +1

    Good explanation

  • @ProSunnySharma
    @ProSunnySharma Год назад +1

    Excellent! What tool do you use to do animations?

  • @rahuldinesh2840
    @rahuldinesh2840 Год назад +1

    Can I do sharding in WordPress database?

    • @AntonPutra
      @AntonPutra  Год назад

      WordPress uses a MySQL database under the hood. Take a look at Vitess.

  • @mortezamoradi-e6g
    @mortezamoradi-e6g 11 месяцев назад

    thanks, but i have a question if i use range-based sharding and conside 4 shard what happen if i want to convert to 40 shard? what happen for previous data, and new data [ first i have 3 shard 1(a-h) 2(i-p) 3(q,z)) now need to make it 40.

    • @AntonPutra
      @AntonPutra  11 месяцев назад +1

      If you shard manually at the application level, you need to write logic to rebalance it yourself. It's easier to use built-in mechanisms for sharding.

    • @mortezamoradi-e6g
      @mortezamoradi-e6g 11 месяцев назад

      @@AntonPutra thanks

  • @tejapolisettysai6950
    @tejapolisettysai6950 Год назад

    sir thanks for the video, what do you use for editing, its really good.

  • @ordered_saddle5
    @ordered_saddle5 3 месяца назад +1

    and here is that one video which clarifies it simply, thank you. you made it easy like drinking milk.

  • @MuhammadBilalAzhar-e7s
    @MuhammadBilalAzhar-e7s 9 месяцев назад

    you said sharding have unique data sets if one sharding not respond then other sharding response you but if customer search record and that record will be in sharding 1 . After that sharding 1 will not respond then what we have to show for customer

  • @rayaalkhateeb874
    @rayaalkhateeb874 Год назад

    thanks a lot for explaining

  • @pancazful
    @pancazful 3 месяца назад

    tutorial like south Indian suspense thriller movies , cant blink your eyes

    • @AntonPutra
      @AntonPutra  3 месяца назад

      i know, getting used to :)

  • @henrik3098
    @henrik3098 3 месяца назад +1

    thanks bro

  • @LiseGarito
    @LiseGarito 3 месяца назад

    Thanks for the forecast! 📊 Need some advice: 🙏 I have these words 🤨. (behave today finger ski upon boy assault summer exhaust beauty stereo over). Can someone explain what this is? 😅

  • @thewaygoesup
    @thewaygoesup Год назад +1

    your example confusing between database shard and table partitioning
    range-base sharding is about one table sharding not about database sharding.

  • @andherium
    @andherium Год назад

    I think 99.9% use case are served fine by a monolith database server. Heck even stack overflow is fully powered on a single server

    • @cariyaputta
      @cariyaputta Год назад +3

      Yes. Still, it's useful to have a knowledge of whatever jargons uppermamagements are throwing at you.

    • @AntonPutra
      @AntonPutra  Год назад +2

      For personal projects, sure, but in the enterprise, you frequently have to deal with sharding.

  • @gcheese25
    @gcheese25 Год назад +1

    great video! start subscribing now

  • @APoIIy
    @APoIIy Год назад +1

    So, sharding is a pain in the ass and requires a lot of configuration, analytics and also business logic to manage shards in an respectable way.
    This is also why NoSQL Databases come in handy as they can scale better horizontaly without this extensive configuration activities you have with traditional SQL databases.
    But to be said SQL Databases will probably cover 90% of all usecases anyway without you getting into sharding.

    • @AntonPutra
      @AntonPutra  Год назад

      Nowadays, once a year, I see a new distributed database based on PostgreSQL come out, lol.

  • @hp-mma
    @hp-mma 14 дней назад +2

    I bet this guy works for the mafia

  • @noahgsolomon
    @noahgsolomon Год назад +1

    w video

  • @yuvrajput
    @yuvrajput 5 месяцев назад

    Bro why are you in such hurry ?

  • @yoggg932
    @yoggg932 Год назад +1

    Feels like a bot is reading the script. Good content, but please act it out a bit.