How Discord Reduced Traffic Bandwidth by 40%

Поделиться
HTML-код
  • Опубликовано: 25 ноя 2024

Комментарии • 160

  • @kikisbytes
    @kikisbytes  27 дней назад +182

    This video took 6,089 lines of code to bring to life. Here’s the breakdown if you’re curious:
    S1: 507
    S2: 386
    s3: 285
    S4: 546
    S5: 771
    s6: 414
    S7: 531
    s8: 572
    s9: 783
    s10: 598
    S11: 256
    S12: 219
    s13: 221
    And it's built using Motion Canvas - motioncanvas.io/

    • @faytruefireside
      @faytruefireside 27 дней назад +7

      What program and language are you using?

    • @NiFreZ
      @NiFreZ 26 дней назад +1

      @@faytruefireside good question

    • @unclejoe8310
      @unclejoe8310 26 дней назад

      Manim

    • @trashcan3958
      @trashcan3958 26 дней назад

      ​@NiFreZ yeah I'm also interested

    • @kikisbytes
      @kikisbytes  26 дней назад +5

      I use motion canvas to make my animation!
      motioncanvas.io/

  • @MelroyvandenBerg
    @MelroyvandenBerg 26 дней назад +1172

    Trust me, engineers don't get the bonus for this

    • @HolyOllie
      @HolyOllie 26 дней назад +169

      youre right, they get paid 300k a year. this is literally their job. a surgeon doesnt get a bonus at every successful heart transplant

    • @kikisbytes
      @kikisbytes  26 дней назад +78

      one can dream hehe. Exceeded performance and get some extra cash into 401k / RRSP

    • @Catge
      @Catge 26 дней назад +51

      I saved my company 30k a year in cost with some optimizations and I all got a nice job.

    • @strawbaria
      @strawbaria 25 дней назад +56

      @@HolyOllie Who the hell gets paid 300k unless they're staff+?

    • @Sieabahlpark
      @Sieabahlpark 25 дней назад +27

      They literally get paid with stock, if discord makes more money their stock is worth more. They can then sell it when discord goes public and get paid. smh

  • @Linkyboy
    @Linkyboy 25 дней назад +527

    "I hope the engineers get a big bonus for this" - Yeah probably, a big bonus time off to look for a new job after the layoffs.

    • @kikisbytes
      @kikisbytes  25 дней назад +24

      that would be very upsetting :(

    • @MelroyvandenBerg
      @MelroyvandenBerg 23 дня назад +6

      True story.

    • @thegrandnil764
      @thegrandnil764 7 дней назад +5

      @@kikisbytes "I love capitalisim" tech bros hgahahahahaha

  • @Hacking-Kitten
    @Hacking-Kitten 26 дней назад +453

    The bonus went to management for sure 😂

    • @kikisbytes
      @kikisbytes  26 дней назад +10

      hahah yea....one can dream

  • @vilian9185
    @vilian9185 26 дней назад +213

    If they ban half the predators there it would also half the traffic lmao

    • @Reyhz
      @Reyhz 25 дней назад +30

      And half those pradators pays for nitro, they'd also halve their revenues

    • @simdimdim
      @simdimdim 25 дней назад +14

      They might end up banning themselves

    • @vilian9185
      @vilian9185 24 дня назад

      @@simdimdim LOL

    • @ropoxdev
      @ropoxdev 7 дней назад

      So you mean everyone on Discord is predators?

    • @autumnblaze6267
      @autumnblaze6267 6 дней назад

      @@Reyhz I'd say banning 1/2 of the predators would cut more than just 1/2 of the revenue, more like 3/4+
      ya assume predators and non-predators buy Nitro at (roughly, we're all doing very vague estimations here, so it's not about the precision) the same rate, but I'd say: what normal person buys Nitro? it seems useless for normal people who just want to have a chat with friends or participate in a kinda forum every once in a while, you need to be a bit of a no-life to pay for fluff like extra emojis or whatever, it's really for the minority of users for whom Discord is their whole life AKA mostly predators, lol - now that I think about it, the profit cuts would prolly be even worse, it's probably more like "1% of predators make up 90% of the Nitro purchases", lol
      [I'm just joking, ofc, no hard feelings]

  • @namaefumei
    @namaefumei 26 дней назад +38

    Its fascinating to see what a big company does to implement a new feature or a fix. So many factors, so many elements to check for.
    Great video. Thanks.

    • @noobgam6331
      @noobgam6331 26 дней назад

      No. It's fascinating to see how people are unaware of zstd existing that was released 10 years ago and I haven't seen anyone who has every worked with logs being unaware of it and actually using gz/zlib for anything where bandwidth/compressed file size is actually important

    • @kikisbytes
      @kikisbytes  26 дней назад +1

      thank you for watching!

    • @autumnblaze6267
      @autumnblaze6267 6 дней назад

      @@noobgam6331 nah, I just send the full raw data, all the music is in WAV, images in BMP (hell, the background is just one color, but I don't just send CSS with background color, you get an 8k uncompressed BMP image of #ffffff, you can check every pixel individually, each one was served with love and care - hell, my engineers are working on a new system where we'll send everything as a JSON/plaintext file in the background, images will just be a giant array with all RGBA values going like [[255, 255, 255, 1], [255, 255, 255, 1]] and so on and then a giant JS script with lots of dependencies will render the images on runtime, it'll be awesome, we have the best pixels out there, it improves accessibility cos every user can just open their Dev Tools and have all the pixels laid out in a human-readable format (images aren't human-readable, they make my eyes hurt), they can read it like a novel, ofc we send all the multimedia first and only once the user loads everything (we'll test it with a checksum), we send the actual content of the website - I need to be sure users experience everything in all its glory, not loading everything ruins the user experience, the new system helps to fight all the hacks like using wget or telling the browser to just download the plain HTML file, now it'll all be sent as JSON on fly, you need to have JS enabled and actually load everything, the script keeps sending randomized unique hashes to check if you're really downloading everything and established a proper session instead of trying to just download the text, ruining your user experience in the process), all with very high bitrates and resolution - my users deserve only the best: the most pixels, the most waves, the most data!
      if you can't access the site, tough luck bucko, buy a better internet connection or if it's on the server due to the load, wait for another time of the day, check the downtime info, estimate based on the timezones of the users and try to access it during the few hours in a day when the servers aren't overwhelmed - the very limited exclusive access only adds to the perceived luxury, it's only accessible to the people with the best internet connection during only a few exclusive hours of the day (if you're from the US, tough luck bucko, you'll need to stay up all night and keep refreshing; in fact, I'm considering closing off the website for the Shabbat [from Friday's sunset to Saturday's sunset, according to the Israel timezone (the servers will be down) + an additional JS script blocking it according to your local timezone] to be respectful towards religious Jews), people like the feeling of being the select few who managed to access and see the website (all bots are blocked, ofc, no archiving, no cache), the exclusivity makes it precious, we're the Apple of the webdev world: only the select few, dedicated rich people, get to see it, but once you access it, you get the best experience, the website runs super smoothly cos no decompressing is happening on the user's side, everything's pre-rendered and sent as raw data, so it's rendered instantly, a super smooth user experience

  • @dingus1720
    @dingus1720 26 дней назад +135

    exceptional video quality, unfortunately i think i'm too orange cat brain to understand compression 💀

    • @kikisbytes
      @kikisbytes  26 дней назад +7

      haha ty for taking the time to watch this video though! Any suggestions on how I can make it easier to understand?

    • @williamine
      @williamine 26 дней назад +9

      @@kikisbytesI didn’t find it particularly hard to understand, and I feel like you did a pretty decent job of explaining compression. I do that wish you went into more depth about what context is and how it’s used to improve the efficiency of streaming compression, though. Otherwise I thought the video was great!

    • @Skaffa
      @Skaffa 24 дня назад +5

      ​@@kikisbytestoo many technical terms were left unexplained to non-programmers i think.

    • @randoguy7488
      @randoguy7488 3 дня назад +1

      @@Skaffa Also there weren't enough pauses. Felt like someone was reading a wall of text without line breaks or paragraphs

    • @kanjakan
      @kanjakan День назад +1

      @@kikisbytes randoguy7488 is right on the money. Your video is great, but not very engaging. The lack of pauses and monotone voice give off a school presentation vibe (no offense). Other than that, the content of the video itself was fantastic and I love seeing deep dives into postmortems like this.

  • @Jamesaepp
    @Jamesaepp 22 дня назад +65

    5:48
    Woooooow, almost like reducing the amount of data you send in the first place is more important than the compression you apply after the fact. Who could have guessed? Truly the most incredible engineering feat of all time.

    • @kronik907
      @kronik907 18 дней назад +4

      This was my takeaway as well lol

    • @wormsquirm2707
      @wormsquirm2707 12 дней назад +5

      You sound salty

    • @Jamesaepp
      @Jamesaepp 12 дней назад

      @@wormsquirm2707 a little. I take issue with the likes of discord taking away what was once a popular paradigm - people self hosting their VCs with software like ventrillo or team speak. Now all of this user data is going to a company - who knows how it will be used against us long term.

  • @dhillaz
    @dhillaz 26 дней назад +3

    I love the quality and pace of this video - engaging explanation of the underlying concepts, use case story, test outcomes, deployment strategy...all explained in 8 minutes!
    Edit: Plus citations and Chapter markers in description too! 🥳

    • @kikisbytes
      @kikisbytes  25 дней назад +1

      Thank you! Definitely a lot to cover so hopefully I was able to explain the most important concepts.

  • @AniketSingh-nx4ds
    @AniketSingh-nx4ds 26 дней назад +9

    Man please upload more regularly if possible, just love your content, especially the breakdown of architecture and design decisions of tech companies

    • @kikisbytes
      @kikisbytes  26 дней назад

      I'll try my best!! Might be a little slow with a full time job though, but definitely more to come!

  • @APDesignFXP
    @APDesignFXP День назад +1

    I really like those videos, very informative and interesting. You got a new subscriber.

  • @nessitro
    @nessitro 24 дня назад +3

    Really nice video!
    big bonuses -> large pizza party hahaha

  • @kowaihana
    @kowaihana 26 дней назад +8

    the goofy ahh sound effects are too contrasting with your bored/tired voice tbh

    • @kikisbytes
      @kikisbytes  26 дней назад +5

      Thank you for the feedback

  • @vihaanthebot429
    @vihaanthebot429 2 дня назад

    The animations look awesome!

  • @Treviath
    @Treviath 9 дней назад

    One thing discord could do to save bandwidth would be to hash all files and checking the hash against a file database before actually sending the file to the server. Firstly it would reduce bandwidth usage and secondly it would reduce storage usage.

  • @fgardt
    @fgardt 26 дней назад +5

    2:20 the resulting huffman codes for a and c are swapped

    • @kikisbytes
      @kikisbytes  26 дней назад +1

      ahhhh fk me. Thank you for catching this!

  • @lokylee7872
    @lokylee7872 27 дней назад +3

    Another great video! Keep up the good work!

  • @StingSting844
    @StingSting844 26 дней назад +35

    The video seems to intentionally skip fast while showing text content? You show a graph and then before even i could glance at the axes, it's gone.

    • @kikisbytes
      @kikisbytes  26 дней назад +18

      Thank you for the feedback! I usually make the animations of how I envision the scene to look like then I try to match it with the voice recording. I find that this 2-3x my development speed. This means that sometimes the animation can be fast because the voice recording section is a lot shorter than I had anticipated.
      Will try my best to leave another second or two for you to absorb the graphs.

    • @StingSting844
      @StingSting844 26 дней назад +5

      @@kikisbytes I'm glad to know it's not intended. Great content 👍

    • @Asdayasman
      @Asdayasman 23 дня назад +2

      @@kikisbytes I thought it was perfect, even at 2x. Honestly really good pace - usually I start a video and scroll to the commemts while the idiot talking takes 9x longer than he needs to to say anything, but this had me actually watching the video. Thank you.

  • @wamellow
    @wamellow 26 дней назад +1

    really good creation! I just love such types of videos

    • @kikisbytes
      @kikisbytes  26 дней назад +1

      Will definitely!! Thank you for tuning in!

  • @lakshmanshankar
    @lakshmanshankar 26 дней назад +2

    Top quality content, do make more like this bro

    • @kikisbytes
      @kikisbytes  26 дней назад

      Thanks! Glad you enjoyed it, more coming soon!

  • @dacjames
    @dacjames 24 дня назад +3

    Yeah, engineers won’t be getting bonuses for doing their job.

  • @arthurguedes2461
    @arthurguedes2461 5 дней назад

    Such an amazing Video! Thanks

  • @RupamKarmakar-s8z
    @RupamKarmakar-s8z 26 дней назад +4

    Please make more and more videos of big tech company's system design architecture 🎉🎉🎉🎉🎉

  • @pardal_bs
    @pardal_bs 26 дней назад

    Great video. I loved the little animations. What software do you use to make them?

  • @AdamBrown-dr7vd
    @AdamBrown-dr7vd 25 дней назад +13

    they reduced traffic by making the app shitty 👍

  • @WillFuI
    @WillFuI 9 дней назад +1

    This is cool, but I’m still upset. I can’t upload 25 MB files anymore.

    • @SlavTiger
      @SlavTiger 5 часов назад

      i cant even send an pic from my phone without compressing it first now

  • @radhithyarkondaveeti
    @radhithyarkondaveeti 22 дня назад

    Good video. But, I thought youll would talk more than the compression algorithms, aren't they only used for the files? Just a question.

  • @MikoSciencia
    @MikoSciencia 10 часов назад

    I ain't no math guy but that thumbnail makes it look like more than 40%

  • @graemepicataggio9814
    @graemepicataggio9814 10 дней назад

    They got a pizza party and a nice work pal🎉

  • @MightyElemental
    @MightyElemental 7 дней назад +1

    It's a shame discord is crap now, especially the android app

    • @eyepatchonmyear1318
      @eyepatchonmyear1318 2 дня назад

      I get so many glitches during voice calls it's ridiculous. UI buggy as hell

  • @simdimdim
    @simdimdim 25 дней назад +2

    They did it by failing to send 40% of my messages lately

    • @kikisbytes
      @kikisbytes  24 дня назад

      ahahahah I had a good laugh at this!

  • @annyone3293
    @annyone3293 21 день назад +3

    By getting banned in Russia, haha!

  • @eliyahubasa9401
    @eliyahubasa9401 23 дня назад

    Great video, thank you

  • @gabrielverle9469
    @gabrielverle9469 27 дней назад +2

    good content!

    • @kikisbytes
      @kikisbytes  26 дней назад

      Thank you for watching!

  • @ayo4008
    @ayo4008 26 дней назад

    The timing of this right after Russia ban of Discord is just perfect

    • @kikisbytes
      @kikisbytes  26 дней назад

      oh actually? TBH I didn't even know that was happening.

  • @Stefan-qk8sw
    @Stefan-qk8sw 6 дней назад

    tldr: They switched libraries and removed unnecessary payload.

  • @dan2800
    @dan2800 26 дней назад +4

    Classic W switching from zlib to zstd

  • @skylark.kraken
    @skylark.kraken 26 дней назад +6

    0:08 costs per what? Week? Day? Hour? It's not cumulative, it's not monthly, the points don't line up with months and looks like it's every 2 weeks but that's a weird unit of time

  • @RupamKarmakar-s8z
    @RupamKarmakar-s8z 26 дней назад +1

    please make more and more, big tech company videos

  • @vippy40
    @vippy40 26 дней назад

    Good video, great hook

  • @lightning_11
    @lightning_11 12 дней назад

    Wait, people still optimize their electron apps? Truly a company of cuture.

  • @LawZist
    @LawZist 24 дня назад

    isn't all this compressions are built in already in the streaming frameworks / protocols?

  • @nomadshiba
    @nomadshiba 23 дня назад

    6:06 yeah i realized that years before.
    i was like, "why?"

  • @notanoption1793
    @notanoption1793 6 часов назад

    And we're stuck to showing clips of maximum 7 seconds long instead of 15 -_-

  • @yofukashino_
    @yofukashino_ 5 дней назад

    5:18 company of culture with no linux support ;-;

  • @alexandrosang21
    @alexandrosang21 26 дней назад +2

    I'm not smart enough to be here 😄

    • @kikisbytes
      @kikisbytes  26 дней назад +1

      Well, your curiosity already proves you're more than smart enough to be here! 😊

  • @protogen_1414
    @protogen_1414 3 дня назад

    HEEELLL YEAHH got me rolling for some reason

  • @feritejakusuma8450
    @feritejakusuma8450 24 дня назад

    i'm here till the end although i don't understand a single thing 🙃😅

  • @iohin
    @iohin 26 дней назад +4

    6:17 will save you guys some time watching the first 6 min: they were sending useless crap to clients. They then stopped.

    • @kikisbytes
      @kikisbytes  26 дней назад +1

      hahha maybe this should be a short :p

  • @waste-of-time2584
    @waste-of-time2584 26 дней назад

    real clap. excellent

  • @TracyGal
    @TracyGal 5 дней назад

    40% reduction in traffic huh?
    Can I have my 25mb upload limit back discord?

  • @farpurple
    @farpurple 26 дней назад +4

    They forkwd it to github?? ON public repo??? They are angels!

  • @Cyclically
    @Cyclically 25 дней назад

  • @SlavTiger
    @SlavTiger 5 часов назад

    cant even send photos anymore, 10mb is too small for most modern camera photos. that change sucks.

    • @SlavTiger
      @SlavTiger 5 часов назад

      for the record, zip drives from the 90s went all the way up to 750mb, a floppy esk disk from 30 years ago can hold more than a discord attatchment, and nitro's 150mb limit isnt much of an improvement, I still have some photos considered too large to send.

  • @aarvlo
    @aarvlo 11 часов назад

    the first part of the video makes a lot of sense to me but 5:46 is baffling, why did they do it that way to begin with? seems wasteful and unnecessary

  • @hill2750
    @hill2750 24 дня назад

    I wish I could improve by 40%

  • @OhhCrapGuy
    @OhhCrapGuy 9 дней назад +1

    1:35 That's not exactly how it would compress, interestingly. The more efficient way is
    A B C (3, 6, EOF)
    This looks like "Write A, then B, then C. For the next 6 characters, copy the character that's 3 characters back from the position you're writing."
    (0 indexed)
    When you write the [3] letter, you're copying whatever letter is 3 positions back, [0]. Then for [4], [1] is 3 positions back. [5] is [2], and this is key:
    [6] is copied from [3], data that has not been written when we started copying!

  • @weshuiz1325
    @weshuiz1325 25 дней назад

    How to reduce bandwith:
    Ban more users

  • @45545videos
    @45545videos 2 дня назад

    Common zstd W

  • @klazzera
    @klazzera 9 дней назад

    discord lost all of its turkish users that can also skew data

  • @vlc-cosplayer
    @vlc-cosplayer 12 дней назад

    They banned PDF files? 🤔

  • @MAXderskibidirizzler
    @MAXderskibidirizzler 7 дней назад

    Interesting

  • @RamkrishanYT
    @RamkrishanYT 26 дней назад

    Neetcode pog

  • @Dactarium
    @Dactarium 5 дней назад

    Bc Discord banned in Turkey and Russia 😢

  • @Ktoyatakoiskazhimne
    @Ktoyatakoiskazhimne 10 дней назад

    They could reduce traffic bandwidth by even further with p2p networking.
    Just send CRC of an image, and then force clients to find the image by themselves with a timeout, if there's no image among the other peers, send one from server.
    So in a large servers, the more people saw some image, the more you have peers for this image

  • @alimahdi1012
    @alimahdi1012 25 дней назад

    2:21 I bet

  • @gibarel
    @gibarel 22 дня назад +1

    Some day discord will managed to reduce their bandwidth usage by 100% by making their clients so bloated and with every feature behind a paywall so everyone will just use something else.

  • @honkhonk8009
    @honkhonk8009 25 дней назад

    So thats why my discord randomly disconnects and is pure TRASH lately

    • @kikisbytes
      @kikisbytes  25 дней назад

      oh is that a consistent issue? Haven't experienced any disconnects

  • @jaysonp9426
    @jaysonp9426 21 день назад

    Anyone complaining about the engineers not getting their share... Go start your own company or accept that you're owned by someone else.

    • @yunis5097
      @yunis5097 7 дней назад

      Sure that’s how we solve problems, everybody should have their own company. That’s a great Idea, everybody should have their own country too, so they can make their own laws. At that point just have your own planet. That’s a stupid way to think. Just accepting obvious injustices and not unionising is the first mistake.

  • @RupamKarmakar-s8z
    @RupamKarmakar-s8z 26 дней назад +1

    Please make more and more videos of big tech company's system design architecture 🎉🎉🎉🎉🎉