Reimplementing Apache Kafka with Golang and S3

Поделиться
HTML-код
  • Опубликовано: 31 янв 2025

Комментарии • 28

  • @harukostein
    @harukostein 8 месяцев назад +10

    I watched a few videos on this channel. This is the best channel about software engineering I've ever listened to. I'm like "wait, he didn't just ask that hard question?" then I'm "wait, the other one gave a great answer?". So much to learn here.

  • @marcusjpotter
    @marcusjpotter 7 месяцев назад +1

    Your such a refreshing thought provoker. Really appreciate your channel and your guests. Curious on S3 to replace Kafka. Watching now.

  • @FnordFandango
    @FnordFandango 8 месяцев назад +14

    This is a terrific topic - thank you

    • @vikingthedude
      @vikingthedude 8 месяцев назад +1

      Terrific is awfully close to terrifying

  • @BR-lx7py
    @BR-lx7py 8 месяцев назад +4

    My major takeaway is how crazy expensive inter-AZ traffic is at AWS. 43% of the total cost of running Kafka...

  • @scottpasnikowski6060
    @scottpasnikowski6060 8 месяцев назад +2

    Very useful for me as a dev dealing with Kafka and AWS

    • @peppybocan
      @peppybocan 8 месяцев назад +1

      AWS MKS is not a good option?

  • @drasticfred
    @drasticfred 8 месяцев назад

    More than a decade i always replicate and backup my audit logs through aws s3, via good old kafka-ish trigger based replication approach, similar to this.☺

  • @nexovec
    @nexovec 8 месяцев назад +16

    My minio is going strong, I don't understand why that wouldn't be cloud native. Why does everything have to be some kind of thinly provisioned resource? The program still runs on a computer, and where exactly it runs is often super important.
    Storage as a service, network as a service, cpu as a service, your mom as a service. I'm tired of this.
    I feel so sorry for newcomers to programming having to learn this stuff, which noone tells them is completely useless for 99.99% of people, ending up not learning it anyways, because they would need to understand non-cloud in order to even realize why people are doing this.
    Seldom ever do I feel this strongly about a topic. Now where's keys to my bare metal postgres cluster...
    EDIT: great video, just can't fit more into this comment.

    • @pookiepats
      @pookiepats 8 месяцев назад +3

      Same man, i blame Python. So many people start there and never progress into more capable languages so to them-frameworks are their way of contributing but there is just too damn much.
      Never once has it ever been anything near seamless, i enjoy these videos but hearing the descriptions of how these tools “work” is always an eye roller.
      You just know there’s a mountain of bu** sh** you will still have to decipher to make it work for you.

  • @richcole157
    @richcole157 8 месяцев назад

    I’m half way through the video. No mention yet about Kinesis yet. Isn’t that an AWS scalable competitor to Kafka?

  • @TheSokilV
    @TheSokilV 8 месяцев назад

    Класно, коли людина реально оцінює свої сили: "не граю, бо боюсь, що затягне"). Повага.

    • @paulfunigga
      @paulfunigga 7 месяцев назад

      Россия победит

  • @nevgeniev
    @nevgeniev 8 месяцев назад

    have you heard about red panda guys??? just struggling to understand what's the point beyond "just having fun' ??

  • @arubaga
    @arubaga 8 месяцев назад +2

    Classic move to maximize profitability ending up painting yourself into a corner.

    • @jeffreyhymas6803
      @jeffreyhymas6803 Месяц назад +1

      This guy's take seems absolutely bonkers to me, including when he talked about "cloud native". If I understand him correctly he's basically saying if you don't use resources exclusively created and managed by cloud providers, that can't be reasonably recreated on local hardware then it's not "cloud native" and being cloud native is good. All I see here is people locking themselves into services and infrastructure they can't control, making them completely dependent on the provider.
      This is like building a business entirely on Facebook or RUclips or some other platform. They decide to pull the plug or change a policy that affects you and you're just completely fucked.

  • @akaalkripal5724
    @akaalkripal5724 8 месяцев назад +1

    Why not Rust? Why Go?

    • @abduljabbarazam943
      @abduljabbarazam943 7 месяцев назад

      Because Ryan said that his team knew Go inside out and they could, with some work, get the performance they wanted.

  • @peppybocan
    @peppybocan 8 месяцев назад +3

    why? There is Kafka. There is Red Panda. Do you need more?

    • @jc-aguilar
      @jc-aguilar 8 месяцев назад +8

      Why not? Someone wants to learn something, we should celebrate people wanting to learn and try new things. There is plenty of room to explore new approaches to same problems.

    • @peppybocan
      @peppybocan 8 месяцев назад +2

      @@jc-aguilar and that's fine. I will prefer to use Kafka/Red panda regardless.

    • @spoonikle
      @spoonikle 8 месяцев назад +2

      @@peppybocan - Build a wheel. Then use wheels bought from the store.
      Build the wheel to appreciate and understand the wheel.

    • @BR-lx7py
      @BR-lx7py 8 месяцев назад +1

      This is ~80% cheaper at the cost of some added latency.

  • @dokkenrox
    @dokkenrox 8 месяцев назад +1

    Vocal fry

  • @Algoritmik
    @Algoritmik 6 месяцев назад

    Why black nails?

    • @DeveloperVoices
      @DeveloperVoices  6 месяцев назад +1

      Why black? That was just the colour I had on at that week. It's always changing.
      Why paint them at all? Because I like it. 🙂

    • @LucasBatistussi
      @LucasBatistussi 5 месяцев назад

      @@DeveloperVoiceslove the response! You put the guy in his place with class! Btw ! Liked the color !