Transition to Apache Kafka on Kubernetes with Strimzi

Steffen Wirenfeldt Karlsson, Lead Software Engineer, Maersk
Sai Charan Madhvaraj, Lead Software Engineer, Maersk
Ravikanth Mallappa, Lead Platform Architect, Maersk
This is a classic migration case study (the past, current and the future) at scale from a world-wide company transitioning from Confluent Platform and Confluent Cloud to self-managed Apache Kafka on Kubernetes using Strimzi. At Maersk, we have been architecting, designing and implementing our 3rd generation Event Streaming Platform. This platform is based on Kubernetes in Azure and using Strimzi to operate Apache Kafka at large scale, highly reliable, segregating data based on isolated use cases. Our 2nd generation was based on OnPrem Confluent Platform and Confluent Cloud and this presentation is the story of this migration and reasoning behind it. Furthermore, we would get into details on how we monitor (Grafana, Prometheus), alert (GoAlert and alert as code), operate and provide self-service solutions on top of Strimzi to enable business critical application in Maersk, implemented in GoLang using the GitOps deployment model with Flux and Kustomization among others. Finally, if time allows we will end with a demo of an open-source self service tool to monitor and explore the cluster with most wanted features such as topic message browsing and configuring and restarting connectors.

[@charansai2356]

Not Yet .. We started our journey with 0.27.1 where Kraft was not on the cards
Will be upgrading our clusters soon to Kraft

[@premierde]

When deploying Strimzi on K8s you used Kraft?

[@charansai2356]

Not Yet .. We started our journey with 0.27.1 where Kraft was not on the cards
Will be upgrading our clusters soon to Kraft