In-depth Intuition on Kafka Rack Awareness with Practical Demo

The Rack Awareness feature in Kafka spreads replicas of the same partition across different racks to minimize data loss in the event of a rack failure.
This video explains about this concept with in-depth intuition & hands-on.
Prerequisite:
--------------------
Rack Awareness in Hadoop
• Rack Awareness in Hadoop
Kafka Cluster with Multiple Brokers
• Kafka Cluster with Mul...
Topic with Replication in Multiple Broker Kafka Cluster
• Topic with Replication...
Code used in this video:
-------------------------------
Start Zookeeper:
------------------
F:/kafka_2.12-3.3.1/bin/windows/zookeeper-server-start.bat F:/kafka_2.12-3.3.1/config/zookeeper.properties
Create 3 server properties file.
Change broker ids,listeners,log directory
Start Kafka-server:
--------------------
F:/kafka_2.12-3.3.1/bin/windows/kafka-server-start.bat F:/kafka_2.12-3.3.1/config/server.properties
F:/kafka_2.12-3.3.1/bin/windows/kafka-server-start.bat F:/kafka_2.12-3.3.1/config/server1.properties
F:/kafka_2.12-3.3.1/bin/windows/kafka-server-start.bat F:/kafka_2.12-3.3.1/config/server2.properties
F:/kafka_2.12-3.3.1/bin/windows/zookeeper-shell.bat localhost:2181 ls /brokers/ids
Create a topic:
----------------
F:/kafka_2.12-3.3.1/bin/windows/kafka-topics.bat --create --topic demo_testing2 --bootstrap-server localhost:9092,localhost:9093,localhost:9094 --replication-factor 2 --partitions 1
F:/kafka_2.12-3.3.1/bin/windows/kafka-topics.bat --bootstrap-server localhost:9092,localhost:9093,localhost:9094 --describe --topic demo_testing2
set broker.rack=1 for broker 0 & 1 and broker.rack=2 for broker 2
F:/kafka_2.12-3.3.1/bin/windows/kafka-topics.bat --create --topic demo_testing3 --bootstrap-server localhost:9092,localhost:9093,localhost:9094 --replication-factor 2 --partitions 1
F:/kafka_2.12-3.3.1/bin/windows/kafka-topics.bat --bootstrap-server localhost:9092,localhost:9093,localhost:9094 --describe --topic demo_testing3
F:/kafka_2.12-3.3.1/bin/windows/kafka-topics.bat --create --topic demo_testing4 --bootstrap-server localhost:9092,localhost:9093,localhost:9094 --replication-factor 2 --partitions 1
F:/kafka_2.12-3.3.1/bin/windows/kafka-topics.bat --bootstrap-server localhost:9092,localhost:9093,localhost:9094 --describe --topic demo_testing4
F:/kafka_2.12-3.3.1/bin/windows/kafka-topics.bat --create --topic demo_testing5 --bootstrap-server localhost:9092,localhost:9093,localhost:9094 --replication-factor 2 --partitions 1
F:/kafka_2.12-3.3.1/bin/windows/kafka-topics.bat --bootstrap-server localhost:9092,localhost:9093,localhost:9094 --describe --topic demo_testing5
(like this multiple topics can be created & tested...)
Check this playlist for more Data Engineering related videos:
• Demystifying Data Engi...
Apache Kafka form scratch
• Apache Kafka for Pytho...
Snowflake Complete Course from scratch with End-to-End Project with in-depth explanation--
doc.clickup.co...
🙏🙏🙏🙏🙏🙏🙏🙏
YOU JUST NEED TO DO
3 THINGS to support my channel
LIKE
SHARE
&
SUBSCRIBE
TO MY RUclips CHANNEL

[@pardhutummala]

you are best kafka teacher i ever gone through. thank you.

[@KnowledgeAmplifier1]

Thank you for your kind words @user-ix4dl6zo1w! Happy Learning

[@ye__in]

Truely amazing!! Thank you sooooo much for making this high quality lecture!

[@KnowledgeAmplifier1]

You are welcome @ye__in! Happy Learning

[@shreyas-s5g]

how come you are so underrated sir. You have simply amazing teaching skills.

[@KnowledgeAmplifier1]

Glad you think so!

[@eduardoIquadrat]

Awesome content!!!!!!! I owe you one.

[@KnowledgeAmplifier1]

Thank you so much @user-ew6de9gq3u! 🙌 I'm glad you found the content helpful!

[@AmarjeetAnandsingh]

Thanks for such nice content.
Really appreciate your efforts. 🙏

[@KnowledgeAmplifier1]

Thank you Amarjeet Anand! Happy Learning

[@sdef719]

Don't know who you are, but Orz to your contents, I have learned the theory till now, never seen this things practically in real life. You made it possible. Kudos. 🙏

[@KnowledgeAmplifier1]

Thank you so much for your kind words and appreciation Bibhash Ghosh! Happy Learning

[@RajbirSingh-eg9qr]

your content is awesome bro

[@KnowledgeAmplifier1]

Thank you so much 😀

[@akashshelke5330]

Beautifully explained.

[@KnowledgeAmplifier1]

Thank You!

[@bravewarrior236]

I have a request. I am trying to build a VCO ADC model in Simulink. Could you try to provide some guidance through your videos. Many thanks!