Kafka Tutorial - Node.js Producer & Consumer

Learn how to create a kafka producer & consumer using node.js.
We will be running apache kafka in a docker container. We'll create a producer & consumer using node.js which will read & write events to the stream. This is useful when building event driven systems.
Check out the code on Github: github.com/kriscfoster/node-k...
node-rdkafka - www.npmjs.com/package/node-rd...
avsc (for serialization & de-serialization) - www.npmjs.com/package/avsc
Don't forget to Subscribe here: / @krisfoster1

[@pankaj_9998]

Thank you so much. Great explaination. Can you do a simple microservices project in nodejs with kafka being used for async communication among the services. That would be great.

[@CodeCraftsman]

This is awesome man! I was looking for short and crisp explanation of Kafka! You nailed it!

[@TheCouncellor7]

Thanks Kris, what a fantastic clear instructions and clean code, wonderful to watch, thanks again for sharing your time and your knowledge, much appreciated

[@NIKOLAZ56]

impeccable way of explaining how kaftka works

[@objectObject212]

Great tutorial, exactly what I was looking for!

[@MegaJagveer]

Incredible video with very clear steps and instructions. Looking forward to more stuff!

[@KrisFoster1]

Thank you @Jagveer Singh!

[@JosuaLeonard]

Thank you Kris! Really clear and straightforward explanation. 🔥

[@nishanthanr605]

no annoying music no extra talks.
very clear content

[@andreassef3656]

Thank you for this content, it help me so much to understand how kafka works!

[@burtreynolds111]

Awesome video Kris!! So helpful.

[@kal_node]

I found it quite explanatory and helpful, thanks!

[@amirmohammadmirzaeirad3]

Thanks man, great tutorial for those who are just getting started with kafka with nodejs.

[@muratasarslan2359]

Thank you Kris, very clear & helpful.

[@mohammadpartovi1813]

This was fantastic. Great explanation. Thank you very much for sharing your knowledge.

[@ayushpgupta5755]

Very clearly explained with bare minimum code.

[@Cyberspider76]

Well done… simple and too the point!

[@KrisFoster1]

Thank you!

[@raviparashanaboina4644]

Thank you So much it is one of the best tutorial for producer and consumer great work!

[@fraewn2617]

Great tutorial, thank you!

[@ryanhill101]

Nice video Kris and super easy to follow.

[@KrisFoster1]

Thank you Ryan, glad you enjoyed!

[@surajjoshi5503]

Great Explanation

[@elad3958]

I subscribed to your channel after this awesome tutorial

[@raymondoyinlola7765]

This is really helpful.

[@sreeananthakannan]

It was the good video to understand about basic of kafka

[@panjijayasutra1573]

Very usefull, thankyou verymuch Kris

[@babanmalhi6168]

Thank you so much. Very very helpful

[@hegeltron]

Thanks Kris!

[@thetrooper6]

great demo! thank you!

[@KrisFoster1]

Glad you enjoyed, thank you!

[@ntwariegide5993]

Thanks for this great video

[@imsarvesh_]

Finally I understood Kafka.

[@KrisFoster1]

Thank you, glad you enjoyed!

[@AnkitKPunjabi]

Crisp and clear

[@Kreator321RG]

Thanks a lot! You are the best))

[@eternalharmony0]

Thank you so much.

[@mahendranath2504]

Thank you so much ❤️👍🏼🎉⭐🙏🤛, subscribed and 👍🏼

[@karmakiller360]

Good Job. Thank U

[@atiqbaqi]

great example

[@vitorsantana2795]

Thanks a lot!!

[@chathurangabandarapc]

Thanks you very much

[@afshinrahmati786]

Thank you ,

[@manikjain6048]

Nice work

[@KrisFoster1]

Thank you @Manik Jain, glad you enjoyed!

[@herbertk9266]

Thanks for sharing

[@KrisFoster1]

Glad you enjoyed!

[@get_carrot]

like your voice

[@K_H-Channel]

Hi there, my company has Kafka built in Java. I am looking for a way to automate the events and use assertions to verify the events. Events can be in thousands. Is there a way to automate this using JavaScript/ node js/ Jest/Mocha/Chai or anything else that’s JavaScript? Thank you.

[@abderrahmane_24m]

Hello, thank for this tutorial! I'm facing a problem using docker-compose, when producer gets triggered, I'm getting LibrdKafkaError: Local: Broker transport failure, any suggestion?

[@daesvideo]

Hi, thank you very much. Can you do another exaple with multiple consumers please?. I tryed but just only one consumer can receibe message. thank you.

[@Mvrck44]

Thanks a lot for this. Quick hint, at the time of writing this comment, the 'npm i' won't work as is: new minor version of node-rdkafka(2.11) is available, need to be updated manually.

[@joaovitorsilveira256]

Hey man, how did you overcome this problem?

[@Mvrck44]

@@joaovitorsilveira256 after couldn't get neither 'npm ci', nor 'npm i' to work, I installed both dependencies listed in package.json manually, like 'npm i '. This installed node_modules and updated both .json files.

[@kishorrathva4097]

Which Nodejs version are you using ?

[@_slier]

so from your demo, kafka is like messaging system like rabbitmq?

[@karusalavinaykumar7399]

Thank you for the Tutorial.
npm i node-rdkafka is failing for us. Please make a video on how to install the pre-requisites and install node-rdkafka.
Thanks.

[@GM-je3zp]

nice tutorial... Kafka container doesnt run for me though and had to get the compose file from an exmaple from cofluetn

[@mohamatnasri9619]

i use type: modules still get same error, so how?

[@thong3233]

i can't install rdkafka, please help me

[@lynnegaming7328]

can i make it without docker??

[@andriizilnyk4927]

does not wok on node 14.17.5

[@shivammishra-sj8jo]

Hi Kris i am not able to install node-rdkafka can you help me out ?

[@daesvideo]

Hi, search some Dockerfile with RUN apt intsall cmake, and other tools, that's the good one.

[@navyaraj5316]

getting error while installing rdkafka 😢 anyone help please

[@rules654]

If in windows and facing issues in creating topic use this command --------> docker exec -it [container_Id] /opt/bitnami/kafka/bin/kafka-topics.sh --create
--bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test
you can get the kafka container Id by running ---> docker ps
replace [container_Id] with the containerId of the kafka container

[@olorundareagnes934]

Great tutorial.
Thanks a lot!