I thing it is Publish-Subscriber approach that you have shown. Event-driven approach is for instance (Event Notification approach) 1. book service got new book 2. then send message NEW_BOOK (not Entity) to TOPIC (NEW_BOOK is our Event that means: "Bro we got new book we have to notify all our friends") 3. book and author get messge NEW_BOOK and realise "Ok then we have to make REST call f.e and find last added book" microservices io said Use an event-driven, eventually consistent approach. Each service publishes an event whenever it update its data. Other service subscribe to events. When an event is received, a service updates its data. It looks you change state in subcriber services (book, author) according to Book entity then your explanation fits definition and it looks like Event Carried state transfer pattern (What do you think) (If I am mistaken let me know)
I'll do my best to give a good answer 💪 You're absolutely right that the mechanism we're using Kafka for could be described as publish-subscribe. We put something on the topic, but we don't care what picks it up, nor do we expect a response. I see this as an implementation detail of our approach to being event driven. The term "event driven" is pretty broad, it doesn't prescribe a particular implementation approach above emitting events and taking actions as a result (or not): martinfowler.com/articles/201701-event-driven.html. However, there are several patterns . The one shown in this video could fit under the "Event-Carried State Transfer" banner of the above article. But as always, what you build will depend on the requirements. Patterns exist as common solutions to common problems. I've this approach in 3 commercial projects so far and it worked wonderfully for distributing data in an eventually consistent way, but that's not to say it will work in all scenarios. The pattern you've identified as the Event Notification approach is totally viable and may be a better fit in some projects, especially when the data involved is too large to fit in a Kafka message e.g images or videos. However it also has the drawback of being synchronous in places -- requiring your REST services be up at all times, even under heavy load. That being said, these patterns along with the Actor pattern, Event-Sourcing and CQRS could all be described as Event-Driven and great solutions for different challenges 🙂
What about real-life scenarios, when the user want to see books list with author data? Looks like that for every book you are going to make HTTP GET from book-service request to author-service to get author by id. It's kind of bottleneck, what do you think?
As always, it depends 🙂 Microservices come with a premium: martinfowler.com/bliki/MicroservicePremium.html Therefore, typically the choice to use them is influenced by Conway's law e.g in order for multiple Dev teams to better work together. Or for horizontal scaling reasons. (This list isn't exhaustive, but should hopefully communicate the point I'm trying to make 😁) . In these scenarios the benefits likely outweigh the the increased latency of making multiple network calls. That being said, you can optimise on this by combining the data on an eventually consistent basis, which would reduce the network calls to 1 -- like a materialised view in a database. But, I'd suggest optimising only when necessary as premature optimisation has its own pitfalls. It all depends on the requirements. It's a big conversation, but hopefully this helps 🙂
Such a good video for Kafka that provides not theory but code! I really needed this, thank you so much! Subscribed, hope you'll get more views and subscribers!
I understand the idea you presented in the video, but what about situations where you have to, for instance, query all the books with its authors. So the Book service will have to call Authors service for the details about authors, but it makes those services highly coupled that makes we should add an abstraction between them like a kafka topic to decouple them but when Book service calls the Author service via Kafka how the Author will know where to send data back?
How will this exact scenario look like when deployed on AWS infrastructure? Would be great if you could cover it using AWS Lambda, RDS ,API Gateway and SNS.
Why are you using in your code in service layer interface service and impl of this interface? Proxy? I thought in new versions of spring there is no need for that
You can absolutely just use concrete classes and inject them where needed. There is a benefit to coding to interfaces instead of concrete classes though. Let's say you had a "FileSaver" interface that had an implementation that saves to disk and then you decided you wanted to save to the cloud. If you used the interface everywhere, you're simply creating a new implementation and then injecting that instead, rather than changing everywhere to use the new concrete class. It's not so much a Spring thing as a general OOP software thing. Nothing forces you to do it that way, but you soon learn to appreciate it 😁 Hopefully this helps!
Just few minutes into the Tutorial, I'm like this is what i have been looking for... The proof of concept is top-notch. But why not use Spring cloud stream?? Thanks so much!!
so for notification sending to frontend only kafka is enough? i can see u send the notification to that topic directly so if the frotnend will be the consumer for that notification so it will display on forntend right ??? plz let me clear
Really nice tutorial. I am more interested in the Devops part of this though. How are you able to have more than one service in a single project. Is this a Monorepo kind of thing. How are you able to deploy the services separately. Is there specific spring configurations for this purpose. Thank you in advance
Great tutorial, devtiro! Your clear, detailed guide on building event-driven microservices with Spring Boot & Kafka is incredibly helpful. You make complex concepts easy to understand. Thanks for sharing your expertise-looking forward to more content like this!
Thanks for the brief yet informative video, can I ask how we can debug this services in IntelliJ? I tried mapping the port 5005 for each service in docker-compose and made a remote-debug configuration, but when trying to start debug, I get address already in used :(
Thank you! Rather than use maven, in intelliJ I usually right click the main class in the file navigator and run it in debug from there. IntelliJ will handle connecting to the debug port for you.
Great video! Thanks for sharing this!!! The examples, content, approach and teaching process is amazing! Can't wait for more Kafka with Spring boot content... Maybe on how to test it with integation tests :)
It depends 🙂 When anything is shared between microservices then it couples them to some extent. I fallback to Conway's law here. If there's a single Dev team working on the project, then the impact of coupling via a shared domain is lessened e.g a distributed monolith could be a viable solution (assuming you actually need it distributed). In the case of multiple Dev teams, I would suggest being less DRY* isn't all that bad, if you can deploy your microservices independently. *Don't Repeat Yourself
![Tee Grizzley - Blow for Blow (feat. J. Cole) [Official Video]](http://i.ytimg.com/vi/wNLN5xsx5dc/mqdefault.jpg)
Really great. Thank you ☺️ sir.
Thank you for the kind words!
Your help to the community or people really great can say in words. I'm thankful for ever.
You are very welcome! 😁
SPRING CLOUD STREAMS
I thing it is Publish-Subscriber approach that you have shown.
Event-driven approach is for instance (Event Notification approach)
1. book service got new book
2. then send message NEW_BOOK (not Entity) to TOPIC (NEW_BOOK is our Event that means: "Bro we got new book we have to notify all our friends")
3. book and author get messge NEW_BOOK and realise "Ok then we have to make REST call f.e and find last added book"
microservices io said
Use an event-driven, eventually consistent approach. Each service publishes an event whenever it update its data. Other service subscribe to events. When an event is received, a service updates its data.
It looks you change state in subcriber services (book, author) according to Book entity then your explanation fits definition and it looks like Event Carried state transfer pattern (What do you think)
(If I am mistaken let me know)
I'll do my best to give a good answer 💪
You're absolutely right that the mechanism we're using Kafka for could be described as publish-subscribe. We put something on the topic, but we don't care what picks it up, nor do we expect a response.
I see this as an implementation detail of our approach to being event driven.
The term "event driven" is pretty broad, it doesn't prescribe a particular implementation approach above emitting events and taking actions as a result (or not): martinfowler.com/articles/201701-event-driven.html.
However, there are several patterns .
The one shown in this video could fit under the "Event-Carried State Transfer" banner of the above article.
But as always, what you build will depend on the requirements. Patterns exist as common solutions to common problems.
I've this approach in 3 commercial projects so far and it worked wonderfully for distributing data in an eventually consistent way, but that's not to say it will work in all scenarios.
The pattern you've identified as the Event Notification approach is totally viable and may be a better fit in some projects, especially when the data involved is too large to fit in a Kafka message e.g images or videos. However it also has the drawback of being synchronous in places -- requiring your REST services be up at all times, even under heavy load.
That being said, these patterns along with the Actor pattern, Event-Sourcing and CQRS could all be described as Event-Driven and great solutions for different challenges 🙂
What about real-life scenarios, when the user want to see books list with author data? Looks like that for every book you are going to make HTTP GET from book-service request to author-service to get author by id. It's kind of bottleneck, what do you think?
As always, it depends 🙂
Microservices come with a premium: martinfowler.com/bliki/MicroservicePremium.html
Therefore, typically the choice to use them is influenced by Conway's law e.g in order for multiple Dev teams to better work together. Or for horizontal scaling reasons. (This list isn't exhaustive, but should hopefully communicate the point I'm trying to make 😁) .
In these scenarios the benefits likely outweigh the the increased latency of making multiple network calls.
That being said, you can optimise on this by combining the data on an eventually consistent basis, which would reduce the network calls to 1 -- like a materialised view in a database. But, I'd suggest optimising only when necessary as premature optimisation has its own pitfalls.
It all depends on the requirements.
It's a big conversation, but hopefully this helps 🙂
Such a good video for Kafka that provides not theory but code! I really needed this, thank you so much! Subscribed, hope you'll get more views and subscribers!
dude your videos are so clear, concise and you are very articulate! I would subscribe 1000 times if I could
I understand the idea you presented in the video, but what about situations where you have to, for instance, query all the books with its authors. So the Book service will have to call Authors service for the details about authors, but it makes those services highly coupled that makes we should add an abstraction between them like a kafka topic to decouple them but when Book service calls the Author service via Kafka how the Author will know where to send data back?
How will this exact scenario look like when deployed on AWS infrastructure? Would be great if you could cover it using AWS Lambda, RDS ,API Gateway and SNS.
nice
Why are you using in your code in service layer interface service and impl of this interface? Proxy? I thought in new versions of spring there is no need for that
You can absolutely just use concrete classes and inject them where needed.
There is a benefit to coding to interfaces instead of concrete classes though.
Let's say you had a "FileSaver" interface that had an implementation that saves to disk and then you decided you wanted to save to the cloud.
If you used the interface everywhere, you're simply creating a new implementation and then injecting that instead, rather than changing everywhere to use the new concrete class.
It's not so much a Spring thing as a general OOP software thing. Nothing forces you to do it that way, but you soon learn to appreciate it 😁
Hopefully this helps!
@Devtiro Thank you for explanation
Just few minutes into the Tutorial, I'm like this is what i have been looking for...
The proof of concept is top-notch. But why not use Spring cloud stream??
Thanks so much!!
so for notification sending to frontend only kafka is enough? i can see u send the notification to that topic directly so if the frotnend will be the consumer for that notification so it will display on forntend right ??? plz let me clear
Really nice tutorial. I am more interested in the Devops part of this though. How are you able to have more than one service in a single project. Is this a Monorepo kind of thing. How are you able to deploy the services separately. Is there specific spring configurations for this purpose. Thank you in advance
You're the man! Thanks for sharing this. You earned a new subscriber!
Same guy from the Kafka Basics Tutorial Video
Forget everything I said, You're perfect as is.
Great tutorial, devtiro! Your clear, detailed guide on building event-driven microservices with Spring Boot & Kafka is incredibly helpful. You make complex concepts easy to understand. Thanks for sharing your expertise-looking forward to more content like this!
Devtiro Sir, your truly Underrated and should have many subscribers
Great video, I've been looking for a spring boot x kafka rundown for a while now and this really ticks all the boxes. Thank you!
Thanks for the brief yet informative video, can I ask how we can debug this services in IntelliJ? I tried mapping the port 5005 for each service in docker-compose and made a remote-debug configuration, but when trying to start debug, I get address already in used :(
Thank you! Rather than use maven, in intelliJ I usually right click the main class in the file navigator and run it in debug from there. IntelliJ will handle connecting to the debug port for you.
Have you considered using Kafka Confluent, with avro schema?
A video using avro is a great idea, thanks! I'll add this to the video backlog!
Great video! Thanks for sharing this!!! The examples, content, approach and teaching process is amazing! Can't wait for more Kafka with Spring boot content... Maybe on how to test it with integation tests :)
Thank you! Great suggestion, I'll add it to the video backlog 💪
very well explained, can you maybe share about handling concurrent request and or async examples? thank you
Thank you! 🙏 I'll see what I can do!
Great content. I am waiting for more videos regarding software architecture and other approaches like CQRS :)
Thank you! I'll add CQRS to the video backlog 👍
Is that good way to create domains module and use as one place where we have entity instead has entities in each module (service)
It depends 🙂
When anything is shared between microservices then it couples them to some extent.
I fallback to Conway's law here.
If there's a single Dev team working on the project, then the impact of coupling via a shared domain is lessened e.g a distributed monolith could be a viable solution (assuming you actually need it distributed).
In the case of multiple Dev teams, I would suggest being less DRY* isn't all that bad, if you can deploy your microservices independently.
*Don't Repeat Yourself
How you have created that beautiful architecture diagram.
Thanks! That one was done in Keynote 🙂
great tutorial
Great work ,looking forward more from you on these. Keep it up Bro, How can i acces the source code.
oh sorry git hub link is there.
👍
soo greatt more content like this wow ur killing it bro
Thank you so much! 🙏
Neat.
Awesome! Please continue the series.
Thank you so much 🙏I'll see what I can do!
Great content as usual! Keep it up :D
Thank you 😁 Lots more to come!