Modular Monoliths: How To Build One & Lessons Learned

Поделиться
HTML-код
  • Опубликовано: 8 сен 2024

Комментарии • 124

  • @MilanJovanovicTech
    @MilanJovanovicTech  Год назад +6

    Want to master Clean Architecture? Go here: bit.ly/3PupkOJ
    Want to unlock Modular Monoliths? Go here: bit.ly/3SXlzSt

    • @reinhardlackner1925
      @reinhardlackner1925 Год назад +3

      If you enjoyed the inflow of new subscribers, how about including all links from the talk in the video description or a link to the talk slides directly?
      Sorry, I love your're content, but I hate how you have to navigate around (my guess) the youtube algorithm and be as un-direct as possible to attract more subscribers by leaving out the direct links included in the video already.

    • @MilanJovanovicTech
      @MilanJovanovicTech  Год назад +1

      @@reinhardlackner1925 That's a passive aggressive comment if I ever saw one 🤣 I added the repo link to the description, thanks for pointing that out. And I'll see about the slides, for now they aren't public.

    • @reinhardlackner1925
      @reinhardlackner1925 Год назад

      oh, and, yearh, I'm moving into the Q&A section and suddenly the video is cut off 😞

    • @reinhardlackner1925
      @reinhardlackner1925 Год назад

      even though, what I remember from most other talks online, nobody cares to either give out mics or repeates questions so late viewers of a talk can still have the full picture.

    • @JoseCapistrano
      @JoseCapistrano 5 месяцев назад

      Is it possible to get a copy of your presentation on the video?

  • @nielshoogev1
    @nielshoogev1 Год назад +17

    Great talk. With respect to data isolation, there is one more option I have found to be useful. Each module owns a set of tables and only that module is allowed to write to those tables. Each module exposes a set of views that other modules are allowed to read. Using database views allows the owning module to change the factual organization of data, while still exposing a stable view over that data to consuming third parties. This is more or less similar to exposing a public interface.

    • @MilanJovanovicTech
      @MilanJovanovicTech  Год назад +2

      That's a really cool way to do it, Niels. I might even add it to my presentation next time I do this talk 😅

    • @RomainBriand
      @RomainBriand Год назад +2

      Sounds very smart!

  • @duncan-dean
    @duncan-dean Год назад +5

    Loved this talk. To the point with no fluff and gives actual examples.

    • @MilanJovanovicTech
      @MilanJovanovicTech  Год назад

      Thanks a lot! I'll also start covering the practical sides of Modular Monoliths on the channel soon.

  • @branislavpetrovic7486
    @branislavpetrovic7486 Год назад +5

    Great talk! Down-to-earth introduction to modular monoliths, great for beginners in this topic.
    Thanks!

  • @theaugmenter
    @theaugmenter 3 месяца назад +1

    This has to be the best presentation I have viewed on any developer topic!

    • @MilanJovanovicTech
      @MilanJovanovicTech  3 месяца назад

      Check out the entire Modular Monolith playlist, I think you'll enjoy it :)

  • @stephendgreen1502
    @stephendgreen1502 Год назад +4

    Modular monoliths seems to be best of both worlds of clean and vertical slices. Each module is vertical. But can be clean too, with multiple projects for tech layers. So one module serves both functional and technical cohesion.

    • @MilanJovanovicTech
      @MilanJovanovicTech  Год назад +4

      And you can also apply Vertical Slice architecture to an entire module

  • @stephendgreen1502
    @stephendgreen1502 Год назад +2

    Excellent, very clearly thought-out, explained and presented

  • @pilotboba
    @pilotboba Год назад +3

    Just an observation. You started with Flowler's quote about starting with a monolith. You explained very well why you thought he said that. Because you don't know where your bottlenecks will be and what needs to be split out.
    Then in your lessons learned you did a 180 and said you wish you spent more time defining your modules upfront.
    It seems like you broke the first rule, you didn't start with a monolith, you started with co-located microservices and only after doing that did you realize where your bottlenecks were and had to rework your module responsibilities.
    Correct me if I interpreted your learnings wrong.
    Otherwise very good information and description of some of the patterns this type of architecture might contain.

    • @MilanJovanovicTech
      @MilanJovanovicTech  Год назад +1

      Fowler's quote was just to bring up an argument for starting with Monolith first.
      I guess that final lesson learned does come to the same conclusion as Fowler's quote, in essence, but it's something we didn't do well at the start. It doesn't hurt to repeat it twice.

  • @Adrianvdh
    @Adrianvdh 4 месяца назад +1

    Please do a video on Eventual Consistency vs Transactional Consistency

  • @humbertoh9252
    @humbertoh9252 9 месяцев назад

    Good talk! Monoliths are also horizontal scalable as you can have multiple nodes with the same app. What is harder to horizontal scale is the database.

    • @MilanJovanovicTech
      @MilanJovanovicTech  9 месяцев назад

      The DB is always hard to scale, maybe I should do some more videos on that

  • @kodindoyannick5328
    @kodindoyannick5328 7 месяцев назад +1

    Great talk. All the content is useful for me. Thanks you Milan.

  • @vilkazz
    @vilkazz 11 месяцев назад +2

    For us the biggest pain with this architecture was to keep the api layer in check.
    Our team opted for hard-coupled api layer between the domains... Which caused every api update to require dependency update in multiple modules.

    • @MilanJovanovicTech
      @MilanJovanovicTech  11 месяцев назад

      How did you fix that?

    • @devolajide
      @devolajide 10 месяцев назад

      I am interested in how you fix this because I am building a system now and trying to use this approach. But my problem is any change in one module, I will need to build all modules and the core before redeploying.

  • @josedesousa5789
    @josedesousa5789 Год назад +1

    Très très bon contenu comme d'habitude ! Merci beaucoup Milan pour ton travail ! Keep up the good work !

  • @stephendgreen1502
    @stephendgreen1502 Год назад +2

    Modular monolith seems to lead to lots of connection strings and projects being added to the legacy monolith, which might not be geared up for this (nor its developers). Microservices seems to lead to lots of API web apps with their configuration added to the original legacy monolith and lots of extra solutions in the repo. But devs love it on their CV, like you say. So microservices wins despite grim warnings from the elders (like Uncle Bob and Martin Fowler). Fools rush in. The outcome: support calls inundating devs, and eventual consistency in their nightmares.

    • @MilanJovanovicTech
      @MilanJovanovicTech  Год назад

      You could still organize the configuration inside each module, so that they're not mixed

    • @stephendgreen1502
      @stephendgreen1502 Год назад

      @@MilanJovanovicTech How would you do that?

  • @cesarcofg
    @cesarcofg 10 месяцев назад +1

    Awesome presentation MIlan thanks , very clear.

  • @JaumeSabater
    @JaumeSabater 3 месяца назад

    I think that a good first step is using one database schema per module, keeping referential integrity but with no joins among schemas. Then you can break that referential integrity when you need to move a module out of the monolith.

  • @danielgruner
    @danielgruner Год назад +1

    I really love how you went through the problems and different solutions for each problem. I stumbled over the repo you mentioned in your video and was a little bit overwhelmed because I didn't get the inter-module communication by event bus. You gave a perfect explanation and a kind of smooth transition from method calls to messaging.
    One question (perhaps you have an idea, otherwise perhaps there are others here in the chat with some ideas):
    How would you code the module communication by method calls in Node.js? In Node you can do the same thing in tens or hundreds of ways. People coming from OOP languages tend to use classes and patterns like Dependency Injection while others just allow other modules to use an exported function. Is there a way to do it right or wrong?

    • @MilanJovanovicTech
      @MilanJovanovicTech  Год назад +1

      As long as you are following some sort of dependency rule, you should be fine. I'm not sure if there is a way to expose an interface in Node.js - but exposing a set of functions seems to be a possible solution.

  • @saddamhossaindotnet
    @saddamhossaindotnet Год назад +1

    Excellent my friend. 🙂

    • @MilanJovanovicTech
      @MilanJovanovicTech  Год назад +1

      Thank you. I think for the next talk, I'm going to add a slide for "fallacies of distributed computing". And also some examples of code.

    • @saddamhossaindotnet
      @saddamhossaindotnet Год назад

      Thank you very much for the community. I greatly appreciate your efforts.

  • @tirgo007
    @tirgo007 Год назад +1

    Great content as always .

    • @MilanJovanovicTech
      @MilanJovanovicTech  Год назад

      You're a fast watcher 😁 Were you familiar with Modular Monoliths from before?

  • @JoseFelipeBlumdeAraujo
    @JoseFelipeBlumdeAraujo Год назад +1

    Great content!

    • @MilanJovanovicTech
      @MilanJovanovicTech  Год назад

      If there's any question, I'm more than willing to clear up any confusions :)

  • @georgesngandeu9115
    @georgesngandeu9115 9 месяцев назад

    Milan you are very good. Thanks for sharing

  • @aseneda
    @aseneda Год назад +2

    Hi Milan, thanks for sharing your knowledge, that is great. Two question:
    * Considering the database isolation (same DB diff schema or diff DB): Do we continuously use the concept of column relationship among tables using their FK's ? e.g.: (Customer Id) » (Order CustomerId)
    * When we face a complex query from diff schema or DB (through EF e.g.), How could it be done using MM?
    Thanks

    • @MilanJovanovicTech
      @MilanJovanovicTech  Год назад +4

      1) You can use FK's in the same DB, but it only helps with referential integrity. I'd keep using it if it's the same schema. If it's a different schema (other module) you're probably better of without it
      2) Execute multiple queries and patch the results together

    • @aseneda
      @aseneda Год назад

      @@MilanJovanovicTech thanks for your reply.
      So, do we may get data among different schemas using referential column without integrity?

  • @stephendgreen1502
    @stephendgreen1502 Год назад +1

    It was a challenge to start modularising a legacy app by adding class libraries compatible enough to sit inside the legacy solution but I find it is doable with .NET 4.8, EF6, and a few other legacy technologies.

    • @MilanJovanovicTech
      @MilanJovanovicTech  Год назад +1

      I can imagine the difficulties you ran into with .NET Framework

    • @stephendgreen1502
      @stephendgreen1502 Год назад

      @@MilanJovanovicTech Yes, you hit the nail on the head. The weakness of modular monoliths is the inability to break away from and modernise the framework of the monolith and the version of libraries it supports.

    • @stephendgreen1502
      @stephendgreen1502 Год назад

      @@MilanJovanovicTech The strength of the modular monolith architecture is it allows you to break away from and modernise the domains of the monolith. It also lets you share the database and its transactions.

  • @stephendgreen1502
    @stephendgreen1502 Год назад +2

    Major problem with eventual consistency is data loss (received out of order so cannot process second message because first not yet received)

    • @MilanJovanovicTech
      @MilanJovanovicTech  Год назад +1

      You can always buffer the messages on the receiving end, and make sure to process them in order

  • @i.t.9015
    @i.t.9015 Год назад +2

    Great content! 👏
    Can we somehow have transaction atomicity between modules?

    • @MilanJovanovicTech
      @MilanJovanovicTech  Год назад +1

      Two Phase Commit? In general, you don't want to have atomic transactions between modules. If you need it, consider merging modules.

  • @thebowshock7729
    @thebowshock7729 8 месяцев назад +1

    Hey Milan. At 42:30 do you mean data denormalization and duplication across multiple isolated databases/ schemas belonging to individual modules?

  • @AndreiDzimchuk
    @AndreiDzimchuk Год назад +1

    With module boundaries being well understood and defined, with modules owning their data and communicating over a message bus, what are the benefits of keeping them in one process other than to simplify deployment?

    • @MilanJovanovicTech
      @MilanJovanovicTech  Год назад +1

      Distributed systems are really hard, there's all sorts of headaches you need to solve when doing microservices. I think people take it too much for granted 🤷‍♂️

    • @AndreiDzimchuk
      @AndreiDzimchuk Год назад

      @@MilanJovanovicTech There is no doubt about, but in your example you’ve already done the hardest parts (identified boundaries, split data, decided on integration approaches), is there a reason to not take full advantage of microservices (independent scalability, release cycles, tech choices, etc)?

    • @spstc967
      @spstc967 Год назад

      @@AndreiDzimchuk I think biggest challenges of microservices will occur after deploying the services. like containerization,monitoring,network latency and the maintenance cost .Also it requires relatively more experienced developers to build services which increase overall cost of the application

  • @infotips2475
    @infotips2475 9 месяцев назад +1

    Do I still need to implement a saga (or something for consistency) if I use a message broker?, rigth.

  • @Alex-gj2uz
    @Alex-gj2uz 7 месяцев назад

    Great explanation, in good detail and also with some realworld "handson" tipps, like the example with internal keyword in c#, the comparison of method calls and message bus etc.
    I have one question though:
    You say you don't want to share data between modules directly, only go over the modules boundry. Imagine a system where you have a timeseries database, its an industrial edge usecase:
    You have one PC which is getting data from 50-100 sensors or controllers, it will collect all the data into the timeseries db every n seconds. Now one other module would like to get some data from the timeseries database, resp. the module which is having this DB.
    This means you could only get data via the module interface (messagebus, RPC, REST, what ever), would you really strictly stick to this even you get some performance and complexity drawbacks or would you make some exceptions and access the database directly?

    • @MilanJovanovicTech
      @MilanJovanovicTech  7 месяцев назад

      This is where we SHOULD make some tradeoffs, depending on our requirements.
      - Is performance critical? Then we can merge these modules together. The dependent module has direct access to the timeseries database. Maybe it only makes sense to move part of one module, into the module with timeseries data? Lots of options to explore IF performance is critical.
      - If we can live with some latency - then we can make tradeoffs based on complexity. Duplicating the time series data in the other module would give us excellent performance, at the cost of some latency and data duplication.
      Where would you take this further?

    • @Alex-gj2uz
      @Alex-gj2uz 7 месяцев назад

      @@MilanJovanovicTech
      "Is performance critical? " I wouldnt say the performance is that critical or my biggest concern, I'm just a bit scared of wrapping all the Data which comes from the database in a API Call, in case of Protobuf Messages or REST I would have to put it into maybe json like data. I guess that gets bloated and memory intense quite quickly, I would also have to do the serialization and deserialization. By using binary formats in a protobus message directly from a language like java I loose language agnostic communication. And for every "type" of request I would have to build an "API fassade" around the DB which is always some effort. I hope you know what I mean? Using the DB directly would be fairly smooth, assuming there is a good library for the language of choice to communicate to the DB, which most probably is in most languages.
      "If we can live with some latency" --> Not a big issue, the application is living on one PC and maybe has permanent 2-3 viewers via browsers concurrently and some data exchange to the cloud and the customer network (for example if there is an ERP connection or similar).
      I'm not yet sure which approach I will take, It will take a little bit more time in my case to get a more bigger picture from the customers requirements and where he wants to go. I like the "merge" approach, resp. make a little bit bigger modules as you anyway mentioned you tented to have to much smaller modules in the first place which were chatty and you end up merging them anyway.
      As always with architecture, its all a tradeoff, but I really enjoyed your video, it stood out to me. Lots of other videos can stay very shallow or abstract.

  • @alabidavid4674
    @alabidavid4674 5 месяцев назад +1

    I intend to try this out on a nestjs app as I believe the knowledge is transferrable.
    My question is this: should every module have only a table? If not, can the tables that are under each modules share foreign keys?

    • @MilanJovanovicTech
      @MilanJovanovicTech  5 месяцев назад

      They can share foreign keys, as long as you're using one database.

    • @alabidavid4674
      @alabidavid4674 5 месяцев назад

      @@MilanJovanovicTech I appreciate your response.
      May I share the project with you here once I'm done? I would like to get your input on it. If that wouldn't be possible is there a means to get your input on something like this?

    • @MilanJovanovicTech
      @MilanJovanovicTech  5 месяцев назад

      @@alabidavid4674 Best shoot me an email, or I'll forget about it

    • @alabidavid4674
      @alabidavid4674 5 месяцев назад

      @@MilanJovanovicTech all right.
      Can I get it on your website?

  • @RN-jo8zt
    @RN-jo8zt Год назад +1

    can we use kafka in modular monolithic architecture?
    also you mentioned monolithic is communicated by method call. is this means by REST or SOAP api. is not it a network call?
    my question releted to modular monolithic not only monolthic .

    • @MilanJovanovicTech
      @MilanJovanovicTech  Год назад

      Yes - but you really don't need it. Even an in-memory queue should be sufficient to start off
      Communication by method call is just that - an in-memory method call

  • @Olosann
    @Olosann 7 месяцев назад +1

    Circle of creation ;) So it's simillar to how SAP or othe big vendors structured their ERP since 80's ?

  • @user-ow6gy6xj5v
    @user-ow6gy6xj5v Год назад +1

    Do you have a video for the modular monolith project setup? Thank you.

    • @MilanJovanovicTech
      @MilanJovanovicTech  Год назад +7

      No yet, unfortunately. Didn't have the time to get started on this. But it's coming.

    • @Stokmanne
      @Stokmanne Год назад +1

      @@MilanJovanovicTech - Awesome! I am excited to learn more about it.

    • @MXDMND_
      @MXDMND_ Год назад

      ​@@MilanJovanovicTechthanks Milan! Can't wait to see it :)

  • @phoenicianathletix2866
    @phoenicianathletix2866 Год назад +1

    for a video page if the video is served from Database A and the stats(views, likes,comments etc) of that video was served from Database B is that doable in the Modular Monolith without breaking anything?

  • @ml_serenity
    @ml_serenity Месяц назад

    I wouldn't call one deployment artifact to be "the best of both worlds" from the CICD perspective. I mean it's fine for the small projects, but for the medium and large ones it can make proper CICD nearly impossible or impossible at all.

  • @waggerdagger2831
    @waggerdagger2831 5 месяцев назад

    A MM cannot scale horizontally, unless the MM is stateless, is that correct?

    • @MilanJovanovicTech
      @MilanJovanovicTech  5 месяцев назад

      In theory, statefulness doesn't prevent you from scaling out. But it will cause a lot of headaches and probably a few bugs.
      But doesn't that apply to any type of architecture, really? (the statelessness remark, I mean)

  • @ml_serenity
    @ml_serenity Месяц назад

    I really fail to understand how modular monolith with inter-module communication implemented via the message bus is any better than the actual microservices? The advantage is clear when something like MediatR and CQRS is used due to the minimal overhead, but if we use a message bus then probably there is no point in having these modules inside a single deployment unit as we lose the ability to scale and deploy them independently as well as reducing the availability... Why give up the immediate consistency and minimal latency of the monolith?

    • @MilanJovanovicTech
      @MilanJovanovicTech  Месяц назад

      The message bus doesn't have to be distributed, to start. It could be an in-memory bus (like MediatR). It's an approach to control coupling, or rather make the modules less dependent on each other. Async communication is helpful when you know you'll be moving to a distributed system but don't want the overhead (of microservices) from the start. We found it quite helpful on my last project. There was a legacy system acting as the source of truth, so we had to translate from the old system's data model to an improved one in the new system, so we were already using messaging. Figuring out the module boundaries with (physically) separate service would've been a huge mess. With a monolith we could move quickly, and reorganize the design as needed.

  • @IlyaPermyakov
    @IlyaPermyakov 5 месяцев назад

    Thanks for the video, how do you deal with a lot of UI tasks/pages that need to read data from multiple different modules?

    • @MilanJovanovicTech
      @MilanJovanovicTech  5 месяцев назад

      As in, too many requests needed to serve a page?

    • @IlyaPermyakov
      @IlyaPermyakov 5 месяцев назад

      @@MilanJovanovicTech more like when you need to read from different bounded contexts , data is “isolated” but you always need it. Yeah you don’t read it from databases directly, you use API but eventually you create more coupling and your FE or BFF layer becomes itself a big ball of mud(messy monolith)

    • @IlyaPermyakov
      @IlyaPermyakov 5 месяцев назад

      @@MilanJovanovicTech vertical slices are not so vertical anymore

  • @ackuthex
    @ackuthex 7 месяцев назад +1

    I am also software eng.(good one), tried to start with Microservices my app, then realized half-road it is going to fail and fallback to monolith.. don't do this mistake.. before everything it will cost more

  • @vanasisbaboomi7819
    @vanasisbaboomi7819 Год назад

    what is difference between modular monolithic and SOA ( Service-oriented architecture )? is it same?

    • @MilanJovanovicTech
      @MilanJovanovicTech  Год назад

      Modular Monolith is still a single executable application. SOA has more than one executable, but not as fine-grained as Microservices.

  • @stephendgreen1502
    @stephendgreen1502 8 месяцев назад

    It would be nice if .Net catered better for modular monoliths. They are the youngest brother and the older ones, database-centric layered UI/BL/DAL, get all the attention and good stuff. Be nice if having a lot of modules projects within one solution worked better - if the solution could be compiled both in IDE and in Cloud without all the projects having to be at top level within the solution physical folder/files. That sort of thing. To make it more scalable. After all modular monolith arguably is the original and purest architecture - how modules were intended to work.

    • @MilanJovanovicTech
      @MilanJovanovicTech  8 месяцев назад

      Let's see if they manage to improve on this in .NET 9, since Aspire is a thing now

    • @stephendgreen1502
      @stephendgreen1502 8 месяцев назад

      @@MilanJovanovicTech Yes, we want something like Aspire but not just for microservices and Cloud Native apps but modular monolith too

  • @mdsafikulislam9113
    @mdsafikulislam9113 Месяц назад

    What is Message Bus?

    • @MilanJovanovicTech
      @MilanJovanovicTech  Месяц назад

      It's a components used for inter-process communication by sending messages. You'll often see RabbitMQ used for this.

  •  2 месяца назад

    It's coming...

  • @emreaka3965
    @emreaka3965 6 месяцев назад +1

    Thanks!

    • @MilanJovanovicTech
      @MilanJovanovicTech  6 месяцев назад +1

      Hey, thank you so much! I'm glad you enjoyed this talk on Modular monoliths :)

  • @dave7244
    @dave7244 Год назад +14

    I like how this is called a "Modular Monolith" these days it used to be just called organising your code in your project.

    • @MilanJovanovicTech
      @MilanJovanovicTech  Год назад +8

      It's more than just project structure

    • @dave7244
      @dave7244 Год назад +2

      @@MilanJovanovicTech A good 80% of this is literally the project structure and just SOLID.
      Also a lot of this stuff is overkill for even medium sized projects.
      Also a lot of us don't think Microservice scale at all and they aren't good for big teams (a team really shouldn't be big in the first place). There is a lot of things said in this presentation that are just plain wrong in practice.

    • @UserDirk
      @UserDirk 11 месяцев назад +1

      @@dave7244What things would you deem plain wrong?

    • @dave7244
      @dave7244 11 месяцев назад

      @@UserDirk I watched this like 6 months ago.

    • @UserDirk
      @UserDirk 11 месяцев назад +1

      @@dave7244 Okay too bad, I hoped you could say what you thought was plain wrong. I agree with everything in this video, but it's good to also hear other sides of the story. What points were missed etc.