How to use Multithreading with "worker threads" in Node.js?

Поделиться
HTML-код
  • Опубликовано: 31 янв 2025

Комментарии • 114

  • @sohaibnoor6610
    @sohaibnoor6610 7 месяцев назад +22

    I have 3.5 years of Experience as a Node js Developer. But I feel I knew nothing. Thank you so much man. This is mind blowing. Love every of it

  • @guytonedhai
    @guytonedhai Год назад +13

    This is gold! I have been binge watching your channel. I wish I had discovered your channel much before.

  • @leeshey99
    @leeshey99 10 месяцев назад +2

    The most intuitive video of multithreading nodejs out there

  • @Dev-Siri
    @Dev-Siri 2 года назад +6

    This is the best video I have found that teaches how to use multithreading in Node

  • @raviagheda5564
    @raviagheda5564 10 месяцев назад

    Awesome content, Thanks for the creating video Multi Threading. Explanation was too smooth, and especially loved the optimization part.

  • @codingispower1816
    @codingispower1816 Год назад +1

    Great video! Excited for this whole series!

  • @Mainak908
    @Mainak908 Год назад +1

    Best video I have ever seen on node multithreading👍

  • @KinoPM2
    @KinoPM2 7 месяцев назад +2

    Google is playing a joke on me. I spent a month fiddling with the first topic, with the promise of all cycles. I just solved the problem and RUclips immediately suggested your video...

  • @pabloroque8268
    @pabloroque8268 9 месяцев назад +1

    Great content, i hope you to do more videos with advanced concepts and proyects with nodejs, best regards from Argentina

  • @vitto_pincharrata
    @vitto_pincharrata Год назад +15

    An amazing oportunity to use os.cpus()!!
    + import os from "os";
    ...
    - const THREAD_COUNT = os.cpus().length;
    and reduce as well!
    - const total = thread_results.reduce((x, y) => x + y);
    Thanks for the video! :3

    • @TheLexa
      @TheLexa 7 месяцев назад +2

      const { cpus } = require("os");
      const THREAD_COUNT = cpus().length;
      maybe better option

  • @gillesashley9314
    @gillesashley9314 20 дней назад

    This man is too good.

  • @subhamshaw1726
    @subhamshaw1726 6 месяцев назад

    Wow This is amazing. very well explained as well. I am gonna use it where ever possible. Thank you for making this video. loved it. 😍😍

  • @jefersoncosta2621
    @jefersoncosta2621 8 месяцев назад

    This playlist is awesome! Thank you!

  • @Pareshbpatel
    @Pareshbpatel Год назад +1

    An excellent tutorial on Multithreading in Node using Worker Threads. Thank you very much.
    {2024-01-12}, {2024-01-15}, {2024-01-19}.{2024-11-20}

  • @OleksandrDanylchenko2k
    @OleksandrDanylchenko2k 2 года назад +2

    Really nice guide! Thanks 🙌🏻

  • @Jberv
    @Jberv 2 года назад +1

    Deym, I thought i'm on my limits for optimizing my api, Thank you so much for this

  • @arkadiptadas4148
    @arkadiptadas4148 2 года назад +1

    Super helpful. Thank you so much!

  • @Blossemcraft
    @Blossemcraft 11 месяцев назад

    You solved a great problem. Thanks a lot

  • @ahmedsoran4710
    @ahmedsoran4710 2 года назад +1

    thank you for sharing this was so helpful

  • @Abdulkalam-he7wo
    @Abdulkalam-he7wo Год назад

    great work , explanation at its best..!

  • @shivanshpatel4072
    @shivanshpatel4072 Месяц назад

    Awesome tutorial

  • @rasulrukh2907
    @rasulrukh2907 8 месяцев назад

    great job man!

  • @carlosdelgado5632
    @carlosdelgado5632 2 года назад

    Excellent explanation, thank you

  • @acluster3411
    @acluster3411 Год назад

    Very well done!

  • @vicTor-ft8xv
    @vicTor-ft8xv Год назад +1

    Thank you brother

  • @gregoax5474
    @gregoax5474 11 месяцев назад

    great content!

  • @caotichcuc
    @caotichcuc 8 месяцев назад

    God, it's super helpful. Please make more videos about APIs optimization and security for Nodejs, pleaseeeeeeeeeeeeeeeeeeeeeeeeeee

  • @s.mehradi
    @s.mehradi 6 месяцев назад

    Thanks a million! 😊

  • @samiranroyy1700
    @samiranroyy1700 10 дней назад

    as a fresher, mern stack developer..should i learn this playlist?

  • @ronnykgalema
    @ronnykgalema Год назад +3

    Imagine combining multithreading with clustering? That would be a bomb 💣

  • @ahmeterdonmez9195
    @ahmeterdonmez9195 Год назад +2

    hello brother. Can we use this to handle 2-3 different tasks at the same time? I subscribed your channel ;)

  • @_digital_nomad_
    @_digital_nomad_ Месяц назад

    It's very cool, but how to understand how many threads will you need? Just check it manually?

  • @frontend_ko
    @frontend_ko Год назад

    okay, understood, thanks for nice tip

  • @MykhailoKlym
    @MykhailoKlym 10 месяцев назад

    Technically, node does not use threads for network operations. It uses different mechanism called async io. But the idea is similar - main thread is also not blocked.

  • @Aleks-fp1kq
    @Aleks-fp1kq 10 месяцев назад

    Hello. why do you need the number of cores? If I understand it correctly, the new loop is located in the same process but on a different thread.

  • @realanikahmed
    @realanikahmed 9 месяцев назад

    Sir Which theme you are using? It's cool for the eyes.

  • @RealEstateAPBRDev
    @RealEstateAPBRDev 11 месяцев назад +1

    I think while creating worker threads , new main thread is not created. Correct me if I am wrong.

  • @labidyramzi4840
    @labidyramzi4840 Год назад

    Question: What if I will be using "docker + kubernetes" to horizontally scale my node app and at the same time i have a file upload + file processing intensive task in my app should I still use "worker threads" to manage the intensive tasks or "docker+kubernetes" will solve the problem anyway ?
    Thank you for this amazing explanation !

  • @parassharma7041
    @parassharma7041 8 месяцев назад

    Great video

  • @alirezamosavi6185
    @alirezamosavi6185 Год назад

    Great bro

  • @atmai1026
    @atmai1026 10 месяцев назад

    I has a question that when we use Multithreading with "worker threads" or real applications of it. I have saw many Nodes courses and they just introduce about it, not talk about the real application

  • @ayeshazahid2421
    @ayeshazahid2421 Год назад +1

    awesome video
    but I have a question, can you elaborate on the difference between background jobs and worker threads. I have a situation in which I need to call db recursively then call third-party API which takes some time to respond then refactor data then filter specific data and again call third-party API and then bulk insert into db
    + I need to perform this action inside cron jobs which get triggered after every 3 hours
    so what should I need to use???? and I'm using ec2 which has only 1 core so I think I can't divide it if my solution is worker thread

    • @SoftwareDeveloperDiaries
      @SoftwareDeveloperDiaries  Год назад +3

      Thank you! :)
      Sounds like you need a queueing mechanism for such a bulky operation. Take a look at RabbitMQ or Apache Kafka and how you can integrate them with your existing API.

    • @ayeshazahid2421
      @ayeshazahid2421 Год назад

      @@SoftwareDeveloperDiaries okay 🤩 thanks again sir

  • @zoeycvu
    @zoeycvu 4 месяца назад

    perfect!

  • @vaibhavdani4470
    @vaibhavdani4470 6 месяцев назад

    Though, one thing I could not digest,
    20M loops by one thread Vs 4 threads counting 5 M loops simultaneously / parallelly :->
    1. addition of their count will be 5M + 5M + 5M + 5M = 20 M loops for sure as mathematically it is correct. But,
    As these 4 threads are working parallelly time taken by them is almost equal which is of counting only 5M loops.
    Diagrammatically,
    1 Thread ---------------- 20 M loops ---------------------> 12 Sec (let's say)
    1/4 Thread --------5M loops ------------> 3 Sec (let's say)
    2/4 Thread --------5M loops ------------> 3 Sec
    3/4 Thread --------5M loops ------------> 3 Sec
    4/4 Thread --------5M loops ------------> 3 Sec
    Here, we simply reduced our task to count form 20 M to 5 M, hence time taken describes to count 5M only not 20M.

    • @vaibhavdani4470
      @vaibhavdani4470 6 месяцев назад

      Are they counting like
      1st Thread => 1M ---> 5M , 2nd Thread => 6M ---> 10 M , 3rd Thread => 11M ----> 15 M, 4th Thread => 16M ---> 20M .... ??? No , all threads are counting form 1 to 5M and parallelly hence time taken describes to count 5M only not 20M

  • @vb4190
    @vb4190 Год назад

    and what will happen when you run out of all your worker threads? in case of multiple requests to blocking endpoint by different clients

    • @SoftwareDeveloperDiaries
      @SoftwareDeveloperDiaries  Год назад

      The machine will have to launch a fair amount of worker threads and get noticeably slow, before it throws an error that it can't launch a new thread. But this is probably the sequence that you'd see.

    • @vb4190
      @vb4190 Год назад +1

      looks like a case that should be handled)

  • @jsarabia_dev7371
    @jsarabia_dev7371 9 месяцев назад

    If our app is inside of pods of kubernetes is not possible the Multithreading if has not have more 1 CPU for pod?

    • @SoftwareDeveloperDiaries
      @SoftwareDeveloperDiaries  9 месяцев назад

      That's a great question! In a Kubernetes environment, each pod typically runs a single container, and the number of CPUs available to a pod depends on the resources allocated to it. If a pod has only one CPU allocated, then indeed, multithreading might not provide significant benefits since there's only one CPU core available for processing.

  • @muhammedhuzaifa1392
    @muhammedhuzaifa1392 7 месяцев назад

    I have a question,
    If I cluster node js app on 8 cors and if I use thread on each computation in my code,
    So assumption is my same code is running on 8 clusters and all have threads in them mean to say the api I wrote in my single code is running on 8 different cors, does it improve performance and if it improve performance? Can you tell me pros and cons of it?

  • @vaibhavm1986
    @vaibhavm1986 Год назад

    Hi , one question is like I have scenario in which have to do file operation many times we are using some third party library to club or combine small doc files to one big final pdf , but that has to be in particular order , there are 20 sections , each section is tree containing further sections and so on and so forth ,each section contains docx file that has to be merged, so can I divide workers to club 20/ thread-count and then finally club the result of all of them to final file, only thing is I would have to find thread count , programmatically
    do you agree? is there any other way you can recommend

    • @SoftwareDeveloperDiaries
      @SoftwareDeveloperDiaries  Год назад

      Maybe try worker threads? I have a video on it too. If that doesn't help, you can try asking ChatGPT, I'm pretty sure it knows everything :d

    • @vaibhavm1986
      @vaibhavm1986 Год назад

      @@SoftwareDeveloperDiaries thanks

  • @verdynandreyhans9592
    @verdynandreyhans9592 Год назад

    thanks bro,

  • @reactjs1900
    @reactjs1900 Год назад

    What is the difference between workers and fork or child process

    • @jperusm
      @jperusm 11 месяцев назад

      Processes take much more resources to spawn while threads are lighter and are handled by the same process

  • @vaibhavm1986
    @vaibhavm1986 Год назад

    Hi,
    I did not understand the idea of 7 threads , kindly correct me If this is wrong from what I know is UV_THREADPOOL_SIZE = 4 , by default and these are the only threads used by libuv for some of the operations! i.e except CPU intensive tasks

    • @SoftwareDeveloperDiaries
      @SoftwareDeveloperDiaries  Год назад

      Correct. When we refer to a "thread", we actually mean a "thread pool". With UV_THREADPOOL_SIZE you define the size of the pool, which by default consists of 4 "child threads". You can still increase the size of the pool up to 1024.

    • @vaibhavm1986
      @vaibhavm1986 Год назад

      @@SoftwareDeveloperDiaries is it practically possible to have 1024 threads ?

    • @vaibhavm1986
      @vaibhavm1986 Год назад

      my cloud setup is having 7 nodes each node has 2 vcpu , virtual CPU
      I am not sure how many threads each instance has , I printed cpus length using os module it printed 2 , so only 2 threads ? do you suggest any other way or to know about it

    • @vinaykumarkharwar177
      @vinaykumarkharwar177 Год назад

      @@vaibhavm1986 It can't, the UV_THREADPOOL_SIZE can't be bigger than the number of total logical threads on your CPU. If you have an Octa-Core CPU each of them have 2 threads, then the logical threads at your disposal will be 8 * 2. It can't be more. To automate the code, you can use -
      const os = require("os")
      process.env.UV_THREADPOOL_SIZE = os.cpus().length
      Also, the threads used in the worker pool are not real threads. They are circumventive solution for multithreading. Possibly very close to the Co-routines of Kotlin language. Or GoRoutines from Go lang. In kotlin, you can have thousands of coroutines with multiple tasks.
      If you are looking for multithreading, you can use cluster module from node.js, which spawns many worker instances on each cpu thread, but yeah they are still memory intensive and mixing worker_thread with clustering is bad idea.

  • @firoj.siddiki
    @firoj.siddiki Год назад

    subscribed.

  • @muharief3885
    @muharief3885 2 года назад +9

    great for single good server, bad for container. This way was just make container suffer bcose the resources already taken up by the io operation.

    • @davitti7036
      @davitti7036 2 месяца назад

      What do you suggest then

  •  8 месяцев назад

    0:19 - wait, I'm not ready yet!

  • @aki1840
    @aki1840 Год назад

    Thanks for this video! All Node haters - in your face! :D

  • @swoopertr
    @swoopertr 2 месяца назад

    where is the source code link?

  • @dukim632
    @dukim632 6 месяцев назад

    Great video!
    but most likely you work in a company that uses containerized environment (unless you live under a rock) and if thats the case - dont bother, just put 1.25 cpu requests and add more replicas!
    its safer and much more scalable, also in containers environment you cant trust the core count since it works on CPU time

  • @esp2644
    @esp2644 2 года назад

    How download code example?

    • @SoftwareDeveloperDiaries
      @SoftwareDeveloperDiaries  2 года назад +1

      Hey, there's none. I was hoping you could reproduce it by watching the video

    • @swagz6440
      @swagz6440 Год назад

      How hard is it to write that

  • @chavdarajan7428
    @chavdarajan7428 9 месяцев назад

    ACCHA KAM 👍 - HANCOCK

  • @KinoPM2
    @KinoPM2 7 месяцев назад

    Just learn my friends, and you will get where you want

  • @sreeraghuvardhanvangipuram9547
    @sreeraghuvardhanvangipuram9547 Месяц назад

    Audio is slightly off-sync with the video. Great vid tho!

  • @meandradethiago
    @meandradethiago Год назад +1

    Jesus loves you, man. Thanks for this incredible video.

  • @innoventstack
    @innoventstack Год назад

    Hey Use cluster module instead of this much better approach

  • @brooklyn6683
    @brooklyn6683 4 месяца назад

    fuck bro 😶‍🌫

  • @Flonfl0n
    @Flonfl0n 9 месяцев назад

    Great video, but you should have used a reduce to sum your results. I'm triggered haha

    • @SoftwareDeveloperDiaries
      @SoftwareDeveloperDiaries  9 месяцев назад

      Check this out, it's better to use good old for loops when working with a lot of data :)
      leanylabs.com/blog/js-forEach-map-reduce-vs-for-for_of/

    • @Flonfl0n
      @Flonfl0n 9 месяцев назад

      @SoftwareDeveloperDiaries I meant to sum the result of each thread.
      Total = thread_results[0] + thread_results[1]....

    • @SoftwareDeveloperDiaries
      @SoftwareDeveloperDiaries  9 месяцев назад

      @@Flonfl0n oh i see ^^

    • @Flonfl0n
      @Flonfl0n 9 месяцев назад

      @@SoftwareDeveloperDiaries hehehe