Great explanation. When all requests are fast - throughput is CPU bound. When some/all requests are slow, they change from being limited by thread-count to becoming CPU bound or bound by the next bottleneck. For example, even with virtual threads, if I am limited to 50 DB connections, then all DB-blocked virtual threads will be processed 50-at-a-time. However, even if 10,000 virtual threads are waiting for 50 DB connections, still small requests like container health checks will still go through with Virtual Threads. That last part is important! If you are resource-bound your tests may show no improvement with Virtual Threads! But, we may miss the fact that container health checks will still go through and container will not get killed/restarted when Virtual Threads are used. Also, in case of old thread pools, we would hit external APIs at a resonable rate. With Virtual threads, we may flood the external API and overwhelm them or get blocked. Need to take care of rate-limiting when moving to Virtual Thread.
Do you guys think that with virtual threads we might be able to be done with the gore of callbacks, no matter how much we try to mask said gores with frameworks like promises? To be clear, I'm not looking for a debate on whether callbacks / promises are gory; I have respectfully reached an opinion *currently*, and it might change in the future. I'm simply wondering if you guys think that going back to blocking I/O will now be feasible with super-fast context switching offered by virtual threads.
Great explanation. When all requests are fast - throughput is CPU bound. When some/all requests are slow, they change from being limited by thread-count to becoming CPU bound or bound by the next bottleneck. For example, even with virtual threads, if I am limited to 50 DB connections, then all DB-blocked virtual threads will be processed 50-at-a-time. However, even if 10,000 virtual threads are waiting for 50 DB connections, still small requests like container health checks will still go through with Virtual Threads. That last part is important! If you are resource-bound your tests may show no improvement with Virtual Threads! But, we may miss the fact that container health checks will still go through and container will not get killed/restarted when Virtual Threads are used. Also, in case of old thread pools, we would hit external APIs at a resonable rate. With Virtual threads, we may flood the external API and overwhelm them or get blocked. Need to take care of rate-limiting when moving to Virtual Thread.
Nima (Helidon WebServer) need to be released as standalone artifact, so we can explore other frameworks on top it.
You should reneme the video from Virtual Threads to Helidon in Action
My thoughts exactly. I expected to learn about implementing stuff WITH virtual threads and/or how they work in action. Not a showcase of Helidon...
Superb explanation..!!
Do you guys think that with virtual threads we might be able to be done with the gore of callbacks, no matter how much we try to mask said gores with frameworks like promises? To be clear, I'm not looking for a debate on whether callbacks / promises are gory; I have respectfully reached an opinion *currently*, and it might change in the future. I'm simply wondering if you guys think that going back to blocking I/O will now be feasible with super-fast context switching offered by virtual threads.