Great video, João. 10 mins videos are just right, thank you for condensing the information and for sharing. It is great you also share where to go to dive deeper. Excellent!
@@CodingMilitia Would it be possible to share these slides used in the presentation? Talking about async/await is something complex, and I really liked your illustration and I intend to present this simplistic view, but it manages to get the message across perfectly.
This is the best explanation of the async/await pattern I've seen so far, great illustration. Can you make a high level video of the Actor model concept ? Thanks João
Thanks! About the actor model, I feel there's already a lot of good info on it (and I'm no expert on the topic), but I can take a look and see if there's a way I could explain things that would make it worth it.
Wow. This has to be the best high level explanation of async its so easy to comprehend and WHY we are doing it. I do find the async pollution an issue and I do wonder if there is a reason why not all applications can be async by default, i'm talking about some global project setting making all our requests asynchronous. So instead of having to declare it everywhere. Thank you so much for this video.
Excellent video! It makes me wonder why one would ever want to not have stuff working in the async way? Why do we have the async/await keywords at all? Why not just implement it in the language as the default way of working and maybe have it the other way around with a “sync” keyword?
From what I remember seeing floating around on Twitter recently (twitter.com/shanselman/status/1532982259640201217), there was some work to try to use fibers/green threads/those kinds of concepts, to achieve something like you're saying, but apparently with unsatisfactory results. With Go and Java embracing that though, the .NET team is also looking into it, so maybe we'll get there at some point.
Excellent video ;) . One tiny question, in the presentation you mentioned 'any other thread' picking up the work after the DB process gets done. But in the code there's no .ConfigureAwait(false) , hence the 'captured context' will be true and the code will resume ALWAYS using the same thread, right ? Keep up the good work !
Thanks. That was the behavior in ASP.NET pre-Core, and in desktop and mobile applications (though I don't have much experience with these), but it's not the case with ASP.NET Core. That was related to the synchronization context, which doesn't exist in ASP.NET Core. Here's a post, from the usual suspect, Stephen Cleary 🙂: blog.stephencleary.com/2017/03/aspnetcore-synchronization-context.html Great question though 😉.
That's a great question. I don't have numbers, so if you need them, you'll need to dive deeper, but I can explain from the understanding I have. Using async/await does introduce some overhead, but considering it's normally associated with I/O operations, which is an order of magnitude slower than the code running in process, managing the threads, this overhead becomes negligible.
Greate video! simple but important to grasp the basics. Besides the great information content, equally important, from my point of view, was the last side note mentioning EF Core DB Context not being thread safe. Important the highlight of perhaps, more than desirable mistakes. Could be a good strategy to keep doing shorter and high level videos like this just to point out some aspects. Then who ever wants can check out yout other more in dept videos. Keep up the excellent work you do!
@@CodingMilitia Usage of AsyncLocal as a way of sharing data in an asynchronous control flow; Also do a refresh on proper usage os dependency injection lifetime, meaning perhaps some injected services could be singleton instead of being scoped, or scope instead of transient (basically what questions "asked" to a given DI service to determine in a given context their lifetime).
Underlying it, some thread or process still needs to wait right? Who is that? For example, in NodeJS, this would be a pthread created by a C++ addon, or in the browser JS, it would be a thread created by web browser internally. But here what is it?
As far as I understand, there's no thread waiting on a particular I/O operation to complete. There is an I/O thread pool, from which a thread is borrowed to handle when the I/O operation is complete, then passing it to another thread to actually execute the continuation. An important article on the inner workings, is this one from Stephen Cleary: blog.stephencleary.com/2013/11/there-is-no-thread.html
Not exactly. If something is running, it needs a thread. The thing here is, when, for example, you make a database query, there’s nothing running locally, as the query is executing in the database server. For this reason, while waiting for the database server to respond, there is no need to hold a thread, as there is nothing to do. When the database server responds, then a thread can pick up the result and continue executing the application code.
@@CodingMilitia I probably am having a difficult time understanding what you mean by `while waiting for the database server to respond, there is no need to hold a thread, as there is nothing to do`. In the example in the video, when r1 comes and hits await line, the executing thread is freed up. An available thread from the pool is assigned with the io task. when r2 comes and hits await, the thread pool is checked if there are other threads available, but there should be no other threads available since the io op is running on a thread, i.e. the other thread is busy, thus here the await happens on the executing thread. when r3 comes it's just put on the queue to wait for either of the threads to be available on the pool. I.e. async await is not magically creating new threads that can magically wait for io ops, right ? I.e. the waiting still requires a thread or I'm probably very much missing something.
"I.e. the waiting still requires a thread" this is exactly the point, waiting does *not* require a thread. There are operating system level features that allow for us to start the IO, let the thread go and do something else, then only require a thread back when the IO is completed (e.g., in Windows, I/O completion ports). This post goes into more detail about what's going on under the hood, so it might help you understand the missing piece to make it click for you -> blog.stephencleary.com/2013/11/there-is-no-thread.html
@@CodingMilitia So then if you are referring to io threads, then why not say that clearly in the video example that there are t1, t2, tIO in the thread pool. And the t1 and t2 could be freed up when hitting the await, because the tIO can wait for multiple io ops ? Then the example would be complete, right ? As it stands now, the implication is that we only have 2 worker threads and there is no added benefit of the async/await when there are 3 requests. But if we have the t1, t2, tIO within the example, it makes everything much more clearer. Anyways, just food for thought. Thanks for the discussion.
There's a couple of things. First one, as I state in the beginning of the video, the idea is to be a high-level explanation, not going super deep, just for folks to understand why we're using async/await. For this reason, I left out other things, and focused on the work done by the request handling threads. For folks interested in more detail, I left the excellent Stephen Cleary post in the description. Second, it bears reiterating, no thread is blocked waiting for the result, including I/O threads. As you might have read in the linked post, there's a separate thread pool for I/O work, but those threads don't wait for the result, they're used only when the I/O is completed, to notify the tasks that they're ready to proceed,
Thanks for the explanation. I got it for the usage of async on the signature of apis. But whats the purpose of doing it on entity framework side? like AddAsync and SaveChangesAsync. Thanks
Using async on the signature of the APIs is irrelevant if you don't call async methods when handling the requests. The async on the signature is just like a marker to say that'll you'll be calling async methods when handling the requests (and awaiting said async methods). In the case of EF, what that means is, when you do an operation that does a request over the network to the database, like your examples AddAsync and SaveChangesAsync, the thread will not be blocked waiting for the database to respond, it can be used to do other things (e.g. handle other requests).
@@CodingMilitia oh ok. So the entire backend flow should be used in async way to handle this? Or mainly ef side? (Excep the case where we want to do parallel task on purpose, dont consider that case). So all the thread pool explanation was about EF side?
Yes, for async to work properly, it'll affect all layers of the application, from the entrypoint (the API), to all the I/O that it needs to do, be it access a database, make an HTTP request to another service, or just read/write to local files (in the video I focused on DB access for simplicity, but the same logic applies to the other types of I/O). The thread pool is handled by the .NET runtime, not specific to EF. As long as the libraries use the right primitives to implement I/O, it'll work the same for all of them.
Excellent explanation! I have to admit I had the same question, now I understand it better. ...but not quite yet. I get it when the database is on another (supposedly faster) server, and the CPU of the async/awaited machine is mostly just waiting for something to happen. But I usually run small project on a small budget, so my database server is the very same physical machine where my api endpoints are running , so the Threads, those yellow hat workers have to work also in the database department, so they can't go back to relax and lie by the pool (pun intended). How does it look like in my case, do I still have some advantage of sprinkling those async await structures all over my code? Or are there still things that can just happen in a computer without the CPU involved, (like seek times on a slow disk drive 20 years ago), so those threads can have a rest, I don't think it is the case any more.
That's a great question! Not sure I have the best of answers, but I'll give it a shot. If your small project is of such small scale that you don't have enough concurrent requests to exhaust the thread pool threads, then possibly you don't benefit from the async/await machinery. However, if the small scale is enough to exhaust the pool, potentially causing the thread pool to allocate more threads, then I'd say it might not be a lot, but you could possibly benefit from async/await, because these aren't API exclusive resources, these are machine resources, so if you avoid allocating threads on the API, means more free resources on the machine for the DB (or other processes) to use. Regarding things happening without a CPU involved, for sure there are, and you mentioned possibly the main one. Even with the fast SSDs we have today the CPU is still orders of magnitude faster executing instructions than IO can happen, so when you fetch from the SSD, the threads can still have a "rest". A couple of interesting resources about these latencies: - norvig.com/21-days.html#answers - pbs.twimg.com/media/EgW5_CPXYAE4PYr?format=png&name=900x900
@@CodingMilitia Thank you for the thorough update. two more arguments for using async/await come to my mind: 1. My setup can - hopefully - grow, one day I can outgrow the one machine, I will then need scalability without too much code rewriting. 2. it is more useful in the long term to get used to writing professional code. We use turn signals while driving even when no one is in sight, one may ask why: it is easier to just do it, than to always check before whether it is needed.
Good illustrations! However, the default thread will be the same before the DB call unless stated otherwise with ConfigureAwait(false). [I really don't know about the efficiency of the threadPool]. Also async creates a state machine which is quite expensive too (only use await when it is needed, else use only Task).
Hey, thanks for the comment. Some notes though: "However, the default thread will be the same before the DB call unless stated otherwise with ConfigureAwait(false)" this was true in ASP.NET pre-Core, where there was a SyncronizationContext that enforced that, but not really in ASP.NET Core. We might see that happening, but it's an implementation detail and not something guaranteed at all. As for the state machine being "quite expensive", not sure we could classify it as such, but yes, it adds overhead. Of course it depends on the types of applications we're developing, and their performance requirements. For the typical LOB applications, the code simplicity improvements in my opinion outweigh the performance gains.
Very detailed explanation, Thanks
Best explanation! No abstract examples or console app coding… good explanation for basis
Right to the point 👌 Thanks for this presentation ✌
very nice work! Thanks
Great video, João. 10 mins videos are just right, thank you for condensing the information and for sharing. It is great you also share where to go to dive deeper. Excellent!
Thanks!
Very good, you talked about a complex subject in a very didactic and simple way, even if briefly.
Thanks!
@@CodingMilitia Would it be possible to share these slides used in the presentation?
Talking about async/await is something complex, and I really liked your illustration and I intend to present this simplistic view, but it manages to get the message across perfectly.
Here you go: e1.pcloud.link/publink/show?code=XZym3RZLlGCiB9lkRX2pqoBRmcvsfIwFwey
(link will expire in a month or so)
@@CodingMilitia Many thanks, help me a lot.
Thank you! Very useful.
Great visual sample! Thanks!
Glad it’s useful! 🙂
thank you! very useful info!
This is the best explanation of the async/await pattern I've seen so far, great illustration. Can you make a high level video of the Actor model concept ?
Thanks João
Thanks!
About the actor model, I feel there's already a lot of good info on it (and I'm no expert on the topic), but I can take a look and see if there's a way I could explain things that would make it worth it.
Wow. This has to be the best high level explanation of async its so easy to comprehend and WHY we are doing it.
I do find the async pollution an issue and I do wonder if there is a reason why not all applications can be async by default, i'm talking about some global project setting making all our requests asynchronous. So instead of having to declare it everywhere.
Thank you so much for this video.
Thanks!
Thank you
Muito bom! O canal e o blog! :)
Obrigado!
Excellent video! It makes me wonder why one would ever want to not have stuff working in the async way? Why do we have the async/await keywords at all? Why not just implement it in the language as the default way of working and maybe have it the other way around with a “sync” keyword?
From what I remember seeing floating around on Twitter recently (twitter.com/shanselman/status/1532982259640201217), there was some work to try to use fibers/green threads/those kinds of concepts, to achieve something like you're saying, but apparently with unsatisfactory results.
With Go and Java embracing that though, the .NET team is also looking into it, so maybe we'll get there at some point.
Excellent video ;) . One tiny question, in the presentation you mentioned 'any other thread' picking up the work after the DB process gets done. But in the code there's no .ConfigureAwait(false) , hence the 'captured context' will be true and the code will resume ALWAYS using the same thread, right ?
Keep up the good work !
Thanks.
That was the behavior in ASP.NET pre-Core, and in desktop and mobile applications (though I don't have much experience with these), but it's not the case with ASP.NET Core.
That was related to the synchronization context, which doesn't exist in ASP.NET Core.
Here's a post, from the usual suspect, Stephen Cleary 🙂: blog.stephencleary.com/2017/03/aspnetcore-synchronization-context.html
Great question though 😉.
Nice 🔥🔥
what about the thread switching costs does ..should be consider while polluting simple code with async style
That's a great question.
I don't have numbers, so if you need them, you'll need to dive deeper, but I can explain from the understanding I have.
Using async/await does introduce some overhead, but considering it's normally associated with I/O operations, which is an order of magnitude slower than the code running in process, managing the threads, this overhead becomes negligible.
Greate video! simple but important to grasp the basics. Besides the great information content, equally important, from my point of view, was the last side note mentioning EF Core DB Context not being thread safe. Important the highlight of perhaps, more than desirable mistakes. Could be a good strategy to keep doing shorter and high level videos like this just to point out some aspects. Then who ever wants can check out yout other more in dept videos. Keep up the excellent work you do!
Thanks!
High-level videos are a good idea, just need to come up with topics. I’m open to suggestions 🙂.
@@CodingMilitia Usage of AsyncLocal as a way of sharing data in an asynchronous control flow; Also do a refresh on proper usage os dependency injection lifetime, meaning perhaps some injected services could be singleton instead of being scoped, or scope instead of transient (basically what questions "asked" to a given DI service to determine in a given context their lifetime).
Nice ideas, thanks!
Underlying it, some thread or process still needs to wait right? Who is that? For example, in NodeJS, this would be a pthread created by a C++ addon, or in the browser JS, it would be a thread created by web browser internally. But here what is it?
As far as I understand, there's no thread waiting on a particular I/O operation to complete.
There is an I/O thread pool, from which a thread is borrowed to handle when the I/O operation is complete, then passing it to another thread to actually execute the continuation.
An important article on the inner workings, is this one from Stephen Cleary: blog.stephencleary.com/2013/11/there-is-no-thread.html
So, the long running ops just magically execute somewhere not on a thread, right ?
Not exactly.
If something is running, it needs a thread.
The thing here is, when, for example, you make a database query, there’s nothing running locally, as the query is executing in the database server. For this reason, while waiting for the database server to respond, there is no need to hold a thread, as there is nothing to do. When the database server responds, then a thread can pick up the result and continue executing the application code.
@@CodingMilitia I probably am having a difficult time understanding what you mean by `while waiting for the database server to respond, there is no need to hold a thread, as there is nothing to do`.
In the example in the video,
when r1 comes and hits await line, the executing thread is freed up.
An available thread from the pool is assigned with the io task.
when r2 comes and hits await, the thread pool is checked if there are other threads available,
but there should be no other threads available since the io op is running on a thread, i.e. the other thread is busy,
thus here the await happens on the executing thread.
when r3 comes it's just put on the queue to wait for either of the threads to be available on the pool.
I.e. async await is not magically creating new threads that can magically wait for io ops, right ?
I.e. the waiting still requires a thread or I'm probably very much missing something.
"I.e. the waiting still requires a thread" this is exactly the point, waiting does *not* require a thread.
There are operating system level features that allow for us to start the IO, let the thread go and do something else, then only require a thread back when the IO is completed (e.g., in Windows, I/O completion ports).
This post goes into more detail about what's going on under the hood, so it might help you understand the missing piece to make it click for you -> blog.stephencleary.com/2013/11/there-is-no-thread.html
@@CodingMilitia So then if you are referring to io threads, then why not say that clearly in the video example that there are t1, t2, tIO in the thread pool.
And the t1 and t2 could be freed up when hitting the await, because the tIO can wait for multiple io ops ?
Then the example would be complete, right ?
As it stands now, the implication is that we only have 2 worker threads and there is no added benefit of the async/await when there are 3 requests.
But if we have the t1, t2, tIO within the example, it makes everything much more clearer.
Anyways, just food for thought.
Thanks for the discussion.
There's a couple of things.
First one, as I state in the beginning of the video, the idea is to be a high-level explanation, not going super deep, just for folks to understand why we're using async/await. For this reason, I left out other things, and focused on the work done by the request handling threads. For folks interested in more detail, I left the excellent Stephen Cleary post in the description.
Second, it bears reiterating, no thread is blocked waiting for the result, including I/O threads. As you might have read in the linked post, there's a separate thread pool for I/O work, but those threads don't wait for the result, they're used only when the I/O is completed, to notify the tasks that they're ready to proceed,
Thanks for the explanation. I got it for the usage of async on the signature of apis. But whats the purpose of doing it on entity framework side? like AddAsync and SaveChangesAsync. Thanks
Using async on the signature of the APIs is irrelevant if you don't call async methods when handling the requests. The async on the signature is just like a marker to say that'll you'll be calling async methods when handling the requests (and awaiting said async methods).
In the case of EF, what that means is, when you do an operation that does a request over the network to the database, like your examples AddAsync and SaveChangesAsync, the thread will not be blocked waiting for the database to respond, it can be used to do other things (e.g. handle other requests).
@@CodingMilitia oh ok. So the entire backend flow should be used in async way to handle this?
Or mainly ef side? (Excep the case where we want to do parallel task on purpose, dont consider that case).
So all the thread pool explanation was about EF side?
Yes, for async to work properly, it'll affect all layers of the application, from the entrypoint (the API), to all the I/O that it needs to do, be it access a database, make an HTTP request to another service, or just read/write to local files (in the video I focused on DB access for simplicity, but the same logic applies to the other types of I/O).
The thread pool is handled by the .NET runtime, not specific to EF. As long as the libraries use the right primitives to implement I/O, it'll work the same for all of them.
@@CodingMilitia yeah ok. So async like that works for any kind of I/O. Thanks a lot
Excellent explanation! I have to admit I had the same question, now I understand it better.
...but not quite yet. I get it when the database is on another (supposedly faster) server, and the CPU of the async/awaited machine is mostly just waiting for something to happen. But I usually run small project on a small budget, so my database server is the very same physical machine where my api endpoints are running , so the Threads, those yellow hat workers have to work also in the database department, so they can't go back to relax and lie by the pool (pun intended). How does it look like in my case, do I still have some advantage of sprinkling those async await structures all over my code?
Or are there still things that can just happen in a computer without the CPU involved, (like seek times on a slow disk drive 20 years ago), so those threads can have a rest, I don't think it is the case any more.
That's a great question!
Not sure I have the best of answers, but I'll give it a shot.
If your small project is of such small scale that you don't have enough concurrent requests to exhaust the thread pool threads, then possibly you don't benefit from the async/await machinery.
However, if the small scale is enough to exhaust the pool, potentially causing the thread pool to allocate more threads, then I'd say it might not be a lot, but you could possibly benefit from async/await, because these aren't API exclusive resources, these are machine resources, so if you avoid allocating threads on the API, means more free resources on the machine for the DB (or other processes) to use.
Regarding things happening without a CPU involved, for sure there are, and you mentioned possibly the main one. Even with the fast SSDs we have today the CPU is still orders of magnitude faster executing instructions than IO can happen, so when you fetch from the SSD, the threads can still have a "rest".
A couple of interesting resources about these latencies:
- norvig.com/21-days.html#answers
- pbs.twimg.com/media/EgW5_CPXYAE4PYr?format=png&name=900x900
@@CodingMilitia Thank you for the thorough update. two more arguments for using async/await come to my mind:
1. My setup can - hopefully - grow, one day I can outgrow the one machine, I will then need scalability without too much code rewriting.
2. it is more useful in the long term to get used to writing professional code. We use turn signals while driving even when no one is in sight, one may ask why: it is easier to just do it, than to always check before whether it is needed.
@nick066hu 💯
Good illustrations! However, the default thread will be the same before the DB call unless stated otherwise with ConfigureAwait(false). [I really don't know about the efficiency of the threadPool]. Also async creates a state machine which is quite expensive too (only use await when it is needed, else use only Task).
Hey, thanks for the comment. Some notes though:
"However, the default thread will be the same before the DB call unless stated otherwise with ConfigureAwait(false)" this was true in ASP.NET pre-Core, where there was a SyncronizationContext that enforced that, but not really in ASP.NET Core. We might see that happening, but it's an implementation detail and not something guaranteed at all.
As for the state machine being "quite expensive", not sure we could classify it as such, but yes, it adds overhead. Of course it depends on the types of applications we're developing, and their performance requirements. For the typical LOB applications, the code simplicity improvements in my opinion outweigh the performance gains.