Damn i really struggled with moving a Rc into a closure and then to another function. Seeing you doing it with Arc helped a lot. Never would have thought of simply cloning it lol or using the "move" before the closure.
I mean i went through many other materials for rust but this content is the best for understanding how rust really works... So overwhelmed and hyped for rust on my hands when i know what i am doing...
I'm a C++ veteran among other languages, and I haven't written a single line of rust yet, but I've been shopping for languages for a new project, catching up on what's out there. I've been reading the documentation, watching conference talks (the rush to stay under the time limit is really not helping, lots of hype and buzz words lol) and checking out the ecosystem, and this video is probably the most helpful to understand what I'll be dealing with in terms of sitting down, writing code, and thinking of all the concerns I need to be aware of. You explain things very nicely. Thanks for making this! I'll definitely be checking out your other videos.
I can't imagine you only have 2.5k subscribers. This content is first class, one of the better tutorials I've ever seen. 10/10 from me, great job explaining what in the end are hard subjects.
For anyone else getting errors when following, this could be because you are entering the grey text, which is inline annotaion (from the Rust-Analyzer VS code extension, mentioned @34:09). Just remove/ignore all grey characters (including some colons) and it should work. Also: Great tutorial, thanks so much!
This is one of your best videos I've seen - maybe because it's just at the right level for where I'm at. But all of them are great. Thanks for doing these!
EDIT: You've answered it for me partially a bit later in the video. But what I still don't think I understand, is the difference between Box and &dyn Fn(). Why is there a & needed in the second notation? 23:55 Ryan, is there any way to care, so to speak, about the concrete Fn type in this example, such that we would implement static dispatch instead of dynamic dispatch? Like, could you not Box that Fn trait? Can't you just take a function pointer? I mean can't you write a pool that only works with a single kind of closure type? I'm leaning towards you can't because Rust needs to know the sizes of everything it puts on the stack, right? Thanks!
1:48:56 I don't understand how moving the Arc nref two times into the foo closure by calling pool.execute(foo.clone()) twice, doesn't throw and error. Shouldn't nref be dropped the first time the execute function finishes, the first execute?
1:56:20 question: is it necessary to explicitly handle the Result returned by recv like that (line 19)? Wouldn't recv.unwrap() have the same effect basically? Wouldn't unwrap() invoke the panic macro if the Result maps to an Error, which in turn would end the thread? Or is it just explicitly handled in a match to "gracefully" end the thread (so it doesn't display the panic message)? Correct me if I'm wrong, still learning Rust. Nice video BTW! Really helped me
Came here just after finishing the last chapter (webserver with threadpool) from "The Book". This was a great addendum to that, however towards the end of the stream things got a little rushed. Looking forward to your next stream!
For anyone who might have the same problems as me. At around 1h:25min where tests are ran I had a strange problem. My tests executed OK with no output. It wasn't until I added sleeping at the end of the test I got the output. This is because my virtual machine has only one processor core and threads couldn't even start working before the program was done.
This is awesome. I'm at 1:24:00, but I have to stop the kata and ask, what would have happened if you had followed the Fn() error message (needed + Send), made that fix, and then worked out the Arc solution off new errors. When that error first came up I would have tried to Arc the Fn() passed to execute(work), based on the docs and the error you were looking at. I would still be lost right now I think.
At 1:49:37, I'm looking over the code and thinking about what happens with captured variables inside the closure when the closure itself is cloned. I'm guessing the ARC counter gets increased automatically in this case. Any thoughts?
The demonstration of the 'static bound at 01:02:00 was really useful. Does the fact that execute() on line 27 uses static dispatch mean it would generate a different version for each closure via monomorphisation? Or do the closures share the same underlying type? Also I once had an issue with using Atomics in tests where Cargo was running the tests in parallel and (maybe due to Relaxed ordering?) the different test cases could interfere with eachother - as though it were a static variable. You can tell cargo test not to run them in parallel in this case though.
Probably the same code as far as binary size, but different captured values on the heap. Atomics don't really guarantee exclusive access, they only guarantee that your thread can not be interrupted during the 1 or 2 clock cycles between reading and writing that value. If you need to read a value, process it in some way beyond a single cpu instruction, and then store it back, you really need a critical section or mutex.
I have enough experience with Rust to know that the solution to the sharing of the Receiver is Arc, but I don't really think that follows obviously from the error the compiler gives. In fact, I feel like the most naive thing one might try when getting that error is to change dyn Fn() to dyn Fn() + Send. EDIT: I see now that you got to that after finishing up with the Arc-Mutex thing, so I guess it wasn't left out. I just thought the two concepts came in the wrong order, at least given what the compiler was talking about.
That is a feature of the rust-analyzer extension in VSCode. I think it is enabled by default but you can find it as "inlay hints > type hints" in the settings.
I'm sure Ryan knows this, but for others.... While it's true that the extra indirection makes calls on dyn traits a bit slower, that isn't the only reason it's slower, or even necessarily the most important. Function calls on dyn traits are not inlined, and inlining is often what leads to dramatic optimizations.
Is this executing work in parallell? Since the mutex is locked durring work only one thread at a time is able to work, so if you have 10 threads, one would work and the other nine would wait for the mutex to unlock and when it does, another thread would acquire the mutex and the remaining nine would wait again. Is this correct or am I reading the code wrong? (sorry for the broken english)
How did he get those gray type annotations? Is that a plugin? Also what color theme is that? My default VSC rust theme has for ex. "pub struct" and "impl" blue.
@@honzasedlon3309 So which VSCode settings need to change to get that? Just changing the language server from rls to rust-analyzer didn't seem to do it.
Hey, thanks a lot for these videos. They are very helpful. I followed your example and also had I look at your code in git and I came across an issue. It seems that only one worker is executing all the functions. Is there a reason for this?
having a lot of run!!! Send Trait make me a bit confused even i know for thread safety. At same time ,if receiver unlock made by manual and before work, would something goes wrong? (Forgive me for not being a native speaker of English)
It's ok to call "work" after the Mutex is unlocked, because when we receive work, we get ownership of that work. It's moved out from the Receiver inside the Mutex. The Mutex cannot be accidentally unlocked while the thread is accessing the Receiver, because when we lock the Mutex we get a MutexGuard. This MutexGuard lets us borrow a reference to the data within the Mutex, and Rust enforces that this borrow must end before the guard is dropped. When the MutexGuard is dropped, it unlocks the Mutex. Dropping the guard is the only way to unlock the Mutex.
Any idea why Arc doesn't implement copy?. Passing a cloned arc is so normal that it should surely be clear even if it is implicit. I would have thought the compiler would be smart enough to not call copy when it is not needed, and there are presumably some cases where the compiler could be smart enough to realise that it doesn't need to copy. Explicitly telling it to clone would be a mistake in these cases. The whole point of Arc is that cloning it is easy and safe. It would make teaching a little more involved, but avoiding it for that reason just makes code where it is implemented harder to understand, as it is less normal.
Copy has the simple semantics of bitwise copy. Copying never involves anything other than copying bits. Cloning an Arc requires an incrementing a reference count so it's not just a bitwise copy and thus copy is not the appropriate trait to model this.
@@RyanLevicksVideos Copy is a trait, so surely lives in specification land decoupled from implementation details. Is there anything stopping it being implemented in ways other than a bitwise copy? Is copy hardwired into the compiler to do something specific when it is implemented, and always be automatically implemented? Not trying to make a point with that, just fairly new to rust and actually don't know. I suppose you might find a case where you could get away with borrowing an Arc, avoiding the clone and destroy step (As I said I'm new, so this could be a common pattern). Even then, nothing stops you doing that if copy is implemented. The only other downside I can see is if the compiler copies even at last use, potentially causing more clones than necessary. That should be relatively easily fixable, but if the assumption has been that only bitwise copies are copy then that might be how things are. Bitwise copying it is not appropriate, but you do end up using it in the same way as a value type that is copied on passing. You just end up writing clone everywhere and manually implementing it inline, at least in the code I've come across (may not be representative). Binging the streams btw. Best resource out there after the book, and complements the book fantastically.
@@agsystems8220 Yes Copy is "special" in that it has specific meaning tied to the compiler of bitwise copy. This is how Rust represents changing from the default move sematics to copy semantics. You can borrow an Arc so something requires you to clone it. Rust generally favors being explicit, and I think having to type "clone" is a small price to pay for the clarity it brings.
Gread stream. The only thing I don't agree with it the middle part about channel. It's not like you are wrong just the order kina backward. Make sure the data you send to another thread is Send. Then make sure your sender/receiver is Send. Then if you ever want to share the send/receiver between threads make sure they are also Sync. There are various channels introduced by lots of 3rd party crates and in this way beginners can have a easier time to figure them out.
I wouldn't use mpsc from std as it has aged panic bug which can hurt you in production (github.com/rust-lang/rust/issues/39364). Better to use crossbeam from the scratch. For more info refer to stjepang.github.io/2019/03/02/new-channels.html
I was looking for something more advanced to solve the problems i have with concurrency and closures in my open source project (github.com/sinasalek/clipboard_guardian) and this was incredibly useful. thanks Ryan
Why bother with sending receiver to a spawned thread when you can spawn thread with receiver and return a sender to the main thread? like this gist.github.com/JohnDowson/b607c49ef3aba7bae86bf589c7f91322 I guess that forces you to implement some way to keep track of job queues, but at least you aren't piling wrappers around wrappers in attempts to trick borrow checker into allowing you to compile your questionable design.
Rust really makes every programming concept explicit.
This is gem of content
Killer content -- us beginners trying to level up need more of this :)
Thank you.
I realize I am kinda off topic but does anyone know a good place to watch new series online?
@Boone Samuel thanks, I signed up and it seems like they got a lot of movies there =) I really appreciate it!
@Ariel Krew Glad I could help :)
Damn i really struggled with moving a Rc into a closure and then to another function. Seeing you doing it with Arc helped a lot. Never would have thought of simply cloning it lol or using the "move" before the closure.
I mean i went through many other materials for rust but this content is the best for understanding how rust really works... So overwhelmed and hyped for rust on my hands when i know what i am doing...
I'm 55 minutes in and boy is this stuff hard both to understand and explain! Thank you for taking the time to making it easier I loved it.
one of the best Rust resource is this channel 🖤
I'm a C++ veteran among other languages, and I haven't written a single line of rust yet, but I've been shopping for languages for a new project, catching up on what's out there. I've been reading the documentation, watching conference talks (the rush to stay under the time limit is really not helping, lots of hype and buzz words lol) and checking out the ecosystem, and this video is probably the most helpful to understand what I'll be dealing with in terms of sitting down, writing code, and thinking of all the concerns I need to be aware of. You explain things very nicely. Thanks for making this! I'll definitely be checking out your other videos.
the ownership description at 8:20 clicked for me and I made the self "killing" changes for my code. Thanks for inspiring.
Thank you so much! This is one of the best tutorials I have watched! 💯
Your channel and content is gold. thank you very much and appreciate your dedication. Please make more videos and content
Very good! This style of teaching is exactly what Rustlings need!
I can't imagine you only have 2.5k subscribers. This content is first class, one of the better tutorials I've ever seen. 10/10 from me, great job explaining what in the end are hard subjects.
His tutorial on lifetimes is invaluable too. I finally got a solid grasp on it thanks to his video
For anyone else getting errors when following, this could be because you are entering the grey text, which is inline annotaion (from the Rust-Analyzer VS code extension, mentioned @34:09). Just remove/ignore all grey characters (including some colons) and it should work. Also: Great tutorial, thanks so much!
its so hard to find good rust channels. you've made the list. great channel. thank you!
Finding this channel has been a blessing. Thank you for your work.
This is one of your best videos I've seen - maybe because it's just at the right level for where I'm at. But all of them are great. Thanks for doing these!
This was a really helpful stream. Really appreciated. Thank you.
Thanks for the video Ryan. You’ve helped me understand threads better
I just want to say thank you for this amazing content!
Fantastic explanation of these concepts!!
Thanks for the video! I absolutely love it! We need more of this 💯
Really great content Ryan! Definitely need more of these! :)
Much needed content! And very well presented. Thank you!
Bro, this is awesome! Thank you!
Excellent explanations
Thank you very much for sharing the great content!
Yes just started learning Rust, this is exactly what i needed to get going. Keep it up
EDIT: You've answered it for me partially a bit later in the video. But what I still don't think I understand, is the difference between Box and &dyn Fn(). Why is there a & needed in the second notation?
23:55 Ryan, is there any way to care, so to speak, about the concrete Fn type in this example, such that we would implement static dispatch instead of dynamic dispatch? Like, could you not Box that Fn trait? Can't you just take a function pointer? I mean can't you write a pool that only works with a single kind of closure type? I'm leaning towards you can't because Rust needs to know the sizes of everything it puts on the stack, right? Thanks!
1:48:56 I don't understand how moving the Arc nref two times into the foo closure by calling pool.execute(foo.clone()) twice, doesn't throw and error. Shouldn't nref be dropped the first time the execute function finishes, the first execute?
Only just understand Rust, well explained, and comedy gold.
Thanks for this excellent material. You rock!
youre such an amazing teacher
I really appreciate this, I LEARNT A TON!. Thanks Ryan :)
1:56:20 question: is it necessary to explicitly handle the Result returned by recv like that (line 19)? Wouldn't recv.unwrap() have the same effect basically? Wouldn't unwrap() invoke the panic macro if the Result maps to an Error, which in turn would end the thread? Or is it just explicitly handled in a match to "gracefully" end the thread (so it doesn't display the panic message)? Correct me if I'm wrong, still learning Rust. Nice video BTW! Really helped me
Nice job, first explanation of what 'static is that i understand. Thanks
Incredible!! Amazing!! Awesome tutorial :)))
Links/References/Mentions:
- doc.rust-lang.org/book/
- doc.rust-lang.org/std/sync/mpsc/index.html
- doc.rust-lang.org/edition-guide/rust-2018/trait-system/dyn-trait-for-trait-objects.html
- doc.rust-lang.org/std/marker/trait.Send.html
- doc.rust-lang.org/std/marker/trait.Sync.html
- doc.rust-lang.org/std/sync/struct.Mutex.html
- doc.rust-lang.org/std/sync/struct.Arc.html
- doc.rust-lang.org/std/clone/trait.Clone.html
- doc.rust-lang.org/std/marker/trait.Copy.html#whats-the-difference-between-copy-and-clone
- doc.rust-lang.org/std/borrow/enum.Cow.html
- github.com/rylev/rust-workshop/tree/master/part-3
- doc.rust-lang.org/std/ops/trait.Fn.html
- doc.rust-lang.org/std/ops/trait.FnMut.html
- doc.rust-lang.org/std/ops/trait.FnOnce.html
- doc.rust-lang.org/std/sync/atomic/index.html
- doc.rust-lang.org/std/sync/atomic/struct.AtomicU32.html#method.fetch_and
Honorary mentions:
- rust-analyzer.github.io/
Thank you so much for this tutorial! 💯
subscribed. most of teh Rust community is like that who know basics but are stuck at advanced concepts. Good luck :)
Came here just after finishing the last chapter (webserver with threadpool) from "The Book".
This was a great addendum to that, however towards the end of the stream things got a little rushed.
Looking forward to your next stream!
You are a gem Ryan
Thanks, learned a ton from this one.
For anyone who might have the same problems as me. At around 1h:25min where tests are ran I had a strange problem. My tests executed OK with no output. It wasn't until I added sleeping at the end of the test I got the output. This is because my virtual machine has only one processor core and threads couldn't even start working before the program was done.
This is awesome. I'm at 1:24:00, but I have to stop the kata and ask, what would have happened if you had followed the Fn() error message (needed + Send), made that fix, and then worked out the Arc solution off new errors. When that error first came up I would have tried to Arc the Fn() passed to execute(work), based on the docs and the error you were looking at. I would still be lost right now I think.
At 1:49:37, I'm looking over the code and thinking about what happens with captured variables inside the closure when the closure itself is cloned. I'm guessing the ARC counter gets increased automatically in this case. Any thoughts?
Learned a lot, thanks so much.
Thank you for the amazing content!
Really helpful, thanks!
Great tutorial, excellent explanations
The demonstration of the 'static bound at 01:02:00 was really useful.
Does the fact that execute() on line 27 uses static dispatch mean it would generate a different version for each closure via monomorphisation? Or do the closures share the same underlying type?
Also I once had an issue with using Atomics in tests where Cargo was running the tests in parallel and (maybe due to Relaxed ordering?) the different test cases could interfere with eachother - as though it were a static variable. You can tell cargo test not to run them in parallel in this case though.
Probably the same code as far as binary size, but different captured values on the heap.
Atomics don't really guarantee exclusive access, they only guarantee that your thread can not be interrupted during the 1 or 2 clock cycles between reading and writing that value. If you need to read a value, process it in some way beyond a single cpu instruction, and then store it back, you really need a critical section or mutex.
I have enough experience with Rust to know that the solution to the sharing of the Receiver is Arc, but I don't really think that follows obviously from the error the compiler gives. In fact, I feel like the most naive thing one might try when getting that error is to change dyn Fn() to dyn Fn() + Send.
EDIT: I see now that you got to that after finishing up with the Arc-Mutex thing, so I guess it wasn't left out. I just thought the two concepts came in the wrong order, at least given what the compiler was talking about.
Just awesome! Thanks a lot!
Awesome stream! Thank you
Perfect, thanks
how'd you get the greyed-out "intellisense" types visible? Looks like an editor extension or feature in VSCode.
That is a feature of the rust-analyzer extension in VSCode. I think it is enabled by default but you can find it as "inlay hints > type hints" in the settings.
This is awesome. Thanks!
Thanks
it's a brilliant video - thank you...but one question, why the heck you using Microsoft Edge on a mac
I'm sure Ryan knows this, but for others.... While it's true that the extra indirection makes calls on dyn traits a bit slower, that isn't the only reason it's slower, or even necessarily the most important. Function calls on dyn traits are not inlined, and inlining is often what leads to dramatic optimizations.
Is this executing work in parallell? Since the mutex is locked durring work only one thread at a time is able to work, so if you have 10 threads, one would work and the other nine would wait for the mutex to unlock and when it does, another thread would acquire the mutex and the remaining nine would wait again.
Is this correct or am I reading the code wrong?
(sorry for the broken english)
How did he get those gray type annotations? Is that a plugin?
Also what color theme is that? My default VSC rust theme has for ex. "pub struct" and "impl" blue.
It's rust analyzer
@@honzasedlon3309 Wau, je i pár dalších Čechů co píše v rustu jo? ^^
@@TheNoirKamui Teda jako píše.. No :D Věčný začátečník, mířím spíš na fullstack, ale Rust se mi líbí hrozně a rád bych mu rozuměl :)
I would also like to know that
@@honzasedlon3309 So which VSCode settings need to change to get that? Just changing the language server from rls to rust-analyzer didn't seem to do it.
mind ask what color theme you are using? @Ryan Levick
Don't you have a memory leak when you use 'static lifetime? When is the closure dropped after you pass it to execute?
Thankyou :)
Hey, thanks a lot for these videos. They are very helpful. I followed your example and also had I look at your code in git and I came across an issue. It seems that only one worker is executing all the functions. Is there a reason for this?
I just found that for this to work as it should you need to drop the mutex after recv() to allow other workers to get hold of the receiver
gracias
Why need to put template parameter after :: when calling channel function, and not just after function name?
Does 'static lifetime addition actually move ownership ?
No it does not. It is still a borrow, it's just one that might last for the rest of the program execution.
@@RyanLevicksVideos Thank you! so does a borrow prevent the object from being dropped once it goes out of scope in the owning scope?
Nice!
having a lot of run!!! Send Trait make me a bit confused even i know for thread safety. At same time ,if receiver unlock made by manual and before work, would something goes wrong? (Forgive me for not being a native speaker of English)
It's ok to call "work" after the Mutex is unlocked, because when we receive work, we get ownership of that work. It's moved out from the Receiver inside the Mutex.
The Mutex cannot be accidentally unlocked while the thread is accessing the Receiver, because when we lock the Mutex we get a MutexGuard. This MutexGuard lets us borrow a reference to the data within the Mutex, and Rust enforces that this borrow must end before the guard is dropped. When the MutexGuard is dropped, it unlocks the Mutex. Dropping the guard is the only way to unlock the Mutex.
Cool tutorial, please upload in at least 1080p though.
(720p was the highest I could choose anyway)
Gold
Any idea why Arc doesn't implement copy?. Passing a cloned arc is so normal that it should surely be clear even if it is implicit. I would have thought the compiler would be smart enough to not call copy when it is not needed, and there are presumably some cases where the compiler could be smart enough to realise that it doesn't need to copy. Explicitly telling it to clone would be a mistake in these cases.
The whole point of Arc is that cloning it is easy and safe. It would make teaching a little more involved, but avoiding it for that reason just makes code where it is implemented harder to understand, as it is less normal.
Copy has the simple semantics of bitwise copy. Copying never involves anything other than copying bits. Cloning an Arc requires an incrementing a reference count so it's not just a bitwise copy and thus copy is not the appropriate trait to model this.
@@RyanLevicksVideos Copy is a trait, so surely lives in specification land decoupled from implementation details. Is there anything stopping it being implemented in ways other than a bitwise copy? Is copy hardwired into the compiler to do something specific when it is implemented, and always be automatically implemented? Not trying to make a point with that, just fairly new to rust and actually don't know. I suppose you might find a case where you could get away with borrowing an Arc, avoiding the clone and destroy step (As I said I'm new, so this could be a common pattern). Even then, nothing stops you doing that if copy is implemented. The only other downside I can see is if the compiler copies even at last use, potentially causing more clones than necessary. That should be relatively easily fixable, but if the assumption has been that only bitwise copies are copy then that might be how things are.
Bitwise copying it is not appropriate, but you do end up using it in the same way as a value type that is copied on passing. You just end up writing clone everywhere and manually implementing it inline, at least in the code I've come across (may not be representative).
Binging the streams btw. Best resource out there after the book, and complements the book fantastically.
@@agsystems8220 Yes Copy is "special" in that it has specific meaning tied to the compiler of bitwise copy. This is how Rust represents changing from the default move sematics to copy semantics. You can borrow an Arc so something requires you to clone it. Rust generally favors being explicit, and I think having to type "clone" is a small price to pay for the clarity it brings.
Gread stream. The only thing I don't agree with it the middle part about channel. It's not like you are wrong just the order kina backward. Make sure the data you send to another thread is Send. Then make sure your sender/receiver is Send. Then if you ever want to share the send/receiver between threads make sure they are also Sync. There are various channels introduced by lots of 3rd party crates and in this way beginners can have a easier time to figure them out.
thumbs up
nice stream
Fucking incredible content!
I wouldn't use mpsc from std as it has aged panic bug which can hurt you in production (github.com/rust-lang/rust/issues/39364). Better to use crossbeam from the scratch. For more info refer to stjepang.github.io/2019/03/02/new-channels.html
Did no one notice the Microsoft Edge on a Mac?
Hi, I am receiving memory leakage with this example
PS
Great content still
I was looking for something more advanced to solve the problems i have with concurrency and closures in my open source project (github.com/sinasalek/clipboard_guardian) and this was incredibly useful. thanks Ryan
the guy who has no experience about coding, if he can get rust in his mind then your training material is able to use for all difference people. :)
Why bother with sending receiver to a spawned thread when you can spawn thread with receiver and return a sender to the main thread?
like this gist.github.com/JohnDowson/b607c49ef3aba7bae86bf589c7f91322
I guess that forces you to implement some way to keep track of job queues, but at least you aren't piling wrappers around wrappers in attempts to trick borrow checker into allowing you to compile your questionable design.
thank you :)