Tutorial: Santiago Basulto - Python Concurrency: from beginner to pro
HTML-код
- Опубликовано: 28 авг 2024
- Presented by:
Santiago Basulto
This is the ultimate concurrency tutorial. Aimed for beginners, we won’t skip the ugly parts (OS low level and computer science concepts). In this tutorial you’ll learn:
what is concurrency and why you need it?
what’s the role of the OS in computing and parallelism?
how processes work, what fundamental OS structures are used
what are threads, how are they created and what’s their scope
what modules does Python offer for multithreading (_thread, threading, concurrent.futures) and what’s the difference between them, the same for multiprocessing (subprocess, multiprocessing or concurrent.futures).
what are race conditions? how to avoid them with synchronization primitives and threadsafe collections
What is the GIL? In which situations will it affect our code?
This tutorial will include coding examples for all the concepts in it, along with two main activities, in which we’ll be writing a web server both using multithreading and multiprocessing!
At the end of this tutorial, you’ll feel confident answering the following questions:
Should I even use concurrency?
If I want to use concurrency, threads or processes?
How can I make sure my code is safe?
Code repo: github.com/san...
The very best tutorial on Python concurrency. Thanks!
Such a great video. This helped me a lot understanding how the concurrency actually works at the core and make better decisions writing the concurrent programs. Thanks a lot to Santiago for this great talk.
Agreed, great video. Helped me with how I want to tackle a problem with getting lots of json data and share it accordingly.
Best video on Concurrency... Question : How is Queue of producer-consumer model a thread safe? I understood scaling is achieved but consumer threads can still share the data and which needs synchronization isn't it?
Here's the implementation of the Queue in Python: github.com/python/cpython/blob/2145c8c9724287a310bc77a2760d4f1c0ca9eb0c/Lib/queue.py#L28
It's rather simple, but you can see that it's thread-safe. Anything else you do OUTSIDE of the queue won't be thread safe, so you must synchronize it on your own.
Great tutorial! Thanks for the effort!
I have a concern about "Shared Data and Synchronization" (1:03 in ved).
Per my understanding, pls correct me if I misunderstand join method, as long as you don't call join method then, you will probably run assert statement before your threads finished and COUNTER will not be correct anyway.
The race condition would be better to produce if you used join method.
Even though, issue wasn't shown on my python 3.8 version. I tried to tune setcheckintervel .. etc.
I guess there is an misunderstanding in 'Shared Data and Synchronization' section. Instructor runs lines 27-31 and without waiting threads to finish task he runs 32nd line and gets an error. This is expected to get an error before the execution finished. When you run and wait till execution is finished values of COUNTER is equal to 10000.
Very useful tutorial, thank you very much!
Awesome video. By any chance, Can you share the code you used in the tutorial?
why does queue.task_done() goes into blocking if there is a exception encountered in the program?
Use Ray module
Buenas Santiago, cómo estás? Dónde puedo contactarme con vos, me gustaría poder aprender programación. Soy de argentina también y si podés darme una mano, te lo agradecería hermano.
Here's the link to that gil talk: ruclips.net/video/KVKufdTphKs/видео.html