Python Generators

Поделиться
HTML-код
  • Опубликовано: 26 сен 2024

Комментарии • 223

  • @biffenb7534
    @biffenb7534 Год назад +400

    I love the relatively monotone presentation, which helps clarity, with only a few incredibly well placed jokes sprinkled here and there. Keep it up!

    • @Holphana
      @Holphana Год назад +9

      Monotone? I hear many inflections.
      I would be offended at the implication towards my accent if I were in the RUclipsrs shoes. 😢

    • @biffenb7534
      @biffenb7534 Год назад +22

      @@Holphana You’re right, I just didn’t choose my words carefully enough. What I mean is that the presentation is that of a factual text, instead of creating unnecessary clutter through adding emotion or emphasis like so many other RUclipsrs do.

    • @Ptolemarch
      @Ptolemarch Год назад +1

      Indeed, these are vastly the best videos on coding I've found.

    • @RoamingAdhocrat
      @RoamingAdhocrat Год назад

      What's really missing is a title card with a photo of M with chin in hands looking puzzle, or head-in-hands despairing at some topic that was exhorted in many keynote talks - TDD, say

    • @NewLondonMarshall
      @NewLondonMarshall 3 месяца назад

      What a wholesome reply :) ​@@biffenb7534

  • @l4luchi123
    @l4luchi123 Год назад +148

    No matter how experienced you are, in your videos there is always a new small detail you didn't know about. Love it!

    • @georgplaz
      @georgplaz Год назад +6

      yes! I thought "oh. I know everything about yielding.."
      and then.. bi-directional?! 🤯

  • @dhravya
    @dhravya Год назад +52

    Always struggled with generators. Thanks for the amazing explanation!

  • @adsfwef1331
    @adsfwef1331 Год назад +47

    Dude, you are a genius. I wonder why your vids aren't getting more views than those other millions of python hacks out there. Keep up the good work.

  • @ssholum
    @ssholum Год назад +91

    Async (vs threading vs "other kinda-multitasking methods") video would be greatly appreciated. It's not the most useful stuff for your average script or simple program, so I rarely see good explanations of it for any language.

  • @Fikerus2
    @Fikerus2 Год назад +23

    I write things like "sum(x for x in range(10))" very often and I didn't know that was generator comprehension! I was thinking it was the same as writing "sum([x for x in range(10)])" (with brackets). Thanks a lot for your content!

    • @yazanalhariri3700
      @yazanalhariri3700 Год назад +1

      Me too!

    • @enkryp
      @enkryp Год назад +4

      Why using a comprehension at all? Can't we just do sum(range(10)) ?

    • @MrRyanroberson1
      @MrRyanroberson1 Год назад +1

      @@enkryp i was just about to mention! yeah range is directly iterable

    • @lawrencedoliveiro9104
      @lawrencedoliveiro9104 Год назад +1

      That’s the difference between an “iterator” and an “iterable”. An “iterable” has an ‗‗iter‗‗ method, which returns an iterator when it is called. The iterator is what returns the elements of the sequence, until it is exhausted and can’t be called again.
      Because range() is an iterable, you can reuse a single range object to return any number of iterators over that range.

    • @Geza_Molnar_
      @Geza_Molnar_ Год назад

      @@lawrencedoliveiro9104 Yeah, your explanation helps me a lot.

  • @mabml
    @mabml Год назад +34

    Your async explanation would be much appreciated! :)

  • @Gamecrazy721
    @Gamecrazy721 Год назад +23

    Very insightful video! One other place I've seen generators used frequently is for API pagination. If, for instance, if you want to get all 100 records from an API, and the API limits your query to 10 records per call, you may not want to wait for all 10 calls. Instead you create a generator that calls the API only when you've completed working with the first page and are ready to move onto the second, etc.

    • @lawrencedoliveiro9104
      @lawrencedoliveiro9104 Год назад +4

      Database queries is a good example. A very common sequence is
      * Create a cursor object
      * Execute a query on the cursor
      * Iterate over the results returned from the cursor
      * Close the cursor.
      You can wrap all these steps up in a single generator function, which takes the query and lets you iterate over the results in a single step.

  • @GlutesEnjoyer
    @GlutesEnjoyer Год назад +3

    “Even if a file is gigabytes large you’ll only need enough memory to handle a single line” great point!

  • @vincentperrollaz5261
    @vincentperrollaz5261 Год назад +16

    Outstanding video as always.
    itertools might have deserved a mention since it is very useful with generators.
    The jump from iterator to coroutine was a bit steep and I would personally love an async video

  • @MessLeadingProgramming
    @MessLeadingProgramming Год назад +14

    Great video. It was eye opening for me when I learnt about generator comprehension. I realised that I was doing something very inefficient passing list comprehensions to sum and other functions.
    Also worker example is amazing. Never used it this way.

  • @ahmadhesham1389
    @ahmadhesham1389 Год назад +4

    This is why I like your channel: Even when I think I know a topic well, I still expect to find something new in your videos, and I always do. Keep it up!

  • @69k_gold
    @69k_gold Год назад +2

    Thank you! By learning this I was able to create a toy version of the async coroutines, helped me a lot to grasp the concept of event loops

  • @SwissTHX11384EB
    @SwissTHX11384EB Год назад +138

    Mate, I have years of Python experience, yet everytime you release a video I am floored by some of the stuff you showcase and I feel like I'm learning about the language for the first time. Stellar job, as always, and your presentation style is excellent for the job.

    • @luiz8755
      @luiz8755 Год назад

      same
      it's a little bit overwhelming lol

  • @lex_darlog_fun
    @lex_darlog_fun Год назад +5

    Just as I was about to suggest a video on async, James announced it himself. 👍🏻
    Can't wait to see this convoluted topic explained in mCoding simple comprehensible manner.

  • @wojasek1223
    @wojasek1223 5 месяцев назад

    Thank you so much for this video, and all of others. I've started reading Fluent Python a couple months ago, but that was quite hard for me to go through it. And you on your channel describe those hard for me things and I finally get them somehow. Thought I was dumb, but all i needed was to find a great lecturer. Thank you so much!

  • @TylerLarson
    @TylerLarson Год назад +8

    Yep, definitely interested in an async video. Especially interested in learning the "python native" async features and components so as to detangle it from all the third-party contributed stuff.

  • @trag1czny
    @trag1czny Год назад +7

    craving for the async vid 🤤

  • @jewpcabra666
    @jewpcabra666 Год назад +2

    always love the videos - would love an async video! Always learn a little tidbit from these

  • @technowey
    @technowey Год назад +1

    Thank you for another great video.
    I’ve been using generators for years, and still didn’t know some of this.

  • @dmdeemer
    @dmdeemer Год назад +2

    Watched this video because I'm like, "I know what generators are, but I bet I'll learn something new about them." Found out that yield is an expression. Mind blown. Not disappointed.
    Please make the async video.

  • @pa1nk1ller14
    @pa1nk1ller14 Год назад +1

    Your videos are going into great depth mate! Im awaiting for that async vid of yours

  • @Cookie-mv2hg
    @Cookie-mv2hg Год назад +2

    Every coding courses of yours it like math classes where you start at 1+1 and a simple zone out and suddenly you're on advanced mathematics.

  • @yxh
    @yxh Год назад +1

    I would def like a video going deeper into "yield from" generator uses, with examples

  • @vidlb
    @vidlb Год назад +2

    Thank you ! And yes for the async video

  • @andrewstribling4385
    @andrewstribling4385 7 месяцев назад

    Nice work! I cannot wait to start using these generator techniques!

  • @CrYpt001
    @CrYpt001 Год назад +1

    Finally something about generators. Never quite got them

  • @FlorianLinscheid
    @FlorianLinscheid Год назад +1

    Didn't know about generator chaining, that sounded very useful! Thanks

  • @MoritzWallis
    @MoritzWallis Год назад +2

    Super informative video, thank you, very interesting

  • @murphygreen8484
    @murphygreen8484 Год назад +1

    Made me realize how much I still need to learn. Great video!

  • @PetrSzturc
    @PetrSzturc Год назад +1

    I watched quite few videos about async but I would still like your take as well. It is one of the more complicated/complex? parts of the language to grasp.

  • @ashishjain518
    @ashishjain518 Год назад

    I probably only got 20% of this video. But it was enough for me to fiddle around and learn more about generators' daily use cases to improve my performance...
    Great video, Mcoding 😄

  • @blacky7801
    @blacky7801 Год назад +2

    You can pass things into a generator? Wow, incredible video!

  • @LastTigerEyes
    @LastTigerEyes Год назад +2

    I really appreciate your videos. I always learn something new, even when I don't think I will. TIL parenthesis make generator comprehension. :)

  • @dixonstroi
    @dixonstroi Год назад

    I think this will really help me with lazy execution via generators. This would have really helped if . Also, 'yield from' was hard for me to get from just the docs, so thanks for summing that up tidily.

  • @modest_supreme
    @modest_supreme Год назад

    Thank you for this! Excellent deep dives. Your videos are a great source for understanding the why/how behind functionality.

  • @MichalPlichta
    @MichalPlichta Год назад

    Wow I thought I know almost everything about generators, but i was wrong! Well done!

    • @mCoding
      @mCoding  Год назад

      There's always more to learn!

    • @Plann
      @Plann Год назад

      Same.

  • @DebashishGhoshOfficial
    @DebashishGhoshOfficial Год назад

    "Thank you, next" was a nice touch.

  • @klaasvaak2575
    @klaasvaak2575 Год назад

    not just imagination or/and downloadspeed but also your pride deciding what you appropriate from the interwebs or what you painstakingly program yourself in most likely a less efficient methode.

  • @AByteofCode
    @AByteofCode Год назад +1

    Very nice CTA at the end, hadn't noticed it before, clever :)
    Otherwise great video and can't wait to know how async uses generators, I've been curious about that ever since you mentioned it a while ago.

  • @ren200758
    @ren200758 Год назад

    didn't know about the generator pipeline and was blown away by it. big hype for the async video!

  • @adamstrejcovsky8257
    @adamstrejcovsky8257 Год назад +1

    when I feel on top of the world with my python knowledge, I go here to restore humility :D

  • @redon638
    @redon638 Год назад

    Thanks! I always learn something new and really impressive watching your videos. Wish you twice as much subs

  • @krimsonsun10
    @krimsonsun10 Год назад

    ALL hail @mcoding High Priest of the Python temple!! Your explanations are succinct, and the usage examples help me so much to grasp the concepts. Thank you.

  • @aditya95sriram
    @aditya95sriram Год назад

    That generator pipeline was eye-opening! I've worked with generators plenty, but still this pipeline idea escaped me

  • @BR-lx7py
    @BR-lx7py Год назад

    This video is definitely eye-opening. I thought that generators were just regular functions that you could re-enter, but I never comprehended how python calls them totally differently. Adding something like "if False: yield" to a regular function totally changes how it is called, despite those 2 lines of code being a noop.

  • @darske1
    @darske1 Год назад

    Plot twist: The 1 Gb file you're trying to read, has only 1 line xD
    Awesome video. The send method confuses me a lot every time I see it, I don't know why lol

  • @endoflevelboss
    @endoflevelboss Год назад

    Looking forward to the async video 👍🥳

  • @BR-lx7py
    @BR-lx7py Год назад

    @9:08 It's crazy that you can even have the same name for the generator comprehensions, so all of them called "nums".

  • @gormster
    @gormster Год назад +1

    No, I don’t want a video on asyncio. I want a whole goddamn *series.*

    • @mCoding
      @mCoding  Год назад +1

      Take note of how I'm methodically making videos on all the prerequisites for a deep async discussion 🙃

    • @khalilhadji6489
      @khalilhadji6489 Год назад

      @@mCoding can't wait, i am facing some issue in one of my projects and understanding how asyncio works inside out would be very helpfull

  • @bigmacbeta
    @bigmacbeta Месяц назад

    Thank you. Amazing explanation.

  • @martinc.7424
    @martinc.7424 Год назад

    I was just told that list comprehension where faster than for loops... now it makes sense.

  • @spaghettiking653
    @spaghettiking653 Год назад +2

    4:44 what is this _make method you call? I wasn't expecting this code to run, but it still did, even though it doesn't appear you defined the method? Is this something generated by dataclass?

    • @NostraDavid2
      @NostraDavid2 Год назад +2

      Note that MyDataPoint class is the type NamedTuple. I googled for "_make NamedTuple Python" and got some info on the function. Turns out that _make is part of the NamedTuple class.

    • @NostraDavid2
      @NostraDavid2 Год назад +1

      Also note that's its a NamedTuple and not a data class. This just means that you have different functions that you can use. So no comparison, like you do with data classes. You CAN do it, but you'll have to implement it yourself.

    • @spaghettiking653
      @spaghettiking653 Год назад +1

      @@NostraDavid2 I see, thanks for pointing it out. So, I'm not sure why I thought it was a dataclass, but I guess I just saw the type annotations and didn't read the top part. That makes a lott more sense, big thanks :)

  • @AngryArmadillo
    @AngryArmadillo Год назад

    Looking forward to the asyncio video!

  • @ciCCapROSTi
    @ciCCapROSTi Год назад

    Good stuff, I didn't know most of the advanced stuff here.

  • @dmitrylitovchenko2365
    @dmitrylitovchenko2365 Год назад +1

    In collatz generator checking for n == 1 should be at the top otherwise it behaves funny with input of 1: [4, 2, 1] :)

    • @mCoding
      @mCoding  Год назад

      Good catch! Yes you are absolutely right.

  • @macedmondson782
    @macedmondson782 Год назад

    I'm really liking your videos! I would love to see an async video from you!

  • @HonsHon
    @HonsHon 8 месяцев назад

    Helped me refactor some code and provide some memory savings! Thank you!

  • @vt2788
    @vt2788 Год назад

    Really advanced high value stuff! Thanks!

  • @anibaldk
    @anibaldk Год назад +1

    You cannot see this (hopefully) but I had to give a standing ovation to this one.

    • @mCoding
      @mCoding  Год назад +2

      I saw it!

    • @anibaldk
      @anibaldk Год назад

      @@mCoding 😱😱😱😱😱😱😱

  • @taylorbreutzmann6034
    @taylorbreutzmann6034 Год назад +2

    It’s crazy, being able to tell how intelligent another human is just behind a screen/video.

  • @AIRLZ6
    @AIRLZ6 6 месяцев назад +1

    damn i need to watch this video over and over again :D. Love it!

  • @jullien191
    @jullien191 Год назад

    Muy buen video. Gracias amigo

  • @MaxBloomfield
    @MaxBloomfield Год назад +1

    I enjoy your dives under the hood of stuff I routinely use and thought I knew well.
    It might be worth pointing out that generator comprehensions can have closures like functions can:
    def multiples(value):
    return (value * x for x in [1, 2, 3])
    fives = multiples(5)
    for number in fives: # 5 is now embedded in fives
    print(number)
    RUclips comments need code formatting markup. At least on this channel.

  • @morrispearl9981
    @morrispearl9981 Год назад

    I am a fan of generators. I use "yield from" when the function that generates the next thing happens to be recursive. For example, I have a generator that takes a list of N things, and yields each permutation of those N things, and the underlying algorithm is recursive.

  • @mjdevlog
    @mjdevlog Год назад

    So useful, even yhough i don't understand the last half of the video. Maybe one day i will😅

  • @kylwatson9240
    @kylwatson9240 5 месяцев назад

    Now if only I was doing something complex enough to merit this new found tool in my python bag... One day

  • @laurentmichel4565
    @laurentmichel4565 Год назад

    I would enjoy a video about asyncio a lot

  • @StephenBuergler
    @StephenBuergler Год назад

    Do you think it would be a good idea for the else clause in the for/in/else loop to be the place where the generator's returned value is made available?

  • @JohnFallot
    @JohnFallot Год назад

    Let’s go Patronssss

  • @yomajo
    @yomajo Год назад +1

    Like 4th year (non-full time) working on Python and this channel consistently keeps me feel stupid. I don't even get how you can call "send" method on a function... I wonder how many years author develops software to know these intricacies of language.
    And 42 reference was observed and appreciated, btw.

  • @alexsere3061
    @alexsere3061 8 месяцев назад

    "Did you really think they would add a new keyword just to abbreviate a for loop?"
    This is python so yes, yes I did

  • @greasedweasel8087
    @greasedweasel8087 Год назад +1

    Can you do a video on making __iter__ a generator vs defining a __next__? I’ve seen both in the wild, you seem preferential to the __iter__ generator, and I’ve googled to no avail. I’m mostly interested in which one is more pythonic, but I’d also like to know what you think the use cases for each are. Great videos!

  • @Blubberland
    @Blubberland Год назад

    Thumbs up.
    4:27 This might be a rather be a problem of spell-checking settings than on syntax.

  • @tywky
    @tywky Год назад

    I can't wait for the async video!

  • @mytalala
    @mytalala Год назад

    looking forward for async video, great job

  • @PikalaxALT
    @PikalaxALT Год назад +1

    4:25 - Pycharm thinks your text file is a requirement.txt.

  • @nassehk
    @nassehk Год назад

    Great videos pall. Please make that async video.

  • @Jakub1989YTb
    @Jakub1989YTb 6 месяцев назад

    9:40 - is there a nicer way to chain the generators like this? To not repeat the "nums" etc?

  • @mattlau04
    @mattlau04 Год назад

    Great video as always! And would love to see a video on async

  • @luiztosk
    @luiztosk Месяц назад

    this is awesome, thanks

  • @mayank8387
    @mayank8387 Год назад

    Dude you're amazing! Thanks for sharing.

  • @arcstur
    @arcstur Год назад

    I am async awaiting for the video on async!

  • @fyellin
    @fyellin Год назад +1

    Just out of curiosity. Python allows min, max, sum, etc on both sequences and iterators. Any idea why len() can’t be called on a generator. I’ve been trying to decide what was going through the heads of the Python developers. You really shouldn’t have to write sum(1 for _ in generator).

    • @ccgarciab
      @ccgarciab Год назад +1

      Probably because generators aren't guaranteed to end and having a common function as len() being potentially non terminating on a common type of object might have been considered bad design

    • @fyellin
      @fyellin Год назад +2

      @@ccgarciab But is len() really that different than min() or max()?

    • @lawrencedoliveiro9104
      @lawrencedoliveiro9104 Год назад

      len() requires you to evaluate the entire sequence.
      Addendum: you should be able to define a ‗‗len‗‗ method for your iterator object, if you want. Then len() should work on it.

    • @fyellin
      @fyellin Год назад +1

      @@lawrencedoliveiro9104 Again, how is this different than min(j, max(), sum()? Unless you’ve got infinities, all require knowing the entire sequence.

    • @lawrencedoliveiro9104
      @lawrencedoliveiro9104 Год назад

      @@fyellin Probably because of the ‗‗len‗‗ issue.

  • @con-f-use
    @con-f-use Год назад

    > "Do you think, they would really introduce a whole new set of keywords just to have a shortened for-loop?"
    Yes. The answer is "Yes, I can see the Python devs do that."

  • @samuelthecamel
    @samuelthecamel Год назад

    Start of video: Ah yes, this is a cool useful Python feature that I didn't know about!
    End of video: What is even going on

    • @mCoding
      @mCoding  Год назад +1

      Haha welcome to my channel! A lot of my videos are guilty of this. I try to make the beginning accessible to everyone but then by the end I'm covering technical details and esoteric use cases that experts get held up on.

    • @samuelthecamel
      @samuelthecamel Год назад

      @@mCoding no problem man, it's nice that anyone at any skill level can learn something from these videos

  • @_veikkomies
    @_veikkomies Год назад +1

    Yes please on the async. It's a confusing feature for me

  • @maljamin
    @maljamin Год назад

    Hooray for Python and yield from. Because the code smelled like it wanted to do that and Python does it. I only know C#, which has "generators", and yielding "from" is an obvious thing one would want to do.

  • @drrros
    @drrros Год назад

    Vote for asyncio video! That would be really awesome!

  • @StrangeArrow
    @StrangeArrow Год назад

    Super Content. Thanks for sharing with us 🥰🥰

  • @yxh
    @yxh Год назад

    So at 9:31, the `nums` on line 15 is really a 5th-level nested generator? And whenever sum() wants the next value in the top-level generator, the inner nested generators wind all the way down to the lowest level to read the file and parse it one row at a time?

    • @mCoding
      @mCoding  Год назад

      Yep, that's right!

  • @arthurfelipe544
    @arthurfelipe544 Год назад

    what is the use of the line x: float? i tried to search the use of the : in that line but didn't find anything

  • @efperel
    @efperel Год назад

    Yield from is great for trees and other recursive traversals

  • @zacharyreilly7307
    @zacharyreilly7307 Год назад +1

    Please make an async video!! I would love it if you included best practices for incorporating async into sync func/programs
    I find most times i need something async i want to do sync stuff too or dont know how to convert it all to async

  • @thyllmuller4524
    @thyllmuller4524 Год назад

    9:42 I tried that exact code and I got the "sum is inf" - any idea what might be happening? I triple checked the code, it is identical aside from the function name being different.

  • @reef2005
    @reef2005 5 месяцев назад

    It would help if you say something about the arg NamedTuple in the MyDataPoint class and also the '_make' method that does not exist.

  • @ErikS-
    @ErikS- Год назад

    4:26 - Maybe the parser may think this may be a line-number?

  • @farpurple
    @farpurple Год назад

    ~~I got my dose of dopamine~~
    Thanks you, but i should watch more times/try in code to understand yield and yield from returns, h, im still on way to switch to rust but python is so interesting!

  • @lawrencedoliveiro9104
    @lawrencedoliveiro9104 Год назад

    14:44 Yield-from does nothing that cannot be done by the outer generator doing its own send calls on the inner generator.

    • @mCoding
      @mCoding  Год назад

      True, but take a look at what yield from is defined as in the spec. Due to all the edge cases, it would be an absolute nightmare to implement it yourself every time you wanted to use it. peps.python.org/pep-0380/#formal-semantics

  • @Thr0nSK
    @Thr0nSK Год назад +1

    Man you killed it with the Ariana Grande 😂😂