Doug Mercer
Doug Mercer
  • Видео 14
  • Просмотров 746 099
I made my own animation library
To try everything Brilliant has to offer-free-for a full 30 days, visit brilliant.org/DougMercer .
You’ll also get 20% off an annual premium subscription.
-------------------------------
Sign up for 1-on-1 coaching at dougmercer.dev
-------------------------------
I made my own animation library (in Python). It's awesome.
In this video, I show five of the cool features I'm really excited to use, and five of the tough challenges that I had to solve to implement them.
This library's primary goal is to make animating code easier. It's not nearly as flexible or robust as manim, but it's already helpful (for me) in its current form.
Most animations in the video, including section screens and the thumb...
Просмотров: 26 969

Видео

How Fast can Python Parse 1 Billion Rows of Data?
Просмотров 218 тыс.4 месяца назад
To try everything Brilliant has to offer-free-for a full 30 days, visit brilliant.org/DougMercer . You’ll also get 20% off an annual premium subscription. Sign up for 1-on-1 coaching at dougmercer.dev The 1 billion row challenge is a fun challenge exploring how quickly we can parse a large text file and compute some summary statistics. The coding community created some amazingly clever solution...
Stop using plain Python scripts (do this instead)
Просмотров 15 тыс.8 месяцев назад
Sign up for the totally free tier of Prefect Cloud here: prefec.tv/doug-mercer Sign up for 1-on-1 coaching at dougmercer.dev One of the most frustrating parts of the workday is doing something that you know could be automated, but just… isn’t yet. In this video, we use Prefect to schedule a Python script to run every week. After, we find that scheduling was really only the first of five problem...
Write Python code people actually want to use
Просмотров 13 тыс.Год назад
Sign up for 1-on-1 coaching at dougmercer.dev Writing code that "works" shouldn't be your end goal. It's really just the start... In this video, we adapt a clumsy, non-Pythonic API into an easy to use, easy to understand Pythonic one. We use magic methods such as getitem , len , enter , and exit to make our objects a context manager and support the len() function and square bracket indexing. An...
Compiled Python is FAST
Просмотров 109 тыс.Год назад
Sign up for 1-on-1 coaching at dougmercer.dev Python has a bit of a reputation fast to write, but slow to run. In this video, we focus on a simple to understand dynamic programming problem that would be terribly slow in native Python or numpy. We show that Python can achieve (and actually exceed) C level performance with the help of just-in-time and ahead-of-time compilers such as mypyc, Cython...
ChatGPT made an iPhone app for me
Просмотров 25 тыс.Год назад
Sign up for 1-on-1 coaching at dougmercer.dev ChatGPT and I create an IOS SwiftUI app, but I'm just the idea guy. Since I have no idea how to write Swift, ChatGPT helps me write an basic fitness app with pretty interesting animations without even knowing the language. #chatgpt #ios #swift Chapters 0:00 Intro 0:17 Hello World 1:04 Buttons and Counters 2:25 Title and Success Indicator 3:26 Fixing...
Never use PowerPoint again
Просмотров 290 тыс.Год назад
Sign up for 1-on-1 coaching at dougmercer.dev In this video, we use Marp to create a presentation from our text editor using Markdown syntax. Marp is an excellent way of quickly creating beautiful presentations with syntax highlighted code blocks, TeX typeset math, and stylized images without having to worry about a finicky WYSIWYG editor like PowerPoint. #markdown #marp #marpit I use Marp a lo...
What's new in Python 3.11?
Просмотров 1,5 тыс.Год назад
Sign up for 1-on-1 coaching at dougmercer.dev Python 3.11 is out! In this video, I show you some of the cool features and improvements made in this release. Release notes - docs.python.org/3.11/whatsnew/3.11.html Other notes: - At the time of this video's release, MyPy doesn't support most of the new typing features. If you're eager to use them right away, consider using pyright. - pyperformanc...
Scary good voice cloning | Python Text to Speech
Просмотров 6 тыс.Год назад
Sign up for 1-on-1 coaching at dougmercer.dev In this video, Doug is lazy and lets TorToiSe text-to-speech do his voice-over for him. Specifically, we create 11 custom voices and illustrate how they sound while teaching you how to do the same. We also discuss some of the ethical considerations of such a powerful model, and highlight the precautions taken by the TorToiSe author. Thanks (and sorr...
Best Python speech to text library (OpenAI Whisper)
Просмотров 10 тыс.Год назад
Sign up for 1-on-1 coaching at dougmercer.dev Whisper, OpenAI's new automatic speech recognition model, is *awesome*. In this video, I show you how to use it and present a few interesting examples of transcribing English and translating spoken German and Japanese to English. github.com/openai/whisper #machinelearning #ai #openai #nlp #python Chapters 00:00 Introduction 00:22 Installing Whisper ...
The Magic that Makes Python Tick
Просмотров 2,8 тыс.2 года назад
Sign up for 1-on-1 coaching at dougmercer.dev In this video, we implement Python's range object in pure Python. We use this as a case study to explore Python's data model and learn how to create rich objects that interact naturally with Python's built-in operators and functions. #python #objectorientedprogramming Chapters 0:00 Introduction 1:02 What does range() do? 2:17 How will we test our im...
Your Python code is almost entirely untested.
Просмотров 2 тыс.2 года назад
Sign up for 1-on-1 coaching at dougmercer.dev Property-based testing is an alternative to example-based testing where tests verify that properties of the output are satisfied given randomly generated input. This randomized approach allows us to more thoroughly test against a broad range of input and can detect hard-to-find bugs better than example-based testing. In this video, we discuss the mo...

Комментарии

  • @matheusaugustodasilvasanto3171
    @matheusaugustodasilvasanto3171 7 дней назад

    Would've been interesting to compare the software engineering approaches or alternative solutions to the problems presented. Even this being a sponsored video and the focus being presenting how the tool can help you, I think it's biggest selling points are A) the centralization of many helpful features and B) sensible and very useful default behaviors. Here is my very incomplete list of how one may tackle these issues outside of prefect: - Scheduling: cron jobs or Windows services - Error Handling: now that's just software engineering 🫠; slap a @retry from tenacity or write one yourself; or alternatively, decouple what you want to retry behind an interface and use a Decorator design pattern to add the behavior dynamically. - Monitoring: collecting good telemetry is certainly a monumental task, but it can be done. Again, decoupling core behaviors, logging, and other debugging information is crucial for good software maintainability. - Parallelization: threading, multiprocessing, or asynchronous programming can help. About scaling and taking advantage of computer hardware, I'm out of my depth. - Data Persistence: simplest possible solution: add a timestamp to the generated file, create a job to occasionally delete (or archive) old results. Now here are the advantages I see for using prefect (besides convenience): - Scheduling: platform independence, configurability, all done in-Python; a hand-rolled solution can also be hard to verify/test, so having a reliable library really helps bring confidence in your solution up. - Error Handling: configurability on how to act depending on errors. I also assume the behavior integrates seamlessly with the web UI, so you can monitor, for instance, how many retries it took to get something from the web. Props to them if they also have rate-limiting behavior built-in, that's a hassle to code on your own. - Monitoring: very sensible defaults that log the start, end, and execution time of the tasks. The live monitoring is also mind-blowing, I have no idea how I would even begin to write that on my own. That being said, I'm not sure of its utility in day-to-day life. Some scripts tend to be in the nature of "write, get to work, schedule, then forget about them"; it seems particularly useful for debugging, but I'm a sucker for TDD and the need to debug rarely comes up. The greatest utility I can think of is monitoring in case your service depends on unreliable third-party endpoints, or if you're dealing with multiple services that need to coordinate among themselves. - Parallelization: the parallelization part doesn't seem to have anything mind-blowing. But using the hardware optimally is certainly a challenge, and prefect gets my full respect for doing that for you. It also adds delayed rewards in the form of future optimizations that your code will benefit from, merely by using prefect's service. Two thumbs up. Even three, perhaps. - Data Persistence: Not much praise from me here. I'm sure the service is fantastic, but I'd need to see some more use cases to determine if it's worthwhile or not. I suppose it's an added bonus if you're using prefect for other benefits. In conclusion, prefect seems like an excellent tool. It's an aggregation of a very cohesive set of features that every script-er desires for their day-to-day work. I stand by the generic software engineering advice of weighing the pros and cons of writing the facilities you actually need yourself. If you adopt the tool, and want some liberty in case the project evolves in a direction you couldn't predict, my advice is to do some good design: separate your implementations from prefect-specific stuff, wrap its services around interfaces you can control, and keep the library at an arm's length. Exceptional video, very well-structured. Please don't take anything I'm saying as adversarial, I just love talking about software engineering. Thank you for doing such a great job of highlighting the tool's strengths. I believe a lot of people will benefit from incorporating it into their workflows.

  • @shahaed
    @shahaed 11 дней назад

    Bro said “importing a lib is better than writing java code” then proceeded to write most of his logic in a SQL string 🤦‍♂️

    • @dougmercer
      @dougmercer 11 дней назад

      ... did you look at the Java solution? I would absolutely prefer to write the couple line SQL statement than the 400 line Java solution, even if it's 6x slower

  • @east4ming
    @east4ming 12 дней назад

    I really like this place. Everyone in the comment area is talented and speaks nicely.

  • @harzer99
    @harzer99 15 дней назад

    The whole trick about performant python code is calling as little native python code as possible.

  • @ismailcreatvt
    @ismailcreatvt 18 дней назад

    This is amazing. Do open source this soon!

    • @dougmercer
      @dougmercer 12 дней назад

      Working on it! But got side tracked on a crazy refactor that makes the library even more useful

  • @zashbot
    @zashbot 19 дней назад

    Congratulations on overcoming your disability!

  • @wicktorinox6942
    @wicktorinox6942 19 дней назад

    Have you guys seen "the One Billion Row Challange" ? they end up with only a few seconds!, using Java-GraalVM and some optimization.

  • @BGDMusic
    @BGDMusic 24 дня назад

    good video

  • @buildgreatproducts
    @buildgreatproducts 25 дней назад

    Great channel

  • @cewla3348
    @cewla3348 26 дней назад

    why use python?

    • @dougmercer
      @dougmercer 25 дней назад

      Generally? I enjoy writing it. It has a great ecosystem, and is a high level, multi-paradigm language that is usually very fast to write something good enough. This video is definitely showing something that it's not ideal at. But it still did OK with the help of PyPy

  • @jamesafh99
    @jamesafh99 26 дней назад

    First time watching a video from you and really loved the explanation and animation! Keep going 🔥

    • @dougmercer
      @dougmercer 25 дней назад

      Thanks! Will do =] recently I've been working on making my own animation library so it's easier for me to keep making these videos. More to come soon!

  • @spongsquad
    @spongsquad 26 дней назад

    why would you not just use jupyter notebooks at this point though

  • @hosseinshahabadifarahani1428
    @hosseinshahabadifarahani1428 27 дней назад

    thanks, I enjoyed watching. Unfortunately, Python is the only programming language that I have mastered. The question that is always in my mind is that all these nice tools and features in OOP provided by Python seem to also be available in many other languages, so what quality or features in Python enables one to write a so-called 'pythonic' style code that has superiority in terms of readability???

    • @dougmercer
      @dougmercer 25 дней назад

      That is a good question. I think answering it would be pretty wishy-washy and philosophical, though... Generally speaking "I like enough of the high level language decisions of Python that I generally feel productive and enjoy writing idiomatic Python". I can't say the same for Java or C++. More practical answer-- why not try some other, totally different languages and get a vibe of what they are like? Every time I've learned a new language or new programming paradigm, it kind of gave me a new way of thinking that benefitted me generally Consider Go, Zig, Rust, or if you want functional play around with Ocaml or Haskell. Bit of a cop out answer, but ¯\_(ツ)_/¯

  • @kinuthiamatata6040
    @kinuthiamatata6040 27 дней назад

    Love the production... johnny Harris typa themes

    • @dougmercer
      @dougmercer 27 дней назад

      Thanks so much! ❤️ Johnny Harris, so that compliment means a lot =]

  • @TheHottabych23
    @TheHottabych23 27 дней назад

    The only problem the challenge rules forbid to use third party libraries 😢

    • @dougmercer
      @dougmercer 27 дней назад

      Yeah =[ PyPy is my closest to valid submission (just built in Python language features)

  • @cerulity32k
    @cerulity32k 28 дней назад

    cool! *_now do it in cuda_*

    • @dougmercer
      @dougmercer 28 дней назад

      This person did Dask + cuDF github.com/gunnarmorling/1brc/discussions/487

  • @Slash27015
    @Slash27015 28 дней назад

    Meanwhile nodejs is like what billion rows? The one I done yesterday?

    • @dougmercer
      @dougmercer 28 дней назад

      github.com/gunnarmorling/1brc/discussions/256 Hmm?

  • @pendragon7600
    @pendragon7600 28 дней назад

    I cry for the decade I lost becoming a master at pandas only for people to bandwagon on a new technology that's basically a knockoff and not even better in my opinion and forget about pandas completely. Guess it's time to learn polars. Beyond frustrating

    • @dougmercer
      @dougmercer 28 дней назад

      I still use pandas! It's just not the right tool for the job here

  • @leosh9026
    @leosh9026 29 дней назад

    Thank you Doug, very cool :)

    • @dougmercer
      @dougmercer 29 дней назад

      No prob! Thanks for watching =]

  • @MogesyemaneHadush
    @MogesyemaneHadush 29 дней назад

    nice tutor in a story telling way. I enjoyed it thank you

  • @petterlarsson7257
    @petterlarsson7257 Месяц назад

    the worst crime here was making them be seperated by semicolons

    • @dougmercer
      @dougmercer Месяц назад

      Agreed. (Of course it's from the twisted mind of a Java dev...)

  • @th1nhng0
    @th1nhng0 Месяц назад

    Such a cool video, why did I only find out about this now?

    • @dougmercer
      @dougmercer Месяц назад

      ¯\_(ツ)_/¯ welcome!

  • @bopon4090
    @bopon4090 Месяц назад

    I haven't watched the video but i can say as fast as c++ is.

    • @dougmercer
      @dougmercer Месяц назад

      I wish... (Kind of close, though)

  • @Joanyan
    @Joanyan Месяц назад

    the fastest way to loop in python is to not loop in python

  • @Axewayboy
    @Axewayboy Месяц назад

    Mojo could be interesting to be included

    • @dougmercer
      @dougmercer 29 дней назад

      I plan on doing a video on mojo eventually. Stay tuned!

    • @cerulity32k
      @cerulity32k 28 дней назад

      oooo yea

  • @Dongobog-ps9tz
    @Dongobog-ps9tz Месяц назад

    I'm calling it by the end you'll just be using python as a thin wrapper for a C implementation

    • @dougmercer
      @dougmercer Месяц назад

      Hah! I did consider it...

    • @Dongobog-ps9tz
      @Dongobog-ps9tz Месяц назад

      @@dougmercer The moment you get numpy involved thats basically whats happening anyway :p

  • @olfredos6
    @olfredos6 Месяц назад

    I enjoyed every single bit of this video and seeing how it uses techniques used by others in different languages. Although Java is still way ahead, this makes me super happy. Thanks Doug! Subscribe!!!

  • @finmat95
    @finmat95 Месяц назад

    14:10 🤢🤮👎

  • @ChristopherMichaelWhite
    @ChristopherMichaelWhite Месяц назад

    This was super entertaining man, well done

    • @dougmercer
      @dougmercer Месяц назад

      Thanks =]. I'm glad people are still enjoying this video (even when generative AI has moved so fast since this video came out)

  • @uis246
    @uis246 Месяц назад

    Programming is just applied mathematics. So another version manim.

  • @xXx-un3ie
    @xXx-un3ie Месяц назад

    you removing the global variables and making it less readable, is that a way to improve performance or you really just don't like more readable code? Weird anyways wether against readability or even worse, it improving performance

    • @dougmercer
      @dougmercer Месяц назад

      I thought it improved performance cause I was changing several things at once. In reality, it doesn't change performance, and I regret changing them

  • @j.i.-cruz
    @j.i.-cruz Месяц назад

    So interested. Would totally use it. Would totally want to contribute. Cant wait to see it open sourced!

  • @gurupartapkhalsa6565
    @gurupartapkhalsa6565 Месяц назад

    A lot of python libraries, especially GPU libraries, are actually executing linked C/C++ code. A good example most people should know is "torch," which you access via Python but which is actually calling C++/CUDA code. Obviously, a SIMD operation is going to beat an interpreter performing a bunch of string conversions, even if it's Java with JIT. The benchmark has to be clear about language features and the constraints of which operations or features are being tested, as well as the testing methodology and how to achieve the same (stated) performance of the bytecode assembler (JVM) using Assembler. Rather than making it a pissing contest, it would be more laudable to demonstrate circumstances where normal people can unleash experienced performance.

    • @dougmercer
      @dougmercer Месяц назад

      Yup, understood. I've done another video on Cython, Numba, mypyc, and Taichi. Feel free to try implementing this in Torch... would love to see it. Also, this is just for fun... not a "pissing contest"

  • @userbobak
    @userbobak Месяц назад

    Great video! If I could make a suggestion though, the background music is a little too loud. It was hard to follow some of what you were saying because of it (like for the cython stuff for example). Overall awesome video though and learned a lot!

    • @dougmercer
      @dougmercer Месяц назад

      Thanks! And def agree- I've reduced music levels in later videos. Thanks again for the feedback and comment =]

  • @michaeldamolsen
    @michaeldamolsen Месяц назад

    I am trying to do something similar with C++ for astronomy animations, but I am really struggling with exactly the aspects you cover here. You have given me some additional things to consider, thank you very much! On a side note, this video is very well structured and presented; a real pleasure to watch. I've subscribed for more :)

    • @dougmercer
      @dougmercer Месяц назад

      That sounds awesome. good luck on the project! Let me know if you eventually open source it-- would love to check it out. There's definitely a lot that can go wrong with animation 😂. I'm in the middle of refactoring a core part of my library. Unit tests are a must, and do your best to clearly define any assumptions (e.g., I apply transformations in the order that their applied, not the order of the starting frame).

  • @midnightfuture
    @midnightfuture Месяц назад

    TIL that [some in] the Python community call[s] "__" "dunder"! Like, I get it-double underscore-but I'd never heard it despite 22 years of casual Python programing. Anyway, you should open source your library, and you shouldn't wait to clean it up first. That's what sub-1.0 semvers are for!

    • @dougmercer
      @dougmercer Месяц назад

      Hah, yeah, I remember when I first heard dunder-- I was like, huh? 😂 I think before that I heard them referred to as "magic methods". Not sure which is better ¯\_(ツ)_/¯

  • @PersonManManManMan
    @PersonManManManMan Месяц назад

    (12@3)))

  • @lovetaiwan708
    @lovetaiwan708 Месяц назад

    you make high quality video.

  • @cinderwolf32
    @cinderwolf32 Месяц назад

    Wondering about the efficiency of using the sim(), min(), and max() functions over chunks of the array/file rather than with only two operands.

    • @dougmercer
      @dougmercer Месяц назад

      Give it a shot! You can clone the repo in the description

  • @gustavcoetzee5018
    @gustavcoetzee5018 Месяц назад

    just tell me why.

  • @garrettwill1220
    @garrettwill1220 Месяц назад

    I personally wouldn't recommend polars just yet. its great and very powerful but its python bindings are still highly under construction.

  • @true.evindor
    @true.evindor Месяц назад

    I'm learning Zig and decided to practice this benchmark and see how fast it could go. If we use the same optimization as the last taichi variant (pre-allocating all necessary memory), it takes 1150ms. If we leave allocation inside (creating and zeroing the matrix, cleaning the memory after we got the result) its about 1800ms (i7-14700k, Ubuntu).

    • @dougmercer
      @dougmercer Месяц назад

      I wanna learn Zig this year...

  • @specy_
    @specy_ Месяц назад

    Instead of manim, have you ever seen "motion canvas"?

  • @AdamS-lo9mr
    @AdamS-lo9mr Месяц назад

    Shocker

  • @glungusgongus
    @glungusgongus Месяц назад

    BRO SUMMOMING SALT

  • @juvenildr
    @juvenildr Месяц назад

    in this video; ruclips.net/video/VioxsWYzoJk/видео.html python just counts to 1 billion 1 minute 52 seconds. but how can you parse that 13 gb's of data in less than just counting?

    • @dougmercer
      @dougmercer Месяц назад

      PyPy interpreter (compiles to C) + multiprocessing (so multiple cores doing work instead of one)

  • @ruroruro
    @ruroruro Месяц назад

    14:11 instead of raising a TypeError, you should return NotImplemented

    • @dougmercer
      @dougmercer Месяц назад

      Doh, yeah -- that's the preferred way of doing it. Not sure what I was thinking (this feature was added in a bit of a rush)

  • @Goliath83
    @Goliath83 Месяц назад

    Ekspecially

  • @VioletJewel1729
    @VioletJewel1729 Месяц назад

    btw the lofree flow feels so much nicer than the nuphy

    • @dougmercer
      @dougmercer Месяц назад

      Ah interesting-- it does seem nice. I recently went off the keyboard deep end and got a ZSA Voyager. I really like it (but miss wireless operation =[)

  • @tonas3843
    @tonas3843 Месяц назад

    I really love these kind of tools, so happy to see more being created. This really reminds me of Aarthficial's MotionCanvas typescript library