For #17 there is a minor gotcha being you can't end a raw string with a lone backslash (i.e. r"windows\path" is legal, r"windows\path\" is not). Trivial to workaround but something that might catch people out the first time they try to prepend with r.
@@DanCojocaru2000 So it makes sense when you realise it’s not a raw string, it’s a string with the escape sequences left unparsed. But they still have to be legal escape sequences. So “abc\” isn’t legal because it assumes \” is the unescaped text and the string is now missing the closing quote. It’s also why “abc\\” is legal.
I agree about map and filter. Comprehensions are usually more readable and avoid ugly function expressions, but there is a niche use case for map: if your generator comprehension would look like this: (f(a, b, c) for a, b, c in zip(A, B, C)), map(f, A, B, C) is a lot more elegant, and often more readable. For example, (a + b for a, b in zip(A, B)) becomes map(add, A, B). (need to import add from operator)
I started learning python a couple years ago to fill a gap left by depression. I'm close to 50 now (not that I wasn't 2 years ago), in that time I've managed to write an AOL3.0+ compatible server - for nostalgia - in python utilizing your videos for guidance, and other content creators as well. I've recently switched to using #pycharm from vscode. Thank you very much for your content, it has helped me tremendously in improving my project and my comprehension of python.
I've just found out that I never really understood the behavior of "and" and "or" for non-boolean types before this video. Awesome content, as always! #pycharm
worth mentioning for no 8: that non-empty collections are considered Truthy , plus the fact that and operator returns the LAST EVALUATED value if all values are true
I love the “and” and “or” explanation, it’s so much easier to understand compared to all the resources out there on the web that have some convoluted explanation of it that’s really difficult to understand. #pycharm
I personally usually prefer filter or map if possible over list comprehensions, I think filter and map have quite an unfortunate syntax in python, as at least in my opinion the order is completely flipped compared to how you read it, I read it as “iterator, operation, function” and instead it reads as “operation, function, iterator”. This also means chaining them is a nightmare because it’s nested instead of appearing one after another, and neither one is ergonomic nor readable when chaining more than two… also the lambda syntax is overly verbose, and not particularly accessible especially to non native speakers as it’s quite an obscure word
I agree that map and filter's syntax in Python isn't the best. As much as I like list comprehensions, sometimes I find myself wanting to use map or filter, but I'm not a huge fan of Python's lambda syntax. I wish there were a better way to do multi-line inline functions in Python (plus don't even get me started on the Callable[] type hint). You might appreciate the PyFunctional library which adds a convenient way of chaining map/filter. Unfortunately, it doesn't currently have support for type hints so it's maybe not the best if you use Pylance's type checker.
it makes sense when you're used to writing math things right-to-left, M_fi, which would be an amplitude to go from the initial state (i) to the final state (f)
Regarding #14: Mostly, one should go for generator expressions like you did, as wirting out "lambda x: ...." as function doesn't save many characters and makes the thing harder to read - granted. However, When stacking multiple of those expressions (map, filter, reduce, etc.), they can be quite cool. Especially when all those variable names used in your generator would happen to be longer than the usual line together (so you're force to line-break), it looks cleaner to line-break between arguments of a function (effectively creating an easily traceable staircase of transformations functions) that to line break before the "if" of a generator - the latter looks much uglier when stacked into each other. One could argue that I should define my generator expression, then use it when defining the next generator expression and so on, until I did all my transformations - fair point. However, this encourages defining multiple useless variables which serve so purpose and uses up the same amount of lines a stacked reduce(sum(map(fm, filter(ff, myList))) does. I agree, that real use-cases with enough transformations to make this worthwhile are rarer that isolated cases of a map for filter transformation. When confronted with the latter, you should obviously choose the generator-expression approach.
For that I'm currently writing a library. That'll help to handle this nesting mess without needing to define many variables. It's typed-stream on pypi. It's usable, but hasn't reached 1.0 yet. e.g.: Stream.counting().limit(100).map(operator.mul, 2).filter(lambda x: x%5).peek(print).sum()
Awesome tips! Love your videos ☺️ Personally I prefer to iterate over a copy of the dict instead of the extra for loop. While it's less memory efficient, I don't mind it too much since it's a shallow copy and in most cases, I prioritize the readability of the code over efficiency. Also, I'm a bit torn on the filter and map tips (well, less on the map part and more on the filter part 😉) When there is no lambda involved, and depending on the specific variable names we are working with, the filter function can sometimes be closer to human language (though I admit this doesn't happen often). I wonder if in the future there will be a clear runtime difference between the two, there is already a PEP open to make comprehensions more efficient. #pycharm
#13 deleting while iterating - one can use for key, val in list(d.items(): del d[key] In fact you have shown this in one of your videos and I have been using that ever since. It makes code much more readable that collecting the things to delete. #pycharm
Honestly, I am disagreeing with filter and lambda. I got so used to these functions (also due to other languages maybe) that I now find them more readable than list comprehensions, at least usually. Sometimes I still think list comprehensions are better, but nowadays it only rarely happens.
I'm happy that I've learned a few things again here, it's not always being new at the language that makes you do less optimal stuff. sometimes it's being old at it. Format strings are, for me, fairly recent, I know they've been here for years, but I've been coding python for way longer than they've been here, and it's not always easy to let old habits die and to make sure you know exactly all the new ways to do things in the new version you're starting using, at least I've been using format strings, but now I'll have a look into the doc for those extra formating parameters to up my game a little. And thanks for the io.stringio trick, I didn't knew that one before. #pycharm
3: I'm ALWAYS trying to tell people to use pathlib.Path instead of strings!! 10: Ooh divmod looks awesome! Can't wait to use it! 14: Interesting take on filter/map, I think I could agree. 17: I *roughly* knew about raw strings but feel more confident with them after you're 20 second explanation (which I watched at 2x speed) 21: Yes, forcing prints at import is the worst!! Love all your videos, thank you! #pycharm
pathlib just seems to be a sop to Microsoft Windows. String manipulation is perfectly straightforward for POSIX paths. 7:45 Fun fact: “/” works as a path separator even on Python on Windows.
@@lawrencedoliveiro9104 I've never worked in a pure POSIX environment, and now I work in an environment where code is deployed on both Windows and Linux systems. I don't know the difference between all different types of forward and backward slashes and which operating systems uses which types in which cases and that's the way I want it.
@@lawrencedoliveiro9104 it's fine, but still pathlib is very convenient, it helps with splitting paths into their components and has a nice functional interface that allows chaining. Also, it helps a lot if you need to handle paths that need to follow specs of a foreign OS (handling POSIX paths in Windows or vice versa).
@@lawrencedoliveiro9104 not true. I had a case recently where I needed to handle Windows paths on a POSIX system and pathlib was perfect for that. No, I couldn't use POSIX paths, I explicitly needed Windows paths.
Even though I mainly code for Widows OS. If stopped using backslashes as a directory delimiter a long time ago. All python paths can work if you use forests-slashes and even Widows explorer handles forward-slashes so there's really no need to use backslashes in file paths in python.
This is Good Stuff! 17 years of assembler followed by 20+ years of C have left me unprepared for these newfangled gadgets. Thanks ever so much for helping me see through the blur ....
I experimented with concStringPlus and concStringIO and got a result that, at first, contradicted your assumption, since for small and non-chaotic strings the plus method is faster. So I added some complexity to the strings and the numbers got crazy for the plus, jumping from 0.55s to 1.61s, while remaining the same for the StringIO func (near 0.61s for 100000 runs with 100 operations each). My conclusion is: if you're incrementing your string with very small parts, stick to +=. If it's random-sized or chaotic, StringIO will keep the excellent work throughout.
CPython optimizes str+= if there are no other references to the left hand string, but that relies on it being able to reallocate the string data; if your other code was allocating new objects, the risk that reallocation requires a copy goes up drastically. StringIO can reduce these reallocations by keeping a margin for growth, much like list does. Your small and non-chaotic strings may have been previously allocated (e.g. interned), making the ideal case for str+= to reallocate without copying.
As far as I know, something was changed in string concatenation. Concatenation of 10 million elements with StringIO beats basic string concatenation just by 10%
been programming in python for years and consider myself somewhere between intermediate and advanced, and even your beginner-oriented videos have something new to me in them. there are so many beginner python creators out there and i'm thankful that it's so accessible now, but i definitely appreciate the higher-level "niche" you tend to make videos in, even when you claim it's for noobs. also #pycharm gimme that license mDaddy
No 7: also yes. If I want that functionality, but think it may need to me flexible, a store the default as a module constant: STRICT = True def number7(*args, strict=STRICT):
Adding persons together at 6:53 is hillarious! Making friends is not an associative (or even commutative) operation! The friendship relationship is also not reflective, symmetric or transitive. People are not numbers!
I once was creating a parser in python and, naturally, was using bs4 for finding stuff. This, however, resulted in a bottleneck and replacing bs4 with regex made a 12x speed improvement from a couple of seconds for each operation to a couple dozens of milliseconds
I have a question about #7. If I have some global constants, is it ok to use them directly in functions, or should I pass them through few layers of functions from main?
Great question! ALL these nooby habits are okay and they all have situations where they are, in fact, quite reasonable things to do. Just as you suspected, using a global constant is a perfectly reasonable thing to do.
#5 is slightly misleading, since it depends on the python implementation. The statement is true for PyPy, but CPython cleverly uses a reference count of one to make the string mutable. The proposed fix has the exact same performance in CPython (replace 100 with 10000000 to get a proper measurement).
Small annotation to 10: The reason you do this is that most processors use a divmod that is built in already. That means the processor will most likely perform the operation twice only to discard the mod in the first go around and the result in the second.
Totally agree with all of these, but the 2 that irk me the most is altering the iterator and using eval. There's almost always a better way to do it than to use eval, but I think in the age of computer generated code it's especially important to understand the dangers of.
I think filter/map functions are not that bad to read. List comprehension is good, but I would rather not rush rewriting all filter calls to list comp style #pycharm
2:17 for this particular example (though I'm not criticizing the tip), the following is even better (20% faster for n = 100000 for me -- and clearer): s = "".join(f"some string {i}" for i in range(n))
started using dataclasses because I rely heavily on sending dicts around as messages, and then I immediately got the issue of exporting/importing to json. Kind of an annoyance that whole pickle business. So why is using raw dicts a noobie practice btw? You just said not to do it, but not why.
I disagree with #14. You shouldn't be using lambdas as arguments to map and filter, but functions not created just for the purpose of this statement are fine. A lot of people don't know you can pass goodies from the `operator` module(such as `itemgetter`) and unbound methods(such as `str.upper`) as the first argument.
I also agree that eval should not be used for parsing data, but you can pretty much completely sandbox it eval("{'h':6}", {"__builtins__":{}}) Note: someone might still find a way to evaluate unsafe code, so don't do this. It's just a nice thing to know.
No global exists within this sandbox, if a vulnerability exists, it would abuse the built-in datatypes like `(5).__mul__(8)` But I can't think of a real exploit for this.
Regexes really can't parse mathematical expressions or xml, I give you that. I learned that from people more formally educated in computer science than I am. I learned it AFTER presenting a working example that they indeed can, while being simpler and a whooooole lot faster than parsers. Yet somehow, this new realization didn't break my already written nooby code. 🤣
"Concatenating strings with plus" I doubt it gives a meaningful performance improvement if you have string concatenation once or twice in a program. Sure, when I will be writing extra long strings I might actually use this, otherwise it's just unnecessary code obfuscation.
I am a big fan of iterators, and I really disagree about map and filter. I get that sometimes they are less readable, but they also make it way easier to build pipelines of lazy data manipulation. Also, generators and comprehensions have poorer performance, due to the for keyword, afaik. I think this isn't a n00b habit, as most n00bs don't understand iterators and functional / declarative styles, just seems more like a question of taste.
A "nooby" numpy habbit is also doing inp = np.array(inp) at the beginning of a function to ensure that the input is numpy for the rest of the function. This is bad because the np.array() operation actually makes a new copy of the array even if it is already a numpy array, so if there is a lot of data it will take some time. #pycharm
On #5, searching for answers gives some mixed messages, where the immediate evidence is that += is faster than the alternatives in most cases due to some python interpreter trickery, but in testing it's pretty clear that io.StringIO is substantially faster. For 50,000 appends I get 0.00428s using perf_counter, and with StringIO I get 0.00299s. That enormously faster.
9: Some weird audio from the other side during this one from me? Maybe the ghosts like single letter variables. 13: d = {key: val for key, val in d.items() if not val % 2 == 0}
No 4: yes! But....your method names are nooby. If an object writes to a file: def write(self, fdst).: wrapped with: def tofile(self, filename): Methods are verbs that tell you what they do. Not sentences: separation of concerns.
Man, that behavior of and/or with falsy/truthy values is really strange - it almost reminds me of some of the type coercion weirdness that can occur in Javascript. Much as I like using Python (#pycharm) for certain projects, I definitely appreciate languages that can give compile-time errors for this sort of thing - same goes for deleting items from an iterable during iteration as well.
Sometimes, even after 8 years of python, you have some nooby habits :) . As always great video !
Год назад
For iterating and delete, I often just iterate the copy of it. d.items().copy(). Therefore, I dont need the second loop and the copied object will be deleted after loop anyway, so memory ussage should be the same.
#11 I don't actually like @property, when someone creates a setter it usually has a side effect. If I'm looking at where it is used, `obj.x = something` it is really surprising when I find out that this does something more than just changing the x variable. But if I see an `obj.set_x(something)` that immediately raises the alarm: `set_x()` probably does something more - why would anyone bother to write this function otherwise? I suppose if your setter really has no side effect it is safe to use property, but in my experience that is rare (maybe you want to log where it changed or something like that, it's fine). Usually however, it re-computes related values or notifies other objects that a value has been updated - you should be aware of these.
Thanks for the video! Agree on most points, but I do like map and filter though. It feels like I should switch to Elixir though. And thanks for organizing the contest! #pycharm
Never knew about Python getters and setters having special properties so that will be my next amendment I have to make for my existing project #pycharm
thanks mate, pretty useful information, some of these I’ve been doing myself, some I didn’t even know about. keep going with this great content. #pycharm
Not very bad. Path objects include the context that they are paths. For instance, path / subpath instead of os.path.join(path, subpath), and you can read an entire file with path.read_text(). pathlib is implemented using os.path and friends (path._flavour links to os.path, posixpath or ntpath).
Nice, easy and fast way of looking at python along with leaving with the viewer to practice and test these mistakes/habits on their own , makes me want to learn and use this language more and more. Thanks mCoding . #pycharm
i noticed that for #15 the first line was actually line 15, and i think it would be a neat little easter egg if the first visible line would always match the current number.
Definitely enjoyed this, and picked up some great tips! I have to say, I’m not a fan of main() as a construct. __main__ is great, but then having it just call a main() to do everything feels unnecessary unless there’s a really good reason, and then the person importing your code has to weed it back out to avoid pulling it in. (Plus, if I’m being honest, main() feels a bit too Java-like. :) )
the point of main() is to keep code out of the if/then block for dunder main...devs should see the module end in that block and it should be short, and then they know a main() or usage() [for scripts] is right above it.
1 kinda don't agree 7 I'm conflicted on this one. I've been in JS land and got used to the "file-as-module" pattern(?). Now to me it seems so wasteful to define a dedicated class just so I can avoid global variables. Happy to proven wrong about this ofc. Maybe I should just embrace this instead lol 12 the results can also be stored in a "private" variable, yes? Assuming other conditions are taken care of 14 sometimes I use map/filter when I'm too lazy to think of a loop variable name lol. Though more often than tht I consider hsing "_" instead. Dk which is worse heh
Hey James, how do you feel about accepting using `or` to give a value to an attribute if it not set yet like `self.some_dependency = dependency or DefaultDependency()` in the constructor when the parameter `dependency` defaulted to `None` . Or used in a property to lazy load and cache an expensive object: `self._connection = self._connection or DatabaseConnection()`?
When I've just started to learn python, my mentor forced me to use filter, map, reduce etc. List comprehension always seems more readable for me #pycharm
6:30 I actually faced this recently with a project I’m working on. Do they both do the same thing under the hood (same byte code) or is one preferred over the other for different reasons? (In reference to using functional functions like map or filter over comprehensions).
I'll defend facilitating variable rw with global instead of adding it as an input and outputting it again. In some cases, anyway, it's the same thing in shorter syntax. Problem where?
Love the "don't parse HTML with regex" reference. One of my favorite pieces of programming humor.
For #17 there is a minor gotcha being you can't end a raw string with a lone backslash (i.e. r"windows\path" is legal, r"windows\path\" is not). Trivial to workaround but something that might catch people out the first time they try to prepend with r.
That's a very weird restriction.
@@DanCojocaru2000 So it makes sense when you realise it’s not a raw string, it’s a string with the escape sequences left unparsed. But they still have to be legal escape sequences.
So “abc\” isn’t legal because it assumes \” is the unescaped text and the string is now missing the closing quote.
It’s also why “abc\\” is legal.
@@notenoughmonkeys Quite weird design.
@@DanCojocaru2000 Completely agree. Caught me completely off guard when I first encountered it, but once you know, you know.
Not the first time I've prepended with r, but this actually happened to me today and I couldn't see why it shouldn't work. What a timely coincidence.
I agree about map and filter. Comprehensions are usually more readable and avoid ugly function expressions, but there is a niche use case for map: if your generator comprehension would look like this: (f(a, b, c) for a, b, c in zip(A, B, C)), map(f, A, B, C) is a lot more elegant, and often more readable. For example, (a + b for a, b in zip(A, B)) becomes map(add, A, B). (need to import add from operator)
idk see anything wrong with map(int, arr) tbh
idk, at returning and passing in 3+ tuples you really should start thinking about using a namedtuple at that point imo
@@thirtysixnanoseconds1086 yeah, that's another good use for map. I guess I should have said there are few niche use cases for map.
@@cleverclover7 none of these examples are returning 3 tuples. They all return iterators. Can you elaborate on the use of named tuples here?
not to mention that in certain situations map is a tiny bit faster...
I started learning python a couple years ago to fill a gap left by depression. I'm close to 50 now (not that I wasn't 2 years ago), in that time I've managed to write an AOL3.0+ compatible server - for nostalgia - in python utilizing your videos for guidance, and other content creators as well. I've recently switched to using #pycharm from vscode. Thank you very much for your content, it has helped me tremendously in improving my project and my comprehension of python.
Congratulations for your hard work!
Thanks for making a sequel to the first Nooby Habits video, would be awesome if you could do another one for C++ as well.
please keep the comments python pure. I don't need to hear the J word(s), nor anything involving the letter between B and D.
I've just found out that I never really understood the behavior of "and" and "or" for non-boolean types before this video. Awesome content, as always! #pycharm
It's called "Short Circuit Logic".
worth mentioning for no 8: that non-empty collections are considered Truthy , plus the fact that and operator returns the LAST EVALUATED value if all values are true
I love the “and” and “or” explanation, it’s so much easier to understand compared to all the resources out there on the web that have some convoluted explanation of it that’s really difficult to understand. #pycharm
I personally usually prefer filter or map if possible over list comprehensions, I think filter and map have quite an unfortunate syntax in python, as at least in my opinion the order is completely flipped compared to how you read it, I read it as “iterator, operation, function” and instead it reads as “operation, function, iterator”. This also means chaining them is a nightmare because it’s nested instead of appearing one after another, and neither one is ergonomic nor readable when chaining more than two… also the lambda syntax is overly verbose, and not particularly accessible especially to non native speakers as it’s quite an obscure word
I agree that map and filter's syntax in Python isn't the best. As much as I like list comprehensions, sometimes I find myself wanting to use map or filter, but I'm not a huge fan of Python's lambda syntax. I wish there were a better way to do multi-line inline functions in Python (plus don't even get me started on the Callable[] type hint). You might appreciate the PyFunctional library which adds a convenient way of chaining map/filter. Unfortunately, it doesn't currently have support for type hints so it's maybe not the best if you use Pylance's type checker.
I'm currently writing a library called typed-stream. That allows you to easily chain such operations. It's fully typed and checked with mypy
it makes sense when you're used to writing math things right-to-left, M_fi, which would be an amplitude to go from the initial state (i) to the final state (f)
Regarding #14: Mostly, one should go for generator expressions like you did, as wirting out "lambda x: ...." as function doesn't save many characters and makes the thing harder to read - granted.
However, When stacking multiple of those expressions (map, filter, reduce, etc.), they can be quite cool. Especially when all those variable names used in your generator would happen to be longer than the usual line together (so you're force to line-break), it looks cleaner to line-break between arguments of a function (effectively creating an easily traceable staircase of transformations functions) that to line break before the "if" of a generator - the latter looks much uglier when stacked into each other.
One could argue that I should define my generator expression, then use it when defining the next generator expression and so on, until I did all my transformations - fair point. However, this encourages defining multiple useless variables which serve so purpose and uses up the same amount of lines a stacked reduce(sum(map(fm, filter(ff, myList))) does.
I agree, that real use-cases with enough transformations to make this worthwhile are rarer that isolated cases of a map for filter transformation. When confronted with the latter, you should obviously choose the generator-expression approach.
For that I'm currently writing a library. That'll help to handle this nesting mess without needing to define many variables. It's typed-stream on pypi. It's usable, but hasn't reached 1.0 yet.
e.g.:
Stream.counting().limit(100).map(operator.mul, 2).filter(lambda x: x%5).peek(print).sum()
So... Functional programming?
Awesome tips! Love your videos ☺️
Personally I prefer to iterate over a copy of the dict instead of the extra for loop. While it's less memory efficient, I don't mind it too much since it's a shallow copy and in most cases, I prioritize the readability of the code over efficiency.
Also, I'm a bit torn on the filter and map tips (well, less on the map part and more on the filter part 😉)
When there is no lambda involved, and depending on the specific variable names we are working with, the filter function can sometimes be closer to human language (though I admit this doesn't happen often).
I wonder if in the future there will be a clear runtime difference between the two, there is already a PEP open to make comprehensions more efficient. #pycharm
#13 deleting while iterating - one can use
for key, val in list(d.items():
del d[key]
In fact you have shown this in one of your videos and I have been using that ever since. It makes code much more readable that collecting the things to delete.
#pycharm
This is a gem! I love this
I’ve coded python for about 3 years now and never knew divmod existed. I learn something knew every time I watch your channel
Honestly, I am disagreeing with filter and lambda. I got so used to these functions (also due to other languages maybe) that I now find them more readable than list comprehensions, at least usually. Sometimes I still think list comprehensions are better, but nowadays it only rarely happens.
pathlib is definitely something I could be using more and StringIO was completely new to me! Thank you, love your content #pycharm
I'm happy that I've learned a few things again here, it's not always being new at the language that makes you do less optimal stuff. sometimes it's being old at it. Format strings are, for me, fairly recent, I know they've been here for years, but I've been coding python for way longer than they've been here, and it's not always easy to let old habits die and to make sure you know exactly all the new ways to do things in the new version you're starting using, at least I've been using format strings, but now I'll have a look into the doc for those extra formating parameters to up my game a little.
And thanks for the io.stringio trick, I didn't knew that one before. #pycharm
Old habits be like *dr_who_in_da_rain.gif"
totally, and my company is late with the versions for security reasons, so I'm always behind.
3: I'm ALWAYS trying to tell people to use pathlib.Path instead of strings!!
10: Ooh divmod looks awesome! Can't wait to use it!
14: Interesting take on filter/map, I think I could agree.
17: I *roughly* knew about raw strings but feel more confident with them after you're 20 second explanation (which I watched at 2x speed)
21: Yes, forcing prints at import is the worst!!
Love all your videos, thank you! #pycharm
pathlib just seems to be a sop to Microsoft Windows. String manipulation is perfectly straightforward for POSIX paths.
7:45 Fun fact: “/” works as a path separator even on Python on Windows.
@@lawrencedoliveiro9104 I've never worked in a pure POSIX environment, and now I work in an environment where code is deployed on both Windows and Linux systems. I don't know the difference between all different types of forward and backward slashes and which operating systems uses which types in which cases and that's the way I want it.
@@lawrencedoliveiro9104 it's fine, but still pathlib is very convenient, it helps with splitting paths into their components and has a nice functional interface that allows chaining.
Also, it helps a lot if you need to handle paths that need to follow specs of a foreign OS (handling POSIX paths in Windows or vice versa).
@@slash_me Everything does POSIX these days. It’s no point developing for anything else.
@@lawrencedoliveiro9104 not true. I had a case recently where I needed to handle Windows paths on a POSIX system and pathlib was perfect for that. No, I couldn't use POSIX paths, I explicitly needed Windows paths.
Even though I mainly code for Widows OS. If stopped using backslashes as a directory delimiter a long time ago. All python paths can work if you use forests-slashes and even Widows explorer handles forward-slashes so there's really no need to use backslashes in file paths in python.
This is Good Stuff! 17 years of assembler followed by 20+ years of C have left me unprepared for these newfangled gadgets. Thanks ever so much for helping me see through the blur ....
Syntactic Sugar is GOOD FOR YOU.
I experimented with concStringPlus and concStringIO and got a result that, at first, contradicted your assumption, since for small and non-chaotic strings the plus method is faster. So I added some complexity to the strings and the numbers got crazy for the plus, jumping from 0.55s to 1.61s, while remaining the same for the StringIO func (near 0.61s for 100000 runs with 100 operations each). My conclusion is: if you're incrementing your string with very small parts, stick to +=. If it's random-sized or chaotic, StringIO will keep the excellent work throughout.
CPython optimizes str+= if there are no other references to the left hand string, but that relies on it being able to reallocate the string data; if your other code was allocating new objects, the risk that reallocation requires a copy goes up drastically. StringIO can reduce these reallocations by keeping a margin for growth, much like list does. Your small and non-chaotic strings may have been previously allocated (e.g. interned), making the ideal case for str+= to reallocate without copying.
As far as I know, something was changed in string concatenation. Concatenation of 10 million elements with StringIO beats basic string concatenation just by 10%
been programming in python for years and consider myself somewhere between intermediate and advanced, and even your beginner-oriented videos have something new to me in them. there are so many beginner python creators out there and i'm thankful that it's so accessible now, but i definitely appreciate the higher-level "niche" you tend to make videos in, even when you claim it's for noobs. also #pycharm gimme that license mDaddy
Still found areas to improve. Thank you for your videos 👍
#pycharm
Not sure about avoiding map/filter all the time but awesome tips, thanks #pycharm
I like hiding loops, but I've written some itertools/map/filter constructions that are abominable
No 7: also yes. If I want that functionality, but think it may need to me flexible, a store the default as a module constant:
STRICT = True
def number7(*args, strict=STRICT):
Adding persons together at 6:53 is hillarious! Making friends is not an associative (or even commutative) operation! The friendship relationship is also not reflective, symmetric or transitive. People are not numbers!
I once was creating a parser in python and, naturally, was using bs4 for finding stuff. This, however, resulted in a bottleneck and replacing bs4 with regex made a 12x speed improvement from a couple of seconds for each operation to a couple dozens of milliseconds
I have a question about #7. If I have some global constants, is it ok to use them directly in functions, or should I pass them through few layers of functions from main?
I think constants are fine, denote them by writing in UPPER_CASE_SNAKE_CASE
Great question! ALL these nooby habits are okay and they all have situations where they are, in fact, quite reasonable things to do. Just as you suspected, using a global constant is a perfectly reasonable thing to do.
I like:
STRICT = True
def func(*args, strict=STRICT):
so:
It works like a global
It can be overridden if necessary
the interface is exposed to users
#5 is slightly misleading, since it depends on the python implementation. The statement is true for PyPy, but CPython cleverly uses a reference count of one to make the string mutable. The proposed fix has the exact same performance in CPython (replace 100 with 10000000 to get a proper measurement).
Small annotation to 10: The reason you do this is that most processors use a divmod that is built in already. That means the processor will most likely perform the operation twice only to discard the mod in the first go around and the result in the second.
yes definitely microoptimize your assembly code in python, the performance increase will be tremendous
Damned. I taught being proficient in my python skills but some nooby habits are still sticking. Thanks for making us realising that. #pycharm
omg you actualy a solved a python issue I am currently facing lol. Always a greeat watch. Thank you :D
Damn, I thought I was being proficient in my python skills but some nooby habits are still sticking. Thanks for making us realising that. #pycharm
Totally agree with all of these, but the 2 that irk me the most is altering the iterator and using eval. There's almost always a better way to do it than to use eval, but I think in the age of computer generated code it's especially important to understand the dangers of.
I think filter/map functions are not that bad to read. List comprehension is good, but I would rather not rush rewriting all filter calls to list comp style #pycharm
Exactly
2:17 for this particular example (though I'm not criticizing the tip), the following is even better (20% faster for n = 100000 for me -- and clearer):
s = "".join(f"some string {i}" for i in range(n))
I'll do old school:
func = "some string {}".format
s = ''.join(map(func, range(n)))
but I hate loops.
Yes! Only 3 of 21. But honestly, some points are so advanced I never had the chance to do them wrong :D
My man lost his voice for me! Hell yes what a guy. Great vid bro thanks a lot!
started using dataclasses because I rely heavily on sending dicts around as messages, and then I immediately got the issue of exporting/importing to json. Kind of an annoyance that whole pickle business.
So why is using raw dicts a noobie practice btw? You just said not to do it, but not why.
those filter and map improvements were way harder to understand than the filter and map examples. o_O
I disagree with #14. You shouldn't be using lambdas as arguments to map and filter, but functions not created just for the purpose of this statement are fine. A lot of people don't know you can pass goodies from the `operator` module(such as `itemgetter`) and unbound methods(such as `str.upper`) as the first argument.
I also agree that eval should not be used for parsing data, but you can pretty much completely sandbox it
eval("{'h':6}", {"__builtins__":{}})
Note: someone might still find a way to evaluate unsafe code, so don't do this.
It's just a nice thing to know.
No global exists within this sandbox, if a vulnerability exists, it would abuse the built-in datatypes like `(5).__mul__(8)`
But I can't think of a real exploit for this.
I noticed you used single letter variables in the example prior to your point about not using single letter variables.
Regexes really can't parse mathematical expressions or xml, I give you that. I learned that from people more formally educated in computer science than I am. I learned it AFTER presenting a working example that they indeed can, while being simpler and a whooooole lot faster than parsers. Yet somehow, this new realization didn't break my already written nooby code. 🤣
Well, it's nice to see I'm not as much of a noob as I was when the last video was released. #pycharm
"Concatenating strings with plus" I doubt it gives a meaningful performance improvement if you have string concatenation once or twice in a program. Sure, when I will be writing extra long strings I might actually use this, otherwise it's just unnecessary code obfuscation.
I am a big fan of iterators, and I really disagree about map and filter. I get that sometimes they are less readable, but they also make it way easier to build pipelines of lazy data manipulation. Also, generators and comprehensions have poorer performance, due to the for keyword, afaik. I think this isn't a n00b habit, as most n00bs don't understand iterators and functional / declarative styles, just seems more like a question of taste.
"A lot of noobs, especially people coming from Java" - love it (4:35)
Hey Java was my first language so I can say that!
Great tips, man! One of my favorite channels. Keep up the good work. #pycharm
A "nooby" numpy habbit is also doing inp = np.array(inp) at the beginning of a function to ensure that the input is numpy for the rest of the function. This is bad because the np.array() operation actually makes a new copy of the array even if it is already a numpy array, so if there is a lot of data it will take some time. #pycharm
time to refactor.
#pycharm
"... That tech job you somehow got" oof, felt that!
Great videos though, these videos never fail to help!
Love your channel! Have you considered making a video on 'async for' and 'async with'?
On #5, searching for answers gives some mixed messages, where the immediate evidence is that += is faster than the alternatives in most cases due to some python interpreter trickery, but in testing it's pretty clear that io.StringIO is substantially faster. For 50,000 appends I get 0.00428s using perf_counter, and with StringIO I get 0.00299s. That enormously faster.
My favorite series is back again! #pycharm
9: Some weird audio from the other side during this one from me? Maybe the ghosts like single letter variables.
13: d = {key: val for key, val in d.items() if not val % 2 == 0}
No 4: yes! But....your method names are nooby. If an object writes to a file:
def write(self, fdst).:
wrapped with:
def tofile(self, filename):
Methods are verbs that tell you what they do. Not sentences: separation of concerns.
Man, that behavior of and/or with falsy/truthy values is really strange - it almost reminds me of some of the type coercion weirdness that can occur in Javascript. Much as I like using Python (#pycharm) for certain projects, I definitely appreciate languages that can give compile-time errors for this sort of thing - same goes for deleting items from an iterable during iteration as well.
short circuit logic reduces cyclomatic complexity.
Whenever these come up all I can see are things i do :)
Sometimes, even after 8 years of python, you have some nooby habits :) . As always great video !
For iterating and delete, I often just iterate the copy of it. d.items().copy(). Therefore, I dont need the second loop and the copied object will be deleted after loop anyway, so memory ussage should be the same.
Big fan of this channel. Can't tell you the number of times I've revisited the args, kwargs, /, and * video! #pycharm
Very nice!! When's part 3?? 😂
Been waiting for a Mooney habits vid for a while #pycharm
Map and filter are fine. I like functional programming with functools and itertools. #pycharm
fewer loops is better, imho.
Cool, I've done some of these in the past, will try to remember to use the better way :D #pycharm
#11 I don't actually like @property, when someone creates a setter it usually has a side effect. If I'm looking at where it is used, `obj.x = something` it is really surprising when I find out that this does something more than just changing the x variable. But if I see an `obj.set_x(something)` that immediately raises the alarm: `set_x()` probably does something more - why would anyone bother to write this function otherwise?
I suppose if your setter really has no side effect it is safe to use property, but in my experience that is rare (maybe you want to log where it changed or something like that, it's fine). Usually however, it re-computes related values or notifies other objects that a value has been updated - you should be aware of these.
#pycharm
"Keep that coll tech job you somehow got" pretty much sums it up :D
Thanks for the video! Agree on most points, but I do like map and filter though. It feels like I should switch to Elixir though.
And thanks for organizing the contest! #pycharm
I didn't know about the divmod and String io. Those are some good tips 👍👍
you got to know the builtins.
Really cool video, as always! :) #pycharm
Never knew about Python getters and setters having special properties so that will be my next amendment I have to make for my existing project #pycharm
don't forget about "@attr.deleter" too.
My takeaway as a non-python programmer: Oh dear god, python has support for multiple inheritance.
thanks mate, pretty useful information, some of these I’ve been doing myself, some I didn’t even know about. keep going with this great content.
#pycharm
Nice one, I got a kick out of a lot of those. Remember back when we all were n00bs?
Can you make a video about #pycharm itself? Why should I find it exciting?
Good luck in your path of learning
and a #pycharm license would be nice, yes
Thank you so much! I didnt even know about the named tuple! #pycharm
Didn't knew about pathlib, this video is going on my playlist #pycharm.
The amount of times I've used s += text with thousands of documents. Totally explains the high memory now. #pycharm
Thanks for all the work you do, always enjoying! #pycharm
I'm a noob and I've been using os.path to handle paths, how bad is it as opposed to using pathlib?
Not very bad. Path objects include the context that they are paths. For instance, path / subpath instead of os.path.join(path, subpath), and you can read an entire file with path.read_text(). pathlib is implemented using os.path and friends (path._flavour links to os.path, posixpath or ntpath).
@@0LoneTech thanks! what are the benefits of learning pathlib, then?
These videos are a huge help, man. Thank you. #pycharm
Nice, easy and fast way of looking at python along with leaving with the viewer to practice and test these mistakes/habits on their own , makes me want to learn and use this language more and more. Thanks mCoding .
#pycharm
i noticed that for #15 the first line was actually line 15, and i think it would be a neat little easter egg if the first visible line would always match the current number.
I want to make all of my junior engineers watch this video. Especially after the round of code reviews I went through today
7:03 But what if I want to use the "+" operator? Then, wouldn't I HAVE to implement the "add" dunder method, instead of just a regular method?
Definitely enjoyed this, and picked up some great tips! I have to say, I’m not a fan of main() as a construct. __main__ is great, but then having it just call a main() to do everything feels unnecessary unless there’s a really good reason, and then the person importing your code has to weed it back out to avoid pulling it in. (Plus, if I’m being honest, main() feels a bit too Java-like. :) )
the point of main() is to keep code out of the if/then block for dunder main...devs should see the module end in that block and it should be short, and then they know a main() or usage() [for scripts] is right above it.
Thanks for the video! As always informative! I'd love to see your review for Codon compiler and it's advantages and disadvantages! #pycharm
1 kinda don't agree
7 I'm conflicted on this one. I've been in JS land and got used to the "file-as-module" pattern(?). Now to me it seems so wasteful to define a dedicated class just so I can avoid global variables. Happy to proven wrong about this ofc. Maybe I should just embrace this instead lol
12 the results can also be stored in a "private" variable, yes? Assuming other conditions are taken care of
14 sometimes I use map/filter when I'm too lazy to think of a loop variable name lol. Though more often than tht I consider hsing "_" instead. Dk which is worse heh
Hey James, how do you feel about accepting using `or` to give a value to an attribute if it not set yet like `self.some_dependency = dependency or DefaultDependency()` in the constructor when the parameter `dependency` defaulted to `None` . Or used in a property to lazy load and cache an expensive object: `self._connection = self._connection or DatabaseConnection()`?
Hi, im watching your video lately. I have suggestion as nooby. Can you put more visual like dont and do sign? It can be helpful for someone, i’m sure
When I've just started to learn python, my mentor forced me to use filter, map, reduce etc. List comprehension always seems more readable for me #pycharm
2:59 Weird to promote pydantic as a parsing library. It's just a data mapper with some poorly designed validation features.
Thanks for the #pycharm giveaway!
great content as always, mCoding, really looking into Python recently! Number 3 tips is really good! #pycharm
6:30 I actually faced this recently with a project I’m working on. Do they both do the same thing under the hood (same byte code) or is one preferred over the other for different reasons? (In reference to using functional functions like map or filter over comprehensions).
Thank you for yet another super useful and interesting video! #pycharm
I'll defend facilitating variable rw with global instead of adding it as an input and outputting it again. In some cases, anyway, it's the same thing in shorter syntax. Problem where?
As always very high quality content. I love it. Keep it up 👍
#pycharm
I didn't know that raw strings could be used for more than just regex, so much time could have been saved. #pycharm